Error detecting/correcting code activity

This activity is designed to give students some insight into how linear algebra and, in particular, the viewpoint of matrix/vector product as producing linear combination of the column vectors of the matrix, crops up in yet another area of computer science: error detecting/correcting codes.
  1. explain setup, including need to expand message
  2. show the tool with gaussian elim calculator and Z_29 calculator
  3. calc $Gm=b$ (have them do a row each for you) transmit $b$ as text (to other white board), recive $b'$ w/o errors, how to recover message? Solve $Gx = b'$ for $x$.
  4. show single error, show how solving $Gx=b'$ fails
  5. show $Hb' \neq 0$, show $H(b'-e) = 0$, where $e$ is the unkown error, rewrite as $He = Hb'$.
  6. hypothesize single error in position $i$, get $\left[ \begin{array}{c} h_{1,i}\\ h_{2,i} \end{array}\right] \cdot x = Hb'$
  7. have students try the different "$i$" hyptheses and see what works
  8. recover $b$ and solve back to textual message
  9. Challenge for them: break into groups and find message ajkqd (note: answer should be ode!)

Sequences

A sequence is just an ordered list of numbers
sequence $c=$  1,  2,  3,  2,  3,  4,  3,  4,    ← a finite sequence
0 1 2 3 4 5 6 7 8
The elements of the sequence are indexed as 0, 1, 2 ... We start at 0, because computer scientists know better! Often we refer to the sequence element at index $i$ using a subscript. So, for example, $c_6 = 3$. Things can be more interesting, though, with something like
sequence $b=$  1,  2,  3,  2,  3,  4,  3,  4,  5,  ...   ← an infinite sequence
0 1 2 3 4 5 6 7 8
That "..." makes all the difference! Now we have infinite sequences.

Important! Note that an infinte sequence is a function with domain $\mathbb{N}$ (i.e. the non-negative numbers). If we take that view, we would say that $b(6) = 3$. We will freely go back and forth between viewing a sequence as a list, and viewing a sequence as a function with domain $\mathbb{N}$.

Explicitly defined sequences

Sometimes we have an explicit formula for the elements of the sequence (or, equivalently, for the function). For example, I might say: consider the sequence $$a_i = i\cdot i+1$$ This would define the sequence: $$a = 1,2,5,10,17,26,37, \ldots$$ We call "$a_i = i\cdot i+1$" a closed form expression for the elements of sequence $a$. (We would also say that, in this case, the sequence is explicitly defined.) It's great when you have a closed form expression for the elements of a sequence. You can instantaly compute $a_i$, even if $i$ is huge. For example, $a_{100} = 10001$.

Implicitly defined sequences

More often, and more interestingly, the elements of a sequence aren't given explicitly. Instead, they are given implicitly. This means we get an expression for one sequence element in terms of earlier elements of the sequence. For example: $$a_i = a_{i-1} + 2\cdot i - 1$$ Now, on it's own, this doesn't define a sequence. The problem is we can't use this rule to compute $a_0$. Do you see why not? So we have to add an explicit value for $a_0$ $$a_0 = 1$$ Put together, we get the implicit representation
$a_i = a_{i-1} + 2\cdot i - 1$
$a_0 = 1$
We should be able to compute as many of the sequence elements as we want from this. Start with $a_0$, which we are given is 1. Then compute $a_1$ as $a_1 = a_0 + 2i-1 = 1 + 2\cdot 1-1=2$ then compute $a_2$, etc. [Do it!]
	a =  1 ,  2 ,         5 ,        10 ,        17 ,  ...
             |    |           |           |           |
	   a0=1  a1=a0+2i-1  a2=a1+2i-1  a3=a2+2i-1  a4=a3+2i-1
                   =1+2*1-1    =2+2*2-1    =5+2*3-1    =10+2*4-1
                   =2          = 5         =10         =17      
It looks like we are getting the same sequence as in the previous section. Can we prove it? I.e. can we prove that the sequence given implicitly by $$ \begin{array}{l} a_i = a_{i-1} + 2\cdot i - 1,\\ a_0 = 1 \end{array} $$ has closed form $a_i = i\cdot i + 1$? Of course we can verify it by computing as many elements of the sequence as we want, but to prove that this is true for any non-negative $i$ ... well it requires induction, of course!

Let $a$ be the sequence defined by $a_i = a_{i-1} + 2\cdot i - 1$, $a_0 = 1$. Then $a_i = i\cdot i + 1$.
We will prove this by induction on $i$.
Base case $i=0$. $a_0 = 1$ and $i\cdot i+1 = 0\cdot 0+1=1$. ✓
Inductive case $i \gt 0$: In this case $$ \begin{array}{rcl} a_i &=& a_{i-1} + 2\cdot i - 1\ \ {\color{red}{\leftarrow\text{ By the inductive hypothesis, } a_{i-1} = (i-1)\cdot(i-1)+1}}\\ &=& {\color{red}{(i-1)\cdot(i-1)+1}} + 2i-1 \\ &=& i^2-2i+1 + 1 + 2i - 1 \\ &=& i^2+1 \end{array} $$ ... which takes care of the inductive case. ✓

Just call me Recurrence Relation

In the previous section, we used the recurrence relation to construct the sequence in a "bottom up" way. We started with a0, then found a1, then found a2, and so on.

[Do by hand recursive style (building a stack)]

Doesn't this look just like recursion?!?!?! In fact, we can translate the implicit form directly into a recursive function:
int a(int i) {
  if (i==0)
    return 1;
  else
    return a(i-1) - 1 + 2*i;
}
So the implict form of a sequence and recursion in programming have a very close relationship. In fact, an implicit form of a seqeuence (or, equivalently, the implict form of a function with domain $\mathbb{N}$) is called a recurrence relation. Hopefully the above discussion helps you see why it has this name! We won't give a formal definition of "recurrence relation" yet, because there are actually many different kinds of recurrence relations and we're not ready to distinguish between all the different types and/or give a super general definition.

Important! Pretty much the central problem related to recurrence relations is this: given a recurrence relation for function f, find a closed form representation for f.

In the above we did just that. Of course we were lucky. We generated some terms in the sequence from the recurrence relation, and we noticed that they matched a sequence we'd seen before and knew a closed form for. Then all we had to do was prove that the two matched not only for the few terms we'd generated, but for all terms ... which of course required induction. What happens if we don't get lucky, and the sequence we have a recurrence relation for doesn't match something we already know in closed form? Well ... we'll spend the next several classes scratching away at the tip of that particular iceberg!

Why we computer scientists should care about recurrence relations

Let's look at a simple example of a problem that gives rise to a recurrence relation. Assuming we have a linked list "Node" class defined as you are used to from IC210/SI204, the following function prints out the first $n$ elements of a linked list in reverse order.
1: void prn(Node* L, int n) {
2:   if (n == 0)
3:     return;
4:   Node* p = L;
5:   for(int i = 0; i < n-1; i++)
6:     p=p->next;
7:   cout << p->data << ' ';
8:   prn(L,n-1);
9: }
To understand how efficient/inefficient this algorithm is, we want to count, for example, how many ->s to check values from linked list nodes are executed. Of course, this depends on $n$, so the number of ->s executed is a function of $n$: let's call it $T(n)$. If we look at the code we get n-1 ->s from the loop on lines 5-6, and one -> from line 7. Plus, we get $T(n-1)$ ->s from the call on line 8. Since the $n=0$ case on line 2 just returns immediately, $T(0)=0$. So: $$ \begin{array}{l} T(n) = n+T(n-1)\\ T(0) = 0 \end{array} $$ Viola! Trying to understand how much work our code does leads us straight to a recurrence relation. This is one of many reasons that recurrence relations are important to computer science.


Christopher W Brown