UNIVERSITY OF MINNESOTA 
SCHOOL OF MATHEMATICS

Math 5251
The Mathematics of Coding:
Information, Compression,
Error Correction and Finite Fields

Spring 2022

Prerequisites: Single-variable calculus, and a solid background in linear algebra.
Some familiarity with modular arithmetic might help, but is not required.
We will eventually understanding something about finite fields, their structure, and matrices/linear algebra using them.
For this reason, the course has a slightly higher mathematical level than Math 5248.
Instructor: Victor Reiner (You can call me "Vic"). 
Office: Vincent Hall 256
Telephone (with voice mail): (612) 625-6682
E-mail: reiner@math.umn.edu 
Classes: 4:00-05:15 PM on Mondays, Wednesdays in-person in Vincent Hall 113
COVID policy: Here is the explanation of University COVID policy from the University Senate.
Office hours: 5:20-6:00pm on Mondays, Wednesdays in-person in VinH 256,
and Tuesdays 9:05-9:55am at this Zoom link.
Class Discord server: Use our class Discord server to introduce yourself, ask questions, form study groups, etc.  
Course content: This is an introductory course in the mathematics of codes for communication
designed to achieve compression of information and error-detecting/correction.
We intend to cover much of the text by Garrett listed below, including treatment of
  • Elementary information theory and entropy
  • Simple compression schemes and noiseless coding: Kraft-McMillan inequality Shannon's noiseless coding theorem, Huffman coding
  • Error-detection/correction and Shannon's noisy coding theorem
  • Error-correcting codes, with an emphasis on linear codes, parity check matrices, syndrome decoding
  • Bounds on efficiency of error-correcting codes: Hamming, Singleton, Plotkin, Gilbert-Varshamov
  • Finite fields and their structure
  • Cyclic linear codes, such as Hamming, Reed-Solomon, BCH codes.
  • A few other codes, e.g. Golay, Reed-Muller codes.
  • (Not discussed: codes from algebraic curves)
What this course is NOT: This course should not be confused with a course in
  • codes designed for secrecy (see Math 5248 Cryptology and Number Theory)
  • compression via wavelets (see Math 5467 Introduction to the Mathematics of Wavelets)
  • abstract algebra of fields at a more theoretical level (see Math 5286H Fundamentals of Abstract Algebra)
  • the details of serious engineering applications/implementation
    (see EE5501 Digital communication, EE5581 Information theory and coding, EE5585 Data compression)
Text. materials, resources:

Lecture notes
Date(s) discussed Lecture topics
Wed Jan. 19, Mon Jan. 24 Intro and Noiseless Coding (Sec. 3.1)
Mon Jan. 24, Wed Jan. 26 Kraft-McMillan inequalities (Sec. 3.2)
Wed Jan. 26, Mon. Jan. 31 Probability and Entropy (Sec. 1.4, 1.5. 2.2)
Mon. Jan. 31, Wed Feb. 2 Entropy and Shannon's Noiseless Coding Theorem (Sec. 3.3)
Wed, Feb. 2, Mon Feb. 7 Huffman coding (Sec. 3.4)
Mon Feb. 7, Wed. Feb. 9 Noisy coding (Sec. 4.1, 4.2)
Mon Feb. 14 Hamming distance, Shannon's Noisy Coding Thm (Sec. 4.3, 4.4, 4.5)
Wed Feb. 16, Mon. Feb. 21 Cyclic Redundancy Checks (Chap. 5)
Mon Feb. 21, Wed Feb. 23, Mon Feb 28 Rings and Integers mod m (Chap. 6.5, 6.7)
Mon Feb 28, Wed Mar 2, Mon Mar 14 Polynomials, Euler's and Fermat's Theorems (Chap. 10, Sec. 6.9, 6.10)
Wed Mar 16, Mon Mar 21 Minimum Distance and Linear Codes (Chap. 12, Appendix A.1, A.2)
Mon Mar 21, Wed Mar 23, Mon Mar 28, Wed Mar 30 Encoding, Decoding with Linear Codes (Sec. 12.4, 12.7, 12.8, 14.1)
Mon Apr 4 Reed-Muller Codes (Roman Sec. 6.2)
Mon Apr 4, Wed Apr 6 Bounds on Codes (Chap. 13)
Mon Apr 11, Wed Apr 13 Cyclic Codes (Sec. 14.2)
Wed Apr 13, Mon Apr 18, Wed Apr 20 Finite Fields and Primitive Roots (Chaps. 11, 15)
Mon Apr 25, Wed Apr 27 Reed-Solomon Codes (Sec. 17.1-17.3)
Wed Apr 27, Mon May2 Finite Fields and Frobenius map (Secs. 15.2, 17.5)
Homework and exams: There will likely be 6 homework assignments due in the class Canvas site by midnight on Wednesdays, generally every other week, except for
  • the 6th homework will be only one week (see the schedule below),
  • 2 weeks where there will be a week-long take-home midterm exam,
  • a week at the end with a week-long take-home final exam.
Dates for the assignments and exams are in the schedule below, to give you an idea of what will happen. The take-home midterms and final exam are open-book, open-library, open-web, but no collaboration or consultation of human sources is allowed.
Late homework will not be accepted. Early homework is fine, and can be left in my mailbox in the School of Math mailroom near Vincent Hall 105. Collaboration is encouraged as long as everyone collaborating understands the solution thoroughly, and you write up the solution in your own words, along with a note at the top of the homework indicating with whom you've collaborated.
Homework solutions should be well-explained-- the grader will be told not to give credit for an unsupported answer.
Grading:  Homework = 50% of grade
Each of 2 midterms = 15% of grade
Final exam 20% of grade.
Complaints about the grading should be brought to Vic.
Policy on incompletes:  Incompletes will be given only in exceptional circumstances, where the student has completed almost the entire course with a passing grade, but something unexpected happens to prevent completion of the course. Incompletes will never be made up by taking the course again later. You must talk to me before the final exam if you think an incomplete may be warranted.  
Other expectations  This is a 4-credit course, so I would guess that the average student should spend about 8 hours per week outside of class to get a decent grade. Part of this time each week would be well-spent making a first pass through the material in the book that we anticipate to cover in class that week, so that you can bring your questions/confusions to class and ask about them.

Homework/exam schedule and assignments
Assignment
or Exam 		Due date
by midnight at the
course Canvas site		 Problems due,
mainly from Garrett's text
Homework 1 Wed Feb. 2 From Garrett's text:
1.28, 1.31, 1.33
(In Problem 1.31, assume drawing with replacement,
that is, after each time you draw a ball of either color,
you put it back into the urn before doing the next drawing.)
2.03
3.02

Not from text:
A. Consider a source W={A,B,C,D,E} and encoding maps f:W→Ci mapping the 5 letters in order onto these three collections C1, C2, C3 of codewords:
C1={0,10,110,1110,1111}
C2={0,10,110,1110,1101}
C3={0,01,011,0111,1111}
Indicate for each (with explanation) whether or not f is
(a) uniquely decipherable,
(b) prefix (=instantaneous).

B. Does there exist a binary code which is instantaneous and has code words with lengths (1,2,3,3)?
If not, prove it. If so, construct one.
Homework 2 Wed Feb. 16 From Garrett's text:
2.04
3.05
4.01, 4.02, 4.04, 4.05, 4.06, 4.11
Midterm exam 1 Wed. Feb. 23 Midterm 1, to be turned in at the class Canvas site.
Homework 3 Wed Mar. 16 From Garrett's text:
5.01, 5.02, 5.03, 5.04, 5.05, 5.08
6.01, 6.03, 6.07, 6.22, 6.37, 6.49, 6.50, 6.57, 6.52, 6.80, 6.81
8.17
Homework 4 Wed Mar. 30 From Garrett's text:
6.30, 6.31
9.11, 9.12
10.04, 10.08, 10.11
11.11
12.06, 12.10, 12.12, 12.14, 12.15
Midterm exam 2 Wed. Apr. 6 Midterm 2, to be turned in at the class Canvas site.
Homework 5 Wed. Apr. 20 From Garrett's text:
12.01, 12.02, 12.04, 12.17, 12.19, 12.20
13.02, 13.05, 13.07, 13.09, 13.10
Homework 6 Wed Apr. 27
(note 1-week due date!) 		From Garrett's text:
11.04, 11.05
14.02, 05
15.03, 13
Final exam Wed. May 4 Final exam, to be turned in at the class Canvas site.

Back to Reiner's Homepage.