UNIVERSITY OF MINNESOTA 
SCHOOL OF MATHEMATICS

Math 5251
The Mathematics of Coding:
Information, Compression,
Error Correction and Finite Fields

Fall 2021

Prerequisites: Single-variable calculus, and a solid background in linear algebra.
Some familiarity with modular arithmetic might help, but is not required.
We will eventually understanding something about finite fields, their structure, and matrices/linear algebra using them.
For this reason, the course has a slightly higher mathematical level than Math 5248.
Instructor: Victor Reiner (You can call me "Vic"). 
Office: Vincent Hall 256
Telephone (with voice mail): (612) 625-6682
E-mail: reiner@math.umn.edu 
Classes: Monday, Wednesday, Friday 1:25-2:15pm, in-person in Vincent Hall 211
COVID policy: The University currently requires all students, staff, faculty to wear masks when indoors regardless of vaccination status. As your instructor, I should be wearing either a face shield or mask when in front of the room. The University strongly encourages all members of its community to get vaccinated. Resources are available for accessing vaccines. Please stay home if you experience symptoms of COVID-19, and consult with your healthcare provider about an appropriate course of action. An absence due to symptoms of COVID-19 is an excused absence, and I will work with you to find the best course of action for missed work and/or class experiences.
Office hours: Wed 12:20pm in VinH 256
Tues 9:05am, Thur 9:05am at this Zoom link.
Class Discord server: Here is our Discord server.
Let me know if you have ideas for channels, particularly those that could facilitate some study groups.  

Course content: This is an introductory course in the mathematics of codes for communication
designed to achieve compression of information and error-detecting/correction.
We intend to cover much of the text by Garrett listed below, including treatment of
  • Elementary information theory and entropy
  • Simple compression schemes and noiseless coding: Kraft-McMillan inequality Shannon's noiseless coding theorem, Huffman coding
  • Error-detection/correction and Shannon's noisy coding theorem
  • Error-correcting codes, with an emphasis on linear codes, parity check matrices, syndrome decoding
  • Bounds on efficiency of error-correcting codes: Hamming, Singleton, Plotkin, Gilbert-Varshamov
  • Finite fields and their structure
  • Cyclic linear codes, such as Hamming, Reed-Solomon, BCH codes.
  • A few other codes, e.g. Golay, Reed-Muller codes.
  • (Not discussed: codes from algebraic curves)
What this course is NOT: This course should not be confused with a course in
  • codes designed for secrecy (see Math 5248 Cryptology and Number Theory)
  • compression via wavelets (see Math 5467 Introduction to the Mathematics of Wavelets)
  • abstract algebra of fields at a more theoretical level (see Math 5286H Fundamentals of Abstract Algebra)
  • the details of serious engineering applications/implementation
    (see EE5501 Digital communication, EE5581 Information theory and coding, EE5585 Data compression)
Text. materials, resources:
  • The required text is Paul Garrett's book,
    The Mathematics of Coding: Information, Compression, Error Correction,and Finite Fields
    and don't miss his very useful coding theory page, with his lecture slides, quiz solutions, etc.
  • A non-required but useful text is Steven Roman's Coding and Information Theory (only Chaps 1,2 here; whole book available elsewhere). A few extra topics we may touch on from Roman's book: non-binary Huffman coding, Plotkin bound, Reed-Muller codes, Golay codes, latin squares.
  • Richard Ehrenborg's parlor trick using the Hamming binary [7,4,1]-code, from first day of class.
  • Eugenia Cheng's short Wall Street Journal article on QR codes in relation to Reed-Solomon codes.
  • 3Blue1Brown's nice video titled "How to send a self-correcting message", teaching you about Hamming codes.
  • John Baez's very cool blog post on Golay codes, with great pictures.
  • A recent Quanta article on a breakthrough in new codes.
  • My old syllabi from Spring 2006, Spring 2007,

Lecture notes
Date Lecture topics
Wed Sept. 8
Fri Sept. 10
Intro and Noiseless Coding (Sec. 3.1)
Mon Sept. 13 Kraft-McMillan inequalities (Sec. 3.2)
Wed Sept. 15
Fri Sept. 17
Probability and Entropy (Sec. 1.4, 1.5. 2.2)
Fri Sept. 17
Mon Sept . 20
Entropy and Shannon's Noiseless Coding Theorem (Sec. 3.3)
Mon Sept. 20
Wed Sept. 22
Fri Sept 24
Huffman coding (Sec. 3.4)
Mon Sept. 27
Wed Sept. 29
Noisy coding (Sec. 4.1, 4.2)
Wed Sept. 29
Fri Oct. 1
Hamming distance, Shannon's Noisy Coding Thm (Sec. 4.3, 4.4, 4.5)
Mon Oct. 4
Wed Oct. 6
Cyclic Redundancy Checks (Chap. 5)
Fri Oct. 8
Mon Oct. 11
Wed Oct. 13
Fri Oct. 15
Rings and Integers mod m (Chap. 6.5, 6.7)
Mon Oct. 18
Wed Oct. 20
Fri Oct. 22
Mon Oct. 25
Polynomials, Euler's and Fermat's Theorems (Chap. 10, Sec. 6.9, 6.10)
Wed Oct. 27
Fri Oct. 29
Minimum Distance and Linear Codes (Chap. 12, Appendix A.1, A.2)
Mon Nov. 1
Wed Nov. 3
Fri Nov. 5
Mon Nov. 8
Wed. Nov. 10
Encoding, Decoding with Linear Codes (Sec. 12.4, 12.7, 12.8, 14.1)
Fri Nov. 12 Reed-Muller Codes (Roman Sec. 6.2)
Mon Nov. 15
Wed Nov. 17
Bounds on Codes (Chap. 13)
Fri Nov. 19
Mon Nov. 22
Wed Nov. 24
Cyclic Codes (Sec. 14.2)
Mon Nov. 29
Wed Dec. 1
Fri Dec. 3
Finite Fields and Primitive Roots (Chaps. 11, 15)
Mon Dec. 6
Wed Dec. 8
Reed-Solomon Codes (Sec. 17.1-17.3)
Fri Dec. 10
Mon Dec. 12
Finite Fields and Frobenius map (Secs. 15.2, 17.5)

Homework and exams: There will likely be 6 homework assignments due in the class Canvas site by midnight on Wednesdays, generally every other week, except for
  • the 6th homework will be only one week (see the schedule below),
  • 2 weeks where there will be a week-long take-home midterm exam,
  • a week at the end with a week-long take-home final exam.
Dates for the assignments and exams are in the schedule below, to give you an idea of what will happen. The take-home midterms and final exam are open-book, open-library, open-web, but no collaboration or consultation of human sources is allowed.
Late homework will not be accepted. Early homework is fine, and can be left in my mailbox in the School of Math mailroom near Vincent Hall 105. Collaboration is encouraged as long as everyone collaborating understands the solution thoroughly, and you write up the solution in your own words, along with a note at the top of the homework indicating with whom you've collaborated.
Homework solutions should be well-explained-- the grader will be told not to give credit for an unsupported answer.
Grading:  Homework = 50% of grade
Each of 2 midterms = 15% of grade
Final exam 20% of grade.
Complaints about the grading should be brought to Vic.
Policy on incompletes:  Incompletes will be given only in exceptional circumstances, where the student has completed almost the entire course with a passing grade, but something unexpected happens to prevent completion of the course. Incompletes will never be made up by taking the course again later. You must talk to me before the final exam if you think an incomplete may be warranted.  
Other expectations  This is a 4-credit course, so I would guess that the average student should spend about 8 hours per week outside of class to get a decent grade. Part of this time each week would be well-spent making a first pass through the material in the book that we anticipate to cover in class that week, so that you can bring your questions/confusions to class and ask about them.

Homework/exam schedule and assignments
Assignment
or Exam
Due date
by midnight at the
course Canvas site
Problems due,
mainly from Garrett's text
Homework 1 Wed Sept. 22 From Garrett's text:
1.28, 1.31, 1.33
(In Problem 1.31, assume drawing with replacement,
that is, after each time you draw a ball of either color,
you put it back into the urn before doing the next drawing.)
2.03
3.02 (moved 3.05 to HW 2)

Not from text:
A. Consider a source W={A,B,C,D,E} and encoding maps f:W→Ci mapping the 5 letters in order onto these three collections C1, C2, C3 of codewords:
C1={0,10,110,1110,1111}
C2={0,10,110,1110,1101}
C3={0,01,011,0111,1111}
Indicate for each (with explanation) whether or not f is
(a) uniquely decipherable,
(b) prefix (=instantaneous).

B. Does there exist a binary code which is instantaneous and has code words with lengths (1,2,3,3)?
If not, prove it. If so, construct one.
Homework 2 Wed Oct. 6 From Garrett's text:
2.04
3.05
4.01, 4.02, 4.04, 4.05, 4.06, 4.11
Midterm exam 1 Wed. Oct. 13 Midterm 1, to be turned in at the class Canvas site.
Homework 3 Wed Oct. 27 From Garrett's text:
5.01, 5.02, 5.03, 5.04, 5.05, 5.08
6.01, 6.03, 6.07, 6.22, 6.37, 6.49, 6.50, 6.57, 6.52, 6.80, 6.81
8.17
Homework 4 Wed Nov. 10 From Garrett's text:
6.30, 6.31
9.11, 9.12
10.04, 10.08, 10.11
11.11
12.06, 12.10, 12.12, 12.14, 12.15
Midterm exam 2 Wed. Nov. 17 Midterm 2, to be turned in at the class Canvas site.
Homework 5 Wed Dec. 1 From Garrett's text:
12.01, 12.02, 12.04, 12.17, 12.19, 12.20
13.02, 13.05, 13.07, 13.09, 13.10
Homework 6 Wed Dec. 8
(note 1-week due date!)
From Garrett's text:
11.04, 11.05
14.02, 05
15.03, 13
Final exam Wed. Dec. 15 Final exam, to be turned in at the class Canvas site.

Back to Reiner's Homepage.