UNIVERSITY OF MINNESOTA 
SCHOOL OF MATHEMATICS

Math 5251 The Mathematics of Coding:
Information, Compression, Error Correction and Finite Fields

Fall 2024

Prerequisites: Single-variable calculus, and a solid background in linear algebra.
Some familiarity with modular arithmetic might help, but is not required.
We will eventually understanding something about finite fields, their structure, and matrices/linear algebra using them.
For this reason, the course has a slightly higher mathematical level than Math 5248.
Instructor: Victor Reiner (You can call me "Vic"). 
Office: Vincent Hall 256
Telephone (with voice mail): (612) 625-6682
E-mail: reiner@math.umn.edu 
Classes: 12:20-2:15 PM on Mondays, Wednesdays in-person in Vincent Hall 211
Office hours: To be determined. Initial proposal:
Mon, Tues 4:40-5:30 in my office,
and via this Zoom link.
Class Discord server: Use our class Discord server to introduce yourself, ask questions, form study groups, etc.  

Course content: This is an introductory course in the mathematics of codes for communication
designed to achieve compression of information and error-detecting/correction.
We intend to cover much of the text by Garrett listed below, including treatment of
  • Elementary information theory and entropy
  • Simple compression schemes and noiseless coding: Kraft-McMillan inequality Shannon's noiseless coding theorem, Huffman coding
  • Error-detection/correction and Shannon's noisy coding theorem
  • Error-correcting codes, with an emphasis on linear codes, parity check matrices, syndrome decoding
  • Bounds on efficiency of error-correcting codes: Hamming, Singleton, Plotkin, Gilbert-Varshamov
  • Finite fields and their structure
  • Cyclic linear codes, such as Hamming, Reed-Solomon, BCH codes.
  • A few other codes, e.g. Golay, Reed-Muller codes.
  • (Not discussed: codes from algebraic curves)
Some student
learning outcomes:
By the end of the course, among other things, we would like you to know ...
  • why Morse code makes "E" one of the shortest letters to encode, and "Q" longer, as well as how to improve this design based on known letter frequencies,
  • how to mathematically quantify the amount of information conveyed when told the outcome of some random event,
  • as well as how to quantify the capacity of a noisy channel to transmit information,
  • who are all of the finite fields, and how to construct them,
  • a bit about the trade-off between error-detection/correction and efficiency of codes,
  • how to efficiently decode words sent using a linear code,
  • a few famous error-correcting codes: Hamming, Reed-Muller, Golay, Reed-Solomon.

  • What this course
    is NOT about:
    This course should not be confused with a course in
    • codes designed for secrecy (see Math 5248 Cryptology and Number Theory)
    • compression via wavelets (see Math 5467 Introduction to the Mathematics of Wavelets)
    • abstract algebra of fields at a more theoretical level (see Math 5286H Fundamentals of Abstract Algebra)
    • the details of serious engineering applications/implementation
      (see EE5501 Digital communication, EE5581 Information theory and coding, EE5585 Data compression)

    Text,
    materials,
    resources:

    Lecture notes
    Date(s) discussed Lecture topics
    (with section from Garrett text indicated)
    Wed Sept 4, Mon Sept 9 Intro and Noiseless Coding (Sec. 3.1)
    Mon Sept 9, Wed Sept 11 Kraft-McMillan inequalities (Sec. 3.2)
    Wed Sept 11, Mon Sept 16 Probability and Entropy (Sec. 1.4, 1.5. 2.2)
    Mon Sept 16, Wed Sept 18 Entropy and Shannon's Noiseless Coding Theorem (Sec. 3.3)
    Wed Sept 18, Mon Sept 23 Huffman coding (Sec. 3.4)
    Mon Sept 23, Wed Sept 25, Mon Sept 30 Noisy coding (Sec. 4.1, 4.2)
    Mon Sept 30, Wed Oct 2 Hamming distance, Shannon's Noisy Coding Thm (Sec. 4.3, 4.4, 4.5)
    Mon Oct 7 Cyclic Redundancy Checks (Chap. 5)
    No lecture -- watch these videos: Video 1, Video 2, Video 3
    Wed Oct 9, Mon Oct 14 Rings and Integers mod m (Chap. 6.5, 6.7)
    Wed Oct 16, Mon Oct 21 Polynomials, Euler's and Fermat's Theorems (Chap. 10, Sec. 6.9, 6.10)
    Wed Oct 23, Mon Oct 28 Minimum Distance and Linear Codes (Chap. 12, Appendix A.1, A.2)
    Encoding, Decoding with Linear Codes (Sec. 12.4, 12.7, 12.8, 14.1)
    Reed-Muller Codes (Roman Sec. 6.2)
    Bounds on Codes (Chap. 13)
    Cyclic Codes (Sec. 14.2)
    Finite Fields and Primitive Roots (Chaps. 11, 15)
    Reed-Solomon Codes (Sec. 17.1-17.3)
    Finite Fields and Frobenius map (Secs. 15.2, 17.5)

    Homework and exams: There will be 6 homework assignments due in the class Canvas site by midnight on Wednesdays, generally every other week, except for
    • the 6th homework will be only one week (see the schedule below),
    • 2 weeks where there will be a week-long take-home midterm exam,
    • a week at the end with a week-long take-home final exam.
    Dates for the assignments and exams are in the schedule below, to give you an idea of what will happen.
    The take-home midterms and final exam are open-book, open-library, open-web, but no collaboration or consultation of human sources is allowed.
    You must name any sources that you used, beyond Garrett's text, including AI sources such as ChatGPT.
    Use your judgement when consulting such AI sources, as they sometimes spout nonsense, and relying on them too much will hamper your own learning.

    Late homework will not be accepted. Early homework is fine.
    Collaboration is encouraged as long as everyone collaborating understands the solution thoroughly, and you write up the solution in your own words, along with a note at the top of the homework indicating with whom you've collaborated.
    This includes AI sources such as ChatGPT.

    Homework solutions should be well-explained-- the grader will be told not to give credit for an unsupported answer.
    Grading:  Homework = 50% of grade
    Each of 2 midterms = 15% of grade
    Final exam 20% of grade.
    Complaints about the grading should be brought to Vic.
    Policy on incompletes:  Incompletes will be given only in exceptional circumstances, where the student has completed almost the entire course with a passing grade, but something unexpected happens to prevent completion of the course. Incompletes will never be made up by taking the course again later. You must talk to me before the final exam if you think an incomplete may be warranted.  
    Other expectations  This is a 4-credit course, so I would guess that the average student should spend about 8 hours per week outside of class to get a decent grade. Part of this time each week would be well-spent making a first pass through the material in the book and the posted lecture notes that we anticipate to cover in class that week, so that you can bring your questions/confusions to class and ask about them.

    Homework/exam schedule and assignments
    Assignment
    or Exam
    Due date
    by midnight at the
    course Canvas site
    Problems due,
    mainly from Garrett's text
    Homework 1 Wed Sept 18 From Garrett's text:
    1.28, 1.31, 1.33
    (In Problem 1.31, assume drawing with replacement,
    that is, after each time you draw a ball of either color,
    you put it back into the urn before doing the next drawing.)
    2.03
    3.02

    Not from text:
    A. Consider a source W={A,B,C,D,E} and encoding maps f:W→Ci mapping the 5 letters in order onto these three collections C1, C2, C3 of codewords:
    C1={0,10,110,1110,1111}
    C2={0,10,110,1110,1101}
    C3={0,01,011,0111,1111}
    Indicate for each (with explanation) whether or not f is
    (a) uniquely decipherable,
    (b) prefix (=instantaneous).

    B. Does there exist a binary code which is instantaneous and has code words with lengths (1,2,3,3)?
    If not, prove it. If so, construct one.
    Homework 2 Wed Oct 2 From Garrett's text:
    2.04
    3.05
    4.01, 4.02, 4.04, 4.05, 4.06, 4.11
    In Problem 4.05, a minor typo correction:
    after parity bits are added, the codewords should be
    00, 101, 1100, 11101, 11110.
    Also, in the same problem, before adding the extra parity bit,
    any digit that gets flipped in transmission is considered an undetected error.
    Midterm exam 1 Wed Oct 9 Midterm 1, to be turned in at the class Canvas site.
    Homework 3 Wed Oct 23 From Garrett's text:
    5.01, 5.02, 5.03, 5.04, 5.05, 5.08
    6.01, 6.03, 6.07, 6.22, 6.37, 6.49, 6.50, 6.57, 6.52, 6.80, 6.81
    8.17
    Homework 4 Wed Nov 6 From Garrett's text:
    6.30, 6.31
    9.11, 9.12
    10.04, 10.08, 10.11
    11.11
    12.06, 12.10, 12.12, 12.14, 12.15
    Midterm exam 2 Wed Nov 13 Midterm 2 will appear here in PDF,
    to be turned in at the class Canvas site.
    Homework 5 Wed Nov 27 From Garrett's text:
    12.01, 12.02, 12.04, 12.17, 12.19, 12.20
    13.02, 13.05, 13.07, 13.09, 13.10
    Homework 6 Wed Dec 4
    (note 1-week due date!)
    From Garrett's text:
    11.04, 11.05
    14.02, 05
    15.03, 13
    Final exam Wed. Dec 11 Final exam will appear here in PDF,
    to be turned in at the class Canvas site.

    Back to Reiner's Homepage.