# EE5581: Information Theory and Coding

## University of Minnesota, Fall 2005

### Professor Nihar Jindal

This course covers the basics of information theory and coding. The two fundamental questions that information theory considers are:
1. What is the ultimate limit to data compression?
2. What is the ultimate limit of reliable communication over a noisy channel?
By the end of this course we will have answered both questions, as well as provided some basic guidelines for designing practical systems that come close to the fundamental limits.
The course begins by defining the fundamental quantities in information theory: entropy and mutual information. Then we consider data compression (source coding), followed by reliable communication over noisy channels (channel coding). The final topic of the course will be rate distortion theory (lossy source coding).

## Course Information

• Instructor: Nihar Jindal, 6-119 EE/CS, nihar@umn.edu, 625-6306
• Lecture: Mon/Wed/Fri, 12:20 PM - 1:10 PM, MechE 102
• Office Hours: Tu/Th, 10:30 AM - 12:00 PM, 6-119 EE/CS
• TA: Hakim Alhussien, 6-158 EE/CS, hakimh@umn.edu, 626-7178
• TA Office Hours: MW 4-5 PM, 6-158 EE/CS
• Required Textbook: Elements of Information Theory, Thomas Cover & Joy Thomas, John Wiley & Sons
• Prerequisite: EE 5531, Probability and Stochastic Processes (or equivalent).
• Homework: There will be weekly homework assignments.
• Exams: There will be two midterm exams (Oct. 14 and Nov. 18) and a final exam (Wed, Dec. 21, 4-6 PM).
• Project: A research project is a required portion of this course. The project can either be a literature survey of a few information theory papers, or an original research idea. The project will be due towards the end of the semester.
• Grading: Final grade will be 15% project, 15% homework, 20% each midterm, and 30% final.

## Homework

• Homework 1: C&T 2.16, 2.8, 2.2, 2.5, 2.1(a), 2.10
• Homework 2: C&T 2.6, 2.12, 2.18, 2.20, 2.28, 2.29, 2.14
• Homework 3: C&T 6.3, 3.2, 3.5, 3.6, 4.4, 4.8, 4.9
• Homework 4
• Homework 5: C&T 8.1, 8.2, 8.3, 8.5, 8.6, 8.7
• Homework 6: C&T 8.4, 8.9, 8.10, 8.11, 8.12
• Homework 7
• Homework 8: C&T 9.1, 9.2, 9.3, 9.4 (a,b), 10.2
• Homework 9: C&T 10.1, 10.3, 10.4, 13.1, 13.3
• Homework 10: C&T 13.2, 13.5, 13.6, 13.7, 13.8