EE8591: Course Information

Syllabus

pdf file

Lectures

• Time: Tuesdays and Thursdays, 11:15am–12:30pm

• Location: Ackerman Hall 319

Course description

Methods for estimating dependencies from data have been traditionally explored in such diverse fields as: Statistics (multivariate regression and classification), Engineering (pattern recognition, system identification) and Computer Science (artificial intelligence, machine learning, data mining).

Recent interest in learning methods triggered by the widespread use of computers and database technology has resulted in the development of biologically motivated methodologies, such as (artificial) neural networks, fuzzy systems and wavelets.

Unfortunately, developments in each field are seldom related to other fields.

Many data mining applications attempt to estimate predictive models, when estimated models are used for prediction or decision making with new data.

This course will first provide general conceptual framework for learning predictive models from data, and then discuss various methods developed in statistics, pattern recognition and machine learning.

Course Project

• Each student is expected to complete a project (of research nature).

• A list of project topics will be distributed during the 2nd week of class.

• Students will receive close supervision and feedback from the instructor.

• Students may propose their own project topics, subject to the instructor's approval.

Course outline

• Concepts and Theory

  • Introduction/motivation (Ch 1, 0.5 week)

  • Formulation of the learning problem & classical methods (Ch 2, 1 week)

  • Adaptive learning: concepts & inductive principles (Ch 2, 1 week)

  • Regularization and complexity control (Ch 3, 1 week)

  • Statistical Learning Theory (Ch 4, 1 week)

  • Nonlinear optimization (Ch 5, 0.5 week)

• Learning Methods

  • Clustering/VQ/Self-organizing networks (Ch 6, 1 week)

  • Methods for regression (Ch 7, 1 week)

  • Classification (Ch 8, 1 week)

  • Support Vector Machines (Ch 9, 1.5 weeks)

• Advanced/Non-standard Learning Formulations

  • Transduction, Universum Learning, LUPI (Ch 10 and lecture_notes, 2 weeks)

(All chapters refer to the textbook Learning from Data: Concepts, Theory and Methods, by V. Cherkassky and F. Mulier)

Prerequisites

• Graduate standing in EE or IT, or consent of instructor.

• Familiarity with computer programming, using software of your choice, for homework assignments.
  MATLAB or similar environment is recommended but not required.

• Undergraduate courses on probability/statistics and linear algebra.

Course Material

• Textbook
  Learning from Data: Concepts, Theory and Methods, by V. Cherkassky and F. Mulier, Second edition, Wiley 2007.
  The textbook can be ordered at Amazon.com, and a digital version is available at University of Minnesota Library.

• Textbook
  Predictive Learning, by V. Cherkassky, 2013.
  The textbook is available at Amazon.com and the University bookstore.
  Note: most homework assignments will be problems from this textbook.

• Lecture notes and selected papers are available on the Lecture notes and Reading pages.

Grading

• Four homework assignments 40%

• One midterm exam 25%

• Course project

  • midterm progress report 10%

  • final report 25%

• Extra credit for class participation, up to 5%