Lecture 1: 

Overview of Machine Learning

Math background overview: Calculus, Algebra, Probabilities/Statistics (look also at other links from Kari Torkkola)

Reading Assignment: Ch. 1.1 - 1.5, 2.1 - 2.3 from the textbook

Lecture 2: 

Supervised learning; 

Standard accuracy measures; 

Optimal predictors.

Homework 1 (due on Mon, Sep 17): Problems 1.5, 1.6, 1.8, 2.4, 2.8, 2.24, 2.34 from the textbook; two problems from lecture 2 notes

Lecture 3: 

Equivalence between optimal regression and classification; 

Extreme approaches to minimizing MSE (nearest neighbor algorithm and linear regression), 

Linear regression (solution, statistical results); 

Nonlinear regression (gradient descent optimization); 

Homework 2 due on Mon, Sep 24. (download file hw2.m). In addition, solve the 5 homework problems highlighted by blue font in lecture 3 notes. 

Lecture 4: 

Logistic regression by minimizing MSE

Maximum Likelihood (ML) approach for unsupervised learning (density estimation), regression, classification;

Lecture 5: 

Feedforward Neural Network (NN) Architecture

Simple O(NW²) Method for NN Training

O(NW) Method for NN Training: Backpropagation

Homework 3 due on Mon, Oct 8

Lecture 6: 

Machine Learning Process

NN Overfitting (# epochs, #hidden nodes)

Regularization/ Weight Decay for overfitting prevention

Bias-Variance decomposition; Bagging

Learning Curve

Homework 4 due on Mon, Oct 15

Lecture 7: 

Support Vector Machines

Lecture 8: 

Support Vector Machines - continued

Midterm

Course project discussion:

Instructions for Project Proposal (Proposal is due Oct 29, in class)

class presentation instructions

Useful Reading:

How to give a bad presentation

Ian Parbery, "How to Present a Paper in Theoretical Computer Science: A Speaker's Guide for Students"

Homework 5 due on Mon, Oct 29

Lecture 9: 

Bayesian Networks. Reading assignment: Section 8.1, 8.3, 8.28.2.1, 8.2.2, 8.4, 8.4.1. Consult the lecture notes about Bayesian Networks prepared by Prof. Hauskrecht from U. Pitt: Bayesian belief networks. Bayesian belief networks II, Bayesian belief networks. Inference and Learning, Bayesian belief networks. Learning.

15-minute presentations:

DeCoste, D., Mazzoni, D., Fast Query-Optimized Kernel Machine Classification Via Incremental Approximate Nearest Support Vectors, Proceedings of the Twentieth International Conference on Machine Learning, Washington, DC (2003).  (Presentation by Vuk Malbasa)

Lin, C.F.,  Wang, S.D., Fuzzy Support Vector Machines, IEEE Transactions on Neural Networks, Vol. 13, No. 2 (2002). (Presentation by Zhuang Wang)

Lecture 10: 

Bayesian Networks - continued.

15-minute presentations:

Nonparametric Density Estimation (Chapter 2.5 from the textbook)  (Presentation by Riu Baring)

Mixture Density Networks (Chapter 5.6 from the textbook) (Presentation by Qiang Lou)

D. M. J. Tax and R. P. W. Duin. Support Vector Data Description. Machine Learning, 54:45–66, 2004.. (Presentation by Mihajlo Grbovic)

Homework 6. Due on Nov 12, 2007. Download: data1.zip, data2.zip

Lecture 11: 

Continuous Latent Variables. Reading Assignment: Sections 12.1, 12.3 from the textbook.

15-minute presentations:

Mitra, C. Murthy and S. Pal, "A Probabilistic Active Support Vector Learning Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no.3, pp.413 - 418, 2004. (Presentation by Siyuan Ren)

K. Q. Weinberger, F. Sha, and L. K. Saul. "Learning a Kernel Matrix for Nonlinear Dimensionality Reduction." In Proceedings of the Twenty First International Conference on Machine Learning (ICML-04), pages 839-846, Banff, Canada, 2004. (Presentation by Michael Baranthan)

Lecture 12: 

Bayesian Regression. Reading Assignment: Sections 1.2.3, 1.2.5, 1.2.6, 2.1.1, 2.2.1, 2.3.1-2.3.3, 2.3.6, 3.3, 3.5 from the textbook.

15-minute presentations:

G. Camps-Valls, L. Gomez-Chova, J. Calpe, E. Soria, J. D. Martýn, L. Alonso, and J. Moreno, “Robust Support Vector Method for Hyperspectral Data Classification and Knowledge Discovery,” IEEE Transactionson Geoscience and Remote Sensing 42, pp. 1530–1542, July 2004. (Presentation by Haidong Shi)

Rennie J., Shih, L., Teevan, J., & Karger, D, “Tackling the Poor Assumptions of Naive Bayes Text Classifiers,” In Proceedings of the 20th International Conference on Machine Learning, Washington D.C., 2003. (Presentation by Liang Lan)

Boosting (Chapter 14.3 from the textbook) (Presentation by Xin Lin)

Homework 7. Due on Nov 26/29, 2007. Download: data1.zip, data2.zip

Lecture 13: 

Mixture Models and EM. Reading Assignment: Sections 9.1 - 9.3.

15-minute presentations:

Z. Huang, H. Chen, C. J. Hsu, W.H. Chen, and S. Wu. "Credit Rating Analysis with Support Vector Machines and Neural Networks: a Market Comparative Study." Decision Support Systems, 37:543-558, 2004. (Presented by James Joseph)

Yu, L., Liu, H. (2003) Feature Selection for High-Dimensional Data: a Fast Correlation-Based Filter Solution. Proceedings of the International Conference on Machine Learning, 856-863. (Presented by Jingting Zeng)

Decision Trees (Chapter 14.4. from the textbook. Presented by Ping Zhang)

Lecture 14: 

Sequential Data. Reading Assignment: Chapter 13. 

15-minute presentations:

Linear Models for Classification - Discriminant Functions (Section 4.1, presentation by Gregory Johnson)

Gaussian Processes for Regression (Section 6.4, 6.4.1-4, presentation by Li An)

Markov Random Fields (Section 8.3, presentation by Vladan Radosavljevic)