Outline of Today

Unsupervised Learning

This course, and likely most of your prior statistical education, has been on supervised learning
- We typically have n observations with p features (X) and responses (Y)
In unsupervised learning, we don’t have responses (Y)
- This means we generally don’t care about predictions
This chapter:
- PCA: see other stats courses, used for dimension reduction, pre processing, etc
- Missing values/matrix completions: Also not this course. Mean of column, imputing using pca and corelation, LDA, etc
- Clustering: What we will be doing

I like the layout from Professor Miller better than this text, so we will be using labs based on his work this week.

Small decisions have big consequences
- Should we standardize?
- what dissimilarity measure should be used?
- What linkage should be used?
- Where should we cut the dendrogram/how many clusters?
Outliers
- Outliers can massively change clusters
Book Recommendations
- Since clustering is not robust, cluster subsets to get an idea of robustness
- Be careful in reporting
- Change hyperparameters

Reading 1: what is the major difference between supervised and unsupervised learning? How does KNN work?

Reading 2: what is a dendrogram and why do we care?

Answers are not included since the other class also uses these labs.

In this week we covered the following:

Both labs this week are due Friday at 10pm
Some of you haven’t sent me a proposed final group. You have until tonight at 10pm before I start assigning groups.
Please fill out the “Project 2 group” assignment by Friday at 10pm
- Will be announced on Tuesday (Week 9)
- Will be due 1.5 weeks from release (Week 10 Friday)
- Technically available on the website right now, although submissions are not open
Final Project
- Proposal Due Next Friday (Week 9)
- Data Exploration Due Week 11
- Draft Report Due Week 12
- Presentation Week 14 in class
- Final Report Due Week 14 at 5pm
Readings are due 2 days before class