Statistical Learning

Modulo Unsupervised Learning

1.  Knowledge and understanding. The first “Unsupervised Learning” module mainly concerns the fundamentals of two of the main methods used in unsupervised learning: principal component analysis and cluster analysis.

2. Applying knowledge and understanding. On completion, the student will be able to: i) implement the main methods used in unsupervised learning; ii) summarize the main features of a dataset, and extract knowledge from data properly.

3. Making judgements. On completion, the student will be able to choose a suitable statistical model, apply it, and perform the analysis using statistical software. 

4. Communication skills. On completion, the student will be able to present the results from the statistical analysis, and which conclusions can be drawn.

5. Learning skills. On completion, the student will be able to understand the structure of unsupervised learning.

Lectures via slides. The freely available R statistical software will also be used.

Basic notions in statistics, linear algebra, and computing.

Highly recommended.

Statistical Models for Univariate Random Variables. Discrete and continuous random variables. Basic distribution functions. Expectation and variance. Statistical models for random variables. Parametric inference: classical properties of estimators; the maximum likelihood approach and its properties. Goodness-of-fit tests. Model Selection. R functions and packages. Illustration in R. (Slides) 

Basics of Matrices. Matrices. Special matrices. Basic matrix identities. Trace. Inverse and determinant. Eigen-decomposition. Quadratic forms and definite matrices. (Bishop 2007, Appendix C) 

Basics of Multivariate Modelling. Random vectors and their distributions. Mean vector, covariance matrix, and correlation matrix. Multivariate normal distribution: properties and effect of the covariance matrix on the shape of the contours. Data Matrix, centered data matrix, and standardized data matrix. (McNeil, Frey, and Embrechts 2005, Chapter 3) 

Principal Component Analysis. The goal of Principal Component Analysis (PCA). PCA as a tool for data visualization. Definition of principal components (PCs). PCA and Eigen-decomposition. Computing PCs. PCA: Geometrical interpretation. Choosing the number of PCs. Biplot. Illustration of PCA in R. (James G., Witten D., Hastie T., Tibshirani R. 2017, Chapter 10) 

Cluster Analysis. Clustering distance/dissimilarity measures. Data types in Clustering. Data standardization. Distance matrix computation. R functions and packages. (Kassambara, 2017, Chapter 3) 

Hierarchical clustering methods. Peculiarities. Agglomerative hierarchical clustering. Algorithm. Dendrogram. Linkage methods. Simplified example. Agglomerative hierarchical clustering methods using the data matrix. Illustration in R. (Kassambara, 2017, Chapter 7) 

Partitioning (or partitional) clustering methods. Peculiarities. K-means clustering. Algorithm. R functions and packages. Illustration in R. K-medoids clustering. PAM Algorithm. R functions and packages. Illustration in R. (Kassambara, 2017, Chapters 4–5) 

Cluster Validation. Overview. Assessing Clustering Tendency. R functions and packages. Illustration in R. Determining the Optimal Number of Clusters. R functions and packages. Illustration in R. Cluster Validation Statistics: Internal and external measures. R functions and packages. Illustration in R. Choosing the Best Clustering Algorithm(s). Measures for comparing clustering algorithms. Cluster stability measures. R functions and packages. Illustration in R. (Kassambara, 2017, Chapters 11–14) 

Fuzzy Clustering. Preliminaries. Fuzzy clustering. Fuzzy K-Means. Fuzzy methods: Cluster validation. Cluster validation: fuzziness measures. Cluster validation: compactness/separation measures. Gustafson-Kessel Extensions of Fuzzy K-Means. Gustafson-Kessel-Babuska Fuzzy K-Means. Entropic Fuzzy K-Means. Fuzzy K-Means with Noise cluster. Combining entropy and a noise cluster. PAM algorithm in a fuzzy view: the Fuzzy K-Medoids. Non-Euclidean Fuzzy Relational Data Clustering. R functions and packages. Illustration in R. (Giordani et al., 2020, Chapter 5).

Model-Based Clustering. Preliminaries. Mixture models. Clustering with mixture models. Maximum a posteriori probability criterion. Gaussian mixtures. Parsimonious modeling via eigendecomposition. Choosing the number of mixture components and the best parsimonious configuration: the Bayesian information criterion. R functions and packages. Illustration in R. (Kassambara, 2017, Chapter 18). 

·         Bishop C. M. (2007). Pattern Recognition and Machine Learning, Springer, Cambridge.

·         Giordani P., Ferrero M. B. and Martella F. (2020). An Introduction to Clustering with R, Springer.

·         James G., Witten D., Hastie T., Tibshirani R. (2017). An Introduction to Statistical Learning with Applications in R, Springer, New York.

·         Kassambara A. (2017). Practical Guide to Cluster Analysis in R.

·         McNeil A. J., Frey R., Embrechts P. (2005). Quantitative Risk Management Concepts, Techniques and Tools. Princeton University Press, Princeton, New Jersey.

The purpose of the examination is to assess the achievement of the learning objectives.

The examination consists of a multiple-choice test. Questions will cover both:

• the methodological aspects discussed during the course (statistical modelling, principal component analysis, cluster analysis, cluster validation, fuzzy clustering, and model-based clustering);
• the use of the R statistical software for the practical implementation of these methods, including the interpretation of R functions, outputs, and graphical displays presented during the lectures.

The test is designed to assess both the student's theoretical understanding of the methods and the ability to identify and correctly interpret the appropriate R procedures used to perform the analyses.

The final grade is awarded according to the following scheme:

• Not approved: the student has not acquired the fundamental concepts and is unable to answer at least 60% of the questions correctly.
• 18–23: the student demonstrates a basic understanding of the course contents, while the ability to connect concepts is limited.
• 24–27: the student demonstrates a good understanding of the course contents and is able to connect concepts appropriately.
• 28–30 cum laude: the student demonstrates an excellent understanding of all course contents, is able to integrate and critically discuss them, and shows a thorough mastery of both the theoretical concepts and their practical implementation.