Statistical Learning

Modulo Supervised Learning

Expected Learning Results

1. Knowledge and understanding. The first “Supervised Learning” module primarily focuses on the fundamentals of supervised learning techniques within the framework of predictive approaches, utilizing regression and classification methods.
2. Applying knowledge and understanding. On completion, the student will be able to: i) implement the main methods used in supervised learning; ii) summarize the main features of a dataset, and provide models from data properly.
3. Making judgements. On completion, the student will be able to choose a suitable statistical model, apply it, and perform the analysis using statistical software.
4. Communication skills. On completion, the student will be able to present the results from the statistical analysis, and the conclusions can be drawn.
5. Learning skills. On completion, the student will be able to understand the structure of unsupervised learning.

General Synthetic Description

The course provides notions on supervised learning techniques mainly in the framework of regression and classification approaches for both parametric (Linear Regression, logistic regression, discriminant analysis) and non-parametric (tree-based methods) approaches.

Basics of statistical learning. Estimation of Dependencies Based on Empirical Data. General Model of Learning from Examples. The problem of risk minimization. Assessing Model Accuracy, The Bias-Variance Trade-Off. Regression vs Classification, Bayes Classifier, KNN. Learning Paradigms in Statistics Exploratory Data Analysis. Lab with R.

Linear regression. Introduction to Linear Regression Models. Estimating Model Parameters. Model Adequacy Checking. Assessing the Accuracy of the Coefficient Estimates. Properties of least-squares estimators. Confidence intervals and hypothesis testing. Multiple regression. Parameter estimation. Model assessment. Subset selection. Qualitative predictors. Extension of the linear model. Diagnostics for Multiple regression models. Lab with R. 

Resampling methods. Basics of Sampling. Validation set approach. Leave-One-Out Cross-Validation and k-Fold Cross-Validation. Bias-Variance Trade-Off for k-Fold Cross-Validation. Cross-Validation on Classification Problems Lab with R. 

Logistic regression. Logistic regression. Simple logistic regression. Parameter estimation and interpretation Diagnostic checking in Logistic Regression. Multiple logistic regression. Linear model selection criteria. Generalized Linear Models. Lab with R.

Generative Models for Classification. Linear discriminant analysis. Quadratic discriminant analysis. Naïve Bayes. Comparison among methods. Lab with R.

Tree-based methods. Basics of tree-based methods. Regression trees. Growing regression trees. Tree pruning. Classification trees Bagging, Random Forests Boosting. Lab with R.

1. James G., Witten D., Hastie T., Tibshirani R. An Introduction to Statistical Learning with Applications in R, Springer 2021

2. Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning, Springer 2008.

3. Teaching material that will be made available during the lessons on Studium.

The purpose of the examination is to assess the achievement of the learning objectives.

The examination consists of a multiple-choice test. Questions will cover both:

• the methodological aspects discussed during the course;
• the use of the R statistical software for the practical implementation of these methods, including the interpretation of R functions, outputs, and graphical displays presented during the lectures.

The test is designed to assess both the student's theoretical understanding of the methods and the ability to identify and correctly interpret the appropriate R procedures used to perform the analyses.

The final grade is awarded according to the following scheme:

• Not approved: the student has not acquired the fundamental concepts and is unable to answer at least 60% of the questions correctly.
• 18–23: the student demonstrates a basic understanding of the course contents, while the ability to connect concepts is limited.
• 24–27: the student demonstrates a good understanding of the course contents and is able to connect concepts appropriately.
• 28–30 cum laude: the student demonstrates an excellent understanding of all course contents, is able to integrate and critically discuss them, and shows a thorough mastery of both the theoretical concepts and their practical implementation.

Sample multiple-choice questions:

1. What is the effect of collinearity on the estimated regression parameters?

A)    -  It causes the standard error to decrease.

B)    -  It causes the standard error to remain stable.

C)     - It reduces the accuracy of the estimates of the regression coefficients. ✓

D)     - It increases the accuracy of the estimates of the regression coefficients.

2. Which R function is commonly used to perform the linear regression model?

A)     - lm() ✓

B)     - hclust()

C)     - prcomp()

D)    - kmeans()

3. In the context of decision trees, which of the following is commonly used for building regression trees??

A)    - The squared-error node impurity. ✓

B)    - The classification error rate.

C)    - The Gini index.

D)    - The entropy.