I Basic statistics
We aim in this lecture to present some basic results from statistics underlying machine learning theory. In forthcoming lectures we shall present more involved results from multivariate statistics.
Mean squared error of an estimator
Estimators of expectation and variance
Hypothesis tests for Gaussian random variables
Statistic and test statistic
II Linear fitting and regression
Linear fitting and regression are classical problems in statistics, which permit to illustrate the basic ideas underlying machine learning theory. Indeed, linear fitting is one of the simplest examples of machine learning problems.
Multidimensional linear fitting
Multidimensional linear regression
Multidimensional linear prediction
Maximum likelihood estimators
III Logistic regression
Logistic regression is also one of the simplest examples of machine learning problems. It permits to illustrate the ideas underlying some machine learning algorithms. It is an adaptation of linear regression to the case when the output variable is discrete.
Binary logistic regression
Binary logistic regression model
Binary logistic prediction
Multiclass logistic regression
Multiclass logistic regression model
Multiclass logistic prediction
IV Principal component and factor analysis
We shall present two techniques, called ‘principal component analysis’ and ‘factor analysis’, which aim to reduce the eventual redundancy among some observed random variables Y1, Y2, . . . , Yp by using a smaller number of components (or factors). The objective is the same, but each of these techniques has its own specificities, as we shall see.
Principal component analysis (PCA)
PCA for random observed variables
PCA for deterministic observed variables
Existence of factor analysis model
Scaling in factor analysis
Rotation of factors