Search results for “обучение по прецедентам”

Kohonen networks: clustering as a two-layer neural network, the hard (WTA) and soft (WTM) competition rules, self-organising Kohonen maps and their interpretation; autoencoders: the formulation of the problem, the linear autoencoder and the principal component method, sparse, denoising, contractive, relational and variational autoencoders, reparametrisation, autoencoders for supervised learning, multilayer autoencoders; joint training of several models: pre-training and transfer learning, multi-task learning and few-shot learning, self-supervised learning, distillation and surrogate modelling, learning using privileged information (LUPI), generative adversarial networks (GAN).

Collaborative filtering problems: transaction data, the clients–items matrix, filling in missing values and ranking recommendations, the Netflix Prize competition; user-based and item-based correlation methods, the Pearson, cosine and Jaccard similarity measures, Fisher's exact test; the sparse linear model SLIM; latent models and matrix factorisations: SVD, the alternating least squares method ALS, the latent factor model LFM and stochastic gradient, nonnegative matrix factorisations NNMF, PLSA; taking implicit preferences into account (implicit feedback); matrix factorisation as a linear autoencoder, neural network and denoising autoencoders; taking features of clients and items into account, factorisation machines libFM, a hypergraph topic model; the quality of recommendations: RMSE, precision@k, recall@k, diversity, novelty, coverage, serendipity, bias, offline and online evaluation.

Time series forecasting problems: the time series model, the forecasting horizon, the least squares method; econometrics as the main source of problems — trends, seasonalities, change points; examples of forecasting sales volumes and electricity prices; the linear autoregression model; a brief survey of forecasting methods; adaptive short-term forecasting methods: the exponential moving average and the smoothing parameter, the models of Holt, Theil–Wage, Winters with linear and exponential trend, adaptive autoregression; the tracking signal and a statistical test of model adequacy, the Trigg–Leach model; adaptive selection and adaptive composition of models, the LAVR method — local adaptation of weights with regularisation, the monotonicity constraint as a regulariser; experiments on the sales forecasting problem.

The general scheme of supervised learning for regression, binary and multiclass classification; the margin and continuous approximations of the threshold loss; the linear classifier as a model of a neuron; the stochastic gradient method, momentum, Nesterov's accelerated gradient and heuristics; multicollinearity and L2 regularization; the maximum likelihood principle, its link to empirical risk minimization and the probabilistic meaning of regularization.

The continuity and compactness hypotheses; the Minkowski metric and feature-free distances; the generalized metric classifier, the k nearest neighbours method, the Parzen window and the method of potential functions as a linear classifier; complete cross-validation, the compactness profile and the exact CCV formula for the nearest neighbour method; prototype selection; Nadaraya–Watson kernel smoothing, the choice of kernel and bandwidth.

The optimal separating hyperplane and the width of the separating strip; the hinge loss and L2 regularization; the Karush–Kuhn–Tucker conditions, the dual problem and the types of objects; kernels, rectifying spaces and the constructive building of kernels; SVM as a two-layer neural network; choosing the constant C; SVM regression; L1 regularization, Elastic Net SVM, SFM and RVM.

Multiple linear regression and least squares, the pseudoinverse and the geometric meaning of least squares; the solution through the singular value decomposition; multicollinearity and the condition number; ridge regression, choosing the regularization parameter and the effective dimension; LASSO and non-smooth regularizers for feature selection; principal component analysis as a low-rank matrix decomposition; the spectral method of least squares.

Nonlinear regression: the Newton–Raphson and Gauss–Newton methods as a series of linear regressions; logistic regression and iteratively reweighted least squares (IRLS); the generalized additive model and backfitting; the probabilistic meaning of least squares; the generalized linear model (GLM), the exponential family of distributions and the link function; logistic regression as a special case of GLM; the median, quantile regression and robust loss functions.

The combinatorial theory of overfitting: the probability of overfitting and why the Vapnik–Chervonenkis bounds are loose; experiments with overfitting — a monotone chain of algorithms, the effect of splitting and connectivity of the family on the probability of overfitting and on the complete cross-validation estimate, overfitting of chains, the exact formula for the probability of overfitting when choosing between two algorithms; the splitting–connectivity graph of a family of algorithms, examples for linear classifiers, a monotone chain and a two-dimensional net; generating and forbidden sets, upper connectivity and inferiority of an algorithm; the splitting–connectivity upper bound, the idea of its proof and its properties.

Measures of classification quality: type I and type II errors, the ROC curve and AUC, gradient maximization of AUC, log-loss, precision and recall, micro- and macro-averaging, the F-measure; internal and external criteria of generalization ability; hold-out, leave-one-out, q-fold and t×q-fold cross-validation, stratification; model consistency; regularization, AIC, BIC and the Vapnik–Chervonenkis bound; multi-criteria model selection; feature selection: full search, Add, Add-Del, depth-first search, beam search (GMDH), the genetic algorithm and random search with adaptation.

Sequence-to-sequence problems: the recurrent network, the encoder–decoder architecture and its limitations; the recurrent network with an attention model, interpreting the attention matrix in machine translation and image captioning; similarity functions, the query–key–value attention model, the attention formula, self-attention; multi-head attention and its regularisation, hierarchical attention for document classification, the graph attention model GAT; the transformer for machine translation: the encoder, positional encoding, layer normalisation and skip connections, the decoder with masking, the training criterion, BLEU and WER; the transformer encoder BERT, the MLM and NSP criteria, fine-tuning, BPE and WordPiece tokenisation, model sizes.

Probabilistic topic modelling of text document collections: the bag-of-words and conditional independence hypotheses, the maximum likelihood criterion, three interpretations of the problem (soft clustering, stochastic matrix factorisation, autoencoder); the lemma on maximising a function on unit simplices; the ill-posedness of the problem and the additive regularisation of topic models ARTM, the EM algorithm and its interpretation; PLSA and LDA, the Dirichlet distribution, the maximum a posteriori probability and giving up Bayesian inference; smoothing, sparsing, background topics, decorrelation; multimodal and multilingual models, unigrams and bigrams; regularisers for classification and regression on texts, for word co-occurrence in short texts, for links between documents and for topic hierarchies.

Naive set theory and Russell's paradox, Kuratowski's ordered pair and the Cartesian product, a function as a subset of a product, composition, injections, surjections and bijections, the group of bijections and the embedding of any group into it, equinumerosity and indicator functions, the Cantor–Bernstein and Cantor theorems.

A contour integral as a double integral: the hypotheses of Green's theorem, orientation of the contour, area as a contour integral, closing an open arc, an annulus, a singular point inside the contour.

A closing review of the first semester: empirical risk minimisation and gradient methods; underfitting, overfitting, multicollinearity and regularisation; optimisation formulations of supervised learning (regression, classification, ranking) and unsupervised learning (density, mixtures, clustering, one-class classification, semi-supervised learning, PU-learning, principal component analysis, autoencoders); the six scientific schools of machine learning — symbolism, evolutionism, analogism, Bayesianism, connectionism, compositionism — and their Soviet roots; estimate-calculating algorithms; practice: CRISP-DM, data preprocessing, model evaluation and selection, loss analysis, A/B testing, meta-learning and AutoML.