Konstantin Vorontsov

@konstantin_vorontsov

Machine learning, topic modelling; MSU CMC, MIPT, FRC CSC RAS

1 follower104 works

Collections3

Course

Машинное обучение. Дополнительные лекции

Константин Вячеславович ВоронцовAugust 30, 2026

Дополнительные лекции курса «Машинное обучение» К. В. Воронцова: комбинаторная теория переобучения и обзор оптимизационных задач машинного обучения.

Machine Learning Data Science

2 items5 views

Course

Машинное обучение. Семестр 2: прикладные модели

Константин Вячеславович ВоронцовAugust 30, 2026

Второй семестр годового курса «Машинное обучение» К. В. Воронцова: глубокие нейронные сети и обучение без учителя, векторные представления текстов и графов, модели внимания и трансформеры, тематическое моделирование, обучение ранжированию, рекомендательные системы, поиск ассоциативных правил, прогнозирование, инкрементное и онлайновое обучение, обучение с подкреплением, активное обучение, интерпретируемость.

Machine Learning Data Science

13 items7 views

Course

Машинное обучение. Семестр 1: математические модели

Константин Вячеславович ВоронцовAugust 30, 2026

Первый семестр годового курса «Машинное обучение» К. В. Воронцова: обучение по прецедентам, линейные модели и стохастический градиент, нейронные сети, метрические методы, метод опорных векторов, линейная и нелинейная регрессия, отбор признаков и оценивание моделей, теория обобщающей способности, логические методы, ансамбли и бустинг, байесовская классификация, кластеризация, робастные методы.

Machine Learning Data Science

17 items5 views

Machine Learning. Additional Lecture 2. A Survey of Optimisation Problems in Machine Learning

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

A survey lecture: machine learning as formulations of optimisation problems, a model, a loss function, a regulariser. Regression and classification, continuous approximations of the threshold loss function, SVM and logistic regression, quantile, robust and SVM regression; the L0, L1, L2, Elastic Net, SFM, RFM regularisers and feature selection; learning to rank: ordinal regression, the pairwise approach and AUC, LambdaRank; estimating a density and a mixture of distributions, clustering as a special case of a mixture, the one-class SVM; semi-supervised learning, the transductive SVM, PU-learning; matrix factorisations, autoencoders, multidimensional scaling, graph factorisations, GraphEDM; transfer learning, multi-task and self-supervised learning, distillation, learning using privileged information, generative adversarial networks; a typology of machine learning problems.

210

Machine Learning. Additional Lecture 1. The Theory of Overfitting

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

The combinatorial theory of overfitting: the probability of overfitting and why the Vapnik–Chervonenkis bounds are loose; experiments with overfitting — a monotone chain of algorithms, the effect of splitting and connectivity of the family on the probability of overfitting and on the complete cross-validation estimate, overfitting of chains, the exact formula for the probability of overfitting when choosing between two algorithms; the splitting–connectivity graph of a family of algorithms, examples for linear classifiers, a monotone chain and a two-dimensional net; generating and forbidden sets, upper connectivity and inferiority of an algorithm; the splitting–connectivity upper bound, the idea of its proof and its properties.

220

Machine Learning 2. Lecture 12. Interpretability and Explainability

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Interpretability and explainability of machine learning models: goals, tasks and basic notions, who needs explanations and why; interpretable models; feature importance in linear regression: the t-statistic, net effects, contributions and situational importance; partial dependence plots (PDP) and individual conditional expectation plots (ICE); permutation feature importance; the Shapley value from cooperative game theory, its axioms and Monte Carlo estimation, its use in linear regression; surrogate modelling in the neighbourhood of an object, the LIME, Anchors, SHAP, Shapley Kernel and SAGE methods; the Shapley value for training objects, Data Shapley and Gradient Shapley; counterfactual explanations.

230

Machine Learning 2. Lecture 11. Active Learning

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Active learning: the model itself chooses which objects to give the teacher for labelling. The problem setting, why active learning is faster than passive learning, learning curves; pool-based sampling, query synthesis and stream-based selective sampling, cost-sensitive methods; uncertainty sampling (least confidence, minimum margin, maximum entropy), query by committee, version space reduction, expected model change, expected error reduction; query synthesis by the Nelder–Mead method and by the variance reduction criterion; density weighting; exploratory actions, the ε-active and EG-active algorithms with the exponentiated gradient; active learning in crowdsourcing: reconciling annotators' labels, models of their reliability and of object difficulty, assigning tasks to annotators.

450

Machine Learning 2. Lecture 10. Reinforcement Learning

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Reinforcement learning as learning from the environment's response: the multi-armed bandit problem, estimating the value of an action, the greedy and ε-greedy strategies, the exploration–exploitation trade-off, the UCB method, the exponential moving average, the pursuit method and reinforcement comparison; an environment with states, the Markov decision process, how it differs from ordinary machine learning problems, the discounted return, the state and action value functions, the Bellman equation, the SARSA and Q-learning methods, on-policy and off-policy; deep Q-learning DQN on the example of Atari games, experience replay; gradient optimisation of the policy (policy gradient) and its connection with log-likelihood maximisation; model-based and model-free approaches, a regression model of the reward with an upper confidence bound, LinUCB, recommending news articles, evaluating a policy on historical data.

550

Machine Learning 2. Lecture 9. Incremental and Online Learning

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Learning on a data stream: the formulation of the online learning problem, the learning curve, the issues: efficient model updates, making the model more complex as the sample grows, the equivalence of online and offline, storing the sample and catastrophic forgetting; how online learning differs from incremental learning, continual learning, anytime algorithms; lazy learning and online prototype selection; the online naive Bayes classifier with exponential distributions of features; the perceptron, Passive-Aggressive and online gradient descent; the recursive least squares method and the Sherman–Morrison formula; incremental training of decision trees ID5R; online training of an ensemble by the Hedge(β) algorithm and its financial interpretation; online deep learning and Hedge BackProp; incremental learning of new classes, iCaRL.

200

Machine Learning 2. Adaptive Forecasting Methods

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Time series forecasting problems: the time series model, the forecasting horizon, the least squares method; econometrics as the main source of problems — trends, seasonalities, change points; examples of forecasting sales volumes and electricity prices; the linear autoregression model; a brief survey of forecasting methods; adaptive short-term forecasting methods: the exponential moving average and the smoothing parameter, the models of Holt, Theil–Wage, Winters with linear and exponential trend, adaptive autoregression; the tracking signal and a statistical test of model adequacy, the Trigg–Leach model; adaptive selection and adaptive composition of models, the LAVR method — local adaptation of weights with regularisation, the monotonicity constraint as a regulariser; experiments on the sales forecasting problem.

140

Machine Learning 2. Lecture 8. Association Rule Mining

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Association rule mining as unsupervised learning: binary features, support and frequent itemsets, the association rule, minimum support MinSupp and minimum confidence MinConf; market basket analysis; the link between association rules and logical regularities; the anti-monotonicity of support; the APriori algorithm: breadth-first search for frequent itemsets and recursive extraction of rules; modifications: search on a random subsample, item hierarchies, incremental and decremental algorithms; the FP-growth algorithm: the prefix FP-tree, building it in one pass over the transaction database, computing the support of any itemset from the tree, the conditional FP-tree and recursive search for frequent itemsets; the speed of FP-growth compared with APriori.

160

Machine Learning 2. Lecture 7. Recommender Systems

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Collaborative filtering problems: transaction data, the clients–items matrix, filling in missing values and ranking recommendations, the Netflix Prize competition; user-based and item-based correlation methods, the Pearson, cosine and Jaccard similarity measures, Fisher's exact test; the sparse linear model SLIM; latent models and matrix factorisations: SVD, the alternating least squares method ALS, the latent factor model LFM and stochastic gradient, nonnegative matrix factorisations NNMF, PLSA; taking implicit preferences into account (implicit feedback); matrix factorisation as a linear autoencoder, neural network and denoising autoencoders; taking features of clients and items into account, factorisation machines libFM, a hypergraph topic model; the quality of recommendations: RMSE, precision@k, recall@k, diversity, novelty, coverage, serendipity, bias, offline and online evaluation.

140

Machine Learning 2. Lecture 6. Learning to Rank

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

The learning-to-rank problem: recovering a partial order relation, the pointwise, pairwise and listwise approaches, the linear ranking model, applications; ordinal regression and the ordinal SVM (SVOR); the pairwise criterion, Ranking SVM, RankNet, the transition to the listwise method LambdaRank; ranking search results: query–document objects, assessor ratings, text, link and click features, TF-IDF and its probabilistic meaning, Okapi BM25, PageRank; quality criteria: precision and recall, MAP, the share of defective pairs and its link with AUC, DCG and NDCG, the pFound user behaviour model; the DSSM neural model: a siamese network, word hashing by letter trigrams, training on clicks with negative sampling.

130

Machine Learning 2. Lecture 5. Topic Modelling

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Probabilistic topic modelling of text document collections: the bag-of-words and conditional independence hypotheses, the maximum likelihood criterion, three interpretations of the problem (soft clustering, stochastic matrix factorisation, autoencoder); the lemma on maximising a function on unit simplices; the ill-posedness of the problem and the additive regularisation of topic models ARTM, the EM algorithm and its interpretation; PLSA and LDA, the Dirichlet distribution, the maximum a posteriori probability and giving up Bayesian inference; smoothing, sparsing, background topics, decorrelation; multimodal and multilingual models, unigrams and bigrams; regularisers for classification and regression on texts, for word co-occurrence in short texts, for links between documents and for topic hierarchies.

130

Machine Learning 2. Lecture 4. Attention Models and Transformers

Aug 30, 2026LectureКонстантин Вячеславович Воронцов

Sequence-to-sequence problems: the recurrent network, the encoder–decoder architecture and its limitations; the recurrent network with an attention model, interpreting the attention matrix in machine translation and image captioning; similarity functions, the query–key–value attention model, the attention formula, self-attention; multi-head attention and its regularisation, hierarchical attention for document classification, the graph attention model GAT; the transformer for machine translation: the encoder, positional encoding, layer normalisation and skip connections, the decoder with masking, the training criterion, BLEU and WER; the transformer encoder BERT, the MLM and NSP criteria, fine-tuning, BPE and WordPiece tokenisation, model sizes.

450