Search results for “вероятность переобучения”

14 results

@konstantin_vorontsov

Константин Вячеславович Воронцов — профессор РАН, доктор физико-математических наук. Заведующий кафедрой «Математические методы прогнозирования» ВМК МГУ, где с 2021 года — профессор; заведующий лабораторией «Машинное обучение и семантический анализ» Института искусственного интеллекта МГУ (с 2022 года); заведующий кафедрой «Машинное обучение и цифровая гуманитаристика» МФТИ (с 2021 года); заведующий кафедрой «Интеллектуальные системы» МФТИ (с 2022 года), на которой работает с 2004 года; главный научный сотрудник отдела «Интеллектуальные системы» ВЦ ФИЦ ИУ РАН, где работает с 2001 года. Окончил МФТИ (1994) и аспирантуру ВЦ РАН (1997). Кандидатская диссертация «Локальные базисы в алгебраическом подходе к проблеме распознавания» (1999, специальность 01.01.09); докторская диссертация «Комбинаторная теория надёжности обучения по прецедентам» (2010, специальность 05.13.17). Автор комбинаторной теории надёжности обучения по прецедентам, оценивающей вероятность переобучения с учётом эффектов расслоения и сходства в семействах алгоритмов; основное направление его исследований — теория и методы аддитивной регуляризации тематических моделей (ARTM). С 2014 года — научный руководитель разработки библиотеки тематического моделирования с открытым кодом BigARTM, с 2019 года — поисково-рекомендательной системы SciSearch. Один из идеологов и администраторов ресурса MachineLearning.ru (с 2008 года). Медаль РАН с премией для молодых учёных (1999, совместно с Е. В. Нарыжным); почётное звание профессора РАН (2015); юбилейная медаль РАН «300 лет Российской академии наук» (2024); премия Yandex ML Prize в номинации «Зал славы» (2025). Автор курсов «Вероятностные тематические модели», «Математические методы анализа текстов», «Прикладной статистический анализ данных» и спецкурса по теории надёжности обучения по прецедентам; в 2008–2020 годах преподавал в Школе анализа данных Яндекса. С 2004 года читает годовой курс «Машинное обучение» в МФТИ и ВМК МГУ.

Machine Learning. Additional Lecture 1. The Theory of Overfitting

2026LectureКонстантин Вячеславович Воронцов

The combinatorial theory of overfitting: the probability of overfitting and why the Vapnik–Chervonenkis bounds are loose; experiments with overfitting — a monotone chain of algorithms, the effect of splitting and connectivity of the family on the probability of overfitting and on the complete cross-validation estimate, overfitting of chains, the exact formula for the probability of overfitting when choosing between two algorithms; the splitting–connectivity graph of a family of algorithms, examples for linear classifiers, a monotone chain and a two-dimensional net; generating and forbidden sets, upper connectivity and inferiority of an algorithm; the splitting–connectivity upper bound, the idea of its proof and its properties.

220

Machine Learning 1. Generalization Theory

2026LectureКонстантин Вячеславович Воронцов

The problem of estimating generalization ability; the error matrix and the probability of overfitting; the case of a single algorithm and the hypergeometric distribution; the principle of uniform convergence of frequencies and Boole's inequality; the Vapnik–Chervonenkis bound, the growth function and the capacity of a family of algorithms; inverting the bound and the complexity penalty; structural risk minimization; the splitting and similarity effects; the Occam's razor bound and its link to regularization.

140

Machine Learning 1. Bayesian Classification Theory

2026LectureКонстантин Вячеславович Воронцов

The probabilistic formulation of classification: prior and posterior class probabilities, class likelihood functions, the maximum a posteriori principle; the optimal Bayes classifier and losses from errors; empirical estimation of prior probabilities and class densities; the naive Bayes classifier, features with exponential-family densities and the linear naive Bayes classifier; a survey of Bayesian classifiers: the Parzen window method, the quadratic discriminant, Fisher's linear discriminant, Gaussian mixtures with diagonal covariance matrices, and the radial basis function network.

180

Machine Learning 2. Lecture 12. Interpretability and Explainability

2026LectureКонстантин Вячеславович Воронцов

Interpretability and explainability of machine learning models: goals, tasks and basic notions, who needs explanations and why; interpretable models; feature importance in linear regression: the t-statistic, net effects, contributions and situational importance; partial dependence plots (PDP) and individual conditional expectation plots (ICE); permutation feature importance; the Shapley value from cooperative game theory, its axioms and Monte Carlo estimation, its use in linear regression; surrogate modelling in the neighbourhood of an object, the LIME, Anchors, SHAP, Shapley Kernel and SAGE methods; the Shapley value for training objects, Data Shapley and Gradient Shapley; counterfactual explanations.

230

Probability Theory. Lecture 1

2026LectureСтанислав Шапошников

The finite probability space and five properties of probability, a description of all probability measures on a finite set, equally likely outcomes and the choice of a model, classical paradoxes (two dice, the division of the stakes, Bertrand's paradox), conditional probability and the product rule, the law of total probability as a way to define a measure, Bayes' formula, independence of events.

170

Machine Learning 2. Lecture 5. Topic Modelling

2026LectureКонстантин Вячеславович Воронцов

Probabilistic topic modelling of text document collections: the bag-of-words and conditional independence hypotheses, the maximum likelihood criterion, three interpretations of the problem (soft clustering, stochastic matrix factorisation, autoencoder); the lemma on maximising a function on unit simplices; the ill-posedness of the problem and the additive regularisation of topic models ARTM, the EM algorithm and its interpretation; PLSA and LDA, the Dirichlet distribution, the maximum a posteriori probability and giving up Bayesian inference; smoothing, sparsing, background topics, decorrelation; multimodal and multilingual models, unigrams and bigrams; regularisers for classification and regression on texts, for word co-occurrence in short texts, for links between documents and for topic hierarchies.

130

Machine Learning 1. Lecture 14. Anomaly Detection and Robust Methods

2026LectureКонстантин Вячеславович Воронцов

Anomaly and novelty detection; heuristic anomaly scores: local density, isolation forest, a mixture with a background component, robust principal component analysis; outlier removal in nonparametric regression, the LOWESS algorithm; anomaly as the value of the loss function; the theory of robust learning: slowly growing functions, iterative reweighting of objects with IRS and IRLS, robust loss functions, the median, the quantile and the censored mean, robust aggregation functions and their smoothing, the IR-ERM algorithm; one-class SVM, learning from positive and unlabeled objects (PU-learning), problems with new and unknown classes.

480

Machine Learning 2. Lecture 8. Association Rule Mining

2026LectureКонстантин Вячеславович Воронцов

Association rule mining as unsupervised learning: binary features, support and frequent itemsets, the association rule, minimum support MinSupp and minimum confidence MinConf; market basket analysis; the link between association rules and logical regularities; the anti-monotonicity of support; the APriori algorithm: breadth-first search for frequent itemsets and recursive extraction of rules; modifications: search on a random subsample, item hierarchies, incremental and decremental algorithms; the FP-growth algorithm: the prefix FP-tree, building it in one pass over the transaction database, computing the support of any itemset from the tree, the conditional FP-tree and recursive search for frequent itemsets; the speed of FP-growth compared with APriori.

160

Machine Learning 2. Adaptive Forecasting Methods

2026LectureКонстантин Вячеславович Воронцов

Time series forecasting problems: the time series model, the forecasting horizon, the least squares method; econometrics as the main source of problems — trends, seasonalities, change points; examples of forecasting sales volumes and electricity prices; the linear autoregression model; a brief survey of forecasting methods; adaptive short-term forecasting methods: the exponential moving average and the smoothing parameter, the models of Holt, Theil–Wage, Winters with linear and exponential trend, adaptive autoregression; the tracking signal and a statistical test of model adequacy, the Trigg–Leach model; adaptive selection and adaptive composition of models, the LAVR method — local adaptation of weights with regularisation, the monotonicity constraint as a regulariser; experiments on the sales forecasting problem.

140

Machine Learning 1. Lecture 10. Linear Ensembles

2026LectureКонстантин Вячеславович Воронцов

Ensembles of models: base algorithms, the space of estimates, the decision rule and correcting functions; the problem of diversity of base algorithms; bagging, the random subspace method and a common pseudocode of stochastic ensembling; the unbiased out-of-bag estimate and estimating feature importance; converting simple voting into weighted voting; the random forest; boosting for binary classification with abstentions, the exponential approximation of the threshold loss function, the main theorem of AdaBoost with proof, convergence, classical AdaBoost and heuristics; why boosting does not overfit: experiments, bounding the probability of error through margins, the distribution of margins, comparison with bagging; the ComBoost algorithm.

160

Machine Learning 2. Lecture 3. Vector Representations of Texts and Graphs

2026LectureКонстантин Вячеславович Воронцов

Vector representations (embeddings) of texts and graphs: the distributional semantics hypothesis, syntagmatic and paradigmatic similarity of words, the CBOW and Skip-gram models of the word2vec program, hierarchical SoftMax on a Huffman tree, negative sampling (SGNS) and the connection of word2vec with the matrix factorisation of the PMI matrix, semantic analogies of words, the FastText model on character n-grams; texts as graphs; multidimensional scaling and the stress criterion, stochastic neighbour embedding SNE, perplexity, t-SNE; matrix factorisations of graphs, the random-walk models DeepWalk and node2vec; graph embedding methods as autoencoders, the generalised GraphEDM model, graph neural networks and message passing.

250

Machine Learning 2. Lecture 7. Recommender Systems

2026LectureКонстантин Вячеславович Воронцов

Collaborative filtering problems: transaction data, the clients–items matrix, filling in missing values and ranking recommendations, the Netflix Prize competition; user-based and item-based correlation methods, the Pearson, cosine and Jaccard similarity measures, Fisher's exact test; the sparse linear model SLIM; latent models and matrix factorisations: SVD, the alternating least squares method ALS, the latent factor model LFM and stochastic gradient, nonnegative matrix factorisations NNMF, PLSA; taking implicit preferences into account (implicit feedback); matrix factorisation as a linear autoencoder, neural network and denoising autoencoders; taking features of clients and items into account, factorisation machines libFM, a hypergraph topic model; the quality of recommendations: RMSE, precision@k, recall@k, diversity, novelty, coverage, serendipity, bias, offline and online evaluation.

140

Machine Learning 2. Lecture 10. Reinforcement Learning

2026LectureКонстантин Вячеславович Воронцов

Reinforcement learning as learning from the environment's response: the multi-armed bandit problem, estimating the value of an action, the greedy and ε-greedy strategies, the exploration–exploitation trade-off, the UCB method, the exponential moving average, the pursuit method and reinforcement comparison; an environment with states, the Markov decision process, how it differs from ordinary machine learning problems, the discounted return, the state and action value functions, the Bellman equation, the SARSA and Q-learning methods, on-policy and off-policy; deep Q-learning DQN on the example of Atari games, experience replay; gradient optimisation of the policy (policy gradient) and its connection with log-likelihood maximisation; model-based and model-free approaches, a regression model of the reward with an upper confidence bound, LinUCB, recommending news articles, evaluating a policy on historical data.

550