LibreTimes

Search results for “машинное обучение”

Categories

Machine Learning

33 results

Константин Вячеславович Воронцов
@konstantin_vorontsov

Константин Вячеславович Воронцов — профессор РАН, доктор физико-математических наук. Заведующий кафедрой «Математические методы прогнозирования» ВМК МГУ, где с 2021 года — профессор; заведующий лабораторией «Машинное обучение и семантический анализ» Института искусственного интеллекта МГУ (с 2022 года); заведующий кафедрой «Машинное обучение и цифровая гуманитаристика» МФТИ (с 2021 года); заведующий кафедрой «Интеллектуальные системы» МФТИ (с 2022 года), на которой работает с 2004 года; главный научный сотрудник отдела «Интеллектуальные системы» ВЦ ФИЦ ИУ РАН, где работает с 2001 года. Окончил МФТИ (1994) и аспирантуру ВЦ РАН (1997). Кандидатская диссертация «Локальные базисы в алгебраическом подходе к проблеме распознавания» (1999, специальность 01.01.09); докторская диссертация «Комбинаторная теория надёжности обучения по прецедентам» (2010, специальность 05.13.17). Автор комбинаторной теории надёжности обучения по прецедентам, оценивающей вероятность переобучения с учётом эффектов расслоения и сходства в семействах алгоритмов; основное направление его исследований — теория и методы аддитивной регуляризации тематических моделей (ARTM). С 2014 года — научный руководитель разработки библиотеки тематического моделирования с открытым кодом BigARTM, с 2019 года — поисково-рекомендательной системы SciSearch. Один из идеологов и администраторов ресурса MachineLearning.ru (с 2008 года). Медаль РАН с премией для молодых учёных (1999, совместно с Е. В. Нарыжным); почётное звание профессора РАН (2015); юбилейная медаль РАН «300 лет Российской академии наук» (2024); премия Yandex ML Prize в номинации «Зал славы» (2025). Автор курсов «Вероятностные тематические модели», «Математические методы анализа текстов», «Прикладной статистический анализ данных» и спецкурса по теории надёжности обучения по прецедентам; в 2008–2020 годах преподавал в Школе анализа данных Яндекса. С 2004 года читает годовой курс «Машинное обучение» в МФТИ и ВМК МГУ.

Machine Learning 1. Lecture 1. Basic Concepts and Examples of Applied Problems

2026LectureКонстантин Вячеславович Воронцов

Objects, features and answers; the types of supervised learning problems; parametric models and the learning method; loss functions and empirical risk minimization; the Runge example and overfitting; hold-out, leave-one-out and cross-validation; applied problems from medical diagnosis to sales forecasting; the CRISP-DM standard.
0
210

Machine Learning 1. Lecture 2. Stochastic Gradient and the Linear Classifier

2026LectureКонстантин Вячеславович Воронцов

The general scheme of supervised learning for regression, binary and multiclass classification; the margin and continuous approximations of the threshold loss; the linear classifier as a model of a neuron; the stochastic gradient method, momentum, Nesterov's accelerated gradient and heuristics; multicollinearity and L2 regularization; the maximum likelihood principle, its link to empirical risk minimization and the probabilistic meaning of regularization.
0
130

Machine Learning 1. Lecture 3. Neural Networks: Gradient Optimization Methods

2026LectureКонстантин Вячеславович Воронцов

The neuron as a linear model; implementing Boolean functions and the exclusive-OR problem; two-layer networks as universal approximators, Cybenko's theorem; the multilayer network and the backpropagation method; the adaptive gradient methods RMSProp, AdaDelta, Adam, Nadam; gradient explosion and gradient clipping; Dropout, ReLU activation functions, batch normalization, initialization and network pruning (Optimal Brain Damage).
0
130

Machine Learning 1. Lecture 4. Metric Methods of Classification and Regression

2026LectureКонстантин Вячеславович Воронцов

The continuity and compactness hypotheses; the Minkowski metric and feature-free distances; the generalized metric classifier, the k nearest neighbours method, the Parzen window and the method of potential functions as a linear classifier; complete cross-validation, the compactness profile and the exact CCV formula for the nearest neighbour method; prototype selection; Nadaraya–Watson kernel smoothing, the choice of kernel and bandwidth.
0
150

Machine Learning 1. Lecture 5. The Support Vector Machine

2026LectureКонстантин Вячеславович Воронцов

The optimal separating hyperplane and the width of the separating strip; the hinge loss and L2 regularization; the Karush–Kuhn–Tucker conditions, the dual problem and the types of objects; kernels, rectifying spaces and the constructive building of kernels; SVM as a two-layer neural network; choosing the constant C; SVM regression; L1 regularization, Elastic Net SVM, SFM and RVM.
0
190

Machine Learning 1. Lecture 6. Linear Regression Models

2026LectureКонстантин Вячеславович Воронцов

Multiple linear regression and least squares, the pseudoinverse and the geometric meaning of least squares; the solution through the singular value decomposition; multicollinearity and the condition number; ridge regression, choosing the regularization parameter and the effective dimension; LASSO and non-smooth regularizers for feature selection; principal component analysis as a low-rank matrix decomposition; the spectral method of least squares.
0
130

Machine Learning 1. Lecture 7. Generalizations of Linear Models

2026LectureКонстантин Вячеславович Воронцов

Nonlinear regression: the Newton–Raphson and Gauss–Newton methods as a series of linear regressions; logistic regression and iteratively reweighted least squares (IRLS); the generalized additive model and backfitting; the probabilistic meaning of least squares; the generalized linear model (GLM), the exponential family of distributions and the link function; logistic regression as a special case of GLM; the median, quantile regression and robust loss functions.
0
130

Machine Learning 1. Lecture 8. Feature Selection, Model Evaluation and Model Selection

2026LectureКонстантин Вячеславович Воронцов

Measures of classification quality: type I and type II errors, the ROC curve and AUC, gradient maximization of AUC, log-loss, precision and recall, micro- and macro-averaging, the F-measure; internal and external criteria of generalization ability; hold-out, leave-one-out, q-fold and t×q-fold cross-validation, stratification; model consistency; regularization, AIC, BIC and the Vapnik–Chervonenkis bound; multi-criteria model selection; feature selection: full search, Add, Add-Del, depth-first search, beam search (GMDH), the genetic algorithm and random search with adaptation.
0
150

Machine Learning 1. Generalization Theory

2026LectureКонстантин Вячеславович Воронцов

The problem of estimating generalization ability; the error matrix and the probability of overfitting; the case of a single algorithm and the hypergeometric distribution; the principle of uniform convergence of frequencies and Boole's inequality; the Vapnik–Chervonenkis bound, the growth function and the capacity of a family of algorithms; inverting the bound and the complexity penalty; structural risk minimization; the splitting and similarity effects; the Occam's razor bound and its link to regularization.
0
140

Machine Learning 1. Lecture 9. Logical Classification Methods

2026LectureКонстантин Вячеславович Воронцов

The interpretability of models; decision trees and the greedy ID3 algorithm; the axioms of impurity, the entropy and Gini criteria; handling missing values; the merits and drawbacks of the greedy strategy; tree pruning (post-pruning); CART regression trees and Minimal Cost-Complexity Pruning; rule induction: logical regularities, families of rules (conjunctions, syndromes, half-planes, balls), search metaheuristics, a tree as a covering set of conjunctions; the Pareto front in (p,n) space, logical and statistical regularities, Fisher's exact test, informativeness criteria; building a classifier from regularities.
0
160

Machine Learning 1. Lecture 10. Linear Ensembles

2026LectureКонстантин Вячеславович Воронцов

Ensembles of models: base algorithms, the space of estimates, the decision rule and correcting functions; the problem of diversity of base algorithms; bagging, the random subspace method and a common pseudocode of stochastic ensembling; the unbiased out-of-bag estimate and estimating feature importance; converting simple voting into weighted voting; the random forest; boosting for binary classification with abstentions, the exponential approximation of the threshold loss function, the main theorem of AdaBoost with proof, convergence, classical AdaBoost and heuristics; why boosting does not overfit: experiments, bounding the probability of error through margins, the distribution of margins, comparison with bagging; the ComBoost algorithm.
0
160

Machine Learning 1. Lecture 11. Advanced Ensemble Methods

2026LectureКонстантин Вячеславович Воронцов

Bias–variance analysis: the decomposition of the error into noise, bias and variance, the bias and variance of simple voting, why complex ensembles do not overfit; gradient boosting for an arbitrary loss function as a parametric approximation of a gradient step; stochastic gradient boosting; special cases: regression, AdaBoost, AnyBoost, variants of loss functions; XGBoost: the tree as a linear model, L0 and L2 regularisation, analytical leaf values and the splitting criterion; CatBoost: leakage in the gradients, ordered boosting on subsamples of doubling length, unbiased target statistics for categorical features.
0
190

Machine Learning 1. Lecture 12. Density Estimation and Bayesian Classification

2026LectureКонстантин Вячеславович Воронцов

Estimating a probability density from a sample: the maximum likelihood principle, the multivariate Gaussian density and its geometric meaning, multicollinearity and regularisation of the covariance matrix; separating a mixture of distributions, the EM algorithm as a method of simple iterations, the Gaussian mixture (GMM); the nonparametric Parzen–Rosenblatt estimate, the choice of kernel and window width; Bayesian classification theory: the generative and discriminative approaches, the optimal Bayes classifier, the Parzen window method, the quadratic discriminant, Fisher's linear discriminant, the radial basis function network and its training by the EM algorithm.
0
160

Machine Learning 1. Bayesian Classification Theory

2026LectureКонстантин Вячеславович Воронцов

The probabilistic formulation of classification: prior and posterior class probabilities, class likelihood functions, the maximum a posteriori principle; the optimal Bayes classifier and losses from errors; empirical estimation of prior probabilities and class densities; the naive Bayes classifier, features with exponential-family densities and the linear naive Bayes classifier; a survey of Bayesian classifiers: the Parzen window method, the quadratic discriminant, Fisher's linear discriminant, Gaussian mixtures with diagonal covariance matrices, and the radial basis function network.
0
180

Machine Learning 1. Lecture 13. Clustering and Semi-Supervised Learning

2026LectureКонстантин Вячеславович Воронцов

The clustering problem and its ill-posedness, the aims of clustering, types of cluster structure, sensitivity to the choice of metric; the problem of semi-supervised learning and transductive learning, why it reduces neither to classification nor to clustering; criteria of clustering quality, the silhouette coefficient, the BCubed measures of precision and recall; the k-means method (Lloyd's algorithm) as a simplification of the EM algorithm for a mixture of Gaussians; the DBSCAN algorithm; agglomerative hierarchical clustering, the Lance–Williams formula and its special cases, the dendrogram, monotonicity, contracting and dilating distances, Ward's distance; semi-supervised learning based on classification: self-training, co-training, co-learning, the transductive SVM, expectation regularisation (XR).
0
280

Machine Learning 1. Lecture 14. Anomaly Detection and Robust Methods

2026LectureКонстантин Вячеславович Воронцов

Anomaly and novelty detection; heuristic anomaly scores: local density, isolation forest, a mixture with a background component, robust principal component analysis; outlier removal in nonparametric regression, the LOWESS algorithm; anomaly as the value of the loss function; the theory of robust learning: slowly growing functions, iterative reweighting of objects with IRS and IRLS, robust loss functions, the median, the quantile and the censored mean, robust aggregation functions and their smoothing, the IR-ERM algorithm; one-class SVM, learning from positive and unlabeled objects (PU-learning), problems with new and unknown classes.
0
480

Machine Learning 1. Lecture 15. A Short Review of the Semester

2026LectureКонстантин Вячеславович Воронцов

A closing review of the first semester: empirical risk minimisation and gradient methods; underfitting, overfitting, multicollinearity and regularisation; optimisation formulations of supervised learning (regression, classification, ranking) and unsupervised learning (density, mixtures, clustering, one-class classification, semi-supervised learning, PU-learning, principal component analysis, autoencoders); the six scientific schools of machine learning — symbolism, evolutionism, analogism, Bayesianism, connectionism, compositionism — and their Soviet roots; estimate-calculating algorithms; practice: CRISP-DM, data preprocessing, model evaluation and selection, loss analysis, A/B testing, meta-learning and AutoML.
0
170

Machine Learning 2. Lecture 1. Deep Neural Networks

2026LectureКонстантин Вячеславович Воронцов

Reminder: the neuron model, the multilayer network, stochastic gradient, momentum, vanishing and exploding gradients, ReLU, dropout, batch normalisation; depth matters more than width; feature generation and end-to-end learning; convolutional neural networks: convolution and pooling layers, the standard CNN scheme, AlexNet and ImageNet, residual networks ResNet and Highway Networks; convolutional networks for speech, texts and games; recurrent neural networks, training by BPTT, long short-term memory networks LSTM, the peephole variant, GRU.
1
390

Machine Learning 2. Lecture 2. Neural Networks for Unsupervised Learning

2026LectureКонстантин Вячеславович Воронцов

Kohonen networks: clustering as a two-layer neural network, the hard (WTA) and soft (WTM) competition rules, self-organising Kohonen maps and their interpretation; autoencoders: the formulation of the problem, the linear autoencoder and the principal component method, sparse, denoising, contractive, relational and variational autoencoders, reparametrisation, autoencoders for supervised learning, multilayer autoencoders; joint training of several models: pre-training and transfer learning, multi-task learning and few-shot learning, self-supervised learning, distillation and surrogate modelling, learning using privileged information (LUPI), generative adversarial networks (GAN).
0
210