Travaux

It is shown that computationally tight bounds for the probability of overfitting can be obtained only by simultaneous consideration of the following two properties of classifier sets: splitting into error levels and similarity of classifiers. For a set consisting of only two classifiers, an exact bound is obtained for the probability of overfitting. This is the simplest learning task that exhibits overfitting and the effects of splitting and similarity, which reduce the probability of overfitting. For a more complex case—a chain of classifiers—an experiment is carried out in which the effects of splitting and similarity are estimated separately. It is shown that reasonably low probabilities of overfitting can be obtained only for the sets that possess both properties.

Three general methods for obtaining exact bounds on the probability of overfitting are proposed within statistical learning theory: a method of generating and destroying sets, a recurrent method, and a blockwise method. Six particular cases are considered to illustrate the application of these methods. These are the following model sets of predictors: a pair of predictors, a layer of a Boolean cube, an interval of a Boolean cube, a monotonic chain, a unimodal chain, and a unit neighborhood of the best predictor. For the interval and the unimodal chain, the results of numerical experiments are presented that demonstrate the effects of splitting and similarity on the probability of overfitting.

Topic modelling is an area of text mining that has been actively developed in the last 15 years. A probabilistic topic model extracts a set of hidden topics from a collection of text documents. It defines each topic by a probability distribution over words and describes each document with a probability distribution over topics. In applications, there are often many requirements, such as, for example, problem-specific knowledge and additional data, to be taken into account. Therefore, it is natural for topic modelling to be considered a multiobjective optimization problem. However, historically, Bayesian learning became the most popular approach for topic modelling. In the Bayesian paradigm, all requirements are formalized in terms of a probabilistic generative process. This approach is not always convenient due to some limitations and technical difficulties. In this work, we develop a non-Bayesian multiobjective approach called the Additive Regularization of Topic Models (ARTM). It is based on regularized Maximum Likelihood Estimation (MLE), and we show that many of the well-known Bayesian topic models can be re-formulated in a much simpler way using the regularization point of view. We review some of the most important types of topic models: multimodal, multilingual, temporal, hierarchical, graph-based, and short-text. The ARTM framework enables easy combination of different types of models to create new models with the desired properties for applications. This modular “lego-style” technology for topic modelling is implemented in the open-source library BigARTM.

Probabilistic topic modeling of text collections is a powerful tool for statistical text analysis based on the preferential use of graphical models and Bayesian learning. Additive regularization for topic modeling (ARTM) is a recent semiprobabilistic approach, which provides a simpler inference for many models previously studied only in the Bayesian settings. ARTM reduces barriers to entry into topic modeling research field and facilitates combination of topic models. In this paper we develop the multimodal extension of ARTM approach and implement it in BigARTM open source project for online parallelized topic modeling. We demonstrate the ability of non-Bayesian regularization to combine modalities, languages and multiple criteria to find sparse, diverse, and interpretable topics.

Обзор оптимизационных задач машинного обучения. Лекция курса «Машинное обучение» К. В. Воронцова. Обзор оптимизационных задач машинного обучения

Теория переобучения. Лекция курса «Машинное обучение» К. В. Воронцова. Задача оценивания вероятности переобучения. Матрица ошибок конечного множества алгоритмов.

Интерпретируемость и объяснимость. Лекция курса «Машинное обучение» К. В. Воронцова. Интерпретируемость и объяснимость — цели, задачи, основные понятия.

Активное обучение. Лекция курса «Машинное обучение» К. В. Воронцова. Постановка задачи машинного обучения. Основные стратегии: отбор объектов из выборки и из потока, синтез объектов. Приложения активного обучения.

Обучение с подкреплением. Лекция курса «Машинное обучение» К. В. Воронцова. Задача о многоруком бандите. Жадные и эпсилон-жадные стратегии. Метод UCB (upper confidence bound).

Инкрементное и онлайновое обучение. Лекция курса «Машинное обучение» К. В. Воронцова. Задачи инкрементного и онлайнового обучения. Оценивание инкрементного обучения. Кривые обучения.

Адаптивные методы прогнозирования. Лекция курса «Машинное обучение» К. В. Воронцова. Задача прогнозирования временных рядов. Примеры приложений.

Поиск ассоциативных правил. Лекция курса «Машинное обучение» К. В. Воронцова. Понятие ассоциативного правила и его связь с понятием логической закономерности.

Рекомендательные системы. Лекция курса «Машинное обучение» К. В. Воронцова. Задачи коллаборативной фильтрации, транзакционные данные.