LibreTimes

Search results for “ARTM”

Categories

Political Philosophy

9 results

Mining Ethnic Content Online with Additively Regularized Topic Models

2016Journal articleMurat Apishev, Sergei Koltcov, Olessia Koltsova, Sergey Nikolenko +1

Computación y Sistemas

Social studies of the Internet have adopted large-scale text mining for unsupervised discovery of topics related to specific subjects. A recently developed approach to topic modeling, additive regularization of topic models (ARTM), provides fast inference and more control over the topics with a wide variety of possible regularizers than developing LDA extensions. We apply ARTM to mining ethnic-related content from Russian-language blogosphere, introduce a new combined regularizer, and compare models derived from ARTM with LDA. We show with human evaluations that ARTM is better for mining topics on specific subjects, finding more relevant topics of higher or comparable quality.
0
3

Incremental Learning of Topic Models for Finding Trend Topics in Scientific Publications

2022Journal articleN. A. Gerasimenko, A. S. Chernyavsky, M. A. Nikiforova, M. D. Nikitin +1

Doklady Mathematics

With a soaring number of scientific publications and rapid emergence of new directions and approaches, the scientific community faces the task of timely identification of trends. By a trend, we mean a semantically homogeneous topic characterized by a steady lexical kernel and a sharp, often exponential increase in the number of publications [1]. Examples of trends in machine learning are “LSTM,” “deep learning,” “word2vec,” “BERT,” and “fake news detection.” For real-time detection of trend topics from a stream of scientific publications, we use incremental methods of probabilistic topic modeling. An ARTM-based approach to early trend detection has been shown to outperform popular classical and neural network approaches to this task. A dataset of 91 trends for performance evaluation has been manually collected and made available for public use.

0
7

Convergence of the Algorithm of Additive Regularization of Topic Models

2021Journal articleI. A. Irkhin, K. V. Vorontsov

Proceedings of the Steklov Institute of Mathematics

The problem of probabilistic topic modeling is as follows. Given a collection of text documents, find the conditional distribution over topics for each document and the conditional distribution over words (or terms) for each topic. Log-likelihood maximization is used to solve this problem. The problem generally has an infinite set of solutions and is ill-posed according to Hadamard. In the framework of Additive Regularization of Topic Models (ARTM), a weighted sum of regularization criteria is added to the main log-likelihood criterion. The numerical method for solving this optimization problem is a kind of an iterative EM-algorithm written in a general form for an arbitrary smooth regularizer as well as for a linear combination of smooth regularizers. This paper studies the problem of convergence of the EM iterative process. Sufficient conditions are obtained for the convergence to a stationary point of the regularized log-likelihood. The constraints imposed on the regularizer are not too restrictive. We give their interpretations from the point of view of the practical implementation of the algorithm. A modification of the algorithm is proposed that improves the convergence without additional time and memory costs. Experiments on a news text collection have shown that our modification both accelerates the convergence and improves the value of the criterion to be optimized.
0
3
Константин Вячеславович Воронцов
@konstantin_vorontsov

Константин Вячеславович Воронцов — профессор РАН, доктор физико-математических наук. Заведующий кафедрой «Математические методы прогнозирования» ВМК МГУ, где с 2021 года — профессор; заведующий лабораторией «Машинное обучение и семантический анализ» Института искусственного интеллекта МГУ (с 2022 года); заведующий кафедрой «Машинное обучение и цифровая гуманитаристика» МФТИ (с 2021 года); заведующий кафедрой «Интеллектуальные системы» МФТИ (с 2022 года), на которой работает с 2004 года; главный научный сотрудник отдела «Интеллектуальные системы» ВЦ ФИЦ ИУ РАН, где работает с 2001 года. Окончил МФТИ (1994) и аспирантуру ВЦ РАН (1997). Кандидатская диссертация «Локальные базисы в алгебраическом подходе к проблеме распознавания» (1999, специальность 01.01.09); докторская диссертация «Комбинаторная теория надёжности обучения по прецедентам» (2010, специальность 05.13.17). Автор комбинаторной теории надёжности обучения по прецедентам, оценивающей вероятность переобучения с учётом эффектов расслоения и сходства в семействах алгоритмов; основное направление его исследований — теория и методы аддитивной регуляризации тематических моделей (ARTM). С 2014 года — научный руководитель разработки библиотеки тематического моделирования с открытым кодом BigARTM, с 2019 года — поисково-рекомендательной системы SciSearch. Один из идеологов и администраторов ресурса MachineLearning.ru (с 2008 года). Медаль РАН с премией для молодых учёных (1999, совместно с Е. В. Нарыжным); почётное звание профессора РАН (2015); юбилейная медаль РАН «300 лет Российской академии наук» (2024); премия Yandex ML Prize в номинации «Зал славы» (2025). Автор курсов «Вероятностные тематические модели», «Математические методы анализа текстов», «Прикладной статистический анализ данных» и спецкурса по теории надёжности обучения по прецедентам; в 2008–2020 годах преподавал в Школе анализа данных Яндекса. С 2004 года читает годовой курс «Машинное обучение» в МФТИ и ВМК МГУ.

Communicative Type “Municipal Employee” in the Media Space: Development of an Automatic Information and Analytical Assessment System

2024Journal articleIrina Karabulatova, Konstantin Vorontsov, Daniil Okolyshev, Ludan Zhang

Vestnik Volgogradskogo gosudarstvennogo universiteta Serija 2 Jazykoznanije

The article examines the issue of representing municipal government in the media space, followed by the proposed solution for automatically identifying signs of destructive and constructive positioning of communicative types of municipal employees in the public information space. The definition of the concept of the communicative type “municipal employee” with verification features is introduced. The results of the analysis of the organization of local self-government on the example of the Moscow region allowed us to conclude that the communicative type “municipal employee” reflects a diversified system of territorial communicative position within the regional government. The information obtained during the analysis of public information space attitudes regarding the activities of municipal employees can be automated with the method of identifying linguistic markers of emotivity to determine the communicative position of territorial authorities. The suggested methodology for effective automation of the studied subject area in the humanities has been verified as possessing a high scientific potential for further research. It is concluded that the development of technology for monitoring and forecasting public threats based on “soft power” methods through automatic and expert work to identify markers of evaluative presentation of communicative types of municipal employees is designed to help regional authorities achieve the desired results in ensuring territorial identity.
0
5

Sharpness Estimation of Combinatorial Generalization Ability Bounds for Threshold Decision Rules

2021Journal articleSh. Kh. Ishkina, K. V. Vorontsov

Automation and Remote Control

This article is devoted to the problem of calculating an exact upper bound for the functionals of the generalization ability of a family of one-dimensional threshold decision rules. An algorithm is investigated that solves the stated problem and is polynomial in the total number of samples used for training and validation and in the number of training samples. A theorem is proved for calculating an estimate for the functional of expected overfitting and an estimate for the error rate of the method for minimizing empirical risk on a validation set. The exact bounds calculated using the theorem are compared with the previously known quick-to-compute upper bounds so as to estimate the orders of overestimation of the bounds and to identify the bounds that could be used in real problems.
0
6

RuSciBench: Open Benchmark for Russian and English Scientific Document Representations

2024Journal articleA. Vatolin, N. Gerasimenko, A. Ianina, K. Vorontsov

Doklady Mathematics

Sharing scientific knowledge in the community is an important endeavor. However, most papers are written in English, which makes dissemination of knowledge in countries where English is not spoken by the majority of people harder. Nowadays, machine translation and language models may help to solve this problem, but it is still complicated to train and evaluate models in languages other than English with no or little data in the required language. To address this, we propose the first benchmark for evaluating models on scientific texts in Russian. It consists of papers from Russian electronic library of scientific publications. We also present a set of tasks which can be used to fine-tune various models on our data and provide a detailed comparison between state-of-the-art models on our benchmark.
0
4