Search results for “machine learning”

Categories

7 results

Incremental Learning of Topic Models for Finding Trend Topics in Scientific Publications

2022Journal articleN. A. Gerasimenko, A. S. Chernyavsky, M. A. Nikiforova, M. D. Nikitin +1

Doklady Mathematics

With a soaring number of scientific publications and rapid emergence of new directions and approaches, the scientific community faces the task of timely identification of trends. By a trend, we mean a semantically homogeneous topic characterized by a steady lexical kernel and a sharp, often exponential increase in the number of publications [1]. Examples of trends in machine learning are “LSTM,” “deep learning,” “word2vec,” “BERT,” and “fake news detection.” For real-time detection of trend topics from a stream of scientific publications, we use incremental methods of probabilistic topic modeling. An ARTM-based approach to early trend detection has been shown to outperform popular classical and neural network approaches to this task. A dataset of 91 trends for performance evaluation has been manually collected and made available for public use.

Additive regularization of topic models

2014Journal articleKonstantin Vorontsov, Anna Potapenko

Machine Learning

Константин Вячеславович Воронцов

@konstantin_vorontsov

Константин Вячеславович Воронцов — профессор РАН, доктор физико-математических наук. Заведующий кафедрой «Математические методы прогнозирования» ВМК МГУ; заведующий лабораторией «Машинное обучение и семантический анализ» Института искусственного интеллекта МГУ; заведующий кафедрой «Машинное обучение и цифровая гуманитаристика» МФТИ; профессор кафедры «Интеллектуальные системы» МФТИ; главный научный сотрудник отдела «Интеллектуальные системы» ВЦ ФИЦ ИУ РАН. Области научных интересов: машинное обучение, комбинаторная теория обобщающей способности, вероятностное тематическое моделирование, анализ текстов. Автор годового курса «Машинное обучение», который читается с 2004 года на кафедре «Интеллектуальные системы» МФТИ, с 2007 года на кафедре ММП ВМК МГУ и с 2009 года в Школе анализа данных Яндекса. Один из идеологов и администраторов ресурса MachineLearning.ru.

Improving the Quality of Machine Translation Using the Reverse Model

2022Journal articleN. A. Skachkov, K. V. Vorontsov

Automation and Remote Control

Machine translation is a natural language text processing task that aims to automatically translate input text from one language into another language. The currently known machine translation models show a fairly high quality of translation between large languages, but for smaller language areas, represented by less data, the problem is still not solved. Different methods are used to deal with various errors in automatic translation systems. This paper discusses approaches that use translation models of reverse language directions and improve consistency between translations of the same text using direct and reverse translation models. The paper presents a general theoretical justification for such methods in terms of solving the likelihood maximization problem and also proposes a method for stable training of modern models using cyclic translations.

Reranking Hypotheses in Translation Models Using Human Markup

2024Journal articleK. V. Vorontsov, N. A. Skachkov

Journal of Computer and Systems Sciences International

Modern machine translation systems are trained on large volumes of parallel data obtained using heuristic methods of bypassing the Internet. The poor quality of the data leads to systematic translation errors, which can be quite noticeable to humans. To fix such errors, human-based models for reranking hypotheses is introduced in this study. In this paper the use of human markup is shown not only to increase the overall quality of the translation but also to significantly reduce the number of systematic translation errors. In addition, the relative simplicity of human markup and its integration in the model training process opens up new opportunities in the field of domain adaptation of translation models for new domains like online retail.

Hypotheses re-ranking in translation models using human markup

2024Journal articleK. V. Vorontsov, N. A. Skachkov

Известия Российской академии наук Теория и системы управления

Modern machine translation systems are trained on large volumes of parallel data obtained using heuristic methods of the Internet bypassing. The poor quality of the data leads to systematic translation errors, which can be quite noticeable from the human point of view. To fix such errors a human based models hypotheses re-ranking is introduced in this work. In this paper the use of human markup is shown not only to increase the overall quality of translation, but also to significantly reduce the number of systematic translation errors. In addition, the relative simplicity of human markup and its integration in the model training process opens up new opportunities in the field of domain adaptation of translation models for new domains like online retail.

RuSciBench: Open Benchmark for Russian and English Scientific Document Representations

2024Journal articleA. Vatolin, N. Gerasimenko, A. Ianina, K. Vorontsov

Doklady Mathematics

Sharing scientific knowledge in the community is an important endeavor. However, most papers are written in English, which makes dissemination of knowledge in countries where English is not spoken by the majority of people harder. Nowadays, machine translation and language models may help to solve this problem, but it is still complicated to train and evaluate models in languages other than English with no or little data in the required language. To address this, we propose the first benchmark for evaluating models on scientific texts in Russian. It consists of papers from Russian electronic library of scientific publications. We also present a set of tasks which can be used to fine-tune various models on our data and provide a detailed comparison between state-of-the-art models on our benchmark.