Konstantin Vorontsov

The Universal data markup structures empower the annotation of multiple text fragments (multi-spans or multi-fragments furthermore) to analyze large collections of content. Multi-spans have demonstrated helpful in tackling issues related to the programmed discovery of semantic blunders in school essays or human values in social media writings. Labeling multi-fragment information has made it conceivable to form an interdisciplinary classification of human values. This classifier consists of 105 labels grouped into 7 categories, and a corresponding dataset has been created. Subsequent ML experiments have been designed to demonstrate the effectiveness of the multi-spans structure in recovering annotations of human values. The accuracy of the multi-fragment detector is 0.943 for material values and 0.957 for legal awareness (a subject of civic engagement and citizenship).

Modern machine translation systems are trained on large volumes of parallel data obtained using heuristic methods of the Internet bypassing. The poor quality of the data leads to systematic translation errors, which can be quite noticeable from the human point of view. To fix such errors a human based models hypotheses re-ranking is introduced in this work. In this paper the use of human markup is shown not only to increase the overall quality of translation, but also to significantly reduce the number of systematic translation errors. In addition, the relative simplicity of human markup and its integration in the model training process opens up new opportunities in the field of domain adaptation of translation models for new domains like online retail.

Topic modelling is fundamentally a soft clustering problem (of known objects – documents, over unknown clusters – topics). That is, the task is incorrectly posed. In particular, the topic models are unstable and incomplete. All this leads to the fact that the process of finding a good topic model (repeated hyperparameter selection, model training, and topic quality assessment) can be particularly long and labor-intensive. We aim to simplify the process, to make it more deterministic and provable. To this end, we present a method for iterative training of a topic model. The essence of the method is that a series of related topic models are trained so that each subsequent model is at least as good as the previous one, i.e., that it retains all the good topics found earlier. The connection between the models is achieved by additive regularization. The result of this iterative training is the last topic model in the series, which we call the iteratively updated additively regularized topic model (ITAR). Experiments conducted on several collections of natural language texts show that the proposed ITAR model performs better than other popular topic models (LDA, ARTM, BERTopic), its topics are diverse, and its perplexity (ability to "explain" the underlying data) is moderate.

This paper presents the solution of the LomonosovMSU team for the SemEval-2024 Task 4 "Multilingual Detection of Persuasion Techniques in Memes" competition for the English language task.During the task solving process, generative and BERT-like (training classifiers on top of embedder models) approaches were tested for subtask №1, as well as an BERT-like approach on top of multimodal embedder models for subtasks №2a/№2b.The models were trained using datasets provided by the competition organizers, enriched with filtered datasets from previous SemEval competitions.The following results were achieved: 18th place for subtask №1, 9th place for subtask №2a, and 11th place for subtask №2b.The code for the solutions is available at github 1 .

LLM-based representation learning is widely used to build effective information retrieval systems, including scientific domains. For making science more open and affordable, it is important that these systems support multilingual (and cross-lingual) search and do not require significant computational power. To address this we propose SciRus-tiny, light multilingual encoder trained from scratch on 44 M abstracts (15B tokens) of research papers and then tuned in a contrastive manner using citation data. SciRus-tiny outperforms SciNCL, English-only SOTA-model for scientific texts, on 13/24 tasks, achieving SOTA on 7, from SciRepEval benchmark. Furthermore, SciRus-tiny is much more effective than SciNCL: it is almost 5x smaller (23 M parameters vs. 110 M), having approximately 2x smaller embeddings (312 vs. 768) and 2x bigger context length (1024 vs. 512). In addition to the tiny model, we also propose the SciRus-small (61 M parameters and 768 embeddings size), which is more powerful and can be used for complicated downstream tasks. We further study different ways of contrastive pre-training and demonstrate that almost SOTA results can be achieved without citation information, operating with only title-abstract pairs.

Sharing scientific knowledge in the community is an important endeavor. However, most papers are written in English, which makes dissemination of knowledge in countries where English is not spoken by the majority of people harder. Nowadays, machine translation and language models may help to solve this problem, but it is still complicated to train and evaluate models in languages other than English with no or little data in the required language. To address this, we propose the first benchmark for evaluating models on scientific texts in Russian. It consists of papers from Russian electronic library of scientific publications. We also present a set of tasks which can be used to fine-tune various models on our data and provide a detailed comparison between state-of-the-art models on our benchmark.

The article examines the issue of representing municipal government in the media space, followed by the proposed solution for automatically identifying signs of destructive and constructive positioning of communicative types of municipal employees in the public information space. The definition of the concept of the communicative type “municipal employee” with verification features is introduced. The results of the analysis of the organization of local self-government on the example of the Moscow region allowed us to conclude that the communicative type “municipal employee” reflects a diversified system of territorial communicative position within the regional government. The information obtained during the analysis of public information space attitudes regarding the activities of municipal employees can be automated with the method of identifying linguistic markers of emotivity to determine the communicative position of territorial authorities. The suggested methodology for effective automation of the studied subject area in the humanities has been verified as possessing a high scientific potential for further research. It is concluded that the development of technology for monitoring and forecasting public threats based on “soft power” methods through automatic and expert work to identify markers of evaluative presentation of communicative types of municipal employees is designed to help regional authorities achieve the desired results in ensuring territorial identity.

Modern machine translation systems are trained on large volumes of parallel data obtained using heuristic methods of bypassing the Internet. The poor quality of the data leads to systematic translation errors, which can be quite noticeable to humans. To fix such errors, human-based models for reranking hypotheses is introduced in this study. In this paper the use of human markup is shown not only to increase the overall quality of the translation but also to significantly reduce the number of systematic translation errors. In addition, the relative simplicity of human markup and its integration in the model training process opens up new opportunities in the field of domain adaptation of translation models for new domains like online retail.