Search results for “sequences”

Модели внимания и трансформеры. Лекция курса «Машинное обучение» К. В. Воронцова. Задачи обработки и преобразования последовательностей (sequence to sequence).

Lexical Quantile-Based Text Complexity Measure

2019Conference paperAITHEA, Russia, Maksim Eremeev, Konstantin Vorontsov

This paper introduces a new approach to estimating the text document complexity.Common readability indices are based on average length of sentences and words.In contrast to these methods, we propose to count the number of rare words occurring abnormally often in the document.We use the reference corpus of texts and the quantile approach in order to determine what words are rare, and what frequencies are abnormal.We construct a general text complexity model, which can be adjusted for the specific task, and introduce two special models.The experimental design is based on a set of thematically similar pairs of Wikipedia articles, labeled using crowdsourcing.The experiments demonstrate the competitiveness of the proposed approach.

Three-stage question answering system with sentence ranking

2019Conference paperDaria Soboleva, Konstantin Vorontsov

EPiC series in language and linguistics

We explore a recently proposed question answering system. We developed a high speed modification based on dividing the question answering system into three consecutive stages. The first step is to find the candidate documents that most likely contain the answer to the question. The second step is to rank sentences by the probability of having a correct answer to the question. The third step is to find the exact phrase that answers the question. At the third step we used a recently proposed recurrent bidirectional neural network predicting the beginning and the end of a response. In this paper we showed that the proposed question answering system allows to speed up its work without significant losses in the quality. For each step we also explored the feature space construction techniques allowing to improve the final quality.

ruSciFact: Open Benchmark for Verifying Scientific Facts in Russian

2025Conference paperA. Vatolin, N. Gerasimenko, N. Loukachevitch, A. Ianina +1

Computational Linguistics and Intellectual Technologies

Against the backdrop of active LLM development, their tendency to hallucinate, as well as the growing volume of texts they generate, the validation of facts has become increasingly important and relevant.We propose ruSciFact 1 , a new benchmark for fact-checking scientific claims in Russian.ruSciFact is structured as a NLI task, the goal is to verify whether a fact is confirmed by a given abstract.To generate facts, we used an 3-step pipeline based on LLaMA-405B, validating the resulting sentences with the help of assessors-terminologists.The ruSciFact dataset consists of 1128 pairs in the format , which we are releasing as open source together with the benchmark code.Additionally, we are opensourcing the fact-generation pipeline 2 , which facilitates the expansion of the dataset to specific scientific domains.We evaluated several popular language models on ruSciFact, including text embedders and generative models.The results show that this benchmark allows to effectively assess the fact-checking capabilities of LLMs in Russian.