Search results for “SAGE”

Categories

High-Performance Computing Media and Communication

48 results

Machine Learning 2. Lecture 12. Interpretability and Explainability

2026LectureКонстантин Вячеславович Воронцов

Interpretability and explainability of machine learning models: goals, tasks and basic notions, who needs explanations and why; interpretable models; feature importance in linear regression: the t-statistic, net effects, contributions and situational importance; partial dependence plots (PDP) and individual conditional expectation plots (ICE); permutation feature importance; the Shapley value from cooperative game theory, its axioms and Monte Carlo estimation, its use in linear regression; surrogate modelling in the neighbourhood of an object, the LIME, Anchors, SHAP, Shapley Kernel and SAGE methods; the Shapley value for training objects, Data Shapley and Gradient Shapley; counterfactual explanations.

230

Three-stage question answering system with sentence ranking

2019Conference paperDaria Soboleva, Konstantin Vorontsov

EPiC series in language and linguistics

We explore a recently proposed question answering system. We developed a high speed modification based on dividing the question answering system into three consecutive stages. The first step is to find the candidate documents that most likely contain the answer to the question. The second step is to rank sentences by the probability of having a correct answer to the question. The third step is to find the exact phrase that answers the question. At the third step we used a recently proposed recurrent bidirectional neural network predicting the beginning and the end of a response. In this paper we showed that the proposed question answering system allows to speed up its work without significant losses in the quality. For each step we also explored the feature space construction techniques allowing to improve the final quality.

COMBINING FACTS, SEMANTIC ROLES AND SENTIMENT LEXICON IN A GENERATIVE MODEL FOR OPINION MINING

2020Conference paperD. G. Feldman, T. R. Sadekova, K. V. Vorontsov

Computational Linguistics and Intellectual Technologies

Opinion mining is a popular task, that is applied, for example, to determine news polarisation and identify product review classes. Our task is unsupervised clusterization of opinionated texts, in particular news on political events. Many papers that tackle this issue use generative models based on lexical features. Our goal is to determine the entities defying an opinion amongst lexical, syntactic and semantic features as well as their compositions. More specifically, we test the hypothesis that an opinion is determined by the composition of the mentioned facts (SPO triples), the semantic roles of the words and the sentiment lexicon used in it. In this paper we formalise this task and prove that using a composition of the above features provides the best quality when clusterising opinionated texts. To test this hypothesis we have gathered and labelled two corpuses of news on political events and proposed a set of unsupervised algorithms for extracting the features.

Hierarchical Interpretable Topical Embeddings for Exploratory Search and Real-Time Document Tracking

2020Journal articleAnastasia Ianina, Konstantin Vorontsov

International Journal of Embedded and Real-Time Communication Systems

Real-time monitoring of scientific papers and technological news requires fast processing of complicated search demands motivated by thematically relevant information acquisition. For this case, the authors develop an exploratory search engine based on probabilistic hierarchical topic modeling. Topic model gives a low dimensional sparse interpretable vector representation (topical embedding) of a text, which is used for ranking documents by their similarity to the query. They explore several ways of comparing topical vectors including searching with thematically homogeneous text segments. Topical hierarchies are built using the regularized EM-algorithm from BigARTM project. The topic-based search achieves better precision and recall than other approaches (TF-IDF, fastText, LSTM, BERT) and even human assessors who spend up to an hour to complete the same search task. They also discover that blending hierarchical topic vectors with neural pretrained embeddings is a promising way of enriching both models that helps to get precision and recall higher than 90

Improving the Quality of Machine Translation Using the Reverse Model

2022Journal articleN. A. Skachkov, K. V. Vorontsov

Automation and Remote Control

Machine translation is a natural language text processing task that aims to automatically translate input text from one language into another language. The currently known machine translation models show a fairly high quality of translation between large languages, but for smaller language areas, represented by less data, the problem is still not solved. Different methods are used to deal with various errors in automatic translation systems. This paper discusses approaches that use translation models of reverse language directions and improve consistency between translations of the same text using direct and reverse translation models. The paper presents a general theoretical justification for such methods in terms of solving the likelihood maximization problem and also proposes a method for stable training of modern models using cyclic translations.

Incremental Topic Modeling for Scientific Trend Topics Extraction

2023Conference paperNikolai Gerasimenko, Alexander Chernyavskiy, Maria Nikiforova, Anastasia Ianina +1

Computational Linguistics and Intellectual Technologies

Rapid growth of scientific publications and intensive emergence of new directions and approaches poses a challenge to the scientific community to identify trends in a timely and automatic manner. We denote trend as a semantically homogeneous theme that is characterized by a lexical kernel steadily evolving in time and a sharp, often exponential, increase in the number of publications. In this paper, we investigate recent topic modeling approaches to accurately extract trending topics at an early stage. In particular, we customize the standard ARTM-based approach and propose a novel incremental training technique which helps the model to operate on data in real-time. We further create the Artificial Intelligence Trends Dataset (AITD) that contains a collection of early-stage articles and a set of key collocations for each trend. The conducted experiments demonstrate that the suggested ARTM-based approach outperforms the classic PLSA, LDA models and a neural approach based on BERT representations. Our models and dataset are open for research purposes.

Learning Topic Models with Arbitrary Loss

2020Conference paperMurat Apishev, Konstantin Vorontsov

Topic modeling is an area of text analysis actively developing over the past 20 years. A probabilistic topic model (PTM) finds a set of hidden topics from a collection of text documents. It defines each topic as a probability distribution over words and describes each document as a probability mixture of topic distributions. Learning algorithms for topic models are usually based on Bayesian inference or log-likelihood maximization. In both cases, EM-like algorithms are used. In this paper, we propose to replace the logarithm in the log-likelihood by an arbitrary smooth loss function. We prove that such a modification preserves both the structure of the algorithm and compatibility with any regularizers in terms of additive regularization of topic models (ARTM). Moreover, in the case of a linear loss, the Estep becomes much faster due to the omission of a normalization. We study combinations of the fast and usual E-steps and compare them to regularization using different number of topics in both offline and online versions of EM-algorithm. For an empirical comparison of the algorithms, we estimate perplexity, coherence, and learning time. We use an efficient parallel implementation of the EM-algorithm from the BigARTM open-source library. We show that in most cases the two-stage strategy wins, which uses fast E-steps at the beginning of iterations, then proceeds with usual E-steps.

2026Problem sheet

International mathematics competition for pre-university students, held annually since 1959. Six problems over two days.

380