Search results for “LIME”

Interpretability and explainability of machine learning models: goals, tasks and basic notions, who needs explanations and why; interpretable models; feature importance in linear regression: the t-statistic, net effects, contributions and situational importance; partial dependence plots (PDP) and individual conditional expectation plots (ICE); permutation feature importance; the Shapley value from cooperative game theory, its axioms and Monte Carlo estimation, its use in linear regression; surrogate modelling in the neighbourhood of an object, the LIME, Anchors, SHAP, Shapley Kernel and SAGE methods; the Shapley value for training objects, Data Shapley and Gradient Shapley; counterfactual explanations.

Linear dependence and independence of the rows of a matrix, the criterion via a linear combination, and the connection to rank.

Linear systems in matrix form, the Kronecker–Capelli theorem, homogeneous systems, and the fundamental system of solutions.

The axioms of a linear (vector) space, examples and counterexamples.

A linear operator and its matrix, the action x' = Ax, similar matrices, and the formula B = C⁻¹AC.

Why the definition of a filtered category insists on non-emptiness: the empty colimit fails to commute with the empty limit, and that is the only failure.

Vectors in coordinates: dividing a segment, midpoints of sides, the angle between vectors, the cross product and the scalar triple product.

Integrating a vector field along a directed curve: parametrising a segment and a graph, the work of a force, path dependence, a potential field and its potential, a curl-free field in a domain that is not simply connected.

The general scheme of supervised learning for regression, binary and multiclass classification; the margin and continuous approximations of the threshold loss; the linear classifier as a model of a neuron; the stochastic gradient method, momentum, Nesterov's accelerated gradient and heuristics; multicollinearity and L2 regularization; the maximum likelihood principle, its link to empirical risk minimization and the probabilistic meaning of regularization.

Multiple linear regression and least squares, the pseudoinverse and the geometric meaning of least squares; the solution through the singular value decomposition; multicollinearity and the condition number; ridge regression, choosing the regularization parameter and the effective dimension; LASSO and non-smooth regularizers for feature selection; principal component analysis as a low-rank matrix decomposition; the spectral method of least squares.

Nonlinear regression: the Newton–Raphson and Gauss–Newton methods as a series of linear regressions; logistic regression and iteratively reweighted least squares (IRLS); the generalized additive model and backfitting; the probabilistic meaning of least squares; the generalized linear model (GLM), the exponential family of distributions and the link function; logistic regression as a special case of GLM; the median, quantile regression and robust loss functions.

Ensembles of models: base algorithms, the space of estimates, the decision rule and correcting functions; the problem of diversity of base algorithms; bagging, the random subspace method and a common pseudocode of stochastic ensembling; the unbiased out-of-bag estimate and estimating feature importance; converting simple voting into weighted voting; the random forest; boosting for binary classification with abstentions, the exponential approximation of the threshold loss function, the main theorem of AdaBoost with proof, convergence, classical AdaBoost and heuristics; why boosting does not overfit: experiments, bounding the probability of error through margins, the distribution of margins, comparison with bagging; the ComBoost algorithm.

Linear dependence of vectors, basis and coordinates, dimension, subspaces, and linear span.

Text trees as a data structure occur in numerous machine learning tasks like hierarchical summarization and automatic mind map generation. One of the main methods of quality evaluation in these tasks is comparison with reference hierarchies created by experts. The method used so far to compare text hierarchies, as shown in this work, poorly accounts for their structure and text semantics relative to phrasing. To address this issue, we propose a new metric on the set of text trees — text tree edit distance (TTED), based on tree edit distance with semantic distance between texts measured using a large language model. To evaluate how the metric reflects different aspects of text tree difference, we introduce special quality coefficients that reflect the sensitivity of a metric to paraphrasing relative to structural and semantic differences of text trees. Using these coefficients, we conduct extensive testing of the proposed metric and its modifications compared to a baseline used in previous works to compare text hierarchies, which shows that TTED indeed captures significant differences between text trees more accurately than the previously used method. We also provide a practical implementation of TTED for further usage.

This paper introduces an approach to measuring the cognitive complexity of texts on various language levels. While standard readability indices are based on the linear combination of primary statistics, our general approach allows us to estimate complexity on morphological, lexical, syntactic, and discursive levels. Each model is defined by the tokens for the specific language level and the complexity function of a single token. We then use the reference collection of moderately complex texts and the quantile-based approach to spot the abnormally rare tokens. The proposed supervised ensemble, based on the ElasticNet model, incorporates models from all language levels. Having collected a labeled dataset through crowdsourcing, consisting of pairs of articles from the Russian Wikipedia, we consider several models and ensembles and compare them to common baselines. Suggested models are flexible due to the freedom in choosing the reference collection. The described experiments confirm the competitiveness of the proposed approach, as the ensembles demonstrate the best target metric value.

We explore a recently proposed question answering system. We developed a high speed modification based on dividing the question answering system into three consecutive stages. The first step is to find the candidate documents that most likely contain the answer to the question. The second step is to rank sentences by the probability of having a correct answer to the question. The third step is to find the exact phrase that answers the question. At the third step we used a recently proposed recurrent bidirectional neural network predicting the beginning and the end of a response. In this paper we showed that the proposed question answering system allows to speed up its work without significant losses in the quality. For each step we also explored the feature space construction techniques allowing to improve the final quality.

The problem of probabilistic topic modeling is as follows. Given a collection of text documents, find the conditional distribution over topics for each document and the conditional distribution over words (or terms) for each topic. Log-likelihood maximization is used to solve this problem. The problem generally has an infinite set of solutions and is ill-posed according to Hadamard. In the framework of Additive Regularization of Topic Models (ARTM), a weighted sum of regularization criteria is added to the main log-likelihood criterion. The numerical method for solving this optimization problem is a kind of an iterative EM-algorithm written in a general form for an arbitrary smooth regularizer as well as for a linear combination of smooth regularizers. This paper studies the problem of convergence of the EM iterative process. Sufficient conditions are obtained for the convergence to a stationary point of the regularized log-likelihood. The constraints imposed on the regularizer are not too restrictive. We give their interpretations from the point of view of the practical implementation of the algorithm. A modification of the algorithm is proposed that improves the convergence without additional time and memory costs. Experiments on a news text collection have shown that our modification both accelerates the convergence and improves the value of the criterion to be optimized.

This paper presents the solution of the LomonosovMSU team for the SemEval-2024 Task 4 "Multilingual Detection of Persuasion Techniques in Memes" competition for the English language task.During the task solving process, generative and BERT-like (training classifiers on top of embedder models) approaches were tested for subtask №1, as well as an BERT-like approach on top of multimodal embedder models for subtasks №2a/№2b.The models were trained using datasets provided by the competition organizers, enriched with filtered datasets from previous SemEval competitions.The following results were achieved: 18th place for subtask №1, 9th place for subtask №2a, and 11th place for subtask №2b.The code for the solutions is available at github 1 .