Search results for “FastText”

Vector representations (embeddings) of texts and graphs: the distributional semantics hypothesis, syntagmatic and paradigmatic similarity of words, the CBOW and Skip-gram models of the word2vec program, hierarchical SoftMax on a Huffman tree, negative sampling (SGNS) and the connection of word2vec with the matrix factorisation of the PMI matrix, semantic analogies of words, the FastText model on character n-grams; texts as graphs; multidimensional scaling and the stress criterion, stochastic neighbour embedding SNE, perplexity, t-SNE; matrix factorisations of graphs, the random-walk models DeepWalk and node2vec; graph embedding methods as autoencoders, the generalised GraphEDM model, graph neural networks and message passing.

Real-time monitoring of scientific papers and technological news requires fast processing of complicated search demands motivated by thematically relevant information acquisition. For this case, the authors develop an exploratory search engine based on probabilistic hierarchical topic modeling. Topic model gives a low dimensional sparse interpretable vector representation (topical embedding) of a text, which is used for ranking documents by their similarity to the query. They explore several ways of comparing topical vectors including searching with thematically homogeneous text segments. Topical hierarchies are built using the regularized EM-algorithm from BigARTM project. The topic-based search achieves better precision and recall than other approaches (TF-IDF, fastText, LSTM, BERT) and even human assessors who spend up to an hour to complete the same search task. They also discover that blending hierarchical topic vectors with neural pretrained embeddings is a promising way of enriching both models that helps to get precision and recall higher than 90

, : . , , , . ,