A carregar...
Publicação
Mapintel: enhancing competitive intelligence acquisition through embeddings and visual analytics
| Nome: | Descrição: | Tamanho: | Formato: | |
|---|---|---|---|---|
| 3.34 MB | Adobe PDF |
Orientador(es)
Resumo(s)
Competitive Intelligence allows an organization to keep up with market trends and
foresee business opportunities. This practice is mostly performed by analysts scanning
for any piece of valuable information in a myriad of dispersed and unstructured
sources. Here we present MapIntel, a system for acquiring intelligence from large
collections of text data by representing each document as a multidimensional vector
that captures its own semantics. The system is designed to handle complex Natural
Language queries and visual exploration of the corpus, potentially aiding overburdened
analysts in finding meaningful insights to help decision-making. The searching
module of the system uses a retriever and re-ranker engine that first finds the closest
neighbors to the query embedding, and then sifts the results through a cross-encoder
model that identifies the most relevant documents. The browsing or visualization
module also leverages the embeddings by projecting them onto 2 dimensions while
preserving the multidimensional landscape, resulting in a map where semantically
related documents form topical clusters which we capture using topic modeling. This
map aims at promoting a fast overview of the corpus while allowing a more detailed
exploration and interactive information encountering process. We evaluate the system
and its components on the 20 newsgroups dataset, making use of the semantic
document labels provided, and we demonstrate the superiority of Transformer-based
components. Finally, we present a prototype of the system in Python and show how
some of its features can be used to acquire intelligence from a news article corpus we
collected during a period of 8 months.
Descrição
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics
This work was supported by national funds through FCT (Fundação para a Ciência e a Tecnologia), under the project MapIntel (DSAIPA/DS/0116/2019): https://doi.org/10.54499/DSAIPA/DS/0116/2019
This work was supported by national funds through FCT (Fundação para a Ciência e a Tecnologia), under the project MapIntel (DSAIPA/DS/0116/2019): https://doi.org/10.54499/DSAIPA/DS/0116/2019
Palavras-chave
Natural Language Processing Transformer Architecture UMAP Information Encountering Information Retrieval Competitive Intelligence Sentence Embeddings Topic Modeling Unsupervised Learning