A carregar...
Publicação
Transformers in Time Series Forecasting: A Systematic Literature Review
| Nome: | Descrição: | Tamanho: | Formato: | |
|---|---|---|---|---|
| 2.66 MB | Adobe PDF |
Orientador(es)
Resumo(s)
The growing interest in transformers for time series forecasting has triggered an exponential
surge in publications, making it increasingly challenging to organize and understand
emerging trends and discern future directions. This thesis is a systematic literature review of
the subject, that aims to study how the different components and aspects of the
transformers evolved, finding trends, and highlighting gaps and future directions. The review
focuses on papers in English, peer-reviewed, published in reputable sources, and that focus
exclusively on transformer models for time series forecasting. It spans sources such as ACM,
ARXiv, Elsevier, MDPI, OpenReview, AAAI, IJCAI, IEEEExplore, NeurIPS, Springer, epubs, and
Jstage, from December of 2023 to April of 2024 – selecting a total of 99 papers. To assess the
risk of bias, statements regarding conflicts of interest and sources of funding were
examined. Publication bias was evaluated by comparing transformer results over time. The
reviewed papers reveal a preference for nine datasets in specific but a lot of papers still use
unique private datasets, with some variation of the time-horizons used as well as the
metrics. Recent advancements showcase innovations in Input Representation (Positional
Encoding, Segmentation, Decomposition, and Covariates), Modelling (Attention Mechanism,
Feature Selection, Channel Dependence or Independence, Multi-scale and Hierarchical
approaches, and Hybridization), Model Optimization (Regularization, Loss function, and
Normalization), and Types of Learning. These advancements aim to enhance model
adaptability, accuracy, robustness, and computational efficiency, as well as to improve the
capture of local context and temporal relationships at different granularities. However, there
is a limited understanding of transformer performance across diverse domains and datasets,
as most studies focus on tailored models rather than cross-domain evaluations. The lack of
standardized benchmarks complicates model assessment, necessitating more transparent
and explainable models. Scalability and computational efficiency remain major concerns,
especially for large-scale and real-time implementations.
Descrição
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data Science
Palavras-chave
Transformer Time Series Forecast Attention Mechanism Deep Learning