A carregar...
Publicação
Previsão do preço ótimo para carros usados com Machine Learning: Trabalho desenvolvido para uma empresa de indústria automóvel
| Nome: | Descrição: | Tamanho: | Formato: | |
|---|---|---|---|---|
| 1.16 MB | Adobe PDF |
Autores
Orientador(es)
Resumo(s)
Este documento descreve o trabalho feito durante um estágio de 6 meses na Deloitte. O estágio teve como objetivo desenvolver uma plataforma de analytics na cloud para um cliente no setor automóvel, que visa proporcionar uma infraestrutura flexível e escalável para a análise preditiva de preços de carros usados, permitindo assim aos comerciantes desta empresa retirarem insights importantes sobre os seus dados e, consequentemente, fazer decisões mais estratégicas, otimizando o seu lucro.
É apresentada uma visão geral do projeto, com várias componentes diferentes, embora o foco seja nas tarefas principais da aluna. Sendo esta uma plataforma para ser utilizada por vários mercados diferentes, é apresentada uma metodologia generalista para que a análise consiga ser adaptada a diferentes tipos de dados e comportamentos.
Para avaliar esta metodologia, foi feita uma análise com foco em dados do mercado da Alemanha de vendas diretas, contendo carros retornados à empresa de 3 de maio de 2020 até 30 de junho de 2021, contando com 55 216 carros. Através de uma análise exploratória detalhada, foi possível identificar que, a maioria das variáveis usadas têm uma assimetria positiva muito acentuada, existem variáveis explicativas com dependências elevadas, multicolinearidade, e variáveis categóricas com valores muito elevados de cardinalidade.
Neste contexto, são sugeridas várias técnicas de tratamento de dados e feature engineering, tais como target encoding, com vista a melhorar o desempenho dos modelos utilizados. Para a seleção de variáveis é aplicado o método RFE, Recursive Feature Elimination, com o objetivo de escolher as variáveis que mais contribuem para a previsão do preço.
Procurando por robustez e precisão, optou-se por implementar e comparar modelos ensemble, dado que são conhecidos pela sua capacidade de captar comportamentos complexos nos dados e lidar bem com overfitting. Os resultados destacam a eficácia do modelo XGBoost, obtendo um 𝑅2 de 0.965 na amostra de teste, com 19 variáveis de input. Esta performance sugere que o modelo é capaz de identificar os comportamentos complexos nos dados, porém precisa de ser analisado com mais cuidado e aplicado a outras amostras para perceber a sua generalidade.
This document describes the work done during a 6-month internship at Deloitte. The internship aimed to develop a cloud analytics platform for a client in the automotive sector, which is intended to provide a flexible and scalable infrastructure for the predictive analysis of used car prices, thus, enabling the company's dealers to draw important insights from their data and consequently help them on their decision-making process, optimizing their profit. An overview of the project is presented, with several different components, although with a focus on the student's main tasks. As this is a platform to be used by several different markets, a general methodology is presented so that the analysis can be adapted to different types of data and behaviours. To evaluate this methodology, an analysis was carried out on data from the German direct sales market, containing used cars returned to the company from May 3, 2020 to June 30, 2021, with 55 216 cars. Through a detailed exploratory analysis, it was possible to identify that most of the variables used have a very pronounced positive asymmetry, some of the explanatory variables have high dependencies, multicollinearity, and that there are categorical variables with a very high cardinality. In this context, various data processing and feature engineering techniques are suggested, such as target encoding, to improve the performance of the models used. The RFE method, Recursive Feature Elimination, is used to select the variables that contribute most to price prediction. Seeking for robustness and accuracy, we chose to implement and compare ensemble models, since they are known for their ability to capture complex behaviours in the data and deal well with overfitting. The results highlight the effectiveness of the XGBoost model, obtaining an 𝑅2 0.965 in the test sample, with 19 input variables. This performance suggests that the model is able to identify complex behaviors in the data, but it needs to be analyzed more carefully and applied to other samples to understand its generality.
This document describes the work done during a 6-month internship at Deloitte. The internship aimed to develop a cloud analytics platform for a client in the automotive sector, which is intended to provide a flexible and scalable infrastructure for the predictive analysis of used car prices, thus, enabling the company's dealers to draw important insights from their data and consequently help them on their decision-making process, optimizing their profit. An overview of the project is presented, with several different components, although with a focus on the student's main tasks. As this is a platform to be used by several different markets, a general methodology is presented so that the analysis can be adapted to different types of data and behaviours. To evaluate this methodology, an analysis was carried out on data from the German direct sales market, containing used cars returned to the company from May 3, 2020 to June 30, 2021, with 55 216 cars. Through a detailed exploratory analysis, it was possible to identify that most of the variables used have a very pronounced positive asymmetry, some of the explanatory variables have high dependencies, multicollinearity, and that there are categorical variables with a very high cardinality. In this context, various data processing and feature engineering techniques are suggested, such as target encoding, to improve the performance of the models used. The RFE method, Recursive Feature Elimination, is used to select the variables that contribute most to price prediction. Seeking for robustness and accuracy, we chose to implement and compare ensemble models, since they are known for their ability to capture complex behaviours in the data and deal well with overfitting. The results highlight the effectiveness of the XGBoost model, obtaining an 𝑅2 0.965 in the test sample, with 19 input variables. This performance suggests that the model is able to identify complex behaviors in the data, but it needs to be analyzed more carefully and applied to other samples to understand its generality.
Descrição
Internship Report presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data Science
Palavras-chave
Aprendizagem Automática Supervisionada CRISP-DM Target Encoding Overfitting Métodos de Ensemble Supervised Machine Learning Target Encoding Overfitting Ensemble Methods