A case study in developing an automated ETL solution : concept and implementation
Pham, Phuong (2020)
Pham, Phuong
2020
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2020052614188
https://urn.fi/URN:NBN:fi:amk-2020052614188
Tiivistelmä
The study focuses on the implementation of an automated extract, transform, and load (ETL) solution for the commissioner company of this thesis, a global spares supply company. The commissioner company has an existing automated solution that still has some manual steps in the progress and limitations in extracting transparent tables in its enterprise resource planning (ERP) system and other online sources. The objective of this thesis was to introduce and implement the ETL process that transforms raw data from multiple data sources into meaningful and valuable information in the data warehouse. The research approach for this thesis is action research. This is a combination of taking action and doing research for the given problem. The definition of ETL was examined and its implementation areas were studied based on the combination of quantitative and qualitative methodology. Data cleaning and the application of data mining techniques were also implemented in the process to extract knowledge.
The thesis was carried out within the scope of a global spares supply company. The study was carried out with qualitative research interviews in the commissioning company, study of the existing process along with analyses on the performance of the existing process. The interviews were used to gather information about the views of the interviewees about ETL and its current challenges.
For the development of the system, this thesis explained a deployment process, introduces libraries, and shows how to utilize these libraries for data integration. The deployment process was built and reviewed by the case company and adjusted to better meet the case company needs. The outcome of this thesis is the automated ETL scripts released on production for the case company.
The thesis was carried out within the scope of a global spares supply company. The study was carried out with qualitative research interviews in the commissioning company, study of the existing process along with analyses on the performance of the existing process. The interviews were used to gather information about the views of the interviewees about ETL and its current challenges.
For the development of the system, this thesis explained a deployment process, introduces libraries, and shows how to utilize these libraries for data integration. The deployment process was built and reviewed by the case company and adjusted to better meet the case company needs. The outcome of this thesis is the automated ETL scripts released on production for the case company.