Development of Distributed Cache Strategy for Analytic Cluster in an Internet of Things System
Zhou, Yang (2016)
Zhou, Yang
Vaasan ammattikorkeakoulu
2016
All rights reserved
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2016061012667
https://urn.fi/URN:NBN:fi:amk-2016061012667
Tiivistelmä
This thesis discusses the development of a distributed cache strategy for an analyt-ic cluster in an IoT system. In this thesis, LRU and Proactive Cache and essential distributed system related concepts are discussed. The study about the approaches for performance optimization, nodes and data distributing in the IoT system are also included.
In the IoT system, the cluster for data analysis involves large volume of data and some specific processes such as streaming processing raises a need for real-time data querying, which brings a heavy load to the data center for catering the re-quest. Besides for expanding the capability of the data center, a well-organized cache layer is required, which will noticeably enhance the performance and scalability of the data center.
This thesis mainly contributes to the development of a cache strategy based on the distributed system. In the cache strategy session, LRU and Proactive Cache, and the optimizing of them for the IoT system are discussed. Next, a set of topology model of nodes is introduced and compared. After that, the algorithm of data partitioning is discussed and examined. Redis is used as the main database in the study, Apache Kafka is involved for enabling Proactive Cache and Spring Boot is adopted as the platform for the implementation of algorithms and business logic included in this thesis. The result of this thesis is tested using local development machine and the development server in Wapice Ltd.
It can be concluded that the distributed cache strategy developed in this thesis is functional and well-extendable. It fulfills the need of high performance data querying from analytic cluster and reduces the load from data center in the IoT system.
In the IoT system, the cluster for data analysis involves large volume of data and some specific processes such as streaming processing raises a need for real-time data querying, which brings a heavy load to the data center for catering the re-quest. Besides for expanding the capability of the data center, a well-organized cache layer is required, which will noticeably enhance the performance and scalability of the data center.
This thesis mainly contributes to the development of a cache strategy based on the distributed system. In the cache strategy session, LRU and Proactive Cache, and the optimizing of them for the IoT system are discussed. Next, a set of topology model of nodes is introduced and compared. After that, the algorithm of data partitioning is discussed and examined. Redis is used as the main database in the study, Apache Kafka is involved for enabling Proactive Cache and Spring Boot is adopted as the platform for the implementation of algorithms and business logic included in this thesis. The result of this thesis is tested using local development machine and the development server in Wapice Ltd.
It can be concluded that the distributed cache strategy developed in this thesis is functional and well-extendable. It fulfills the need of high performance data querying from analytic cluster and reduces the load from data center in the IoT system.