The emerging Internet of Things and Smart Cities scenarios also confirm the requirement for real time, large scale data processing.
In this paper, we present a real world case study of a large-scale data processing and analysis system.
Because cloud computing addresses a key difficulties surrounding large scale data processing, and because there are many difficulties that current technology is facing with there is a need to find ways improving the services to the client by introducing new techniques for upgrading the service.
From that necessity ERMA was born, one of the first large scale data processing machines for business.
MapReduce [1] is considered one of the most prominent programming models for problems of large scale data processing of which the cluster analysis.
Although there is a variety of parallel programming models, the MapReduce paradigm is considered as the most prominent model for problems of large scale data processing of which the clustering.
Examples include using a distributed network of video game players to assist in solving protein folding problems (Markoff 2010), and leveraging Amazon's Mechanical Turk crowdsourcing marketplace platform to assist with large scale data processing and coding functions that cannot be automated (Conley & Tosti-Kharas 2010; Chen et al. 2011).
A trend of huge amount of data with characteristics of large scale, complex types, and wide area distribution produced by smart substation makes the traditional relational database more and more difficult to adapt to the requirements of large scale data processing from power enterprises [3, 4].
There have been a lot of large-scale data processing frameworks in the scientific research area.
We use hybrid address spaces for an implementation of MapReduce, a programming model for large-scale data processing, and the implementation of a remote memory access (RMA) model.
Data Artisans was founded in 2014 by the team leading the development of Apache Flink, an open source large-scale data processing technology.
