CLASSIFICATION ALGORITHMS FOR BIG DATA ANALYSIS, A MAP REDUCE APPROACH
- 1Dept. of Electrical Engineering, Pontifical Catholic University of Rio de Janeiro, Brazil
- 2Dept. of Computer and Systems, Rio de Janeiro State University, Brazil
- 3Dept. of Technology of Computers and Communications, University of Extremadura, Spain
- 4Dept. of Electronics, University of Pavia, Italy
Keywords: Big Data, MapReduce Framework, Hadoop, Classification Algorithms, Cloud Computing
Abstract. Since many years ago, the scientific community is concerned about how to increase the accuracy of different classification methods, and major achievements have been made so far. Besides this issue, the increasing amount of data that is being generated every day by remote sensors raises more challenges to be overcome. In this work, a tool within the scope of InterIMAGE Cloud Platform (ICP), which is an open-source, distributed framework for automatic image interpretation, is presented. The tool, named ICP: Data Mining Package, is able to perform supervised classification procedures on huge amounts of data, usually referred as big data, on a distributed infrastructure using Hadoop MapReduce. The tool has four classification algorithms implemented, taken from WEKA’s machine learning library, namely: Decision Trees, Naïve Bayes, Random Forest and Support Vector Machines (SVM). The results of an experimental analysis using a SVM classifier on data sets of different sizes for different cluster configurations demonstrates the potential of the tool, as well as aspects that affect its performance.