Improving Big Data Performance by Improving the Structure of Locality of Reference

Abstract:

Big data has become the norm of data accumulation in various fields of science, technology, education, health, security and others. Obviously, big data phenomenon has been largely associated with the decrease in the cost of hardware required to store and process the data. Figure (1) shows how the cost of a gigabyte has changed over the last 30 years. More importantly, big data has become a natural growth of various technologies such as social networks. Facebook for example hosts more than 2 billion subscribers using all forms of data such as text, images and videos (Ousterhout, et al. 2010). Such networks increase the size of data by the second rather than by the day. Similarly, security needs have demanded the use of video cameras almost everywhere in big cities. According to Chicago Tribune (February 26, 2019), the city of Chicago hosts more than 30,000 cameras which stream trillions of bytes worth of images continuously.

nsdlogo2016