The contemporary age of technology has enabled the world to complete major industrial and household tasks in an astonishingly less amount of time and with exquisite efficiency, as compared to the time only four to five decades back. With a growing population, demand for growth in utilities has surfaced too. The amount of unstructured data generated from various public and private sector departments that is so huge that obtaining useful insights through basic queries is impossible and hence, called big data. In fact, the most sophisticated databases and data warehouses are unable to process big data in a short time with perfection. Besides that, DBs and DWHs are only capable of processing structured real-time or archived data.
The benefits of big data can be explained in two major categories. First, it is used to reduce cost and time by enabling software to become artificially intelligent. This is achieved either by machine learning, where a human train the software with data to become intelligent, or deep learning, where software is made capable of training itself with the help of sophisticated algorithms. Making use of software for performing the tasks that were previously carried out by humans can significantly increase profits. A couple of decades back, tasks that took days or months are now completed in an efficient manner with few clicks on computers (Davenport, Barth, & Bean, 2012).
Second, market trends can be predicted by inducing intelligence in software. AI entitles the software to extract reasonable information from data, and process the information to generate more information that can be used for a range of purposes. By analyzing these trends, management can make technical decisions (Bello-Orgaz, Jung, & Camacho, 2016).
Almost every company and department with millions of customers, users, and clients generate big data. For instance, Google, a popular search engine, generates 24000 terabytes of data alone every day (Davenport et al., 2012). In total, 1000,000 terabytes of data are generated every day and are expected to increase seven times in less than a decade (Bello-Orgaz et al., 2016). The unstructured data of telecommunication companies, banks, streaming, and social media websites can be processed to make predictions and, as a result, evaluate the choices and interests of users.
Public and private sector departments including, but not limited to, healthcare, automobile insurance, utility bills, surveillance cameras, crime analysis, and television broadcasting produce raw data which is used for assessing useful insights and performance enhancements. In the past, the spread of Ebola, H1N1, and other viruses was contained by performing big data analytics (Raghupathi et al., 2014). Cameras on highways and streets produce data that can be used to determine and predict traffic influx.
Although big data provides an abundance of opportunities to the world and technology to automate tasks and predict what matters, it poses some challenges and threats too. Recently, the news about the manipulation of the mindset of American voters for presidential elections by using the data from a popular social network has stirred up a debate over the ethical use of big data (Bello-Orgaz et al., 2016). There is a large number of incidents where privacies and liberties of people were compromised in the name of research and to seek money.
Another challenge that big data has posed to technologists is the efficiency of using the data. Some incidents have proved that the predictions can go extremely wrong if the data is not effectively used. In the United Kingdom, a controversy surfaced where a media organization claimed that the data from cameras is not producing the desired results. Finally, the dearth of data scientists, that are capable of processing big data, is creating a huge lag between the generation and processing of data. This lag, at times, affects the efficiency of algorithms (Davenport et al., 2012).
Great innovations bring great responsibilities. Big data can bring about revolutionary development by improving lifestyles. However, unethical use of it can lead to immensely dire consequences.
References
Davenport, T. H., Barth, P., & Bean, R. (2012). How big data is different. MIT Sloan Management Review, 54(1), 43.
Bello-Orgaz, G., Jung, J. J., & Camacho, D. (2016). Social big data: Recent achievements and new challenges. Information Fusion, 28, 45-59.
Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health information science and systems, 2(1), 3.
