Big data usually means the way data is processed and analysed, rather than large numbers of entries or megabyte volumes (which often come with it, though). This article explores key differences and challenges facing CEOs.
Simple questions – simple answers
Cases where it’s easy to define how a particular answer can be obtained from data are not usually regarded as handling big data even if the number of entries to be processed is large.
If the CEO needs to find out which of his shops has the highest sales volume and which has the lowest, sales data can be imported into an Excel spreadsheet (assuming that raw data is available in good quality) then sorted according to the appropriate column, and the answer will pop up.
When the question gets more complex…
Situations are increasingly arising where there is no simple way of finding a link between the question and available data. For example, if a bank wishes to offer a new service to its customers, but the new service doesn’t directly replace an existing one, the link isn’t obvious. As well as having to define a need and to process data technically, we should think of how to come to a result. In such cases we sometimes have to apply industry expertise (e.g. which services the customer might buy together), sometimes the link is counter intuitive, and we need to look for the true answer in the data, using mathematical, statistical and other modern data processing methods.
Anyway this process begins with setting an objective – what is the business problem we want to solve? For example, a company has some infrastructure and wants to invest in the part of infrastructure that yields the highest return on investment; or a company wants to open a new shop and needs a location with the best growth potential. These are examples of data analysis goals with no obvious way of finding the right answer.
This article doesn’t aim to go into mathematical, statistical or artificial intelligence methods that are applied to data in order to find various links – nowadays these tasks are performed by highly skilled professionals and are less accessible to the wider community than working with an Excel spreadsheet. The CEO should remember that this process is experimental and without a guaranteed result. It might turn out during a research that the available data isn’t sufficient to answer the question raised in the objective of that research.
When analysing data we sometimes have to conclude that the quality of data isn’t appropriate for making a decision. In such cases we often have to transform the processes that result in data accumulation or the systems where data is accumulated. This might mean that data won’t be immediately available and it’s possible that analysis cannot be resumed for many months or even several years.
Big data analytics should be seen as investment. Investing in technical aspects such as IT systems and services, improving data quality, making organisational changes, setting up new roles and transforming the corporate culture in most cases don’t pay off immediately but rather in the long run, and this exercise involves a certain degree of risk, yet the return can be substantial.
Despite the difficulties, many companies have seen big data analytics bring significant benefits.