It isn’t possible to automate data science and decisions because they are dependent on domain expertise and human judgment. According to Columbia historian Matthew Jones, data science isn’t algorithms although it is dependent on it. Data science can’t be reduced to articulated rules because experience and expertise belong to a particular domain and that data scientists have the understanding and experience to apply the rules on data.
One piece of record isn’t really of primary importance. However, if each data is pulled together, such big data becomes valuable. According to Quentin Hardy, the tech industry will be facing a big problem if old data isn’t saved and people will just rely on real-time data. According to Stephen Sorkin of Splunk, big data software doesn’t produce the causes. It only searches for correlations. Humans are needed to make sense of big data. Lithium Principal Scientist of Analytics Michael Wu said that big data doesn’t mean there’s more information. As the volume of big data increases, less information can be gained.
According to Freddie Mac CIO Rob Lux, humans are still needed to refine big data into useful information. The problem is that there not enough human beings with the right experience and expertise to mine the gold from big data. As such, these data scientists command a huge price tag. According to Tom Davenport, data scientists are worth $300,000 in annual salaries.
IDC is estimating that there’ll be some 4 trillion new data gigabytes next year. IBM is working with Ohio State University to create a data analytics center to create course materials for data science. Data scientists can also be trained through a marketing analytics Master’s program at the University of Maryland. Amazon has recently announced that its cloud-based data warehouse service Redshift will be delivering better performance and scalability than the usual data warehouses. It also announced that S3 storage service is now capable of storing 1.3 trillion objects. GE, on the other hand, is geared towards launching “Industrial Internet Quests” with Kaggle in order to improve hospital experience and reduce flight delays. The company is set to invest $1.5 billion on these endeavors.
