Follow Datanami:
January 23, 2019

Data Science in a Box: Tools Attack Critical Skills Shortage

Instrumental to advanced analytics and machine learning, data scientists must have command of a kitchen sink of tasks:  collecting, preparing and organizing large data sets in a variety of formats; developing and testing algorithms; building and implementing machine learning solutions; conducting data pattern analysis; explaining results to line managers, senior management and customers. Their education and qualifications, it include “a graduate degree in computer science and expertise in mathematics, statistics, computer programming and business knowledge.

Oh heck ya, we can do that.

The kitchen sink list of data science skills and knowledge combined with scarcity has driven stratospheric costs: data scientists out of college are supplicated with offers in the high five figure; for experienced ones, pay reaches into the mid-six figures. A McKinsey report projects a U.S. data scientist shortfall of about 250,000 by 2024.

Meanwhile, getting value out of data is getting harder all the time. Panoply, a smart cloud data warehouse vendor, surveyed attendees at Amazon’s recent re:Invent conference and found that 66 percent said that with the growing number of apps, databases and data sources to contend with, data is too difficult to manage; data needs are outpacing their teams’ abilities to keep up; that disparate apps (Google Analytics, Facebook Ads, etc.) and services don’t work together well enough, and so on.

But technology, like nature, hates a void. New tools for automated machine learning, pre-trained AI models, data prep and other data science tasks comprise a growing software category. Deloitte Insights recently issued a report, “Democratizing Data Scient to Bridge the Talent Gap,” (by David Schatsky, Rameeta Chauhan and Craig Muraskin) painting an optimistic picture that begins by citing a study issued a year ago by Gartner predicting that by 2020 more than 40 percent of data science tasks will be automated. This may well be an achievable goal, considering that data mining company Figure Eight (formerly CrowdFlower) has reported that 80 percent of what data scientists do is tedious, repetitive and ripe for some degree of automation.

Please read the rest of the story at www.enterprisetech.com/2019/01/22/data-science-in-a-box-tools-attack-critical-data-science-shortage

Datanami