Wolfram: Software the Hard Part

The development of algorithms that enable non-data scientists and executives the ability to access big data will be the over-arching trend over the next 5 years says luminary Stephen Wolfram in a recent interview.

“It’s going to become possible for executives and others to just ask natural language questions and expect that those questions can be answered using computation and the data that the their organization has accumulated,” suggested the prolific innovator who once predicted that even programming will one day be done in natural language.

The challenge, says Wolfram, will chiefly be on the side of the software and algorithmic infrastructure, and developing the proper predictive systems that take advantage of the data.

“I don’t think [hardware] is going to be that difficult,” explains Wolfram. “The amounts of data that even the largest enterprises have these days, it’s not outrageously big relative to hardware that becomes available.” The real issue, says Wolfram, is the computation that gets done on top of that hardware.

“It’s not going to be just a bunch of SQL databases and people writing SQL queries and things,” says Wolfram. “It’s something where you need more sophisticated tools, more layers of algorithmic computation to be able to take advantage of what’s happening with data.”

No stranger to complex systems, Wolfram says that he finds it shocking how primitive data science is. He notes that Wolfram Research has a team that consists of a lot of ex-experimental particle physicists who have noted that the rate at which we get data into the Wolfram web analytics systems is comparable to the rate that their particle detectors used to get when they were doing particle physics. The startling difference, says Wolfram, is the predictability of the subjects being observed

“When you look at the curves when they were doing particle physics, they’d be really jagged curves,” explains Wolfram. “You look at the data you get now in web analytics, you get perfectly smooth curves. It’s so strange because one would think stuff involving people would be much more complicated – but it isn’t true. Yet can we explain the curves we get from what people do? Now we can’t. That area of science is really not well developed.”

Wolfram laments that there seems to be slow progress in trying to understand humans as simple computational elements, how they interact, and what their aggregate behavior is.

The technology visionary advises that this shortcoming is an opportunity for those prepared to seize it. He explains that the rise of the data scientist will provide potential for innovation and career advancement for those moving into the field.

“Don’t work on the plumbing stuff, because that’s going to get automated – work on the stuff that is going to take humans, and ingenuity and creativity to figure out,” advises Wolfram.

Related Items:

Hellerstein: Humans are the Bottleneck 

Rometty: The Third Era and How to Win the Future 

Chaotic Nihilists and Semantic Idealists