Can the ‘SP Machine’ Straighten Out Big Data?

A form of artificial intelligence variously referred to as “information compression” or the SP theory of intelligence promises to help solve some of the most vexing challenges associated with increasingly fragmented big data.

In a recent article published in the journal IEEE Access, researcher Gerry Wolff of CognitiveResearch.org argues that the human brain in the form of what he calls an “SP machine” could help sift through the mountains of disparate scientific, business, social media and other forms of data that are clogging today’s supercomputers.

“We can save a lot of energy by using probabilities,” Wolff argues. “Instead of doing computations mechanically, we can concentrate our efforts where answers are most likely to be found. We don’t need some special process for gathering information about probabilities because, as a by product of how the SP system works, it creates a statistical model of its environment.”

Part of the big data problem is the way “knowledge” is represented in computers. Wolff asserts computer science in general and artificial intelligence in particular are becoming fragmented.

A prime example is the long list of image formats such as GIF and JPEG. “This jumble of different formalisms and formats for knowledge is a great complication in the processing of big data, especially in processes for the discovery or learning of structures and associations in big data.” Wolff said. The SP system promises to help to simplify things by serving as a “universal framework for the representation and processing of diverse kinds of knowledge.”

What is needed is nothing less than a “Copernican revolution” that makes sense of this jumble of data, the researcher added.

Proponents of this approach also argue that so-called SP machine is capable of “unsupervised learning” to discover the structure of data, pattern recognition and the parsing of natural language. Moreover, “it lends itself to the analysis of streaming data, helping to overcome the problem of velocity in big data,” Wolff said.

The SP machine would operate by what Wolff called “lossless compression of information.” Similar breakthroughs in the compression of video signals in the 1980s helped make digital TV possible. Compression algorithms were used to focus processing on the changing parts of a scene, ignoring those parts of a video image that remained unchanged.

Such an approach could now be applied to big data as a way to make it “smaller” while reducing storage and management requirements, Wolff argued.

Another problem is the sheer size and energy requirements of today’s data crunchers. Wolff maintained that his approach could speed the processing and transmission of data while reducing energy consumption and the size of supercomputers.

Finally, the SP model also promises to help manage errors and the uncertainty of data. “It lends itself to the visualization of knowledge structures and inferential processes,” Wolff said.

He advocates development of a highly parallel version of the SP machine based on the existing SP computer model hosted on a current high-performance computer. This beta version would allow researchers to work out the kinks in the system while creating a new and better version of the SP machine that can help make sense of big data.

The result of this simplification of computing, including software, would be “deeper insights and better solutions,” Wolff said.