The Bigger the Data, the Smaller the World Gets

It seems like a contradiction: that big data would somehow shrink our world. But as data scientist Kirk Borne recently explained, the more we know about our world, the more the unknown parts fall away, and the closer everyone becomes.

Borne, a multidisciplinary data scientist and an astrophysicist at GeorgeMasonUniversity, shared some astounding statistics and insights into big data during a talk at the second annual TEDxGeorgeMasonU conference.

Borne cited a study by researchers at the University of California, Berkeley, that found the amount of data generated from the dawn of time through 2002 was about 5 exabytes. To put that into perspective, that’s about as much data as exists on 5 billion DVDs, or enough to fill a typical stadium with DVDs.

As the big data phenomenon took off, the numbers have gotten downright mind numbing. In 2011, it took about two days for the world to create the same 5 exabytes of data that it took human eons to generate. In 2013, it took the world just 10 minutes to create 5 exabytes.

“The really scary thing is this number itself is accelerating. The prediction is this will be the amount we create every few seconds a few years from now,” Borne said. “So if you don’t think big data is a challenge or a problem, then I’ve failed you.”

While administrators are struggling to store and manage the gobs of bytes, scientists are celebrating the huge potential that big data holds for accelerating the pace of human technological advancement and discovery.

“The promising news is that the potential for big discovery and insights is enormous now,” Borne said. “We have the ability to discover treatments for disease, discover new drugs, just by exploring data sets, believe it or not. Once you have the full genomic sequence of a human, you can now discover from that person what the likelihood they’ll be susceptible to different diseases in life, and how treatable they’ll be using certain drug therapies.”

That is creating a huge demand for data scientists with the skills to navigate big data sets. When Borne got into this business, he said there were 100 applicants for every job opening. The numbers have flipped, and now there are 100 job openings for every qualified data scientist looking for a job, he said. “For young people, it’s the best job security on the planet right now. Job demand is skyrocketing,” he said.

Big data isn’t just impacting the classical sciences, but is having a disruptive influence on many aspects of people’s lives. Retailers like Amazon and Netflix use the power of big data association to make recommendations to customers. Social media websites use the same technology to connect people with similar traits or interest.

Big data is also impacting the theory that everybody on the planet is connected by an average of six degrees of separation. In fact, the average number of degrees of separation has been shrinking due to big data, and was down to 5.28 by 2008 and 4.74 in 2011, according to Borne’s presentation.

Borne also commented on a competing theory, called “Six Degrees of Kevin Bacon,” in which people measure how far they are from the famous actor/musician. Borne is friends with a George Mason professor who wrote a play in which Bacon starred, so he in fact is separated by the actor by two degrees.

“Right now our world is shrinking,” Borne concluded. “The places you visited, websites you go to, things that you purchase, the friends that you like–all these things are narrowing and shrinking your connection to other people in the world. Businesses are using this to make retail sales. Governments are using this to discover terrorists. Scientists are doing this to discover new drugs and new properties about things in our universe. Our world is shrinking through the advent and application of this big data. So face it or not, big data is shrinking our world.”

Related Items:

Zettaset Puts Hadoop on Lockdown 

Data Mining for Human Cognition 

Interplay between Research, Big Data & HPC