Follow Datanami:
January 2, 2019

Industry Speaks: Big Data Prognostications for 2019


“The future isn’t what it used to be” the French poet Paul Valery wrote nine decades ago. The same could be said for the big data trend that began in the mid ’00s. We’re not where many thought we’d be, but the future, in many ways, turned out to be more interesting than many imagined.

Now, as we kick start the New Year, it’s a good time to dust off the old crystal ball. Datanami has opened the floor to industry prognosticators from across the big data, analytics, and IT industries. Let’s hear what they have to say.

According to Ashish Thusoo, the founder and CEO of big data as a service firm Qubole, there’s no question that investments in business intelligence and data analytics solutions will continue to rise in 2019.

“The interesting question,” he writes, “is where will the focus be. I expect to see an uptick in streaming data analytics, as businesses try to leverage real-time information to make smart decisions in areas like customer support, marketing, fraud detection and upselling customers.” Also seeing growth will be ad hoc analytics as the “democratization of data” continues its relentless march.


2018 was “the year of the data catalog,” declared Alation. That trend shows no sign of letting up, as organizations continue with the ongoing struggle to inventory their data assets for the purposes of monetization and regulatory compliance. As 2019 begins in earnest, keep room in your catalog for a particular type of data: behavioral metadata.

“Analysis of this data can be mined to better shine a spotlight on what’s used and what’s useful,” writes Aaron Kalb, the VP of design and strategic initiatives and co-founder of Alation. “This is the same insight that drove Google Search’s ranking prowess two decades ago: the content of a webpage was less predictive of its utility than how often other pages — built by other people — linked to it. As the ML/AI buzz continues to wear thin, we’ll see a strong appetite emerge for this type of impact-driven technology and behavioral metadata among organizations.”

Despite the progress in AI and machine learning, we still won’t have self-driving cars, according to Lexalytics CEO Jeff Catlin.

“Self-driving cars are getting better-0, enough that prototypes are trusted on the roads in California, Singapore and even Western Australia,” Catlin writes. “But while humans have been at fault in the overwhelming majority of accidents involving autonomous vehicles, self-driving cars still have some kinks to iron out. From ‘seeing’ lane markers in snowy conditions to making judgment calls about whether to save a pedestrian or a driver to detecting kangaroos on the road, the technology still hasn’t quite figured out how to handle all of the decision-making required when you’re in traffic.”

(Bradley Blackburn/Shutterstock)

Privacy will emerge as a top priority at the national level according to the folks at Immuta, a Wasington D.C.-area company that develops software aimed at boosting privacy in AI.

“We expect privacy to increase in importance in the new year, highlighting current efforts to create a single, national privacy standard in the U.S.,” Immuta CEO Matthew Carroll tells Datanami. “Given the impact the E.U.’s GDPR has had on how U.S. and global companies operate, this won’t catch all companies off guard, but it will increase the impacts that privacy issues have had – and will continue to have – on businesses’ bottom lines.”

We could even see new data privacy regulations proposed and enacted, foresees Adrian Moir, a senior consultant in product management at Quest Software.

“Whether affected by GDPR or not (most are), companies should be looking to it as a framework, it’s a good starting point for those building out their processes,” Moir writes. “It’s important to have something set-up for how data is kept and used. If we want to continue to have personal information protected, we will need to have more regulation. Next year, I believe we’ll see more regulation proposed and/or put in place, like the Consumer Data Privacy Act recently introduced by Oregon Sen. Ron Wyden.”

(Wright Studio/Shutterstock)

In late 2018, we witnessed a backlash against cloud vendors by open source software vendors. In 2019, tensions between the two parties will continue to simmer, predicts, Karthik Ramasamy, the founder of Streamlio and creator of the open source Heron streaming analytics platform at Twitter.

“The fear has only grown that big cloud providers will undermine open source communities and vendors by launching their own closed cloud services based on open source without contributing back to those communities,” Ramasamy writes. “However, there are signs in these recent moves that big vendors are taking a nuanced approach—in some cases working to co-opt open source to the ecosystem’s detriment while in other cases supporting vibrant open source ecosystems. For instance, the recently released Amazon Managed Streaming for Kafka (Amazon MSK) is likely to have negative repercussions for the Apache Kafka ecosystem even as Amazon’s open source Firecracker aims to establish an open source community and ecosystem around it. This trend will accelerate in 2019 and beyond, and the extent to which these companies act as ‘good citizens’ within open source will bear watching.”

Amazon has been slowly creeping into other ventures, including healthcare, grocery stores, and newspapers. Don’t be surprised if Amazon makes a big acquisition in 2019 that impacts how enterprise software is developed, says Reid Christian of CRV, a venture capital firm.

“In 2019, I believe Amazon will make a big acquisition that will change the enterprise world and enhance Amazon Web Services,” Christian writes. “With storage and compute decisions today more than ever in the hands of developers instead of CIOs, I believe AWS will make >$1B acquisition centered around exceptional DX (developer experience), meaning workflows and UI/UX that are intuitive and consumer like. I expect Amazon will want to have a big enterprise moment in 2019, similar to what Microsoft had by acquiring GitHub in 2018.”

There’s a lot of room for analytics to impact various aspects of everyday business, writes Doug Hillary, a strategic adviser and board member of Fractal Analytics.

“Enterprises will increase the use of Natural Language Processing (NLP) and voice integration with back-end data, analytics and legacy CRM/ERP systems to create more personalized and enhanced customer service for consumers and employees,” he writes.

The push to hybrid and multi-cloud architectures in 2018 will lead to greater cloud interoperability in 2019, according to the folks in IBM Systems.

“Cloud computing has become all but ubiquitous, but running a cloud environment for many enterprises means orchestrating a quagmire of services and hardware that don’t always play well together,” IBM Systems tells Datanami. “With more than 80% of enterprises using five or more different cloud providers, the ability to quickly and seamlessly move data becomes top of mind for any IT department, particularly as AI and other data-intensive workloads become increasingly common. In 2019, expect to see more innovations in storage hardware and software that help companies reign in and better manage their cloud footprint.”

Expect data management and AI development in the cloud to become more automated, writes Atish Gude, chief strategy officer at NetApp.

“A rapidly growing body of AI software and service tools – mostly in the cloud – will make AI development easier and easier,” Gude writes. “This will enable AI applications to deliver high performance and scalability, both on and off premises, and support multiple data access protocols and varied new data formats. Accordingly, the infrastructure supporting AI workloads will be also have to be fast, resilient, and automated. While AI will certainly become the next battleground for infrastructure vendors, most development will start in the cloud.

Tom LaRock, a “head geek” at Solarwinds, has gone out on a limb and already declared that 2019 will be the year of DataOps.

“In today’s increasingly digital world, data cannot be excluded from the agile decision-making process,” LaRock writes. “In fact, we predict that 2019 will be the year that data is recognized as a key business driver. Data culture will become increasingly implemented into tech environments, and organizations will become data-driven and data-first. This shift will also give rise to DataOps as traditional admins start to understand that their days of tuning indexes are ending, one page at a time.”

It’s been a long time coming, but 2019 will finally be the year that AI goes mainstream, according to Zachary Jarvinen, head of technology strategy for AI and analytics at OpenText.

“The long-promised enterprise AI transformation is poised to begin in earnest in 2019,” he writes. “Most enterprises have reached a point of digital maturity, ensuring access to quality data at scale. With mature data sets, AI providers can offer lower cost, easier to use AI tools for specific business use cases.”

The languages you use to build applications in the emerging serverless paradigm may not be the languages you use now, according to Amod Gupta, director of product management for AppDynamics.

“Java and .NET will be overthrown as the de-facto languages for serverless technologies,” Gupta predicts. “We will see more and more enterprises adopt new languages like Node.js and Python for building on new technologies like serverless. So far, Java and .NET ruled the roost in enterprises, but the footprint of the new languages will increase by a lot. Serverless functions, like Lambda functions, have so far been predominantly used in development and pre-production environments, but we’ll see them move to production workloads this year, especially as Node.js and Python catch on in broader adoption.”

Big data means big storage requirements, even for small companies in 2019, according to Douglas Brockett, president of StorageCraft.

“Petabyte-size data management used to be a challenge only large enterprises would face,” Brocket writes. “With data growing ten-fold – according to IDC – the petabyte era will start barreling down on mid-sized organizations too. What used to be an anomaly will start to become the norm for SMBs and mid-size organizations. Mid-sized organizations in particular will find their IT architectures simply can’t scale with their data growth. Unlike large enterprises, they won’t have the skills or budget to cope either. The demand to bring data management, protection and cost-effective scale out storage into a single frictionless environment will rise.”

We’ll finally start to see AI impacting healthcare, writes Gianfranco Lanci, president and COO of Lenovo.

“AI is reducing emergency waiting room times, enabling remote personalized health care delivery and monitoring, offering the availability and accessibility of critical hardware and even freeing up doctors’ time by detecting and diagnosing tumors,” Lanci writes. “These advancements are literally saving lives.”

You’ve heard of AI. But 2019 will see the rise of EI, or ethical intelligence, according to Christian Beedgen, the co-founder and CTO of Sumo Logic.

“Our fascination with the use of computing power to augment human decision-making has likely outgrown even the tremendous advances made in algorithmic approaches, ” Beedgen writes. “In reality, the successful use of AI and related techniques is still limited to areas around image recognition and natural language understanding, where input/output scenarios can be reasonably constructed, and that will not change drastically in 2019.

(Sinart Creative/Shutterstock)

“The idea that any business can ‘turn on AI’ to become successful or more successful is preposterous, no matter how much data is being collected,” he continues. “But the collection of data to support humans and algorithms continues and raises important ethical questions and is something we need to pay close attention to over the next few years. Data is human and therefore is just as messy as humans. Data does not create objectivity. It is well established that data and algorithms perpetuate existing biases and automated decisions are — at best — difficult to explain and justify. Appealing such decisions is even harder when we fall into the trap of thinking data and algorithms combine to create objective truth. With greater decision-making power comes much greater responsibility, and humans will increasingly be held accountable for the impact of decisions their business makes.”

Related Items:

AI Prognostications Plentiful for New Year

What’s Up with 2019? Big Data Predictions

2018: A Big Data Year in Review