The Secret to Data and Analytics Success Is…People
One of the most astonishing facts culled from the data and analytics field is the persistently high rate of failure. Despite the billions of dollars and millions of hours invested, the majority of data analytics projects simply do not succeed. There are many reasons for this, of course. Sometimes the technology is not up to par, and the data is almost always dirty. But arguably, the biggest factor is a lack of investment in people.
There’s a growing realization that people are the key to data and analytics success. Of course, every organization that fancies itself to be “data-driven” would like to have more and better data scientists. It goes without saying they’re investing in self-service tools so mere data analysts can hobnob with the data science elite. And let’s not get started on the dearth of well-trained data engineers to build the pipelines that move the all-important resource.
Up and down the organizational structure, people are the crucial points upon which organizations pivot from success to failure, no matter what the endeavor. And when it comes to data and analytics projects in most organizations, there’s a crying need to drive more and better awareness of data and analytics skills among the rank and file.
Qlik recently commissioned Accenture to produce a report on the state of data literacy. The report paints a rather disheartening picture of the adoption of data and tech. Chief data officers may think they’re boldly moving quickly towards their goal of being data-driven. But in reality, the rapid pace of change is inflicting a serious case of fear, uncertainty, and doubt into the minds of the working men and women.
“We studied the human impact of data literacy across over 9,000 respondents, and we found that people are pretty much overwhelmed,” says Jordan Morrow, who heads up data literacy for Qlik.
The average workers loses a week’s worth of productivity work due to their inability to successfully navigate today’s data and analytics world. On a dollar basis, that translates into $100 billion in costs and lost revenues across the world, according to Qlik.
“People are overwhelmed all the time being told ‘We have new software. We have new technology. We’re investing in data. We’re doing all this,’” Morrow tells Datanami. “You throw that in someone’s face and that overwhelming feeling is real. You see it.”
Morrow runs ultra-marathons when he’s not traveling the world spreading the gospel of data literacy for Qlik, and he knows what physical and mental exhaustion look like. That “deer in the headlights” look is all too common when it comes to an average worker being asked to use data and analytics in ways that they’re not prepared for.
“It can even push them to a fight or flight response where they say ‘This is another thing they’re putting in front me. I’m done. I’m not going to use this,’” Morrow says. “All these advances in technology creates more information, more data. But it also creates what could be toxic cultures at companies because they feel like we have to invest in all this stuff, but the workforce is not ready for it. That can cause that overwhelming feeling, that dire feeling, that human impact that we’re talking about.”
Focus on the People
Aaron Kalb, the co-founder and CDO of Alation, sees a similar phenomenon working its way through his customers. The majority of organizations are making investments to become more data driven, but fewer than half of them are succeeding, he says. “I don’t see 2020 as a year in which that trend flips around and suddenly everybody is data driven, but I think the struggle will continue,” Kalb says.
What would make a real difference, he says, is investing more time and money in training people rather than acquiring more technology.
“My real hope is that boards are going to the human side, investing in training and technology that’s optimized for the people, rather than the data,” Kalb Datanami. “What we see is the reason that a lot of these initiatives are failing is because the people and processes, a lack of data literacy, a lack of data access, a lack of data understanding that kind of gets in the way.”
There’s no denying that algorithms can make decisions faster than humans. And in many cases, the algorithms can make decisions that are more accurate than human decisions. But the inscrutability of deep learning algorithmic techniques, coupled with the growing call to regulate AI to mandate ethical uses, are putting the focus back on empowering the human in the loop.
“One way of being a little more human-centric is to think about human values and human fairness,” Kalb says. “Maybe we get algorithms that are slightly less accurate [but allow] us to see what are the feature and the weights, and we can validate that they are working in accordance with what we care about. The other side of human centricity is to actually have humans in the loop, where the computer is leveraging its power to offer greater speed and scale, maybe do the easy cases automatically and pop out the trickier cases to the humans.”
As a data catalog provider, Alation functions as a portal that gives users access to the world of data. Many of its customers are sophisticated data users, including data analysts and data scientists. But increasingly, the lines are beginning to blur between the sophisticated data haves and the less sophisticated data have-nots.
“Every catalog product should be driving real outcomes so folks who don’t have data in their title can get excited about AI,” Kalb says. It’s about “not only appealing to technical users, but being more appealing to a wider array of non-technical people who just want to be using data to optimize their decisions.”
Democratizing access to data will bring benefits across the organization, Kalb says. “It used to be you need a PhD in computer science to do information retrieval,” he says. “But now, thanks to Google, we do it 10 times a day without even thinking about it. Everybody can be a data user, even if you’re not a ‘data person.’”
Data to the People
Both Alation and Qlik have been strong proponents of data literacy and the need to educate users to be smarter about how they consume data and analytics. Although the vendors work at different levels of the data stack, their messages are remarkably similar.
Qlik has taken its commitment to data literacy one step further by launching a data literacy service. Unveiled last week, the new service involves having one of Morrow’s data literacy disciples working closely with an organization to develop greater familiarity and comfort with data and analytics, through both on-site and virtual engagements.
Organizations that complete the program realize compelling benefits that go beyond just feeling comfortable with a new analytics or visualization tool, according to Morrow.
“It’s that empowerment of people, not just feeling like they need to become a data scientist, but just make them confident in data,” Morrow says. “That way, no matter what’s thrown at them, they’re set. They’re comfortable. They get it. ‘Oh, that’s new technology. I know the specialists use it this way. But I can use it this way to make smarter decisions.’”
When a person is comfortable with data, the odds that they get flustered and stick their heads in the sand decrease, Morrow says. They’re more confident in their ability to bridge the gap between new analytic tools and legacy systems.
“If you’re not data literate, and you say ‘Well that‘s the new system. I’m using the old one,’ that deer in the headlight look is going to be more prevalent than it should be,” he says.
The Importance of Culture
Part of Morrow’s data literacy strategy is understanding the role of technology in building a strong company culture. Instead of buying new data analytics technology and forcing it upon an organization, successful data leaders will understand technological decisions flow from business strategies, not the other way around.
“Historically what companies have done is one person in the company or a group get sold on a new technology, then they try to force their strategy into that technology,” he says. “I say you got to reverse that. Start with the strategy, then the technology will fall into place.”
Unfortunately, many companies do not have a compelling strategy, which leads to haphazard technological implementations. That’s a natural human reaction to the stresses that executive are feeling in our hyper-competitive data age, but it leads to bad downstream decision making and, ultimately, that feeling of powerlessness on the part of the average Joe, Morrow says.
“At a leadership level, there is stress that they need to keep up with the advancement with all this data and technology around it,” Morrow says. “The stress is being bombarded again and again and again with something new, or more training, or a new way you have to do your job.”
This phenomenon played itself out a few years ago with the hype around Hadoop. “You don’t even hear about Hadoop anymore, but so many people were jumping onto that bandwagon because it was a big theme,” he says. “In fact, the company I worked for did. It didn’t work as well as they thought. They just thought they needed it.”
Having a solid business strategy is the key to choosing technologies. And having a healthy business culture is a big part of having a solid business strategy, he says.
“The number one impediment to data and analytic success has nothing to do with sourcing of data, the consumption of data, or technology. It’s culture,” Morrow says. “So when they build these strategies, they need to have pieces to help the culture to absorb it. Not going in and saying, we’re going to change the culture, but let’s evolve culture to use data more.”