Big Data Study Cites Challenges, Predicts Growth

Tapping into big data analytics continues to be appealing for many companies who wish to mine their data for “actionable insights,” or information that can assist decision makers with bringing increased value to their businesses. Though the big data industry abounds with software platforms and products, developers and data professionals, and many zealous devotees, there are still concerns and obstacles involved in implementing a data strategy in a business, for professional data workers and management alike.

Data analytics platform provider Unsupervised has published a study called “Big Data Fears and Predictions 2022.”

Unsupervised collected 226 responses from big data-experienced business leaders and 233 responses from U.S. data professionals (including data scientists, data engineers, data analysts, data developers, and other relevant positions), pre-screened by people research firm Prolific.

The study’s key findings include how 64% of business leaders anticipate big data budget increases this year, how 48% of companies plan to hire data analysts, and how 12 hours per week on average is spent by business leaders working directly with big data.

Big Data Anxiety and Frustration

Source: Unsupervised

The first part of the study focused on asking business leaders what intimidated them the most about big data and which challenges they have encountered in particular. For 32%, data analysis was most intimidating, followed by processing data (21%), implementation of findings (20%), and collection and cleaning both coming in at 13%.

The three biggest challenges cited by business leaders included constantly changing data (44%), volume of data (43%), and dealing with inaccuracies in data (40%). Of the leaders that cited data analysis as their biggest challenge, 57% said they planned to hire a data analyst this year.

Prospective Big Data Plans and Budgets

The second part of the study focused on asking business leaders how their companies plan to use big data in 2022. Identifying opportunities for growth was the most common response at 74%, followed by shaping the customer experience (69%), and creating operational efficiencies (61%).

The study then asked about planned strategies for resolving the largest big data issues the companies face, and 51% answered that they planned to train existing employees to support the data program. Other strategies included spending more for big data software (40%) and allocating a larger budget for their big data strategy (37%).

Business leaders reported allocating 30% of their budgets on average to big data, and as mentioned before, 64% anticipate increased spending. Part of that spending could be set aside for hiring, as the 48% of the leaders polled said they were looking to hire data analysts, followed by those wishing to hire data scientists (44%), and those looking for database managers (36%).

Data Professionals Weigh In

Source: Unsupervised

Now for the insights from the 223 data professionals who were surveyed for their own fears, frustrations, and predictions. Unsupervised learned that even though the average salaries of data analysts and data scientists are $70K and $100K respectively, only 35% would describe their current job as paying well. Of the top ten words to describe current data jobs 40% of respondents used “motivating” and “creative,” while 40% also called their job “repetitive” and 39% “stressful.” “Fun” came in at only 32%.

Perhaps revealing why some of the descriptive words for data jobs were less than encouraging, challenges cited by the data pros included several of those also listed by business leaders, including volume of data (38%), inaccuracies in data (33%), and constantly changing data (31%). Interestingly, some of the biggest problems specified by data pros, but not management, were repetitive tasks (48%), short deadlines (44%), incomplete data (33%), and changing project scopes (30%). Also of note, one out of three data pros said they were working more than 40 hours a week, which could explain why only 34% described their work as “rewarding.”

Of the data professionals surveyed, 66% predicted there will likely be increased dependence on software for data functions, while others expected increased data security investments (57%) and outsourcing (54%).

Big Data Demand Will Persist

As for the future of big data in the business world, the study predicts that its impacts will continue to expand. The value of data analytics insights for businesses will continue driving demand for big data professionals and products. One respondent, a 40-year-old data developer, said, “Companies that gather and use data will outperform companies that don’t. [Not doing so] will be a deal breaker.”

Read the full study at this link.

Related Items:

Data Science and AI Predictions for 2022

The Upcoming Year in Big Data: A 2022 Preview

New Data Gravity Study Shows the Massive Scale of Big Data in 2024