Fivetran Survey Finds 86% of Organizations Struggle to Fully Trust AI

(Peshkova/Shutterstock)

Fivetran has released survey results that show 86% of organizations struggle to fully trust AI to make all business decisions without human intervention, and 90% reported their organizations continue to rely on manual data processes.

The survey’s accompanying report, “Achieving AI: A Study of AI Opportunities and Obstacles,” describes the challenges companies face in the current AI landscape. The report examines how even though 87% of companies recognize AI as the future of business and plan to increase their investment in it, a lack of trust in machine-led decision making is a significant barrier driven by technical issues and lack of education. Only 14% consider their organizations to be “advanced” in AI maturity.

Nearly all surveyed organizations are collecting and using data from operational systems, but data difficulties persist. The research shows that technical data pipelines are a major pain point, with 73% reporting that extracting, loading, and transforming data from different sources into different warehouses is a significant challenge. Also, 71% of those surveyed said they struggle to access all of the data needed to run AI programs, workloads, and models.

Source: Fivetran

This translates to diminished confidence for 73% of respondents when it comes to translating data insights into practical advice for decision makers, which they said forces them to rely on human-led decisions 71% of the time.

The research also shows that data scientists are spending more time working with data than building AI models to improve business outcomes with forecasting and decision making insights. When asked the proportion of time they spend preparing data versus building AI models, data scientists indicated an average of 70% of their time goes into data prep, and 87% disclosed that they felt underutilized in their organization because of this.

Organizations are also contending with data governance issues, with 64% of surveyed U.S. organizations admitting there was room for vast improvement regarding their adherence to data governance roles, policies, and standards to ensure data is being used effectively, securely, and in accordance with government regulations.

Fivetran sees automation of data and AI pipelines as a solution to these issues. “With greater automation, organizations can achieve greater scale and cost-efficiencies while saving time. More importantly, more automation allows data scientists to focus on solving complex problems that matter to the business rather than keeping data pipelines working,” Fivetran’s Brenner Heintz wrote in a blog post.

This graph shows how companies are planning to invest in AI in the coming years. Click to enlarge. Source: Fivetran

Educating business stakeholders to increase trust in AI may also be a solution: “Stakeholders and business users must be made aware of the processes behind AI to fully understand how these decisions are made. But it’s also important that human involvement is focused on the right areas — such as improving data quality and the performance of AI models, which will lead to greater trust,” said Heintz.

Fivetran notes that its automated data pipelines adapt to schema changes, allowing users to ingest multiple data sources into a centralized cloud-based data warehouse or data lake for data transformation in a completely automated way, leading to considerable time savings. Fivetran also says its consumption-based pricing model allows organizations to manage costs by replicating only needed data. Finally, the company says data scientists will spend less time on manual tasks, allowing them to focus on building AI models and launching more data and AI initiatives.

“This study highlights significant gaps in efficient data movement and access across organizations. A successful AI program depends on a solid data foundation, starting with a cloud data warehouse or lake as its base,” said George Fraser, CEO of Fivetran. “Analytic teams that utilize a modern data stack can more readily extend the value of their data and maximize their investments in AI and data science.”

The online survey of 550 senior IT and data science professionals across the U.S., U.K., Ireland, France, and Germany was conducted by Vanson Bourne. Find a copy of the report here.

Related Items:

Bad Data Pipelines Costing Companies Big, Fivetran Finds

In Search of the Modern Data Stack

Monte Carlo Survey Finds Data Engineers Spend 40% of Their Workday on Bad Data