Tool Providers Up Their Data Integration Games

(3dreams/Shutterstock)

Data silos and poor data infrastructure integration are thorns in the side of many digital transformation efforts. Organizations are collecting more data from more sources than ever before, yet making sure that data is accessible in a central location to unlock its value can still be an elusive proposition.

Data integration tools can help businesses transform data siloed across multiple departments in multiple formats into a single source of insight. According to Gartner, the data integration tools market is seeing renewed momentum driven by requirements for multicloud and hybrid data integration and data fabric architecture. The company’s 2022 Magic Quadrant for Data Integration Tools explains that as more data leaders recognize the significance of data ingestion infrastructure as part of their overall data management strategy, they are increasingly seeking a comprehensive range of improved data integration capabilities to modernize their data and application infrastructures.

This week ushered in several new data integration capabilities from several companies highlighted in the Magic Quadrant, including Fivetran, Talend, and Informatica.

Fivetran announced three new offerings extending its Data Movement Platform: Fivetran High-Volume Agent Connectors, Fivetran Private Deployment, and support for AWS.

The new HVA connectors use log-based change data capture to read directly from the source system’s logs with an agent that minimizes replication latency while supporting large volumes of data, the company said in a statement.

For high-security organizations managing sensitive data related to defense, healthcare, or finance, data integration options have been limited due to security concerns. Fivetran’s new Private Deployment option allows for hosting sensitive data on-prem or in a virtual private cloud.

AWS GovCloud is an isolated AWS region designed to host sensitive data and regulated workloads in the cloud, and Fivetran now offers deployment support for this region. Users can deploy Fivetran within AWS GovCloud, as well as use AWS PrivateLink to create virtual private cloud endpoints, ensuring sensitive data is not vulnerable to the public internet.

“Fivetran can now meet any architectural and deployment requirements for secure database replication, whether on-premises or in the cloud,” said George Fraser, CEO of Fivetran. “Companies that have been unable to actualize the value of their siloed data, or have been over-investing in fragile legacy infrastructure, are now able to move their valuable data without compromising on security, efficiency or system performance.”

Another data integration specialist, Talend, has updated its Talend Data Fabric platform with a Winter ‘23 release. The company touts this release’s advanced capabilities and enhancements for simplifying and automating cloud migrations and data management, expanding data connectivity, and improving data visibility, quality, control, and access.

Talend has updated Stitch, its fully managed cloud ETL service, to include new role-based access control, new pipeline monitoring capabilities, and a History mode for Snowflake to track data changes for historical analysis or compliance auditing.

The company is also bringing AI-powered automation into the mix through a feature called Smart Services that is designed to simplify task scheduling and orchestration of cloud jobs, allowing users to pause and resume tasks for an operational efficiency boost. The platform also boasts improved connectivity, adding to its roster of 1,000+ connections and components with certified connectors for SAP S/4HANA and SAP Business Warehouse on HANA. There is also support for cloud databases like Amazon Keyspaces, Azure SQL Database, Google Bigtable, and Neo4j Aura Cloud, as well as an enhanced Apache Kafka connector for streaming use cases.

Talend also added new data observability components that it says will enable data teams to check the validity and usage of data types across datasets, apply contextual data quality rules, and monitor how data quality evolves using an enhanced Talend Trust Score.

“Winter ’23 is based on direct customer feedback and continuing to support those on the front lines charged with the increasingly daunting task of extracting maximum, ongoing value from corporate data,” said Jason Penkethman, chief product officer at Talend. “As well as continuing to drive operational efficiency and expedite data modernization efforts and returns, with Winter ’23, we are also empowering our customers to continuously monitor data throughout its lifecycle and understand and impact how it evolves and moves to fuel positive business outcomes.”

Finally, Informatica joined the new release party with the launch of a free cloud data loading, integration, and ETL/ELT service called Informatica Cloud Data Integration (CDI) with two tiers: Free and PayGo.

CDI-Free is a free service that allows users to process up to 20 million rows for ELT or reach 10 processing hours for ETL per month, whichever comes first. CDI-PayGo gives users the option to pay only for what they use with a credit card and offers the same capabilities as CDI-Free without the limit on processing rows or hours of usage. This tier also comes with essential customer support and SOC2 compliance.

Informatica says this release complements its Informatica Data Loader launched last May, and these free data loading and integration solutions collectively support all major data warehouse and lake solutions, including Amazon Redshift, Azure Synapse, Databricks Delta Lake, Google BigQuery, and Snowflake.

“We are redefining the data integration market by making it free, easy to use, and accessible to everyone. Organizations face the challenge of ingesting huge volumes of data from disparate sources and then making sense of that information,” noted Jitesh Ghai, chief product officer at Informatica. “There is a clear need for no setup and no code SaaS data integration tools that are free and pay-as-you-go to quickly get started serving both business-focused data engineers and non-technical business users and analysts. By giving business and non-technical users access to simple, cost-optimized data integration solutions, organizations can bring the power of data to the masses.”

Related Items:

Data Silos Are Here to Stay. Now What?

Overcoming Data Integration Challenges to Unleash Massive Potential: Knowledge Graphs 101

Hands-Off: Manual Data Integration Tasks Plummeting, Gartner Says