Confluent Moves to Boost Kafka Reliability
Apache Kafka has emerged as a leading platform for the management of data transaction streaming. Hence, users are placing a premium on maintaining Kafka uptime by anticipating and resolving issues before Kafka clusters go down, with the resulting loss of data.
Confluent Inc., the company behind Apache Kafka continues to expand its offerings with a monitoring tool that seeks to reduce the risk of Kafka downtime and data loss by identifying and resolving issues before an instance is lost. It is also promoted as a way of preserving Kafka instances without large infrastructure investments.
The “proactive support” platform dubbed Reliable is part of Confluent’s Project Metamorphosis launched earlier this year as a next-generation event streaming platform running in the Confluent Cloud. The initiative is designed to help users scale Kafka deployments while Reliable aims to keep the event streamer up and running.
Confluent said Tuesday (Nov. 10) its Kafka support platform provides real-time analysis of cluster metadata, alerting operators of potential issues before a crash. The monitoring tool also provides performance metrics that would allow support staff to trace the root cause of a Kafka performance issues.
The company said the impetus for the Kafka platform was about 1,000 support tickets that helped it gain insights into detecting performance issues. “We proactively tell you about issues we find with [customers’ deployments] based on experience running literally thousands of clusters,” said Tim Bergland, Confluent’s senior director of developer advocacy.
Hence, the company developed a framework by which cluster metadata is fed through its collection of algorithms to gauge performance and anticipate potential problems. Real-time “health metrics” are promoted as helping support engineers identify root causes faster, cutting the time to resolve support issues by as much as 25 percent, the company claimed.
Among the most common issues affecting the scaling of Kafka’s real-time performance and the threat of data loss are network and storage bottlenecks. Overloaded networks or storage media often bring down Kafka clusters, taking with them business events and other critical data.
The managed service also is designed to help IT administrators reduce the amount of time and resources devoted to operating the relatively complex, distributed Kafka environment.
In launching Project Metamorphosis this past spring, Confluent said it planned a series of monthly product and feature upgrades through the end of the year as it fleshes out its real-time streaming platform. The managed support service is the latest in that series of offerings.
Customers running Confluent 6.0 can opt in to use the initial release of the managed Kafka service. Alerts can be sent via integration with the Confluence control center. Alternatively, users can set up alerts via email or Slack.
January 25, 2022
- TripleBlind Expands Board of Directors
- Incorta Announces Zero Gravity 2022, Cloud Data Pipeline Event
- Dremio Doubles Valuation to $2B With $160M to Reinvent SQL for Data Lakes
- BigID Announces Splunk Ventures Investment
- Query.AI Launches Federated Search for Splunk
- Neo4j Reviews Its 2021 Momentum
- Canonical Releases Latest Version of Its No Cost MLOps Platform, Charmed Kubeflow
January 24, 2022
- SAS and VCU Massey Cancer Center to Use Health Analytics to Research Cancer Disparities
- Pure Storage Partners with Meta on AI Research SuperCluster
- Meta Introduces New AI Research SuperCluster
- Ataccama Announces Record 2022 First Quarter
- University of Phoenix Launches New Bachelor’s in Data Science
- HPE Selected by German Soccer Club Werder Bremen to Deploy Data Intelligence Platform
- UC San Diego’s Jupyterhub Platform Aids Students with Data-Intensive Computing Needs
- Silicon Labs Brings AI and ML to the Edge with Matter-Ready Platform
January 21, 2022
- Deci Launches SuperGradients, an Open-Source Deep Learning Training Library for CV Models
- Cloudian Partners with WEKA to Deliver Exabyte-Scalable Storage for AI and ML
January 20, 2022
- Ascend.io Announces First Annual Data Automation Summit
- Lockheed Martin Space Selects ESTECO’s VOLTA for Its Total System Model Framework
- Sisense Releases New Study: ‘The Business Intelligence Landscape’
Most Read Features
- All Eyes on Snowflake and Databricks in 2022
- Data Mesh Vs. Data Fabric: Understanding the Differences
- Data Science and AI Predictions for 2022
- Big Data File Formats Demystified
- 2022 Big Data Predictions from the Cloud
- The Future of the Metaverse + AI and Data Looks Bright
- Is Quantum Computing the Future of AI?
- 10 NLP Predictions for 2022
- The Upcoming Year in Big Data: A 2022 Preview
- Bank Replaces Hundreds of Spark Streaming Nodes with Kinetica
- More Features…
Most Read News In Brief
- Mathematica Helps Crack Zodiac Killer’s Code
- Alteryx to Acquire Data Wrangler Trifacta for $400 Million
- Data Prep Still Dominates Data Scientists’ Time, Survey Finds
- Big Growth Forecasted for Big Data
- Google Cloud Attacks Supply Chain Crisis with Digital Twin
- Databricks SQL Now GA, Bringing Traditional BI to the Lakehouse
- ETL Tool Apache Hop Graduates Incubator
- Narrowing the AI-BI Gap with Exploratory Analysis
- Nothing Runs Like a GPU-Powered, Fully Autonomous Deere
- Why Is SAS Going Public?
- More News In Brief…
Most Read This Just In
- Deloitte Launches Metaverse Services and Metaverse Experience Studio
- Collibra Announces Investment from Snowflake to Expand Data Intelligence for Snowflake Data Cloud
- Datanami Reveals Winners of the 2021 Readers’ and Editors’ Choice Awards
- BBVA Teams with Accenture to Streamline Operations with Artificial Intelligence
- HHI Group to Establish Big Data Platform in Partnership with Palantir Technologies
- Anomalo and Snowflake Partner to Help Enterprises Trust Their Data
- WEKA Announces $73M Funding Led by Hitachi Ventures
- Piano Partners with Snowflake
- Databricks Launches Data Lakehouse for Retail and Consumer Goods Customers
- Master Data Management Delivers Secure Data Analytics With Okera Partnership
- More This Just In…