
Confluent Moves to Boost Kafka Reliability

Apache Kafka has emerged as a leading platform for the management of data transaction streaming. Hence, users are placing a premium on maintaining Kafka uptime by anticipating and resolving issues before Kafka clusters go down, with the resulting loss of data.
Confluent Inc., the company behind Apache Kafka continues to expand its offerings with a monitoring tool that seeks to reduce the risk of Kafka downtime and data loss by identifying and resolving issues before an instance is lost. It is also promoted as a way of preserving Kafka instances without large infrastructure investments.
The “proactive support” platform dubbed Reliable is part of Confluent’s Project Metamorphosis launched earlier this year as a next-generation event streaming platform running in the Confluent Cloud. The initiative is designed to help users scale Kafka deployments while Reliable aims to keep the event streamer up and running.
Confluent said Tuesday (Nov. 10) its Kafka support platform provides real-time analysis of cluster metadata, alerting operators of potential issues before a crash. The monitoring tool also provides performance metrics that would allow support staff to trace the root cause of a Kafka performance issues.
The company said the impetus for the Kafka platform was about 1,000 support tickets that helped it gain insights into detecting performance issues. “We proactively tell you about issues we find with [customers’ deployments] based on experience running literally thousands of clusters,” said Tim Bergland, Confluent’s senior director of developer advocacy.
Hence, the company developed a framework by which cluster metadata is fed through its collection of algorithms to gauge performance and anticipate potential problems. Real-time “health metrics” are promoted as helping support engineers identify root causes faster, cutting the time to resolve support issues by as much as 25 percent, the company claimed.
Among the most common issues affecting the scaling of Kafka’s real-time performance and the threat of data loss are network and storage bottlenecks. Overloaded networks or storage media often bring down Kafka clusters, taking with them business events and other critical data.
The managed service also is designed to help IT administrators reduce the amount of time and resources devoted to operating the relatively complex, distributed Kafka environment.
In launching Project Metamorphosis this past spring, Confluent said it planned a series of monthly product and feature upgrades through the end of the year as it fleshes out its real-time streaming platform. The managed support service is the latest in that series of offerings.
Customers running Confluent 6.0 can opt in to use the initial release of the managed Kafka service. Alerts can be sent via integration with the Confluence control center. Alternatively, users can set up alerts via email or Slack.
Recent items:
Step One in Kafka’s Metamorphosis Revealed
Real-Time Data Streaming, Kafka and Analytics, Part 1: Data Streaming 101
Real-Time Data Streaming, Kafka and Analytics, Part 2: Going Beyond Pure Streaming
January 26, 2021
- Logi Analytics Announces New Analytics Dashboard Experience for BlueVolt Platform
- RudderStack Adds New Features, Cloud Extract and Warehouse Actions
- Western Digital and Qumulo Enable Massive Capacity and Scale for COVID Analytics and Vaccine Roll Out
- John Snow Labs Providing Free Academic Licenses to AI Researchers and Educators
- H2O.ai Launches H2O AI Hybrid Cloud
- Confluent Announces New Strategic Alliance with Microsoft for Azure Customers
- Novartis Sees 6-Figure Savings with Qlik
- Reveal And Brainspace Merge to Become a Leading AI Powered eDiscovery Platform for Legal Technology
- Cirrascale Cloud Services and Graphcore Announce Availability of Graphcloud for Machine Intelligence Workflows
- Rapid Images Deploys Storage Solution from HPE and Qumulo to Meet Increased Demand for Digitalization
January 25, 2021
- Quantum Launches New Line of Hybrid Storage Arrays for Data-intensive Workloads
- Lucidum Unveils Community License of IT Asset Discovery Platform for AWS Environments
- Cell Data Used to Study Mobility in Rio de Janeiro During COVID Lockdowns
- Machine Learning Used to Optimize Traffic Signals, Reduce Intersection Wait Times
- Virtual ICM Seminar with David Bader, ‘Solving Global Grand Challenges with High Performance Data Analytics,’ to Be Held Jan 29
- Advances in Facial Recognition Improve Gender, Skin Colour Bias of Algorithms
- Cognizant and XPRIZE Announce Finalists for COVID-19 Pandemic Response Competition
- Facebook and NYU Langone AI Research Helping to Predict COVID-19 Resource Needs from X-Rays
- McKinsey & Company Launches Data Analytics Platform Experience DNA
January 22, 2021
Most Read Features
- Dremio Officially a ‘Unicorn’ As it Reaches $1B Valuation
- Big Data File Formats Demystified
- 2021 Predictions: Data Science
- Peering Into the Crystal Ball of Advanced Analytics
- The Maturation of Data Science
- Why Data Science Is Still a Top Job
- 2021 Prediction from the Edge and IoT
- Is Python Strangling R to Death?
- Big Data Predictions: What 2020 Will Bring
- Predictive Maintenance Drives Big Gains in Real World
- More Features…
Most Read News In Brief
- Update: Elastic Shifts Licensing Terms, Citing Amazon Moves
- War Unfolding for Control of Elasticsearch
- A Virtual Ride Along with John Deere at CES
- Momentum Builds to Break Elasticsearch Licensing Deadlock
- PG&E Deploys Palantir’s Foundry Platform
- Data Prep Still Dominates Data Scientists’ Time, Survey Finds
- Verizon Expands Deal with AWS for 5G Edge Platform
- Databricks Plotting IPO in 2021, Bloomberg Reports
- ACLU Objects to Expansion of Facial Recognition by CBP
- AWS Launches Managed Services for Grafana, Prometheus
- More News In Brief…
Most Read This Just In
- SAS Fulfills Pledge to Support HBCUs with Software and Partnerships
- PrestoSQL Rebranding as Trino
- Datatron Releases New Governance Dashboard
- BlackSky Awarded Govt Contract to Develop AI Platform for Global Construction Monitoring
- O’Reilly Announces 2021 Superstream Series Lineup and Dates
- Datanami Reveals Winners of Fifth Annual Readers’ and Editors’ Choice Awards
- Veritone and Alteryx Partner to Power the Alteryx Analytic Process Automation Platform with New AI Capabilities
- Splice Machine Launches Feature Store to Simplify Feature Engineering
- Fivetran Expands Enterprise Data Integration with Release of New Data Source Connectors
- Fujitsu Signs $8M Contract with Palantir, Becomes First Distributor of Foundry Modules in Japan
- More This Just In…
Sponsored Partner Content
-
Who wins the hybrid cloud?
-
Making the Business Case for a Data Catalog
-
Hear from William McKnight on everything you need to know to modernize your enterprise data warehouse.
-
Free Whitepaper: The Race for a Unified Analytics Warehouse
-
Move beyond extracts – Instantly analyze all your data with Smart OLAP™
-
CDATA | Universal Connectivity to SaaS/Cloud, NoSQL, & Big Data
Sponsored Whitepapers
Contributors
Featured Events
-
LEAP
February 1 - February 3Saudi Arabia -
AI & Big Data Expo Global 2021
March 17 - March 18Kensington London United Kingdom