Confluent Moves to Boost Kafka Reliability
Apache Kafka has emerged as a leading platform for the management of data transaction streaming. Hence, users are placing a premium on maintaining Kafka uptime by anticipating and resolving issues before Kafka clusters go down, with the resulting loss of data.
Confluent Inc., the company behind Apache Kafka continues to expand its offerings with a monitoring tool that seeks to reduce the risk of Kafka downtime and data loss by identifying and resolving issues before an instance is lost. It is also promoted as a way of preserving Kafka instances without large infrastructure investments.
The “proactive support” platform dubbed Reliable is part of Confluent’s Project Metamorphosis launched earlier this year as a next-generation event streaming platform running in the Confluent Cloud. The initiative is designed to help users scale Kafka deployments while Reliable aims to keep the event streamer up and running.
Confluent said Tuesday (Nov. 10) its Kafka support platform provides real-time analysis of cluster metadata, alerting operators of potential issues before a crash. The monitoring tool also provides performance metrics that would allow support staff to trace the root cause of a Kafka performance issues.
The company said the impetus for the Kafka platform was about 1,000 support tickets that helped it gain insights into detecting performance issues. “We proactively tell you about issues we find with [customers’ deployments] based on experience running literally thousands of clusters,” said Tim Bergland, Confluent’s senior director of developer advocacy.
Hence, the company developed a framework by which cluster metadata is fed through its collection of algorithms to gauge performance and anticipate potential problems. Real-time “health metrics” are promoted as helping support engineers identify root causes faster, cutting the time to resolve support issues by as much as 25 percent, the company claimed.
Among the most common issues affecting the scaling of Kafka’s real-time performance and the threat of data loss are network and storage bottlenecks. Overloaded networks or storage media often bring down Kafka clusters, taking with them business events and other critical data.
The managed service also is designed to help IT administrators reduce the amount of time and resources devoted to operating the relatively complex, distributed Kafka environment.
In launching Project Metamorphosis this past spring, Confluent said it planned a series of monthly product and feature upgrades through the end of the year as it fleshes out its real-time streaming platform. The managed support service is the latest in that series of offerings.
Customers running Confluent 6.0 can opt in to use the initial release of the managed Kafka service. Alerts can be sent via integration with the Confluence control center. Alternatively, users can set up alerts via email or Slack.
April 14, 2021
- QlikWorld Online 2021 Virtual Global Conference Will Be Held May 10-12
- Informatica Unveils Industry’s First Intelligent Data Management Cloud
- KIOXIA’s PCIe 4.0 NVMe SSDs Now Qualified with NVIDIA Magnum IO GPUDirect Storage
April 13, 2021
- Kirk Borne to Deliver Keynote at Inaugural DataYap Conference, April 17
- Alation Increases Operational Insight and Agility for Global Insurers with Snowflake
- Fractal Offers AI Software on AWS, Accelerating Digital Transformation with Cloud AI
- Minnesota Department of Transportation Leverages Iteris’ ClearMobility Cloud
- NVIDIA and Partners Collaborate on Arm Computing for Cloud, HPC, Edge, PC
- NVIDIA Announces Availability of Jarvis Interactive Conversational AI Framework
- Cloudflare Partners with NVIDIA to Bring AI to its Global Edge Network
- WekaIO Reveals Key Findings from ‘The State of AI and Analytics Infrastructure 2021’ Study
- NVIDIA Announces CPU for Giant AI and HPC Workloads
- Indicative Unveils ‘Journeys’ to Give Product Teams Better Insight into Customer Behavior
- Logz.io Announces Support for OpenSearch
- Privacera and Dremio Integration Enables Self-Service Analytics Across Hybrid Cloud Environments
- DataCore Announces Swarm Object Storage Software to Help Manage Petabyte-Scale Data
April 12, 2021
- Tredence Launches ML Works, Machine Learning Ops Platform
- Cloudera Collaborates with NVIDIA to Accelerate Data Analytics and AI in the Cloud
- Domino Data Lab Debuts New Solutions with NVIDIA to Enhance the Productivity of Data Scientists
- Riffyn Launches Nexus v4.2 to Strengthen Data Integrity for Biotech Product Development
Most Read Features
- Big Data File Formats Demystified
- Synthetic Data: Sometimes Better Than the Real Thing
- Experts Disagree on the Utility of Large Language Models
- A ‘Glut’ of Innovation Spotted in Data Science and ML Platforms
- He Couldn’t Beat Teradata. Now He’s Its CEO
- Who’s Winning In the $17B AIOps and Observability Market
- Why Data Science Is Still a Top Job
- Is Python Strangling R to Death?
- Big Data Predictions: What 2020 Will Bring
- Cloud Data Warehousing: Understanding Your Options
- More Features…
Most Read News In Brief
- Can PHP Script a Big Data Comeback?
- Data Prep Still Dominates Data Scientists’ Time, Survey Finds
- Global DataSphere to Hit 175 Zettabytes by 2025, IDC Says
- AWS Adds Explainability to SageMaker
- AWS Tackles Real-Time Data Transformation with S3 Object Lambda
- The Union of Salesforce, Tableau Yields Hybrid ‘Business Science’
- Databricks Edges Closer to IPO with $1B Round
- Esri Simplifies Developer Access to Location Data with ArcGIS Platform
- The AI Inside NASA’s Latest Mars Rover, Perseverance
- CDOs Must Shift to Offense, Survey Finds
- More News In Brief…
Most Read This Just In
- Moody’s Analytics Wins Award for Best Use of AI in Banking or FinTech
- Aiven Raises $100M Series C to Expand Global Open Source Innovation
- Alluxio Advances Analytics and AI with NVIDIA Accelerated Computing
- AWS Announced Strategic Partnership with Hugging Face NLP Startup
- GrafanaCONline Returns June 7-17, CFP Is Open Now
- y42 Raises $2.9M to Provide a Scalable and Affordable Data Stack to Companies of All Sizes
- ThoughtSpot Acquires SeekWell to Operationalize Analytics, Push Cloud Data Insights to Business Apps
- Trifacta Announces Industry’s First Data Engineering Cloud
- MinIO Enables IT to Manage Kubernetes-native Object Storage
- Dataminr Announces $475M Financing Round, Raising its Valuation to $4.1B
- More This Just In…
Sponsored Partner Content
May 4 - May 5
May 13 @ 11:00 am - 12:30 pm