Aiven Introduces an Open Source Streaming Ecosystem for Apache Kafka

(Blue Planet Studio/Shutterstock)

Users of the event streaming platform Apache Kafka may be interested to know there are new management services available through Aiven. The company has announced a complete open source streaming ecosystem for Kafka.

Aiven recently acquired Kafkawize, a centralized governance layer on top of Kafka, renaming it to Klaw in September. The company says it was the first to offer managed Kafka in the cloud in 2016, and this recent acquisition allows it to provide a comprehensive portfolio of solutions and services around Kafka to let customers transport, manage, process, operate, and efficiently govern their organization’s streaming data in real-time.

The company listed each component of its complete streaming ecosystem for Kafka:

• Aiven for Apache Kafka: the core event streaming framework, allowing real-time data transport for any organization. Manage, operate and transport your data streams reliably and in one place, with the ability to also integrate with other Aiven services, cloud regions, and external systems.
• Aiven for Apache Kafka Connect: a fully managed, fully open source, distributed service enabling customers to integrate existing data sources and sinks seamlessly with Aiven for Apache Kafka.
• Aiven for Apache Kafka MirrorMaker2: a fully managed, fully open source distributed data replication service for cluster to cluster data replication, disaster recovery and geographic proximity across multiple regions and cloud providers.
• Aiven for Apache Flink (beta): a fully managed, fully open source streaming SQL engine for stateful stream processing over data streams.
• Karapace: a fully open source Kafka Schema Registry that applications can access to serialize and deserialize messages with popular formats such as AVRO, Protobuf and JSON.
• Klaw: an open source data governance tool that helps enterprises exercise Apache Kafka topic and schema governance in a self-service manner.

“As a leader in the open source community, Aiven is on a mission to manage software that makes developers’ lives easier – and with our complete, open source streaming ecosystem of technologies around Aiven for Apache Kafka, we’re able to do just that and more for our users,” said Oskari Saarenmaa, CEO and co-founder of Aiven. “I couldn’t be more excited to share this streaming ecosystem with the community and continue fueling innovative, data-intensive open source technologies.”

Aiven claims its Apache Kafka-as-a-Service allows users to set up fully managed Kafka in less than ten minutes, either directly from Aiven’s web console or programmatically via its API, CLI, Terraform provider, or Kubernetes operator. It enables setting up clusters, deploying new nodes, migrating clouds, and upgrading existing versions. There are over 30 connectors available along with logs and metrics via service integrations, according to the website. Users can deploy to five public clouds with over a hundred supported regions, and each plan includes a minimum of three dedicated virtual machines. There is also a Bring Your Own Cloud account model for larger setups.

Helsinki-based Aiven says its founders built their expertise while managing data infrastructure for companies like F-Secure and Nokia. “They noticed that cloud adoption was increasing but infrastructure solutions were either proprietary or difficult to translate into business results. They struck out on their own and built a true, open source data cloud that organizations could use to build a modern data infrastructure. With Aiven, companies could grow from prototyping to world-wide scale at greater and greater velocities,” says the company’s website.

Aiven raised $210 million in a Series D round back in May at a $3B valuation. The company claims to have over a thousand customers, including Everactive, an IoT platform based on self-powered sensor devices and low-power wireless communication, who set up a sensor data pipeline on top of Aiven for Kafka.

“We’ve been using a variety of solutions from Aiven for our event streaming needs, such as Aiven for Apache Kafka and Aiven for Apache Kafka Connect. The combination of Kafka from Aiven, its APIs, and the Terraform provider from Aiven, has enabled us to automate almost everything: configuration, deployment, and maintenance,” said Carlos Olmos, Senior Principal Software Engineer at Everactive. “Aiven’s support and tooling around Kafka has empowered us to embrace Kafka at Everactive and harness the potential of our streaming data.”

Related Items:

Developing Kafka Data Pipelines Just Got Easier

Is Real-Time Streaming Finally Taking Off?

Intimidated by Kafka? Check Out Confluent’s New Developer Site