Introducing Streamlio Cloud Preview
September 16, 2019 — Streamlio officials announced its new Apache Pulsar sandbox, Stremlio Cloud Preview, in a blog posted on thier website by Jon Bock. The full blog is below:
The announcement of Streamlio Cloud earlier this year was a major milestone for Streamlio, building on years of work we’d done at Yahoo and Streamlio helping to create and advance Apache Pulsar. From the start, we architected Pulsar to deliver scalable performance, multi-tenancy, and always-on operation–all of which made Pulsar ideal for the cloud. Streamlio Cloud delivers those capabilities as an enterprise-class service running in the public cloud, offering a solution that combines messaging, stream processing and event storage in a single unified platform that is managed and operated by Streamlio.
The newly available Streamlio Cloud Preview is the next step in making it easy to leverage Apache Pulsar and its capabilities. Streamlio Cloud Preview provides a ready-to-use cloud sandbox for you to try out the capabilities of Apache Pulsar and Streamlio Cloud, with no upfront cost and no long-term commitment. That’s of course extremely useful for data infrastructure teams evaluating their technology choices, but it’s also extremely valuable for application developers and data engineers, an audience that has become increasingly important due to the evolution of data processing and data applications.
From the start, Apache Pulsar and Streamlio Cloud have a strong value proposition for data infrastructure teams, eliminating and streamlining many of the operations pains of legacy messaging and streaming technologies such as data redistribution, partition rebalancing and disruptive scaling. However, data infrastructure teams are only one of the key stakeholders in data processing. Development teams–application development, application integration and data engineering for example–are also a key stakeholder. That’s become even more true as a result of the growing adoption of microservices architectures, streaming data pipelines and data-driven applications. To date, developers have generally been forced to learn complex processing frameworks (e.g. map-reduce, functional programming) to be able to work directly with data in motion. The unfamiliarity of those frameworks and the complexity of the stream processing and data transport infrastructure underpinning them have made the lives of development teams, and the data infrastructure teams supporting them, painful in many aspects. This has resulted in time-consuming coordination across all aspects of development, testing, deployment, monitoring, scaling and more.
However, the capabilities offered by Pulsar and Streamlio Cloud are changing this relationship. The stream processing model and the messaging capabilities of Apache Pulsar, as delivered by Streamlio Cloud, are giving developers the ability to rapidly develop, test, deploy and iterate without overburdening data infrastructure teams. For one, developers can create data processing functions leveraging a variety of common languages (e.g. Java, Python, Go) without requiring less familiar programming paradigms. In addition, support for stream processing, message queuing and publish-subscribe messaging in one solution consolidates the number of different technologies that developers need to work with to access and process data. Developers also benefit from the fact that Pulsar Functions, the stream processing framework in Pulsar, decouples data processing logic from the execution environment and infrastructure, allowing developers to rapidly develop, deploy and iterate their data processing code while data infrastructure teams manage and tune resources, security and data flow. That decoupling allows both teams to focus more on their core tasks rather than being bottlenecked on low-level coordination, making both teams more productive and efficient.
Streamlio Cloud Preview makes it easy for both data infrastructure teams and developers to become familiar with the capabilities of Apache Pulsar provided in Streamlio Cloud. Developers get immediate access to an environment where they can create, deploy and observe their data processing code at work through a friendly interface, without asking the data infrastructure team to take on deployment and configuration of a collection of new technologies. At the same time, the data infrastructure team can use Streamlio Cloud Preview to become familiar with basic monitoring and operation of Pulsar as well.
Streamlio Cloud Preview leverages a number of Pulsar’s cloud-friendly features to make this possible. Unlike Streamlio Cloud, which runs in a dedicated environment on customers’ public cloud infrastructure, Streamlio Cloud Preview is a shared multi-tenant service running in a cloud environment managed by Streamlio. It takes advantage of Pulsar’s native isolation, resource management, security, and elastic scalability to allow developers and data infrastructure engineers to try out the core capabilities of Apache Pulsar without needing to spend time on deployment, configuration and operations. It supports core Pulsar functionality including messaging, connectors and Pulsar Functions while also providing a preview of functionality under consideration for potential future release in Streamlio Cloud such as a web interface friendly to both data infrastructure and development teams.
What can you do with Streamlio Cloud Preview? Although not a supported environment for production applications, there are a lot of things that you can try and learn by using it. As illustrated in our introductory tutorial, you can:
- Connect one of the built-in data sources to get easy access to a stream of data for your testing, or use the built-in library of connectors to connect it to your own data source
- Try your hand at writing and deploying your own Pulsar Function, including support for composing and deploying a Python function directly from within the web interface
- Connect a stream of data to a target system using one of the built-in connectors
- Get familiar with the information and tools available to monitor data flowing through Pulsar
Best of all, Streamlio Cloud Preview comes with no upfront costs and no obligations. That makes it the perfect place to learn how Streamlio and Apache Pulsar can help you. You can get started by signing up online.
What are you interested in doing with Pulsar? We’d love to hear from you.
Source: Jon Bock, Streamlio