Follow Datanami:
September 11, 2018

Streamsets Gets $35M for DataOps

(FlashMovie/Shutterstock)

StreamSets, which bills itself as the “air traffic control” tasked with preventing collisions from occurring with big data, today announced that it raised $35 million, which it will use to continue building its data operations, or DataOps, platform. The company also announced greater support for cloud environments.

StreamSets says DataOps has grown in importance as companies re-architect their “data supply chain” using new technologies and techniques. Successfully navigating the new data environment requires being able to handle data sources like microservices, API feeds and systems logs, along with the array of data processing platforms and frameworks that customers are using, including Kafka, Spark, Hadoop, NoSQL databases, and cloud environments.

Mark Lotke, founder and managing partner of Harmony Partners, which invested in StreamSets, says the firm is on the leading edge of the new data integration challenge. “We’re seeing enterprise data architectures grow in complexity while use of big and fast data becomes business-critical…” he says in a press release.

StreamSets is gaining traction across industries in a variety of engagements, including customer 360 initiatives, cybersecurity, fraud detection, and industrial IoT use cases. StreamSets’ software is designed to help companies keep a close eye on key data integration tasks, which are often developed and implemented as data pipelines.

The San Francisco company told Datanami earlier this year that some enterprises have already developed more than a million data pipelines, often on the back of next-gen data busses like Kafka. It’s StreamSets’ goal to develop the product that companies turn to when they need to ensure those pipelines are monitored and managed in a professional and scalable manner.

One of the dangers that StreamSets hopes to alleviate is data drift, which can occur when schemas change over time. That’s a tough thing to detect when you’re in the thick of it, and one of the reasons why DataOps is needed, the company says.

The Series C funding round was led by Harmony Partners, and had participation from a new investor, Tenaya Capital, and existing investors Battery Ventures and New Enterprise Associates (NEA). StreamSets, which has raised a total of $65 million to date, plans to use the investment to fund various aspects of its business, including marketing, sales, R&D and customer support.

Separately, the company made a cloud-related announcement today as Strata Data Confernce kicks off in New York City. StreamSets says it’s now supporting cloud environments with several of its DevOps capabilities, including support for data drift handling, continuous integration and delivery (CI/CD) automation, and the ability to centrally manage in-stream data protection policies for security and compliance.

The company, which says a majority of its customers already run in the cloud, is expanding its existing cloud support, which includes the capability to scale dataflows using Kubernetes. Now it’s giving customers better tools for managing pipelines that span cloud and on-prem environments.

“As our customers embark on their hybrid cloud journey, we see first-hand their struggle to orchestrate end-to-end management of data movement across a growing range of on-premises and cloud platforms,” StreamSets CTO Arvind Prabhakar says in a press release. “Our DataOps platform was architected as cloud-native from the start, allowing us to easily evolve with the market.”

Related Items:

StreamSets Balances Streaming Data Demands for Security, Access

Fueled by Kafka, Stream Processing Poised for Growth

Keeping on Top of Data Drift

 

Datanami