Follow Datanami:

Tag: stream processing

How Netflix Optimized Flink for Massive Scale on AWS

When it comes to streaming data, it's tough to find a company operating on a more massive scale than Netflix, which streams more than 125 million hours of TV shows and movies --  per day. Netflix captures billions of pi Read more…

SAS Previews Upcoming Enhancements to Viya Platform

Neural network interpretability, integration with open source analytic libraries, and greater mobility within SAS's product suite are among the capabilities that are coming to the SAS Viya platform later this quarter. Read more…

Confluent Adds KSQL Support to Kafka Platform

The latest version of Confluent’s Kafka-based platform incorporates an open source streaming engine for Apache Kafka designed to allow developers using SQL to build real-time, streaming applications. Confluent, the Read more…

DataTorrent Glues Open Source Componentry with ‘Apoxi’

Building an enterprise-grade big data application with open source components is not easy. Anybody who has worked with Apache Hadoop ecosystem technology can tell you that. But the folks at DataTorrent say they've found Read more…

Fueled by Kafka, Stream Processing Poised for Growth

Once a niche technique used only by the largest organizations, stream processing is emerging as legitimate technique for dealing with massive amounts of data generated every day. While it's not needed for every data chal Read more…

Managing Streaming Flink Apps Is About To Get Easier

Apache Flink has emerged as a powerful platform for building real-time stream processing applications. However, not every organization has the resources to go all in on Flink the way Netflix, Uber, and Alibaba have. That Read more…

A Peek Inside Kafka’s New ‘Exactly Once’ Feature

Here's some great news for Apache Kafka users: The open source software will support exactly once semantics for stream processing with the upcoming version 0.11 release, thereby eliminating the need for application devel Read more…

Yahoo’s Massive Hadoop Scale on Display at Dataworks Summit

Yahoo put its massive Hadoop investment on display this week at Dataworks Summit, the semi-annual big data conference that it co-hosts with Hortonworks. While Hadoop is no longer the conference headliner that it once Read more…

Hortonworks Shifts Focus to Streaming Analytics

Hortonworks started life providing a Hadoop distribution that allowed customers to process big data at rest. But these days, the company has shifted its much of its attention and resources to streaming analytics, or proc Read more…

Sparse Fourier Transform Gives Stream Processing a Lifeline from the Coming Data Deluge

When James Cooley and John Tukey introduced the Fast Fourier transform in 1965, it revolutionized signal processing and set us on course to an array of technological breakthroughs. But today's overwhelming data sets requ Read more…

How Pandora Uses Kafka

As a big Hadoop user, Pandora Media is no stranger to distributed processing technologies. But when the music streaming service decided to transition its ad tracking system from a batch-oriented system into a real-time o Read more…

Google/ASF Tackle Big Computing Trade-Offs with Apache Beam 2.0

Trade-offs are a part of life, in personal matters as well as in computers. You typically cannot have something built quickly, built inexpensively, and built well. Pick two, as your grandfather would tell you. But appare Read more…

The Real-Time Future of ETL

We're on the cusp of a huge uptick in data generation thanks to the IoT, but most of that data will never be landed in a central repository or stored for any length of time. To get a handle on this morass of data, enterp Read more…

Kafka ‘Massively Simplifies’ Data Infrastructure, Report Says

What's behind the rapid rise in Apache Kafka? According to a new survey of Kafka users by Confluent, the commercial venture behind Kafka, the data pipeline's capability to "massively simplify" the IT infrastructure of en Read more…

Flink Aims to Simplify Stream Processing

Apache Flink has emerged as a powerful framework for building real-time stream processing applications that has gained traction by some of the most progressive tech companies in the world, including at Netflix, Uber, and Read more…

How Kafka Redefined Data Processing for the Streaming Age

The Apache Kafka phenomenon reached a new high today when Confluent announced a $50 million investment from the venture capital firm Sequoia. The investment signals renewed confidence that Kafka is fast becoming a new an Read more…

Exactly Once: Why It’s Such a Big Deal for Apache Kafka

Organizations building real-time stream processing systems on Apache Kafka will be able to trust the platform to deliver each messages exactly once when they adopt new Kafka technology planned to be unveiled this spring, Read more…

Kafka Gets Server, API Upgrades

The latest feature release from the Apache Kafka stream processing community includes server support for time-based search along with more than 200 bug fixes and other improvements. Confluent Inc., founded by the team Read more…

Acquisition Validates Concord’s Event-Based Approach

We suggested during the summer that companies looking for a stream-processing engine for fast data applications featuring high-throughput and low-latency may want to check out startup Concord Systems Inc. Someone did: Read more…

MemSQL Delivers an ‘Exactly Once’ Real-Time Pipeline

MemSQL today unveiled a new release of its in-memory relational database that can process a real-time flow of messages from Apache Kafka using "exactly once" semantics. The NewSQL database accomplished the feat by creati Read more…

Older Entries
Newer Entries
Datanami