Tag: stream processing

DataTorrent Glues Open Source Componentry with ‘Apoxi’

Feb 22, 2018 |

Building an enterprise-grade big data application with open source components is not easy. Anybody who has worked with Apache Hadoop ecosystem technology can tell you that. But the folks at DataTorrent say they’ve found a way to accelerate the delivery of secure and scalable big data applications with Apoxi, a new framework they created to stitch together major open source components like Hadoop, Spark, and Kafka, in an extensible and pluggable fashion. Read more…

Fueled by Kafka, Stream Processing Poised for Growth

Jan 18, 2018 |

Once a niche technique used only by the largest organizations, stream processing is emerging as legitimate technique for dealing with massive amounts of data generated every day. While it’s not needed for every data challenges, organizations are increasingly finding ways to incorporate stream processing into their plans — Read more…

Managing Streaming Flink Apps Is About To Get Easier

Sep 11, 2017 |

Apache Flink has emerged as a powerful platform for building real-time stream processing applications. However, not every organization has the resources to go all in on Flink the way Netflix, Uber, and Alibaba have. Read more…

A Peek Inside Kafka’s New ‘Exactly Once’ Feature

Jul 3, 2017 |

Here’s some great news for Apache Kafka users: The open source software will support exactly once semantics for stream processing with the upcoming version 0.11 release, thereby eliminating the need for application developers to code the important feature themselves. Read more…

Yahoo’s Massive Hadoop Scale on Display at Dataworks Summit

Jun 16, 2017 |

Yahoo put its massive Hadoop investment on display this week at Dataworks Summit, the semi-annual big data conference that it co-hosts with Hortonworks.

While Hadoop is no longer the conference headliner that it once was, the platform is still critical for the daily operations of Yahoo, which officially became part of Verizon Communications this week when the $4.5 billion acquisition finally closed. Read more…

Hortonworks Shifts Focus to Streaming Analytics

Jun 14, 2017 |

Hortonworks started life providing a Hadoop distribution that allowed customers to process big data at rest. But these days, the company has shifted its much of its attention and resources to streaming analytics, or processing big data in motion. Read more…

Sparse Fourier Transform Gives Stream Processing a Lifeline from the Coming Data Deluge

Jun 13, 2017 |

When James Cooley and John Tukey introduced the Fast Fourier transform in 1965, it revolutionized signal processing and set us on course to an array of technological breakthroughs. But today’s overwhelming data sets require a new approach. Read more…

How Pandora Uses Kafka

May 31, 2017 |

As a big Hadoop user, Pandora Media is no stranger to distributed processing technologies. But when the music streaming service decided to transition its ad tracking system from a batch-oriented system into a real-time one, it brought in a new technological underpinning to serve as the core foundational element. Read more…

Google/ASF Tackle Big Computing Trade-Offs with Apache Beam 2.0

May 19, 2017 |

Trade-offs are a part of life, in personal matters as well as in computers. You typically cannot have something built quickly, built inexpensively, and built well. Pick two, as your grandfather would tell you. Read more…

The Real-Time Future of ETL

May 8, 2017 |

We’re on the cusp of a huge uptick in data generation thanks to the IoT, but most of that data will never be landed in a central repository or stored for any length of time. Read more…