Tag: Spark

How Disney Built a Pipeline for Streaming Analytics

May 14, 2018 |

The explosion of on-demand video content is having a huge impact on how we watch television. You can now binge watch an entire season’s worth of Grey’s Anatomy at one sitting, if that suits your fancy. Read more…

Presto Use Surges, Qubole Finds

Apr 18, 2018 |

Don’t look now, but Presto, the SQL engine developed by Facebook as a follow-on to Hive, is starting to catch on in a big way. According to a new survey of big data-as-a-service customers by Qubole, Presto logged impressive usage gains during 2017, and outgrew Hive and Spark across many metrics. Read more…

Making Hadoop Relatable Again

Mar 26, 2018 |

There has been much debate over the future of Hadoop in recent months. Should it work more like a cloud object store? Should it support GPUs and FPGAs, Docker or Kubernetes (or both)? Read more…

Weighing Open Source’s Worth for the Future of Big Data

Feb 26, 2018 |

The open source software movement began in earnest 20 years ago, when a group of technology leaders in Silicon Valley coined the term as an alternative to the repugnant “free software.” Read more…

DataTorrent Glues Open Source Componentry with ‘Apoxi’

Feb 22, 2018 |

Building an enterprise-grade big data application with open source components is not easy. Anybody who has worked with Apache Hadoop ecosystem technology can tell you that. But the folks at DataTorrent say they’ve found a way to accelerate the delivery of secure and scalable big data applications with Apoxi, a new framework they created to stitch together major open source components like Hadoop, Spark, and Kafka, in an extensible and pluggable fashion. Read more…

The Hybrid Database Capturing Perishable Insights at Yiguo

Feb 22, 2018 |

Yiguo.com is the largest B2C fresh produce online marketplace in China, serving close to 5 million users and more than 1,000 enterprise customers. We have long devoted ourselves to providing fresh food for ordinary consumers and have gained popularity since our founding in 2005. Read more…

ParallelM Aims to Close the Gap in ML Operationalization

Feb 21, 2018 |

A startup named ParallelM today unveiled new software aimed at alleviating data scientists from the burden of manually deploying, monitoring, and managing machine learning pipelines in production.

Dubbed MLOps, ParallelM‘s software helps to automate many of the operational tasks required to turn a machine learning model from a promising piece of code running nn Spark, Flink, TensorFlow, or PyTorch processing engines into a secure, governed, and production-ready machine learning system. Read more…

Snowflake Taps Qubole for Deep Machine Learning in the Cloud

Feb 13, 2018 |

Organizations storing big data in Snowflake’s cloud data warehouse can now run machine learning and deep learning algorithms against that data thanks to a new partnership with Qubole.

The two companies today announced a partnership that will allow Qubole’s big data processing engines, including Apache Spark and TensorFlow, to read and write data to Snowflake’s data warehouse. Read more…

Dr. Elephant Leads the Performance Parade

Jan 12, 2018 |

I started working on big data infrastructure in 2009 when I joined Cloudera, which at the time was a small startup with about 10 engineers. It was a fun place to work. Read more…

Databricks Puts ‘Delta’ at the Confluence of Lakes, Streams, and Warehouses

Oct 25, 2017 |

Databricks today launched a new managed cloud offering called Delta that seeks to combine the advantages of MPP data warehouses, Hadoop data lakes, and streaming data analytics in a unifying platform designed to let users analyze their freshest data without incurring enormous complexity and costs. Read more…

Older Entries