MIT and Databricks Report Finds Data Management Key to Scaling AI

A new report from MIT Technology Review, in association with Databricks, found that 72% of C-level respondents believe data mismanagement will jeopardize future AI success. The report, “CIO vision 2025: Bridging the Read more…

Databricks Scores ACM SIGMOD Awards for Spark and Photon

Databricks announced it has won two awards at the ACM SIGMOD (Association of Computing Machinery’s Special Interest Group in the Management of Data) Conference in Philadelphia. Apache Spark was awarded the SIGMOD Sy Read more…

Databricks Ships New ETL Data Pipeline Solution

Databricks today announced the general availability (GA) of Delta Live Tables (DLT), a new offering designed to simplify the building and maintenance of data pipelines for extract, transform, and load (ETL) processes usi Read more…

Databricks Releases Lakehouse for Financial Services

Databricks first launched its lakehouse architecture in 2017, combining elements of data warehouses, data lakes, and streaming data analytics in a move that has paid off—literally and figuratively—in ensuing years as Read more…

Big Data Career Notes: January 2022 Edition

In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the big data community. Whether it’s a promotion, new company hire, or even an accolade, we’ve got the details Read more…

Databricks Now on Google Cloud

Databricks customers can now run their big data applications on Google Cloud, the companies announced today. The move gives Databricks’ customers another public cloud option besides Microsoft Azure and Amazon Web Servi Read more…

Data Lake or Warehouse? Databricks Offers a Third Way

In the ongoing debate about where companies ought to store data they want to analyze – in a data warehouses or in data lake -- Databricks today unveiled a third way. With SQL Analytics, Databricks is building upon its Read more…

Databricks Brings Data Science, Engineering Together with New Workspace

Data scientists and software engineers work in different ways and use different tools. But both personas will feel more comfortable developing applications in the new version of Databricks Data Science Workspace, which t Read more…

Databricks Cranks Delta Lake Performance, Nabs Redash for SQL Viz

Today at its Spark + AI Summit, Databricks unveiled Delta Engine, a new layer in its Delta Lake cloud offering that uses several techniques to significantly accelerate the performance of SQL queries. The company also ann Read more…

Will Databricks Build the First Enterprise AI Platform?

Ali Ghodsi might have one of the best jobs in technology right now. As the CEO of Databricks, Ghodsi just completed an oversubscribed $400 million round of funding that gave the company a $6.2 billion valuation. Better s Read more…

A Cloud Twist to the Farm-to-Table Movement

Mail-order services, or what a cloud-enabled industry calls “subscription box” services, have become all the rage for everything from fine wine to pet food. Consumers can also order Kansas City steaks online, but a p Read more…

Databricks Offers Something for Everybody with AutoML Solution

Databricks today took the covers off a new automated machine learning solution that promises to reduce the amount of manual coding required to develop predictive applications. But while other AutoML solutions tend to foc Read more…

Databricks Donates Delta Code to Open Source

Databricks today announced that it's open sourcing the code behind Databricks Delta, the Apache Spark-based product it designed to help keep data neat and clean as it flows from sources into its cloud-based analytics env Read more…

Apache Spark Is Great, But It’s Not Perfect

Apache Spark is one of the most widely used tools in the big data space, and will continue to be a critical piece of the technology puzzle for data scientists and data engineers for the foreseeable future. With that said Read more…

A Decade Later, Apache Spark Still Going Strong

Don't look now but Apache Spark is about to turn 10 years old. The open source project began quietly at UC Berkeley in 2009 before emerging as an open source project in 2010. For the past five years, Spark has been on an Read more…

Databricks Open Sources MLflow to Simplify Machine Learning Lifecycle

Databricks today unveiled MLflow, a new open source project that aims to provide some standardization to the complex processes that data scientists oversee during the course of building, testing, and deploying machine le Read more…

Top 3 New Features in Apache Spark 2.3

It's tough to find a big data project that's had as much impact as Apache Spark over the past five years. The folks at Databricks, who contribute heavily to Spark (along with the wider Spark community) are keeping the pr Read more…

Databricks Puts ‘Delta’ at the Confluence of Lakes, Streams, and Warehouses

Databricks today launched a new managed cloud offering called Delta that seeks to combine the advantages of MPP data warehouses, Hadoop data lakes, and streaming data analytics in a unifying platform designed to let user Read more…

The Data Science Behind Dollar Shave Club

Dollar Shave Club burst onto the men's hygiene scene in 2011 with a hilarious video and preposterous business plan: selling subscriptions for razor blades at a ridiculously low price. Six years later, the company keeps g Read more…