Follow Datanami:

Tag: apache spark

Machine Learning, from Single Core to Whole Cluster

The demand for production-quality software for mining insights from datasets across scales has exploded in the last several years. The growing size of datasets throughout industry, government, and other fields has increa Read more…

Meet Sean Knapp, a 2021 Datanami Person to Watch

Getting data to the right place at the right time has never been more important than it is now. But for many organizations, the data movement task largely remains a manual affair. Sean Knapp founded Ascend.io because he Read more…

ML Scaling Requires Upgraded Data Management Plan

Successful data strategies are built on a foundation of meticulous data management, creating enterprise architectures that “democratize” data access and usage, yielding measurable results from machine learning platfo Read more…

Cloudera, Nvidia Team to Speed Cloud AI via Spark

Cloud access to GPUs for AI development will expand under a partnership between Cloudera and Nvidia that calls for the data cloud provider to integrate Nvidia’s accelerated Apache Spark 3.0 platform as a way to scale d Read more…

No-Coder Upsolver Aims to Ease Use of Cloud Data Lakes

Upsolver, the no-code data lake platform vendor, has closed a $25 million funding round this week, boosting total venture funding for its cloud analytics tools to about $42 million. The financing round announced Tuesd Read more…

Databricks Plotting IPO in 2021, Bloomberg Reports

Databricks, which runs a unified data platform in the cloud and is the driving force behind Apache Spark, is preparing for an initial public offering (IPO), possibly in the first half of 2021, according to a report in Bl Read more…

Big Data Apps Wasting Billions in the Cloud

Many organizations have shifted to a cloud-first mentality for deploying their big data applications. But without expending effort to optimize or tune these cloud apps, customers will waste billions of dollars’ worth o Read more…

To Centralize or Not to Centralize Your Data–That Is the Question

Should you strive to centralize your data, or leave it scattered about? It seems like it should be a simple question, but it’s actually a tough one to answer, particularly because it has so many ramifications for how d Read more…

Google Cloud’s Dataproc Gets a GPU-Powered Spark Boost

Google Cloud’s Dataproc – its big data platform that allows users to run Apache Hadoop and Spark jobs – is getting a boost. Apache Spark 3 and Hadoop 3 have launched general availability, enhancing users’ data an Read more…

Spark 3.0 Brings Big SQL Speed-Up, Better Python Hooks

Apache Spark 3.0 is now here, and it’s bringing a host of enhancements across its diverse range of capabilities. The headliner is an big bump in performance for the SQL engine and better coverage of ANSI specs, while e Read more…

Datanami