Follow Datanami:
Data Engineers: The C-Suite’s Savior

Data Engineers: The C-Suite’s Savior

Today’s competitive marketplace requires companies to be data-driven. Why? Because data has become the fuel for organizations to deliver better and faster decisions, quickly respond to customers, and analyze, understand and act on new opportunities (or threats) ahead of the competition. Read more…

Object and Scale-Out File Systems Fill Hadoop Storage Void

Object and Scale-Out File Systems Fill Hadoop Storage Void

The rapid growth of data and the changing nature of data applications is challenging established architectural concepts for how to store big data. Read more…

The State of Storage: Cloud, IoT, and Data Center Trends

The State of Storage: Cloud, IoT, and Data Center Trends

One of the biggest storylines in data storage this year is the explosion of the cloud and the simplicity that cloud storage can bring. Read more…

Big Data Career Notes: July 2019 Edition

Big Data Career Notes: July 2019 Edition

In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the big data community. Read more…

Datanami Headlines

Azure Data Share Seeks to Streamline Big Data Sharing

Microsoft has unveiled its preview for Azure Data Share, a new data service targeted at sharing data across organizations. According to Microsoft, Azure Data Share is targeted at those cases where data is housed with partners or third parties, but sharing the data simply is challenging for users.  Read more…

Ascend Launches From Stealth With $19M

Ascend, an autonomous dataflow service provider, has emerged from stealth with $19M in funding. Ascend, which is seeking to “de-risk big data projects and accelerate digital transformations,” promises a solution that allows data engineering teams to rapidly build, scale and operate Apache Spark-based pipelines. Read more…

Dagster Emerges to Simplify Data App Development

If you’re frustrated with the fragmented way you build ETL processes or machine learning pipelines, then you might be interested in learning about Dagster, a new open source library developed to provide a common abstraction layer that helps data scientists, analysts, and engineers to create robust data applications in their tools of choice. Read more…

Big Data Is Still Hard. Here’s Why

We’re over a decade into the big data era that emerged from the tectonic collision of mobile, Web 2.0, and cloud forces. Bolstered by progress in machine learning, we stand at the cusp of a new AI era that promises even greater automation of rudimentary tasks. Read more…

Cloudera Commits to 100% Open Source

The old Cloudera developed and distributed its Hadoop stack using a mix of open source and proprietary methods and licenses. But the new Cloudera will be 100% open source, just like Hortonworks, its one-time Hadoop rival that it acquired in January. Read more…

Alluxio Bolsters Data Orchestration for Hybrid Cloud World

Alluxio was conceived at Cal Berkeley’s AMPLab as a virtualization layer that eliminates barriers between data silos, and allows users to access remote data as if it were local. With today’s launch of Alluxio 2.0, the technology has evolved to the next stage, with a focus on simplifying data engineering in today’s emerging multi-cloud and hybrid cloud paradigms. Read more…

Google Adds Elastifile to Cloud Storage Mix

Since assuming command at Google Cloud last fall, Thomas Kurian has aggressively pursued acquisitions designed to help differentiate its services as customers roll out their multi-cloud strategies.

The ex-Oracle (NYSE: ORCL) executive followed up last month’s deal for visualization tool vendor Looker with the acquisition this week of cloud storage specialist Elastafile. Read more…

Argonne Team Makes Record Globus File Transfer

A team of scientists at Argonne National Laboratory has broken a data transfer record by moving a staggering 2.9 petabytes of data for a research project. 

The data – from three large cosmological simulations – was generated and stored on the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), which is currently rated as the world’s fastest supercomputer on the Top500 list at nearly 149 Linpack petaflops.  Read more…

Big Data and AI Converge in Map to Protect Biodiversity

What’s living where? Those are the basic pieces of data that biologists and conservationists are hoping to collect and load into the Map of Biodiversity Importance, a dynamic new map unveiled this week at the Esri User Conference. Read more…

Why You Don’t Need AI

Talk of Artificial Intelligence (AI) is everywhere, and the rush to implement AI is unquestioned. Studies such as those by Gartner and the McKinsey Global Institute, report that over the past four years, AI implementation has grown by as much as 270 percent. Read more…

Program Synthesis Moves a Step Closer to Reality

As data scientists and software developers sort through the plethora of tools and APIs ranging from Python to Apache Spark, automation schemes are emerging to help programmers navigate those tools and the accompanying infrastructure that machine learning and other apps run on. Read more…

Common Sense Makes Progress with Deep Learning

We’ve witnessed incredible progress in the capability of deep learning models to not only understand text, but to generate it too. While the generated text is grammatically sound, the actual meaning of the words leaves something to be desired. Read more…

Data Privacy and Smart Streaming Discovery: Getting Past the Furor

There’s a lot of furor worldwide about all of the data privacy regulations that are either in place, about to go into place, or are being debated as to whether they should go into place.  Read more…


Do NOT follow this link or you will be banned from the site!