Follow Datanami:

Tag: Data engineering

Five Ways to Unshackle Data Science from IT Now

If your company is struggling to get value out of its data science initiative, it's not alone. More often than not, an organization will experience growing pains as it figures out how to gain data science competency thro Read more…

DataOps Hiring Surges Thanks to ML, Real-Time Streaming

Nearly three-quarters of companies plan to hire a data operations (DataOps) professional in 2018, according to a new survey commissioned by Nexla, which says investment and interest in machine learning, artificial intell Read more…

Apache Airflow to Power Google’s New Workflow Service

Apache Airflow, the workload management system developed by Airbnb, will power the new workflow service that Google rolled out today. Called Cloud Composer, the new Airflow-based service allows data analysts and applicat Read more…

Why 2018 Will Be The Year Of The Data Engineer

The shortage of data scientists – those triple-threat types who possess advanced statistics, business, and coding skills – has been well-documented over the years. But increasingly, businesses are facing a shortage o Read more…

Learning IoT Lessons the Big Data Way

Spending on Internet of Things (IoT) projects is expected to triple over the next few years, to more than $450 billion globally, as the technology matures and use cases emerge. As organizations embark upon exciting new I Read more…

The Serverless ETL Nextdoor

When Nextdoor set out to rebuild its data ingestion pipeline so it could better handle billions of files per day, it brought in all the usual suspects in the real-time racket: Spark, Kafka, Flume, etc. In the end, howeve Read more…

Cloudera Unveils Altus to Simplify Hadoop in the Cloud

Running Hadoop, whether on-premise or in the cloud, is neither simple nor easy. Administrators with specialized skills are needed to configure, manage, and maintain the clusters for their clients, who are data scientists Read more…

Data Science and the Decision-maker in the Machine

It’s not the algorithms or techniques that comprise data science, but the core principles that underlie the techniques, argue academics, Foster Provost and Tom Fawcett, who say the conflation between popular data-related buzz terms and actual “data science” could be problematic if not straightened-out soon. Read more…

Datanami