Follow Datanami:

Tag: data pipeline

What Is a Data Cloud? And 11 Other Snowflake Enhancements

Jun 2, 2020 |

Snowflake has changed how the industry thinks about data warehouses with its cloud-native offering, which has been adopted by 4,000 organizations, including 2,000 in the last year alone. Now the company is taking the concept one step further with the introduction of a data cloud, which the company is positioning as a one-stop shop where organizations can execute a full range of data-oriented tasks – not just data warehousing and SQL analytics, but also machine learning, data engineering, and monetization of third-party data. Read more…

Data Pipelines of a Higher Order

Dec 12, 2019 |

Data pipelines are being constructed everywhere these days, moving huge swaths of data for a wide variety of operational and analytical needs. There’s no doubt that all these data pipelines are keeping data engineers busy and in high demand (and highly paid, to boot). Read more…

StreamSets Eases Spark-ETL Pipeline Development

Sep 5, 2019 |

Apache Spark gives developers a powerful tool for creating data pipelines for ETL workflows, but the framework is complex and can be difficult to troubleshoot. StreamSets is aiming to simplify Spark pipeline development with Transformer, the latest addition to its DataOps platform. Read more…

Databricks Donates Delta Code to Open Source

Apr 24, 2019 |

Databricks today announced that it’s open sourcing the code behind Databricks Delta, the Apache Spark-based product it designed to help keep data neat and clean as it flows from sources into its cloud-based analytics environment. Read more…

Data Pipeline Automation: The Next Step Forward in DataOps

Apr 24, 2019 |

The industry has largely settled on the notion of a data pipeline as a means of encapsulating the engineering work that goes into collecting, transforming, and preparing data for downstream advanced analytics and machine learning workloads. Read more…

From Big Beer to Big Data: Inside AB InBev’s Digital Transformation

Apr 11, 2019 |

With more than 500 beer brands and $55 billion in sales, Anheuser-Busch InBev is already the world’s biggest beer company. And if all goes as planned with its digital transformation project, it will be the best beer company in the world, too. Read more…

Google Doubles Down on Cloud Data Migration

Feb 21, 2019 |

Data integration startups have become prime acquisition targets as cloud analytics vendors look to beef up their migration capabilities.

What that in mind, Google Cloud announced this week it intends to acquire data migration specialist Alooma, a five-year-old Bay Area startup. Read more…

Streamsets Gets $35M for DataOps

Sep 11, 2018 |

StreamSets, which bills itself as the “air traffic control” tasked with preventing collisions from occurring with big data, today announced that it raised $35 million, which it will use to continue building its data operations, or DataOps, platform. Read more…

Machine Teaching Will Drive Crowdsourced Cognition into the AI Pipeline

Jun 25, 2018 |

Building high-quality artificial intelligence (AI) is hard work. It’s a specialized discipline that historically has required highly skilled specialists, aka data scientists.

Any time you require some highly skilled, highly paid practitioner to accomplish something of value, you’ve introduced a bottleneck into that process. Read more…

Do NOT follow this link or you will be banned from the site!