Follow Datanami:

Tag: data transformation

Tristan Handy’s Audacious Vision of the Future of Data Engineering

Tristan Handy is a lot of things: co-creator of dbt, founder and CEO of dbt Labs, and self-described “startup person.” But besides leading dbt Labs to a $4 billion valuation, he is one more thing: An audacious dreame Read more…

dbt Labs Tackles Data Project Complexity with Mesh at Coalesce

Adoption of dbt has skyrocketed since it was launched just seven years ago, and today, more than 30,000 organizations use the open source software in production. The complexity of data projects has also increased, with 5 Read more…

Starburst Brings Dataframes Into Trino Platform

Starburst customers who prefer to manipulate data using dataframes as opposed to regular SQL will be happy with a pair of announcements made today. That includes the introduction of PyStarburst, which provides a PySpark- Read more…

In Search of Data Model Repeatability

Everybody wants to be data-driven--that much is clear. But that desire doesn’t necessarily translate into real business results, especially in competitive industries like ecommerce. Data quality has long been a burr Read more…

Semantic Layer Belongs in Middleware, and dbt Wants to Deliver It

The folks at dbt Labs think the data transformation tool is the proper place to implement and manage a semantic data layer, as opposed to the BI tool or the data warehouse, where it has traditionally resided. And later t Read more…

Data Integration and Observability Provider Crux Nabs $50m in Funding

Crux, a cloud-based provider of data integration and observability tools that claims to have more than 250 data connectors, today announced that it raised $50 million in a Series B round of venture capital. The San Franc Read more…

Inside AutoTrader UK’s Data Observability Pipeline

In the course of shifting its analytics estate to the cloud, AutoTrader UK has adopted many new tools and technologies, including BigQuery, Looker, and dbt, which have helped to democratize data access among users. Along Read more…

8 Key Considerations for Embarking on a Data Integrity Journey

Modern enterprises are reliant on data, and as the volume of it increases, making that data useful is absolutely critical. However, more data also means the likelihood of incomplete, inconsistent data sets is on the rise Read more…

The ‘Rage Design’ Behind Flatfile’s Onboarding Success

David Boskovic was excited to join a company called Envoy back in 2016. He had worked with B2B startups since he was 18, and was looking forward to helping another tech startup scale an idea. But that excitement turned t Read more…

AWS Tackles Real-Time Data Transformation with S3 Object Lambda

Cloud object stores like S3 have become the default storage repository for many companies. But operational challenges arise when one tries to use a single object store as a universal repository for multiple applications, Read more…

Cloud Is the New Center of Gravity for Data Warehousing

The great migration of data into the cloud didn’t start in 2020, but it certainly accelerated throughout the year. And according to a new survey from IDG, the overwhelming majority of companies are planning to expand t Read more…

Informatica Likes Its Chances in the Cloud

Quick: Name a company that made its name in the 1990s and 2000s by providing data integration tools for enterprise analytics running in on-prem data centers, but has since pivoted the cloud and was even named Snowflake� Read more…

Why You Need Data Transformation in Machine Learning

Thanks to machine learning and the advancements in software and technology, enterprises can now process and understand their data much faster using modern tools with established algorithms. This effectively allows them t Read more…

How ML Helps Solve the Big Data Transform/Mastering Problem

Despite the astounding technological progress in big data analytics, we largely have yet to move past manual techniques for important tasks, such as data transformation and master data management. As data volumes grow, t Read more…

The Seven Sins of Data Prep

Data preparation is often considered a necessary precursor to the “real” work found in visualizing or analyzing data, but this framing sells data prep short. The ways in which we cleanse and shape data for downstream Read more…

Data Engineers in Hot Demand

The big data community has been dealing with the data scientist shortage ever since big data became a thing. Now we're learning that there's possibly an even bigger shortage of another type of data professional: the data Read more…

How This Instrument Firm Tackled Big Data Blending

Thanks to the ongoing digitalization of the world, we're constantly awash in data of all types. From structured data like sales reports and customers list to semi-structured data like photographs and clickstreams, nearly Read more…

Taming Unstructured Data with Cognitive Computing

Contending with unstructured data is no longer a priority reserved for the most well-financed, IT-savvy organizations, like Google and Facebook. As the world’s data continues to increase at nearly exponential rates, th Read more…

Big Data’s Cold Feet Syndrome

With all the hype around big data, it sometimes seems that everybody and her grandmother are doing it. But the truth is that implementing big data analytics can be almost as difficult as it is potentially profitable. And Read more…

Datanami