Follow Datanami:

Tag: Data engineering

The Upcoming Year in Big Data: A 2022 Preview

The world of big data is a never-ending roller coaster of new technologies, new techniques, and an ever-growing tsunami of data. As we roll into 2022, we turn to the community of big data practitioners and solution provi Read more…

Snowflake Adds Python Support with Winter Release

In a nod to the growing importance of data science and AI development on its platform, Snowflake today unveiled that its upcoming Winter Release will support for executing code written in Python, which is the most popula Read more…

What’s Driving Python’s Massive Popularity?

Earlier this month, Python moved into the number one slot in the TIOBE Index, marking the first time in 20 years that a language named C or Java wasn’t at the top of the list. It’s a nice feather in Python’s cap, a Read more…

AI: It’s Not Just For the Big FAANG Dogs Anymore

It’s been said that AI has a 1% problem, that only the biggest tech firms—the Facebooks, Amazons, Apples, Netflixes, and Googles, or FAANGs, of the world--have the resources required to pull it off.  But thanks to t Read more…

Meet Sean Knapp, a 2021 Datanami Person to Watch

Getting data to the right place at the right time has never been more important than it is now. But for many organizations, the data movement task largely remains a manual affair. Sean Knapp founded Ascend.io because he Read more…

No-Coder Upsolver Aims to Ease Use of Cloud Data Lakes

Upsolver, the no-code data lake platform vendor, has closed a $25 million funding round this week, boosting total venture funding for its cloud analytics tools to about $42 million. The financing round announced Tuesd Read more…

Data Engineering Cloud Launched by Trifacta

Trifacta today launched what might be the world’s first cloud designed specifically for data engineering. Running on AWS, Microsoft Azure, and Google Cloud, the new Trifacta Data Engineering Cloud provides a place for Read more…

Data Labs Look to Boost ‘Data Fluency’

As the role of the data scientist expands, so too does the “data lab” product category which seeks to merge data science with the enterprise plumbing required for data-driven decision-making. Add to the list a new Read more…

Prophecy Spins Up Low-Code Data Pipeline Tool

In recent years, the shortage of data engineers has at times exceeded the shortage of data scientists. To help close the gap, a Silicon Valley startup called Prophecy today unveiled a low-code data engineering tool that Read more…

2021: The Year of the Feature Store

Don’t look now, but feature stores--systems for developing, maintaining, and monitoring the data features used by machine learning algorithms for training and inference--are popping up all around us. Amazon Web Service Read more…

Snowflake Extends Its Data Warehouse with Pipelines, Services

Customers running atop Snowflake’s cloud data warehouse soon will find new functionality, including the ability to build ETL data pipelines , as well as the ability to expose pre-built analytic routines as data service Read more…

Data Transformer Fishtown Raises Funds

Fishtown Analytics, the data engineering tool startup, announced a $29.5 funding round this week to be used for further development of its open source analytics engineering tool. The Series B round was led by Sequoia Read more…

Zaloni Pivots to DataOps

Zaloni once was focused on helping customers to manage data in Hadoop. But under new CEO Susan Cook, the company has broadened its scope and is now aiming to help customers manage the entire supply chain of data, or what Read more…

Fivetran Launches Pay-As-You-Go Option for ETL

Fivetran wants to make it “stupidly simple” for customers to load data into cloud data warehouses, and judging from the company’s rapid growth, it seems to be working. Last week, the extract, transformation, and lo Read more…

Snowflake to Make it SNOW on NYSE

Rumors have been swirling for weeks that Snowflake confidentially filed for an IPO with the Securities and Exchange Commission. Yesterday, the Silicon Valley-based cloud data warehousing company made it official: It inte Read more…

Planning an ETL Proof of Concept? Here Is What You Need to Consider

Picture this: You are trying to track the sentiment of your product using first-party customer data, social data, and social listening data to determine the success of a new feature. Getting a daily report on this can he Read more…

MLB Hits a Home Run with BigQuery Migration

The Padres' Fernando Tatis Jr. may be delighting baseball fans with his energetic style of play during this COVID-shortened season. But there's plenty of excitement in the back office too, now that Major League Baseball Read more…

To Centralize or Not to Centralize Your Data–That Is the Question

Should you strive to centralize your data, or leave it scattered about? It seems like it should be a simple question, but it’s actually a tough one to answer, particularly because it has so many ramifications for how d Read more…

Python Dominates, Usage Survey Confirms

Data scientists, machine learning developers and data engineers are turning decisively to the Python programming language, according to a new study. An annual usage analysis released this week by O’Reilly Media also Read more…

AtScale Tackles Data Engineering with Virtualization Software

It’s often the prep work in analytics that kills you. Before any analysis can be run, engineers must assemble, transform, and standardize the data. In a world where data silos are proliferating in the cloud, on premise Read more…

Datanami