Tag: Data engineering
Are Data Engineers Sleepwalking Towards AI Catastrophe?
Since the earliest days of big data, data engineers have been the unsung heroes doing the dirty work of moving, transforming, and prepping data so highly paid data scientists and machine learning engineers can do their t Read more…
Snowflake Widens Analytics and AI Reach at Summit 25
Snowflake unleashed a torrent of news today at its annual user conference, ranging from a new adaptive compute capability and Openflow data integration to new analytic capabilities and AI enhancements, such as AI SQL and Read more…
Google Cloud Cranks Up the Analytics at Next 2025
Google Cloud made a slew of analytics-related announcements at its Next 2025 conference this week, including a range of enhancements to BigQuery, its flagship database for analytics. BigDATAwire caught up with Yasmeen Ah Read more…
Will GenAI Modernize Data Engineering?
Generative AI is being asked to do many things, from replacing call center workers to enhancing developer productivity. There’s also an intriguing possibility that it will supercharge data engineering, if a new survey Read more…
Overcoming the Financial Implications of Poor Data Quality
Approaches to data quality vary from company to company. Some organisations put a lot of effort into curating their data sets, ensuring there are validation rules and proper descriptions next to each attribute. Others co Read more…
Denodo Brings Data Management to Academia, the Cloud
Data management is hard, and it’s not getting easier, thanks to the continued tsunami of big data (a data-nami, as it were). One company that’s aiming to help with the continued digital onslaught is data virtualizati Read more…
Astronomer’s High Hopes for New DataOps Platform
Astronomer last month rolled out a new observability product called Astro Observe that’s aimed at giving customers the full picture of how their data is flowing using Apache Airflow, the open source data orchestration Read more…
Databricks Unveils LakeFlow: A Unified and Intelligent Tool for Data Engineering
Data engineering is a cornerstone for the democratization of data and AI. However, it faces significant challenges in the form of complex and brittle connectors, difficulty in integrating data from disparate and often pr Read more…
Operational Data Warehouse: Power Real-Time Business Processes
Sponsored Content by Materialize
Have you ever tried driving real-time updates to customer-facing applications? Or optimized a conversion funnel using streaming data? If so, you’ve probably suffered the pain of pushing your data warehouse past its l Read more…
Data Observability in 2024: A Guide
In today's data-driven world, data observability is a critical concept for organizations aiming to effectively manage their data. Simply put, it means having the ability to constantly monitor and understand the status of Read more…
Data Engineering in 2024: Predictions For Data Lakes and The Serving Layer
The data landscape experienced significant changes in 2023, presenting new opportunities (and potential challenges) for data engineering teams. I believe we will see the following this year in the areas of analytics, Read more…
How Airflow 2.8 Makes Building and Running Data Pipelines Easier
Apache Airflow is one of the world’s most popular open source tools for building and managing data pipelines, with around 16 million downloads per month. Those users will see several compelling new features that help t Read more…
Mastering Data Modeling: Insights from a Data Product Developer
Let's not deny it; we've all been captivated by the elegant symmetry of Data Products. If you haven't encountered them yet, you might be living under a rock. But don't worry, before we delve into advanced solutions, let' Read more…
Snowflake Gives Everybody a Little Something at Summit
Whether you’re a data engineer building data pipelines, a data scientist creating AI models, or a CFO trying to minimize cloud spending, Snowflake gave you something today at its annual user conference in Las Vegas, Ne Read more…
Numbers Station Sees Big Potential In Using Foundation Models for Data Wrangling
A startup called Numbers Station is applying the generative power of pre-trained foundation models such as GPT-4 to help with data wrangling. The company, which is based on research conducted at the Stanford AI Lab, has Read more…
Meet Maxime Beauchemin, a 2023 Person to Watch
When it comes to prolific contributors to open source projects in the big data space, Maxime Beauchemin is definitely somebody you should know. As a data engineer at Airbnb, Beauchemin created multiple tools that he subs Read more…
Top 12 Datanami Stories of 2022
With another year almost behind us, it’s time to sit back and consider what we’ve just been through. It’s been another active 12 months in the big data space, with plenty of news for the intrepid big data reader. Read more…
Dataiku 11.1 Update Boosts Data Science and MLOps
Dataiku has unveiled the latest update to its data science and machine learning platform, Dataiku 11.1. This update includes improvements to existing capabilities as well as new features for data scientists, ML engineers Read more…
What’s Holding Up Progress in Machine Learning and AI? It’s the Data, Stupid
The lack of a solid data foundation and solid data workflows is preventing companies from making more progress with machine learning and AI, according to a new Forrester Consulting survey conducted on behalf of Capital O Read more…
No-code Analytics System Redbird Nabs $7.6M Seed
Redbird, a New York-based enterprise analytics operating system, announced it has raised $7.6 million in an oversubscribed seed round. The Redbird platform allows non-technical users to automate and unify analytics wo Read more…
Leading Solution Providers
Tabor Network
