Taking the Pain Out of Buying and Selling Data
We’re well into big data’s second decade, and we’ve made a ton of progress on many fronts. We have cloud-based systems with infinite storage capacity, sophisticated machine learning software that improves by the month, and powerful clusters turbo-charged with GPUs. Increasingly, what differentiates big data outcomes is the quantity and quality of the data we use, which typically means going offsite for additional sources. So why does buying and selling data have to be so hard?
That’s the question that Narrative founder and CEO Nick Jordan found himself asking after going through the rigmarole of the data buying process with a company that bought and sold millions of dollars’ worth of data every year.
“I actually looked for a solution to my problem, because it was a giant pain in the ass on both sides, frankly,” Jordan says. “What I found at the time was a bunch of data brokers, a bunch of people who would sit between buyers and sellers and say ‘Don’t worry, well make it easy.’”
But instead of improving life for buyers and sellers of data, these data brokers mostly were out for themselves, Jordan says. The brokers marked up the cost of data, created opacity between the buy and sell sides, and generally did their best to put themselves in the driver’s seat as the digital middlemen – to the detriment of the actual buyers and the sellers.
“When somebody can basically sit in the middle and say, hey this stuff is so hard, we can make a business entirely out of almost arbitraging the underlying asset–to me, that just shows what an immature and inefficient market it was,” Jordan says.
It also presented a business opportunity, which Jordan is hoping to fill with Narrative, the data streaming platform that he founded five years ago. Like the data brokers, Narrative sits between the buyers and the sellers of data. But unlike typical data brokers, Narrative’s goal is to build transparency, automation, and trust on both sides of the data equation.
“The idea is to build a system that takes all the manual and inefficient bits of buying and selling data and make them more efficient through automation, standardization, and normalization,” Jordan says.
Narrative’s SaaS-based application provides a platform to connect buyers and sellers. On the buy side, it helps companies acquire and integrate second- and third-party data, typically for the purpose of AI or analytics. On the sell side, companies that license Narrative’s software have a mechanism for reaching multiple buyers in an orderly and streamlined fashion.
There’s a lot of work that goes into buying and using, on both sides of the equation, according to Jordan. There are all the usual questions about the format that the data takes (CSV, Parquet, JSON, etc.), the units of measurement (Imperial or metric, GPS or UTM) and the frequency at which it will be refreshed (monthly, daily, hourly, etc.).
Once data scientists or analysts have studied a sample of the outside data and decided that it will work for their particular activity, then data engineers are called in to build the ETL pipelines to move the data, which can often take months.
On top of the logistical questions, there are legalities that must be taken into account. Buyers and sellers both must take measures to assure that they’re not violating regulations for their particular geography. Finance teams typically gets involved to obtain usage data and make the payments. And if anything changes to the data or the contract, all the engineers, analysts, data scientists, lawyers, and finance folks get to drop whatever they’re doing and revisit the matter.
Anybody who says that buying data is easy “are clearly people who have never bought data,” Jordan says. “It takes a village to buy data, so the goal is to give tools to all of those personas to make all of their jobs easier, so it’s not the cluster that it is today.”
Companies that buy data through Narrative can build data pipelines that combine datasets from multiple sources into a single, unique stream of data, which can be pushed to a data lake, data warehouse, or specific application. The company has a range of sellers on its platform today, offering many types of consumer data, including demographic data, geo-location data, behavioral data, and device data, among others.
The New York City-based company is an AWS shop, and leverages a host of big data technologies, including Kinesis, Dynamo, and Spark, to facilitate the movement of large amounts of data. Besides the big data tech, the real advantage of Narrative is the taming of all the other questions, including the management of the relationship with data providers.
One Narrative customer is a hotel chain that needs to know the wine preferences of its clients, so that they can find their preferred variety waiting when they get to their room, Jordan says. The company currently has around 40 data partners selling data, and many more clients buying data, in some cases into the millions of dollars.
Narrative recently added the credit bureau TransUnion to the sell-side of the equation. TransUnion has a long history of providing high-quality data about nearly every American consumer, and that data is now on tap in the Narrative platform, which is targeted at Fortune 2000 firms.
Narrative, of course, isn’t the only company striving to simplify the data buying and selling process. Data marketplaces are popping up all over the place, which is a good thing. The more standardization that comes to this market, the better off we will all be.
April 9, 2021
- Alteryx Global Inspire 2021 Conference to Showcase New Products in Analytics and Data Science
- Trifacta Announces New Community and Certification Programs for Data Workers
- Gary Hagmueller Joins Dgraph Labs as CEO
- Elastic and Confluent Partner to Develop Enhanced Experience for Kafka and Elasticsearch Users
- New UK Fellowship Programme Will Fund 6 Fellows in AI and Data Science to Support Life Science Research
April 8, 2021
- Full Agenda Released for TigerGraph’s Graph + AI Summit 2021
- AWS Announces General Availability of Amazon Lookout for Equipment
- Collibra and EVO Banco Boost Data-Driven Digital Banking
- Swarm64 and EDB Partner to Extend EDB Postgres Advanced with Faster Query Performance
- Alation Delivers Cloud-Based Platform for Data Intelligence
- Comet Raises $13M Series A for Model Development and Monitoring
- Graphistry and Pavilion Partner to Accelerate Graph Analytics Using RAPIDS and NVIDIA GPUs
- New Services in SAP HANA Cloud Lower TCO in Data-Intensive, Highly Regulated Industries
- Snorkel AI Launches Application Studio, Raises $35M Led by Lightspeed Venture Partners
- Streamlit Transforms How Data Scientists Share Data, Raises $35M Led by Sequoia
April 7, 2021
- OmniSci Announces Opportunities to Accelerate Geospatial, GPU Database Knowledge At GTC 2021
- MinIO Enables IT to Manage Kubernetes-native Object Storage
- Ataccama Report: Nearly 8 in 10 Businesses Struggle with Data Quality
- MANTA Partners with Neo4j to Provide Enhanced Graph Technology for Data Pipeline Analysis
- Qlik Collaborates with AWS to Accelerate Cloud Analytics with SAP Data
Most Read Features
- Big Data File Formats Demystified
- Synthetic Data: Sometimes Better Than the Real Thing
- Experts Disagree on the Utility of Large Language Models
- A ‘Glut’ of Innovation Spotted in Data Science and ML Platforms
- He Couldn’t Beat Teradata. Now He’s Its CEO
- Is Python Strangling R to Death?
- Why Data Science Is Still a Top Job
- Who’s Winning In the $17B AIOps and Observability Market
- Big Data Predictions: What 2020 Will Bring
- What’s the Difference Between AI, ML, Deep Learning, and Active Learning?
- More Features…
Most Read News In Brief
- Can PHP Script a Big Data Comeback?
- Data Prep Still Dominates Data Scientists’ Time, Survey Finds
- AWS Adds Explainability to SageMaker
- Global DataSphere to Hit 175 Zettabytes by 2025, IDC Says
- AWS Tackles Real-Time Data Transformation with S3 Object Lambda
- The AI Inside NASA’s Latest Mars Rover, Perseverance
- The Union of Salesforce, Tableau Yields Hybrid ‘Business Science’
- Databricks Edges Closer to IPO with $1B Round
- Data Salaries Get a COVID Bump
- Informatica Accelerates DataOps with Spark, GPUs
- More News In Brief…
Most Read This Just In
- Moody’s Analytics Wins Award for Best Use of AI in Banking or FinTech
- Aiven Raises $100M Series C to Expand Global Open Source Innovation
- Alluxio Advances Analytics and AI with NVIDIA Accelerated Computing
- GrafanaCONline Returns June 7-17, CFP Is Open Now
- AWS Announced Strategic Partnership with Hugging Face NLP Startup
- y42 Raises $2.9M to Provide a Scalable and Affordable Data Stack to Companies of All Sizes
- ThoughtSpot Acquires SeekWell to Operationalize Analytics, Push Cloud Data Insights to Business Apps
- PrivaceraCloud’s New Integrations Automate Identity Management Across Multi and Hybrid-Cloud Environments
- New Tool Increases Transparency and Understanding into Machine Behaviors
- MinIO Enables IT to Manage Kubernetes-native Object Storage
- More This Just In…