In Search of Data Model Repeatability

(everything possible/Shutterstock)

Everybody wants to be data-driven–that much is clear. But that desire doesn’t necessarily translate into real business results, especially in competitive industries like ecommerce.

Data quality has long been a burr in the side of would-be data champions. The need to cleanse and normalize dirty and inconsistent data often consumes the lion’s share of the resources devoted to big data analytics and AI endeavors. Many millions have been spent devising novel ways to shortcut these steps, but data success often comes down to how much time and money a company is willing to spend to build it by hand.

Dan LeBlanc is quite familiar with this situation as it exists in the consumer brands space, particularly as it relates to online commerce. As the head of analytics for Provide Commerce, the $750-million company behind ProFlowers and several other brands, LeBlanc and his team had done the hard work needed to be able to call themselves truly data-driven back in the late 2000s.

“We really had dialed in how to understand lifetime value, channel of acquisition, inventory, what we needed to purchase, what we needed in stock,” LeBlanc says. “We knew about three weeks ahead of Valentine’s Day whether a certain sales channel is going to be profitable.”

Daasity says it helps customers move from the old model (top) to the Daasity way (bottom)

The company had built its own ETL system and ran its own data pipelines. It leveraged an on-prem IBM Netezza system to run the SQL queries and get the answers, but much of the value that Provide Commerce was able to extract from data came from its deep understanding in the various data models at play, and particularly from its expertise in transforming data coming from those systems so they can get consistent answers to questions about customer orders, inventory, and marketing, among others.

LeBlanc noticed something odd happening a decade ago, when members of his team at Provide Commerce, which was acquired by FTD in 2014 for $430 million, started work at other companies.

“They started coming back to me and saying, ‘Hey, I’m on this thing called Shopify and the analytics are terrible and I can’t do anything that I used to be able to do when I was at ProFlowers,” LeBlanc says. “They started asking me to kind of help them build their basic data infrastructure.”

LeBlanc obliged. He started working as a consultant to set up the data pipelines and analytics systems for some well-known Internet brands. Eventually, he came to realize that companies in the space were not only asking the same basic questions, but they kept running into the same core data transformation issues, particularly as it relates to the data models.

“We’re all trying to solve the same problems,” LeBlanc says. “We’re all trying to figure out how do I acquire customers? How much can I spend? How much inventory do I need? What are the products I should re-buy? What should I be marketing? And whether you’re $1 million in revenue or billions in revenue…fundamentally you’re trying to do the same thing.”

Getting accurate answers to these questions requires accurate data. The problem was that there were no pre-built data connectors available that would guarantee companies that they’re using accurate data. Companies essentially would need to build their own custom data connectors, as Provide Commerce had done.

LeBlanc noted a business opportunity here. Because the companies in the commerce space are mostly using the same systems for marketing, inventory management, point of sale (POS) processing, and other needs, he figured there was a market in building these shrink-wrapped connectors himself, along with a standardized data model.

That’s the core business model driving Daasity, which LeBlanc and his co-founders Chris Shimojima and Sean Corson hit  upon while founding the company in San Diego, California, in 2017. In addition to building the data extractors, which is the easy part of the ELT equation, Daasity also provides the business logic required to understand the data model for both source and target systems.

“The easy part is actually how do I get data from Amazon and my retail POS and my e-commerce systems,” LeBlanc tells Datanami. “The hard part is putting it together, because each of those data models are different.”

For instance, Amazon doesn’t even have the concept of a customer, “because their belief is they own the customer, not you,” he continues. “So if they don’t have the concept of customer, how do you create a measure of lifetime value? We had to go create the concept of a customer out of Amazon data, and now we merge it with our, say, Shopify data so you can actually compare lifetime value of a customer that’s on your website versus Amazon.”

Larger companies typically just bite the bullet and invest in their human data analysts to be able to generate the custom SQL code necessary to normalize the data and have a standard data model, LeBlanc says. However, smaller companies in the fast-moving ecommerce space are trying to avoid creating large, expensive data teams and want more of an out-of-the-box experience. Daasity doesn’t automate away all the data transformation work that companies must do, but it does take care of most of it, he says.

“What we’ve said is we’re going to take on ourselves mapping that data into a unified customer-order-product data model, so no matter what you’re selling on, we’re mapping for the connectors that we support. We’re going to map that into this unified schema,” he says. “Now you can go develop off this unified schema, so you know that it’s always going to look exactly the same. That’s kind of the secret sauce, because what usually people are doing in other companies is having to go and say, well, I’ve got different sources and I’ve got to go figure it out what this all is myself and I’ve got to develop that. And so they’re basically reinventing a common sort of concept of a customer order, product, inventory type data model.”

Dan LeBlanc is the CEO and co-founder of Daasity

LeBlanc says Daasity’s shrink-wrapped business logic is able to automate 90% of the work in creating standard business model, leaving data analysts to do about 10% of what they previously were doing. That last 10% can be handled through a configuration process in the Daasity product, LeBlanc says.

“We’re not just going to give you an extraction tool and tell you go hire a data scientist and go build it yourself,” he says. “What we’re going to try and do is say, here, brand, go get your analyst, but have your analysts actually spend time doing analytics versus managing pipelines and  building transformations, because you can leverage what we’ve got out-of-the-box, and then take that extra 10% and kind of build it yourself.”

So far, Daasity has attracted about 300 customers since it started selling its software-as-a-service (SaaS) ELT offering in 2019. Brands like Harper Wilde, American Giant, Vuori, and MANSCAPED are among companies leveraging Daasity’s pre-built business logic.

Daasity supports about 90 different source and target systems, focusing on the applications that are mostly commonly found in the consumer brands space. Currently, the company is seeing a resurgence in interest in brick-and-mortar retail outlets, which is spurring a need to integrate with those retailers’ systems. MANSCAPED, for example, is starting to sell its product in Target stores.

“We all got forced to buy online,” LeBlanc says, “but there’s still a lot of people who, that’s not the natural thing. They like to touch and feel.”

For data warehousing, Daasity supports loading data into AWS Amazon Redshift, Google Cloud BigQuery, and Snowflake, which is by far the most popular with consumer brands, LeBlanc says. On the data source side, it’s a select but growing list. “We continue to expand based upon new players coming out,” he says.

Daasity’s success is due in part to the relatively small number of systems that consumer brands use, which makes it possible to devise standardized SQL transformation code. LeBlanc notes that this business model would not work in a business-to-business environment.

Will it work in other industries? “Eventually I think there’s the possibility, but I’d say short-term there’s still enough for us to do in the consumer brand space,” he says. “We’re probably not going to expand into other verticals for quite some time.”

Related Items:

AWS Seeks an End to ETL

Can We Stop Doing ETL Yet?

50 Years Of ETL: Can SQL For ETL Be Replaced?