Follow Datanami:
December 12, 2017

Heap Helps Avoid ‘Adobe Blackouts’ with Hosted Analytics

Heap today unveiled a new iteration of its cloud-based analytics offering that’s designed to automatically ingest, normalize, and analyze user interaction and customer behavior data from a variety of sources. What’s more, Heap says its new “Insight Platform” can help companies avoid the so-called “Adobe Blackout” period.

Heap CEO and co-founder, Matin Movassate, spent time at Facebook, where he managed the Facebook Messenger product. However, he discovered how difficult it was to stay on top of user metrics and KPIs that were so critical to generating insights about product usage.

“Not only was it time consuming and laborious to spec all that out and write all that tracking code and schematize that,” Heap’s other co-founder, Ravi Parikh, tells Datanami, “but what ended up happening is, as soon as that feature got shipped, they would end up seeing that users were interacting with these features and products in ways they didn’t anticipate.”

That sent Movassate and the Facebook Messenger team back to the drawing board to begin tracking the use of features that they didn’t initially realize they needed. That sucked up valuable engineering resources, and also slowed down the process of extracting valuable insights about how users are interacting the product.

“Matin realized the main barrier to him doing his job was not engineering talent or the amazing data science team he had at Facebook,” Parikh says. “Despite those resources, he wasn’t able to do the job effectively because the data was never there when he had the question that he wanted to ask. Those things always took a lot of time and effort and janitorial work to answer rather than spending time on the interesting questions.”

All In One

In 2013, Movassate and Parikh founded Heap Analytics with the goal of building an all-in-one, end-to-end data collection and analytics system that would simplify life for companies looking to squeeze valuable insights from user interaction data connected to their Web and mobile properties.

Heap claims it can start collecting data from a website or mobile app in as little as 5 minutes

The result is the Heap Insight Platform, a hosted analytic offering aimed at companies who want to track user behavior data, predominantly from websites and mobile apps but increasingly through interactions with physical properties (more on that in a sec).

Simplicity is the name of the game for Heap. Customers only need to insert a few lines of code into their web app (or a simple library for a mobile app) to begin collecting user data. And they don’t need to worry about whether they’ve instrumented the data collection correctly, because the company collects everything.

“We’re storing literally everything people do,” Parikh says. “We’re not pre-specifying what people want to track. We’re storing everything and then analyzing after the fact what are the important things.”

The key to Heap’s “store everything” approach is its virtualization layer. The company collects all the user interaction data, and then uses an ontological system to represent the data into something that’s easier to control and to query. (A good compression algorithm also helps to minimize Heap’s AWS storage bill.)

When a business user wants to ask Heap a question, they’re presented with a WYSIWYG screen, where they can define the action they want to take on the website or mobile app. “I go to the Heap UI. The user will apply a semantic label to that button or whatever it is they care about,” Parikh says. “Then Heap will look at that definition they created and apply that retroactively on top of the raw data we collected.”

Once Heap knows what the business analysts wants to see, it presents a range of different analytic options, including funnel analysis, retention analysis, and cohort analysis. “Whatever they do, that is all applied on top of those semantic definition they’ve created,” Parikh says. “It’s done in a way that’s very accessible to business users. It doesn’t’ really necessarily require an understanding of the underlying code or structure of the website or mobile app.”

Blackout Period

Heap provides an array of pre-built analytics, and also allows users to download the processed data to work their own on-premise tools

This approach also helps to minimize the time it takes to re-engineer the data instrumentation when a website or mobile app is updated, and to reconcile the old data with the new data. This is what Heap employees term the “Adobe Blackout,” in reference to the tech giant’s popular Web analytics platform (which formerly was called Omniture).

“Everyone strives to be data-driven, but it’s almost always an unfulfilled promise,” Movassate says in a press release. “As companies try to become agile and iterate quickly, they rapidly encounter what they call the ‘Adobe Blackout,’ where any defect or reskin results in lost data that has to be reconciled, costing teams weeks or months. Heap virtualizes the underlying data structure, enabling companies to flexibly rewire analytics without losing any data integrity.”

Today the company announced a new iteration of the platform that seeks to incorporate additional data sources beyond Web and mobile apps, such as consumer purchase data from a physical store. “A typical challenge in retail might be tying online behavior to offline purchasing activity, which is typically pretty difficult because that offline purchasing behavior and instore purchasing behavior might live in a different data silo or a transactional data store that’s completely divorced from the Web analysis,” Parikh says. “So it’s our goal to tie those together.”

With 15 new connectors into data sources such as Salesforce, Marketo, email and payment providers, and website optimization software, the company is closer to connecting the dots on the data.

“And as a result,” says Heap CMO Shawn Hansen, “you can ask these questions that are almost impossible to answer, things like ‘How does my new feature that I just launched affect customer satisfaction? Or how does my new product affect my customer retention rate or customer lifetime value?'”

Those question can be answered by munging data from three to four different stovepipes in organizations, Hansen says. “I spent years trying to build high-end petabyte scale data warehouses to solve the problem,” says Hansen, who joined Heap from Microsoft. “But most organizations don’t have the budget or the skills to pull that off. And even if they do, it’s fragile and it takes a lot of maintenance.”

By automating the data collection and the ETL work, Heap eliminates the 80% of data janitorial work for these types of analytic projects. “Most major leaps forward in technology have come as the result of someone abstracting away the complexity,” Hansen says. “I joined Heap because I felt it was a real quantum leap forward in automating away all the underlying pain. It’s not a fancy dashboard, or a fancy widget or visualization layer. It’s really focusing on core infrastuctre, which is where most people spend their time.”