Follow Datanami:
January 19, 2016

How This Instrument Firm Tackled Big Data Blending

Thanks to the ongoing digitalization of the world, we’re constantly awash in data of all types. From structured data like sales reports and customers list to semi-structured data like photographs and clickstreams, nearly everything that people do can be reduced to bits and bytes for storage and analysis on a computer. But getting a handle of this expanding stream of data is a big challenge, and the way that one manufacturer of high-end instruments solved it may surprise you.

National Instruments (NASDAQ: NATI) is an Austin, Texas-based provider of high-end hardware and software that engineers and scientists use to solve a variety of problems. From digital multi-meters to arbitrary waveform generators, its products are used in everything from the jet engine manufacturing and oil exploration to telecommunications.

Like most companies its size ($1.2-billion in annual revenue, 7000 employees), National Instruments runs an enterprise analytics group that’s tasked with helping it make the best decisions. The group work is tasked with a variety of jobs, including analyzing customer behavior for the research and development organization to creating share of wallet calculations for the sales and marketing, among others.

Recreating the Wheel

Overseeing much of that work is James Lewin, NI’s principal business intelligence analyst. According to Lewin, the 20-person enterprise analytics group has demonstrated “pockets of brilliance,” but it has struggled to take the data-driven success enjoyed in one area and replicate it in others.

shutterstock_colorful boxes_whitehoune


“We’ve had analytical silos where people who are subject matter experts get very focused,” he says. “But at times we’re not efficient, because the same type of information was being pulled, or metrics were being created to solve one individual groups’ information need, that could have been leveraged by others. So it would essentially recreate the wheel.”

Lewin found it difficult to keep his group’s data warehouse primed with refined data required to serve his constituents at NI. It wasn’t so much the data volume and variety that troubled Lewin (he reports using between 50 and 100 different data sources), but ensuring that the data is consistently structured and applicable from one use case to another.

“We spent a lot of time acquiring and transforming data to answer questions, and to bridge that gap between the data warehouse and data sources,” Lewin says. “With limited resources and long release cycles, it takes a lot to make changes and add things…to the data foundation or data environment.”

The company has a strategy to consolidate its data into one of three types: foundational, transactional, and informational. While the different data sets will measure different things, NI wants to ensure there are “hooks” between them that allow the company to join the data together in such a way that they can tease out any hidden connections that may exist.

In essence, NI wants to burn down the silos separating the data, and replace it with a bigger data tent where connections can be easily created and consumed. “This is all part of what I’ve developed as an analytical framework where you’re linking all these functional areas together, figuring out where those bridges are and how those relationships exist, and how they tie to your major KPIs,” Lewin explains.

Big Data Blender

The company set out to find a tool that could enable this type of enterprise data transformation and blending strategy.  The software it selected, from Alteryx, takes a somewhat unique approach in that it provides a visual representation of the data transformation process. Users can acquire, transform, and blend multiple data sources essentially by dragging and dropping icons on a screen.

This GUI approach is beneficial to NI employees who aren’t proficient at manipulating data using something like SQL. “Besides our SQL gurus, it really enables the analysts to get in there and raise their ability levels by pulling these engineering data sets that answer their specific needs, as well as to leverage things that were built as a group,” Lewin says.

NI_2NI runs its data warehouse on the Oracle database platform and uses Oracle Business Intelligence Enterprise Edition (OBIEE) to analyze that data, as well as Tableau Software‘s visualization tools. It also previously used IBM‘s Modeler, previously called SPSS Clementine, to hammer structure into data.

The addition of Alteryx to this mix is seen as a force multiplier for NI. “Alteryx does a great job blending,” Lewin says. “We leveraged that to build out not only these analytic data sets, but also these combined results that then get pushed into a tool like Tableau to do the visualization.”

While Tableau supports some data blending capabilities, Lewin prefers to use a tool that was designed for data transformation. “Based on my experience, it’s easier for me to build it and move it upstream using a tool like Alteryx,” he says, “and let Tableau focus on doing the visualization and storytelling. We leverage Alteryx to build out those data sets and do the heavy-duty blending.”

Enabling Analytic Agility

NI is using the desktop version of Alteryx at the moment, as its workload doesn’t (yet) require the scalability and power that’s enabled in the server version. Just the same, the software is having a noticeable impact on the productivity of Lewin’s group, which includes a handful of data scientists who work within R.

“Alteryx allow us to create these prototypes and vet the solution out with the business…in a very quick fashion, versus trying to ask IT to persist something or create something that’s going to allows us to answer the same questions,” Lewin tells Datanami. “That’s saved us a lot of effort and money around engaging an IT resource without having a full, complete picture of…what we’re trying to get after.”



Even though it’s running on just a single desktop machine, Alteryx is helping NI get a handle on its data, and providing a way to transform and blend the data in a standard and governed manner. The capability to quickly model new combinations of data from a GUI, and then execute that model as a workflow that can be called as needed, gives NI the confidence to ramp up its analytic ambitions.

“It allows me to actually grow the amount of data that I’d go after and pull together, and also expand it in breadth and depth, because I know I’m going to add more fields and I’m going to add more records, because I can process it faster than other solutions previously,” Lewin says.

“The goal is to answer 80 percent of the questions that we’ve faced in the past and that we anticipate we’ll get in the future,” he continues. “There’s always going to be a handful of questions of why or how did this happen. We we’re trying to anticipate that by building these data sets, and providing those enhancements by using a tool like Alteryx so we can answer them.”

Related Items:

Alteryx Adds In-Database Blending

The New Data Blending Mandate