Follow Datanami:
March 25, 2014

What Color Is Your Data? Inside the Science of Data Visualization

Alex Woodie

Data is the raw material of the 21st century, the commodity at the base of a booming digital economy. Like oil and gold, people mine it, refine it, and combine it with other things. Data has become the most sought-after substance in the world. But please: Just don’t ask to see it.

For all the importance that data plays in our lives, it’s downright odd that it can be so hard to see. We know it’s there, tucked away in disk drives, stored on memory cards, or sitting up in the cloud. We understand that data exists on the computer as a series of tiny 1s and 0s, but that with the proper application, it’s presented to us in ways we understand, such as English words and Arabic numerals.

The big challenge with data is getting meaning out of it — driving it from unstructured to structured to something in context that matters. A log may tell you how many seconds visitors stayed on a particular webpage, but without context, that data is useless. To really understand what a data set tells us, we have to do something to do the data—combine it with other data or transform it into a more consumable shape. And the bigger the data set, the harder those tasks become.

One way to drive meaning out of ordinary linear data is by transforming it into colorful graphics and other visualizations that humans find easier to comprehend. This is the approach advocated by Tableau Software, the Seattle, Washington software company that was formed to commercialize the visual query language (VizQL) that co-founder Chris Stolte created in his doctoral thesis at Stanford University’s renown Visualization  Group (which has since moved to the University of Washington at Seattle).

Making Data Jump

Tableau recently put together a research team to push the state of the visualization art–to contribute to human’s knowledge of data visualization and to build compelling new features too. “We’re trying to innovate and do research, to work on this really audacious goal of helping everybody to see and understand their data,” says Jock Mackinlay, Tableau’s 

Tableau vice president of visual analysis Jock Mackinlay

and an information visualization expert who is the manager of the new team.

The team has experts from various fields, including experts in natural language processing (NLP), HPC, databases, and human color perception. Mackinlay himself did some of the early work on GUIs while at Xerox PARC in the 1980s. The research group’s activities range from working with Tableau developers to help ensure the products are easy to use, to writing academic research papers on, among other things, the latest thinking around the humble bar chart.

“There’s an incredible amount of work to be done to understand how people work with data and what techniques are effective and what techniques aren’t effective,” Mackinlay tells Datanami. “It is surprising, isn’t it, that bar charts, which have been around from the very beginning, are still a topic on which you can do academic research on and learn new things about?”

Color is another topic that evokes red passion among the data visualization Illuminati. Mackinlay’s group includes Maureen Stone, a world-renown human-color interaction expert who also worked at Xerox PARC. Stone, who also worked with the Stanford Visualization Group (which is now at the University of Washington), will help Tableau determine best practices for using color within data visualization.

“Mapping data to color is a rich, complex space,” Mackinlay says. “There are two major parts to it. If you’re trying to encode quantitative data with color then it’s all about making sure you have appropriate range of variation. And if you’re mapping categorical data to color, what you want are colors that are distinctive from each other so you can identify the different categories.”

SAP’s Project Fiori

Another software vendor that’s worrying about appearances lately is SAP. The German ERP giant hopes to improve the user experience of its many products as part of its Fiori project. “We haven’t gotten it right with user experience all these years,” says Sam Yen, global head of design and user experience at SAP. “It’s OK to bash the enterprise software experience. Overall as an industry it’s an area that quite frankly has been neglected for some time.”

SAP has started embracing colorful visualizations wtih its Lumira product

The stark blue and gray SAP screens started to be replaced with more colorful and user-friendly HTML5 and mobile designs as part of the first wave of Fiori screens last May. SAP is taking a slow approach to overhauling the 300,000 or so screens that are in use among SAP’s various ERP, analytics, and industry-specific products.

Yen is also working to improve the usability of SAP’s business intelligence and analytics products, including Business Suite powered by HANA. The company’s recent update to its SAP Lumira product are specifically geared toward helping regular business users make sense of large amounts of data. And thanks to HANA, analytics can be accessed by users directly within their application screens, eliminating the need to switch to a different system or call in somebody with a different skillset.

UX designers face a special set of problems—for example, helping a retailer figure out a way to enable humans to monitor a million KPIs. “We try as designers to simplify everything as much as possible because the point of what we’re trying to do isn’t to create a beautiful experience. The point of an enterprise system is to help people get their job done,” Yen says. “If that means doing more machine-to-machine interactions and just going from transactional mode where there’s a  human in the loop to something where we have more of a notification framework where humans are notified when important decisions need to be made, of course that’s the better way to go.”

Research Drives Development

Keeping it simple is another development tenet followed by the folks at Tableau. “People try and pack too much data into a single view and make it really, really complicated,” Mackinlay says. “What the visualization research community found was the best way to deal with that is to create multiple views and have some coordination between them. That applies to small data and big data both, and it’s really the core of what Tableau does.”

Tableau is at the forefront with its colorful data visualizations

At Tableau, there is a bit more of a focus on beauty, along with speed and simplicity. Mackinlay hopes his team’s research turns into products every time, but he realizes that probably won’t be the case. “Academic work needs to be done,” he says. “There’s an incredible amount of work to be done to understand how people work with data and what techniques are effective and what techniques aren’t effective.”

One area of research that has proven fruitful is around storytelling, which is an area that SAP is exploring too. Members of Mackinlay’s team helped to build the new “story points” feature that will debut with version 8.2 of the Tableau tool. “We’re dealing with a new medium here,” he says.

The idea is to align Tableau’s research with its product development goals, and hopefully things like “story points” will emerge along the way. “It’s intended to be research that’s consistent with our mission, which is to help people see and understand data,” Mackinlay says. “So even if it ends up with academic papers, there’s also a strong possibility that it could end up with some improvement to one of our features, or possibly whole new products that we might create in the future.”

Related Items:

Tableau Embraces R with Version 8.1 Update

Tableau Throws a Brick at Traditional BI

Big Data Unlocks the Secret Behind Women’s Shoes