Follow Datanami:
October 31, 2013

Army Deploying On Big Data Battlefield

Isaac Lopez

While the Affordable Healthcare Act insurance exchange rollout has provided a stunning show of ineptitude that will serve as coursework for decades on how not to roll-out new technology, some of the political talk that government cannot handle technology of enormous size has been over stated. Case in point: The United States Army, which has gotten down to the business of big data – and business is good.

In recent years, the Army has had a tech surge of their own, getting down to the work of building a big data system of their own design called the Army Enterprise Management Decision Support System (EMDS). According to a report by GCN, the system is aimed at harnessing big data across the Army tech infrastructure in order to help commanders answer questions that were previously unsolvable.

In order to provide this “holistic common operating picture,” Lt. Colonel Bobby Saxon told GCN that the EMDS system gathers data from across 3,500 database systems, gathering it all into one place where analysis can be done. It’s the classic big data challenge that so many enterprises are faced with as it plans to make its own move into big data technologies.

So the Army must have a corps of Hadoop experts putting the elephant to its paces, right? Not so fast. According to the report, all of the data collection and analysis are being done without the well-known tools incubated and developed in the “big data” era such as Hadoop, HBase, Pig, and Hive.

Per GCN:

A variety of tools have been deployed to connect the data and assemble it so that it can be accessed and viewed via dashboards, including Oracle’s WebLogic Server, identity management, business intelligence and Endeca Discovery software; Composite Software’s data virtualization technology (now a part of Cisco); the Apache Tomcat application server; and Internet Explorer 9 and Firefox Web browsers… Through EMDS’ dashboards, senior management has a common picture of equipment, personnel, readiness and training activities. For instance, they’ll know if the 48th Brigade out of Georgia is about to deploy or the latest status of the 82nd Airborne. Prior to EMDS they would have had to pull that information from multiple sources on their own.

While the ability to collect, unify, and manage this level of data across such a vast infrastructure using tools that many in the tech industry have started writing epitaphs for is interesting, it’s not clear that these dashboards contain the deep predictive analytics piece that has become part of the calling card of big data systems. And looking at the tools being used, it’s all seems very structured and relational.

On that front, Saxon indicates that the Army, which he describes as “living in a world of PowerPoint,” is wrestling with unstructured data as much as the rest of the large enterprise world – a fact which could eventually lead to an Army to an installation of Hadoop (provided the elephant meets Army security muster). Saxon said his team is trying to figure out how to get access to siloed unstructured data – a real challenge in the Army’s security rich IT environment.

The other challenge that the Army is wrestling with, say Saxon, is the increasing need for real-time in its operations. “We live in a world of near real-time most of the time,” he said. According to the report, in many cases reports still need to be pulled together by hand overnight for next-day briefing reports.

Still, the progress that the Army seems to have made in meeting the challenges of the modern data age is encouraging. At the very least, it seems to have constructed a platform on which there is opportunity to grow.

We’ll be interested to learn what comes next as the military further integrates with the big data technology trend.

Related items:

The Chief Data Officer’s Time Has Come 

Splunk Pumps Up Big Data with Hunk 

SGI Aims to Carve Space in Commodity Big Data Market 

Datanami