Follow Datanami:
July 31, 2014

Police Push the Limits of Big Data Technology

Big data is changing the way business is done across industries, from financial services and banking to retail and mining. As data analytic technologies mature, state and local law enforcement agencies are now using the types of sophisticated analytic tools that previously only the biggest federal agencies could afford. The question on many American’s minds is how far law enforcement will go.

Technological advances are giving local and state agencies a variety of new data sources and analytical tools to help them solve crimes, and even to prevent new crimes from occurring. Lucrative new data sources include mobile data from smartphones, social media, and automated license plate readers, which can all be used to link people to crimes and crime scenes. Geographic information systems (GIS) have been widely adopted in law enforcement, but they just keep getting more powerful every year. Lately, some agencies have started using predictive algorithms to identify and warn people deemed most likely to become criminals.

In Los Angeles, the LAPD is using a suite of big data tools from Palantir to integrate a range of existing data sources–from case management tools and rap sheets to license plate readers and DMV records–thereby providing a new layer of intelligence atop previously disparate structured and unstructured data sources.

According to LAPD Police Chief Charlie Beck, the Palantir software helps “make sense of all the noise” coming into the 10,000-employee department. “For years we’ve had stovepipe systems that have a lot of information but don’t talk to each other and don’t compare that information,” he says in a YouTube video. “Palantir allows us to do that that. It’s a really important Charlie_Becktool for instigators…The end result is will allow us to keep LA safer and allow investigators and police officers to do their job better.”

Since implementing Palantir, the crime rate in the City of Angels has dropped. However, the crime rate has been dropping for the last 10 years, so it’s unclear what impact Palantir’s software had.

Other big city departments are tapping into big data too, including the New York Police Department, which teamed up with Microsoft in 2012 to implement what’s called the Domain Awareness System (DAS). The DAS integrates police databases, closed captioned TV footage, radiation sensors, license plate detectors, and public data streams. Like the LAPD, the NYPD needed a single interface to traverse all these systems.

Since the 9/11 attacks, the NYPD has grown to resemble a small army, complete with a formidable intelligence agency. As part of its intelligence-gathering efforts, the NYPD collected personal information, including names and addresses, from everybody that officers came into contact with as part of its controversial “stop and frisk” policy, or about 4.3 million people. However, the department was forced to stop using that database a year ago when a judge ruled it unconstitutional. Today, the department discards names and address of people when complaints are dismissed or they involved in noncriminal matters.

The Chicago Police Department is also pushing the state of the art (and ethical boundaries) of big data with its so-called “heat list,” which is an index of the 400 or so people in Chicago who are most likely to be involved in violent crime. The CPD system utilizes elements of graph theory and predictive algorithms, and was developed in part based on a Yale University sociology expert’s research into Windy City crime patterns, according to a recent story in The Verge. The fact that gun violence and drug use spreads in a geographic manner shouldn’t surprise anybody, and CPD is using those repeatable patterns for its proactive policing strategy.

Predictive analytics are nothing new in law enforcement. But the CPD takes it a step further and alerts people to the fact that they’re on the heat list and the legal ramifications of crimes they have not yet committed. It’s all part of the city’s Violence Reduction Strategy (VRS), which it implemented in July 2013 and will continue indefinitely, writes the CPD’s Superintendent Garry F. McCarthy.

“It’s been rewarding to watch as law enforcement has started to embrace this idea of intelligence-led policing,” says Mike King, a former police officer who works in the law enforcement division of GIS developer Esri. “They’re taking data that they’ve very effectively collected for years–everything from suspect information to locations where crimes are occurring to the location big data crime mapwhere assets are currently sitting–and putting it into a understandable format so law enforcement officers in a command center or a police car can make decisions that are based on the most important and up to date information that’s available.”

Esri’s GIS suite incorporates analytic tools that help law enforcement agencies make sense of a range of data, ranging from arrest records and license plate readers to location information gleaned from cell phones and social media.

Social media in particular is poised to play a much bigger role in how police interact with citizens, not just in criminal investigations but in reporting crimes too. During the 2013 shooting in Aurora, Colorado, many more witnesses posted to Twitter rather than make calls to 911, King points out.

“There’s a growing number of police departments who are making social media part of intelligence processing,” he tells Datanami in an interview. “The levels of success are dependent on a bunch of different factors, like if location is turned on. There are some mathematical algorithms we can put in place to try to weight tweets. But in cases where people have turned off location information, it makes it much more difficult.”

Just as retailers are using sentiment analysis to gain an edge in social media, police departments are using every legally available tool to keep them abreast of what’s happening in the online world. In some cases, that involves exploiting peoples’ failure to properly configure privacy settings.

“There’s a weird psychology involved with people who are deeply embedded in social media,” King says. In some cases, that leads people to broadcast their criminal acts to the entire world. So if you take part in trashing somebody’s house during a party or arrange a drug deal online, don’t be surprised if you see a man in blue knocking at your door soon thereafter.

Police aren’t interested in violating people’s rights and becoming an all-knowing Big Brother, King says. “We’re going to continue to test boundaries and I believe that’s what makes the Untied States such a great country,” he says. “Maybe the tactics and technology we’re using today won’t be used tomorrow as the courts refine and clarify things. But I don’t see it changing the way law enforcement goes after this data.”

Related Items:

Fighting Crime with Big Data

IBM Flushes Out Fraud with Big Data Analytics

How BI is Becoming Crime’s Biggest Enemy