Big Data • Big Analytics • Big Insight

April 28, 2014

Farmers Plant for Hyper-Local Forecasts with IBM’s ‘Deep Thunder’

Alex Woodie
ibm_logo.png

In the heat of the Georgia summer, farmers’ fortunes live and die by the weather. A farmer who correctly bets on a pop-up thunderstorm can save thousands of dollars in water, chemical, and labor costs. But bet wrongly, and his entire crop can perish. Now a select group of farmers in the state are placing their bets on big data and “hyper-local” weather forecasting with IBM and its Deep Thunder supercomputer.

Researchers with the Flint River Partnership and IBM are working on a six-month project to test the effectiveness of hyper-local weather forecasting on real-world farms in the Flint River basin in southern Georgia. By aligning irrigation to very precise weather forecasts and other data, including soil moisture sensors, the project could cut water use by 15 percent, serving as a model for wider adoption by agriculture.

While Georgia gets more rain than other parts of the country–especially California, which is in the midst of a major drought–its porous soil make farmers more dependent on irrigation than you might think. “Because of the soil type down here, folks like to say we’re about two weeks away from a drought,” says David Reckford of the Flint River Partnership. “That means that, after two weeks without rain, the crop is going to enter into a stressed state.”

No farmer wants to waste water, especially with the drought that drained aquifers in the Southeast several years ago still on farmer’s minds. But in addition to soil type, the peanut, cotton, and sweet corn farmers in Georgia have one other factor working against them—namely the weather.

Deep Thunder provided detailed weather forecasts for the 1996 Summer Olympic Games in Atlanta, Georgia

Unfortunately, weather predictions from the National Oceanic and Atmospheric Administration (NOAA) just don’t have the resolution or accuracy to tell a farmer in this part of the country whether he should water his crop, and by how much. That is led to the work that IBM is doing with its Deep Thunder supercomputer for the Flint River Partnership, which is composed of the Flint River Soil and Water Conservation District, the USDA’s Natural Resources Conservation Service, and The Nature Conservancy.

Superior Weather Prediction

IBM first started developing Deep Thunder in 1996 specifically to provide so-called hyper-local weather forecasts to clients in various parts of the world. The computer, which originally was based on an RS/6000 SP, got its nickname from fellow IBM supercomputer Deep Blue, which famously beat chess grand champion Gary Kasparov.

Deep Thunder has taken on many tasks over the years, including short-term weather forecasts for utility companies in the US, wind farm operators in China, and rice farmers in Brunei. The supercomputer was called in to help predict mudslides in Brazil following deadly storms there in 2010. Its weather-processing muscle also predicted that Hurricane Irene would be reduced to a tropical storm when it made landfall near New York in 2011.

For clients in the United States, Deep Thunder’s model typically starts with NOAA’s standard North American forecast, and then builds upon that with additional data sources. Whereas NOAA generates standard weather forecasts atop a 12-kilometer grid and one-hour increments going out 84 hours, Deep Thunder can generate forecasts with much greater accuracy–sometimes down to the one-meter level, and with much smaller time intervals as well.

Deep Thunder provides three-dimensional weather forecasts.

In southern Georgia, Deep Thunder had been programmed to help farmers align their activities with the weather in ways they currently cannot. The model is fed with a range of data pulled in from NASA satellites about terrain, land use, vegetation, soil moisture and temperature, and the surface temps of lakes and oceans. It gets vegetation and soil data from the USGS, and real-time weather information from thousands of weather stations in the area operated by the University of Georgia. It also gets additional ground, mid-level, and atmospheric weather data from NOAA weather baloons and the private Maryland-based company Earth Networks.

All this data is ingested into Deep Thunder, which in turn generates forecasts in the Flint River basin at a 1.5-kilometer resolution at 10-minute intervals going out 72 hours. That forecast is orders of magnitude more precise than what NOAA is offering, and more accurate too, says Lloyd Treinish, an IBM Distinguished Engineer and Chief Scientist for Deep Thunder.

“If you’re looking at thunderstorms, even if you’re looking at something with a 4-5 kilometer resolution every hour, you’ll miss the thunderstorm cell because it’s smaller than that,” Treinish tells Datanami. “It’s fast moving, so it passes over the area in less than an hour, and it could only be 1 kilometer in size. The resolution in time and in geography is really very important.”

But pop-up thunderstorms in the summer aren’t the only phenomenon where Deep Thunder will help farmers. “If you have a strong cold front that creates a large number of thunderstorm cells, the amount of the rainfall can still be very localized based on the topography, and then the soil and vegetation will determine how much moisture actually gets into the soil from the rain,” Treinish says. “You have to incorporate all those local features to get that information correct. That’s not something NOAA focuses on, because they’re not looking at a local-scale problem. They’re looking at a country-wide problem.”

Depending on the Weather

The amount of data Deep Thunder is processing for the Flint River basin may not qualify it for big data status, but the variety of the data certainly does. “Big data is in the eye of the beholder,” Treinish says. “It’s not so much the aggregate volume, it’s the diversity. There are many different data coming from many different sources, each of which tells us a bit about the story of what’s going into the weather that we will be forecasting.”

Treinish and his team are experts at predicting thunderstorm, and that takes a lot of data. “Weather is a three dimensional dynamic phenomenon with multiple variables,” he says. “Our calculations go up to a height more than 20 kilometers over the earth’s surface. You need to calculate that if you want to figure out how much rain is going to hit the ground.”

The Flint River valley has been at the cutting edge of agriculture technology for some time. Variable rate irrigation, for example, is a technology that was pioneered here. Most of the world’s center-pivot irrigation systems are utilizing this technology. More recently, the Flint River area has been pushing the state of the art in soil moisture monitoring, which also plays a role in the Deep Thunder project.

Lloyd Treinish, an IBM Distinguished Engineer and Chief Scientist for Deep Thunder

Having a more accurate weather forecast is one thing, but turning that forecast into something farmers can put into action is another challenge. That’s where the Flint River Partnership comes into play. “The data needs to be actionable,” Reckford says. “If it rains half an inch on a field, what does that actually mean? Is it a benefit? Does it actually not affect soil moisture?  What if you get an inch or five inches of rain? What does that do to your decision making?”

The 40 or 50 Flint River farmers who are participating in the project have access to Deep Thunder’s reports via a Web portal. That portal shows them two main products, including a computer animation of the forecast, and a detailed plot-by-plot forecast across the various metrics in 10-minute increments. Over the course of the next six months, IBM and the Flint River farmers will be working on ways to simplify the absorption of all that data for today’s busy farmer.

The hope is that the combination of Deep Thunder’s forecasts, the soil sensor network, and variable rate irrigation equipment will largely automate the irrigation process and make it entirely a data- and forecast-driven process, Reckford says. “By September 30, we will have achieved full automation,” he says.

But deciding when to irrigate and by how much is just part of the equation. For example, Deep Thunder’s ability to forecast wind events and precipitation rates will also inform farmers about other critical processes, including planting, feeding, and harvesting. “You’re essentially changing your entire modus of your operation based on a weather forecast,” Reckford says. “The potential transformation in an agricultural context is really exceptional.”

Early predictions have Deep Thunder helping should cut their current irrigation volumes by 10 to 15 percent, which translates into tens of millions of gallons of water savings per farm per year. Reckford and Treinish say they hope the project will be renewed and that they’ll be able to refine what they’ve learned into something that resembles a more finished product.

Related Items:

A Weather Forecast Specific to your Back Yard?

Amazon Hosting 20 TB of Climate Data

Predictive Analytics Prevailing on the Wind Farm