How Big Data Helps Predict the Weather – Plus Your Weekend Reads

How many times a day do you check the weather? Whether it is on your phone, computer, radio or TV – the need for weather data is nearly constant. In the past few years the scientists at NOAA have pioneered numerous hurricane and wind speed prediction models that have become the bedrock for the nation’s weather forecasters.

The results from the models allow us to make critical decisions about evacuations and other preparations for severe weather events, helping protect billions of dollars in property and save thousands of lives. The man behind the models are Mark DeMaria.

He is a Supervisory Meteorologist at NOAA’s Center for Satellite Applications and Research in Fort Collins, Colorado. His work has made him a finalist for the Service to America Medals – the Oscars for federal employees.

DeMaria told Chris Dorobek on the DorobekINSIDER program that while there is still a lot of room to grow, forecasting models have come along way in the last 10 years.

“The ability to track where a storm is going to go have improved tremendously over the past 10-15 years. The accuracy is twice what it was 15 years ago. The skill of the 4-day weather forecast is what the 2-day cast was a few years ago,” said DeMaria. “The new models give you more lead time and allow forecasters to eliminate some of the false alarms.”

Why Are Storms So Hard to Predict?

“The atmosphere varies from very small things like wind gusts to larger scale things like hurricanes, to things that cover the whole plant like the jet stream. All of those things interact with each other and you need to model them at the same time to get an accurate weather forecast,” said DeMaria.

Why Is NOAA Necessary? We have the Weather Channel?

“There is a bit of a misunderstanding because most people’s interface into weather is the media. The average person dosen’t understand the tremendous amount of infrastructure that goes behind those weather forecasts. TV stations don’t own satellites, they don’t have the budget to maintain them. They don’t run their own computer models, that in itself is a gigantic enterprise that really only the federal government can and is willing to support.”

Big Data

“You need these tremendously large data sets to forecast. We use a grid of points that covers the entire globe, where individual stations are only a few miles apart. Plus there are a bunch of data sets from the ground all the way up to 50 kilometers in the atmosphere. How fast the wind is blowing, what the temperature is, the moisture level, we need that data from pretty much everywhere in the atmosphere over the whole earth. If you can’t measure those stats constantly then you are going to have errors in your models.”

Room to Grow

“There is still considerable room to improve. One of big limitations right now is the ability to forecast whether a storm is going to get stronger. Superstorm Sandy was pretty well behaved near the coast so that wasn’t an issue in that case. But there are other storms that can rapidly intensify or weaken and the ability to forecast that even a day or two in advance is pretty limited right now,” said DeMaria.

No Silver Bullet

“There isn’t a silver bullet that we will discover and then suddenly we will able to predict everything perfectly. It is a bunch of different processes interacting together. We need to be able to study the oceans better and measure the atmosphere better.”

Tornados Vs. Hurricanes

“When it come to predicting these storms it is all about the scale.”

• Tornados are really very small and are associated with individual super cell thunderstorms. The ability to predict those is limited to a couple of hours in advance. You can see the conditions coming a couple of days ahead of time but specifically where the storm is going to form you only have a few hours notice. The size of an individual thunderstorm may be 10 miles across and the tornado even a big one is only a mile across. Tornados are born and die within a couple of hours.
• Hurricanes are larger in scale, their motions are determined by the jet stream so you can see those coming days in advance. A hurricane lifecycle lasts a week or so. It is also hundreds of miles across. So that generally makes them easier to predict.

You can find all our Sammies interviews here.

Weekend reads

• The gift of doubt. I was going to recommend this fascinating piece in The New Yorker by Malcolm Gladwell. The Wall Street Journal’s CIO Journal ended up having this great write-up: Economist Albert O. Hirschman was the author of “The Principle of the Hiding Hand,” which says that people and businesses take on projects because of a “presumed absence of challenge.” It’s only when things go wrong and the shot-callers realize it is too late to stop, that creativity and problem-solving come into account. Mr. Hirschman, who died last December, liked to tell of what happened after an enormous lumber mill went online shortly before the nearby bamboo forest it was built to exploit died. The mill company reseeded the forest with a faster-growing species and diversified its resource base. In this week’s New Yorker, Malcolm Gladwell profiles the economist, “whose eye was drawn to the many ways in which plans did not turn out the way they were supposed to —to unintended consequences and perverse outcomes and the puzzling fact that the shortest line between two points is often a dead end.” Mr. Gladwell follows him from his beginning as a European intellectual to his days fighting Fascism in both Spain and later in France, to a number of government and academic posts in the U.S. In the U.S. he questioned World Bank led-economic development—forget big projects, he said, train the locals at a grass roots level—and tilted with Milton Friedman. In one book, Mr. Hirschman dismissed Friedman’s arguments in favor of school vouchers, saying people need to fight the school system from within. “Beneath Hirschman’s elegant sentences, you can hear a deeper argument,” writes Mr. Gladwell, “Exit is passive. It is silent protest. And silent protest, for him, is too easy.”
• Disruptive technologies:Advances that will transform life, business, and the global economy
  
• 3 Ways To Really Put The “Vacate” Into Vacation
  • If your body goes ‘on vacation’ but your mind never leaves your Inbox, you’re missing the real productivity-boosting bonuses of a break from work
• New York Magazine: The Boss Stops Here
  • The Boss Stops Here
  • A nonhierarchical workplace may just be a more creative and happier one. But how would you feel if the whole office voted on whether to hire you—and when to give you a raise? Consider, for instance, the fact that hiring at Menlo is handled by committee, with each applicant spending a little bit of time with a group of employees, until a consensus can be reached. That same collective decision-making happens during promotions, layoffs, and flat-out firings.

Want More GovLoop Content? Sign Up For Email Updates