A Major Advance Made in Forecasting Local Downpours
Big data assimilation is going to revolutionize weather prediction
Takemasa Miyoshi
Team Leader of the Data Assimilation Research Team, RIKEN AICS
Miyoshi joined the Japan Meteorological Agency (JMA) after graduating from university as a physics major. While still at JMA, he went to study in the US, and in just two years obtained an M.S. and Ph.D. in meteorology at the University of Maryland. Later, he assumed the post of Assistant Professor at that university after quitting JMA. In 2013, he came back to Japan to perform data assimilation on the K computer.
Unexpected torrential rain in local areas can be disruptive and hazardous. Until recently, timely and accurate forecasting of local storms had not been possible. But by integrating the large amount of observed meteorological data that advanced phased-array weather radar can provide with accurate simulation data processed on the K computer, one of the world’s fastest supercomputers, Takemasa Miyoshi and his team have achieved some impressive results in forecasting local downpours. Miyoshi’s advanced forecasting method is called big data assimilation, and he believes it could revolutionize the way local storms are forecast. This new cross-disciplinary approach makes it possible to update the forecast of torrential rain at extremely fast 30-second intervals in high resolution over grid spacings of 100 meters.
What is data assimilation?
Miyoshi, who previously worked for the Japan Meteorological Agency, describes data assimilation in simple terms as feeding various data from real-world observations into computers to produce realistic simulations (Fig. 2).
Weather forecasting is performed based on data obtained by observing the state of the atmosphere at a certain point in time. The data are used as the initial values for simulating future atmospheric states by means of a numerical prediction model. The simulations then serve as the basis for forecasting atmospheric conditions on a long-term basis.
However, the longer computer simulations take to complete, the wider becomes the gap between predicted values and real-world observation data. To rectify this and to improve prediction accuracy, a sequential time-stepping procedure called data assimilation is usually employed. The procedure constantly modifies the predicted values based on the changing observed data so as to provide updated values for further simulation. By modifying predictions in this way, forecasting accuracy has been greatly improved.
K computer plus phased-array weather radar
Improvements in computer performance in recent years have extended the scope of and raised the accuracy of simulations. At the same time, rapid advances in sensor technology have led to more details being obtained from meteorological observations.
“As a researcher of data assimilation, I’m thrilled at the thought of what can now be done when the best possible simulation is combined with state-of-the-art real-world observations,” says Miyoshi. “Better simulations and more accurate observations are both praiseworthy in and of themselves. But when combined, they can take weather prediction to the next level. And it’s our goal to make this happen.”
Miyoshi has focused much attention on observation data provided by phased-array weather radar used to detect localized downpours and tornadoes. A given radar installation can sweep an area of up to 60 km in radius and as high as 14 km in altitude at intervals of 30-seconds, then provide the three-dimensional distribution of rain drops across grid spacings of 100 meters. This helped the researchers detect and depict the first signs of rain through to its passing. Miyoshi hypothesized that by integrating this large amount of detailed data with the simulations performed on the K computer also using grid spacings of 100 meters, the accurate prediction of localized downpours would be possible.
Forecasting local downpours
Intense, localized storms have been on the rise in recent years. They threaten people’s lives and can cause natural disasters. Consequently, the public is demanding more reliable forecasting to predict such storms.
Localized heavy rain tends to be accompanied by a cumulonimbus cloud, which forms and develops rapidly—often within a few minutes. As a result, rain suddenly pours down over a particular area. Because the JMA’s current operational forecasting employs simulations based on 2-kilometer-grid spacing with hourly data assimilation, it is far from effective in determining the spatial and temporal patterns of localized storms.
In 2016, Miyoshi developed a prototype system aimed at predicting localized storms—a world’s first. The system has been tested using a couple of retrospective cases, one example being a storm that lashed Kobe City in Hyogo Prefecture, on September 11, 2014. Fig. 3 shows part of the data describing the storm.
The picture top left shows the result of observations produced by the phased-array radar. The other three pictures show simulation results produced by the K computer. These simulations employed the meteorological data collected immediately before the symptoms of the storm were apparent as initial values, and then the simulations were performed over a period of 30 minutes covering the storm’s progress.
The bottom left picture shows the results obtained without data assimilation. No rain clouds are displayed. The bottom right picture shows the results of data assimilation of a 1-kilometer-grid spacing updated at 30-second intervals. Rain clouds can be seen, but they are different in structure from the observation data.
The picture top right gives the results of big data assimilation. Here the observation data—obtained from a 100-meter-grid spacing updated at 30-second intervals—were used. This reproduces clearly the observation data. Assimilation of data from such a narrow grid spacing at such fast intervals is unprecedented.
“Assimilation of data from a 1-kilometer spacing every several minutes has been regarded as cutting-edge research until now,” says Miyoshi. “So we’ve realized an outstandingly high level of big data assimilation.”
He added, however, “I was fortunate enough to have had access to the K computer and to the data from phased-array radar at the same time. Not many researchers pursuing big data assimilation will be so lucky.”
An animated illustration is available here.
Revolutionizing weather prediction?
Miyoshi’s research findings on localized-storm forecasting were published in the August 2016 issue of the Bulletin of the American Meteorological Society. It appeared under the title “Big Data Assimilation” Revolutionizing Severe Weather Prediction.
Regarding the title, Miyoshi says he expected the journal’s editors and reviewers to object to the use of “revolutionizing,” as the term might be considered an exaggeration. But it was cleared for publication. “Their approval testifies to the importance of our findings as a step towards high-speed, high-accuracy weather prediction, which will ultimately revolutionize forecasting,” he says (Fig. 1).
Two major hurdles still have to be overcome before the system can be considered practical: calculation speeds and prediction accuracy both need to be greatly improved.
As real-world atmospheric observations are updated every 30 seconds, a series of processing steps—feeding the observed data into the computer and integrating it with the ongoing simulations— needs to be completed within each 30-second sequence. Currently, this takes as long as several minutes to execute, while Miyoshi is looking for an about 10-fold improvement. “It once took an hour or more, but we’ve reduced it to several minutes,” says Miyoshi. “And we have ideas for further improvement.”
To achieve such a speed gain, he is collaborating with AICS colleagues who are conversant with the K computer, and who know how best to use it to perform such simulations. “By exploiting their competence, I’m sure we can get the process completed within 30 seconds,” says Miyoshi.
As for prediction accuracy, the researchers are exploring ways to prevent a sharp fall off in accuracy five minutes after the process of simulation begins. In part, this is due to the numerical prediction models used. These were developed for simulations based on a grid spacing of several kilometers. So they do not work well for 100-meter-grid-based simulations.
Miyoshi is currently testing a new numerical model called SCALE developed by an AICS Computational Climate Science research team led by Hirofumi Tomita. Miyoshi is impressed by SCALE’s core concept, which targets high-resolution simulations. Tomita’s team is also helping to upgrade the process of data assimilation. “As these researchers share the same building as us, this helps make AICS an outstanding research environment,” says Miyoshi.
Data assimilation is a rich field for future research
Miyoshi notes that the field of data assimilation is short of researchers. This is largely because few universities and colleges offer courses related to the subject in Japan. But he stresses that data assimilation can be applied to all fields that employ simulation—not just weather forecasting. In particular, he believes big data assimilation will become important in better utilizing big data in the future. “So I hope more scientists will take it up,” he says. “It’s a rich and rewarding area to conduct research.”