New LLM Offers Accurate Weather Forecast

New LLM Offers Accurate Weather Forecast

Scientists at the U.S. Department of Energy (DOE) Argonne National LaboratoryIn close collaboration with researchers Aditya Grover and Tung Nguyen at the University of California, Los Angeles, he began developing large artificial intelligence (AI) models for weather forecasting, known as baseline models.

Plot of six-day forecast of 10-meter wind speed (color fill) and mean sea level pressure (contours) using a high-resolution version of Stomer (HR-Stormer) operating at 30 kilometers horizontal resolution. Image Credit: Troy Arcomano/Argonne National Laboratory

The ability to create reliable weather models for forecasting is critical to all sectors of the U.S. economy, from aviation to shipping. Weather models have traditionally been based on equations of thermodynamics and fluid dynamics in the atmosphere. These models are extremely computationally intensive and are typically run on large supercomputers.

Researchers at private sector companies such as Nvidia and Google have begun developing massive artificial intelligence (AI) models, known as baseline models for weather forecasting.

This model can provide more accurate predictions than numerical weather prediction models with lower computational cost.

Some of these models outperformed existing models in forecasts longer than seven days, giving scientists an extra window into the weather.

The base models are based on “tokens,” which are small amounts of information that AI algorithms use to learn the physics behind weather. Many base models are used for natural language processing, which involves manipulating words and phrases.

For these massive language models, tokens are words or chunks of language that the model predicts in succession. Instead of tokens, this new weather forecasting algorithm uses visuals — pieces of graphics that show humidity, temperature, and wind speed in various layers of the atmosphere.

Instead of dealing with a string of text, you are looking at spatio-temporal data represented in images. When you use these pieces of imagery in the model, because of how they are tokenized, you have an idea of ​​their relative positions and how they interact.

Sandeep Madireddy, Computer Scientist, Argonne National Laboratory

Argonne atmospheric scientist Rao Kotamarthi noted that the scientific team was able to make accurate predictions even with relatively low-resolution data.

The philosophy of weather forecasting for years has been to get higher resolutions for better forecasts. This is because you can solve the physics more precisely, but of course this comes at a huge computational cost. But we are finding that with the method we are using now, even at coarse resolution, we can get results that are comparable to existing high-resolution models.

Rao Kotamarthi, Senior Scientist, Environmental Science Division, Argonne National Laboratory

While reliable near-term weather forecasting using AI may seem like an achievable goal in the short term, applying the same method to climate modeling, which involves assessing weather over time, presents a new hurdle.

Kotamarthi added:In theory, the basic models could also be used for climate modeling. However, there is more incentive for the private sector to pursue new approaches to weather prediction than for climate modeling. Work on basic models for climate modeling will likely continue to be the responsibility of national laboratories and universities dedicated to pursuing solutions for the general public.

One of the reasons climate modeling is so difficult is that the climate is constantly changing, according to Argonne Environmental Scientist Troy Arcomano.

With climate, we’ve gone from a largely stationary state to a non-stationary state. That means all of our statistics for climate are changing over time because of the additional carbon in the atmosphere. That carbon is also changing the Earth’s energy budget. It’s complicated to calculate numerically, and we’re still looking for ways to use AI.

Troy Arcomano, Postdoctoral Research Associate, Argonne National Laboratory

With the launch of Argonne’s new exascale supercomputer, Aurora, scientists will be able to train a massive AI-based model that can operate at extremely high resolutions.

To truly capture a finely detailed model with AI, we need an exascale machine” Kotamarthi added.

The model was run on Polaris, a supercomputer at the Argonne Leadership Computing Facility, a user facility of the U.S. Department of Energy’s Office of Science, and the work was supported by Argonne’s Leadership Research and Development Program.

The study won the Best Paper Award for the “Tackling Climate Change with Machine Learning” workshop. The workshop was held on May 10pearl2024 in Vienna, Austria, in conjunction with the 2024 International Conference on Representation of Learning.