Artificial intelligence gives weather forecasters a new edge

Sigh... don't make me bind a hotkey to paste my spiel about how these non-primitive-equation-based models are bad at long tails and freak weather...

Ed.: too late! https://news.ycombinator.com/item?id=40577332

Seems to be a "pattern" in AI for weather forecasting --

  1. Due to AI hype and funding (even predecessor hype cycles like "big data", ML, even GOF time-series statistics), generate 100s of AI models.
  2. Of these, a few perform better than current models and are seen as the "future".
  3. One year later models in 2. fail catastrophically.
  4. Go back to step 1.

IBM tried with Watson based modeling on weather.com and it was a spectacular failure. Hopefully things have improved enough to beat traditional weather models, but I honestly don't know enough about this area to speculate. Classical modeling, I.E. non neural network models, have been iteratively improved on for decades by the brightest minds in weather science and are barely able to eek out something close to a 50% accuracy in forecast. About 5 years ago that was 10x better than the AI based modeling available, It would be cool to see some advancement here!

Should we not get comparisons and absolute values based on true|false positives|negatives?

The most relevant parts seem to be:

> In seconds... [the ML based] GraphCast can produce a 10-day forecast that would take a supercomputer [crunching through traditional deterministic methods] more than an hour

> GraphCast outperformed the best forecasting model of the European Center for Medium-Range Weather Forecasts more than 90 percent of the time

> Dr. Lam said the study found that GraphCast locked in on landfall in Nova Scotia three days before the supercomputers reached the same conclusion

> the European center was considered the world’s top weather agency because comparative tests have regularly shown its forecasts to exceed all others in accuracy

> Weather experts say the A.I. systems are likely to complement the supercomputer approach because each method has its own particular strengths.

So it's not really that exciting, it's just a new tools for weather forecasting adding to the existing tool chain. I feel like this is only a news story because it's AI. New mathematical models are developed, but don't make it into the mainstream news, because math is boring.

Umbrella marketing terms like this are misleading, and hype adjacent tech like GenAI.

"AI" is a family of statistical techniques, and weather forecasting has been using many of them for decades. Are GNNs displayed here more "AI" than the usual stochastic monte-carlo simulations ? I guess the word "Math" is not sexy anymore

I currently develop a ml model to forecast temperature (spatial resolution 10 m x 10m) by leveraging these models (in this case Climax by Microsoft https://github.com/microsoft/climax ). Feel free to ask!

Quick let's fire all the weather professionals

Honestly I'd be surprised if we weren't already using ML in weather models. Seems obvious enough that it should have popped up years ago, before "AI".

In the U S. There is a lack of data input to create meaningful predictions because there is a lack of Doppler radar stations.

https://www.ncei.noaa.gov/maps/radar/

Need far more sites to gain full coverage, but no doubt they'll be strongly opposed by red disinformation.

Any comments from experts on how much better these models are compared to running base WRF over GFS forecasts?

The images with the forecast don’t look like any forecasts I was watching. They predicted pretty well even on July 1st. The majority of models predicted it would go up like it did. The trend between EPS, GEFS, UKMET, and CMC all had it going up. Not saying an AI LLM won’t help, but this seems a little disingenuous.

I feel the need to insert some "edge compute" joke.

Slightly on-topic:

We have efficiently computable state-update equations available for weather. If neural nets are actually able to outperform those in general (and not just because of some form of sampling bias), why would that be?

1. They're exploiting information not available in the raw inputs to those weather models. Perhaps the number of cars driving vs parked in garages on a given day, contributing some local heat and perturbing PDE solutions away from the optimum. If this is the case, there's probably a fruitful line of research in using neural nets to estimate unmeasured parameters and using those directly in the applicable physics.

2. They just look better, e.g. by comparing raw neural outputs to sanitized weather reports or by using a metric biased toward low prediction variance at the expense of accuracy or something (this could be as simple as grading the neural net's probability distribution against the weather model's raw predicted output too, not considering the epistemological tweaked predictions the physics model also produced, thus implicitly giving the neural net and edge because the physics model can't get partial credit, even when worthy of it's and the AI can). Those sorts of mistakes are shockingly easy to make, even with much thought and careful review. Time will tell, I hope.

3. The gains are in some form of auxiliary data, like cheaply extrapolating a coarse forecast into a fine-grained forecast (superresolution/hallucination), or cheaply approximating a single weather estimate without fleshing out the details of the entire grid. This is potentially incredibly valuable, at the obvious cost of occasionally being very wrong. Use such predictions with care.

I don't think that list is exhaustive, but I'm not holding my breath too much either. Gaining an extra day of hurricane touchdown accuracy, for example, takes immense amounts of data. We've slowly made those gains, but for a neural net to do noticeably better there would have to be major problems in our original problem formulation.

Slightly off-topic:

I'd love an easy way to get conditional probabilities in my forecasts. Some sort of raw, fine-grained data allowing those computations would be a god-send. When a forecast says there's a 30% chance of rain each hour, do they mean we're definitely getting rain and don't know which hour (like the very tall, narrow thunderstorms often traversing west->east in tornado alley), do they mean we're getting a spotty amount of rain that entire period (drizzling off and on like in southeast Alaskan summers), do they mean there's a 30% chance the storm hits us and rains continuously and a 70% chance it skirts by (it's commonly easy to predict there will be a storm but not necessarily exactly where or when with respect to communities/times on the boundaries)? Those have drastic impacts on my plans for the day, and the raw data has that nuance, but it's not obtainable from any weather report I've seen.

Thanks nytime for another polarising article

My niece recently asked me why it's important to make weather forecasts. I had never really thought about it myself. But it is a very important job! Knowing the weather can affect decisions about travel, outdoor events and work schedules; forecasts help people prepare for severe weather conditions, such as storms, hurricanes or extreme temperatures, potentially saving lives and reducing injuries!

Ah yes. The old 0% chance of rain today when it’s currently raining.

Nothing suits a generative AI better, in terms of quotidian prognostication, than telling you whether it will be vaguely sunny-ish in the next 24 to 48 hours.

Whatever next? Perhaps that most egregious example of just about vaguely getting it right enough of the time to seem plausible: AI horoscopes?