Launch HN: Silurian (YC S24) – Simulate the Earth

Glad to see that you can make ensemble forecasts of tropical cyclones! This absolutely essential for useful weather forecasts of uncertain events, and I am a little dissapointed by the frequent comparisons (not just you) of ML models to ECMWF's deterministic HRES model. HRES is more of a single realization of plausible weather, rather than an best estimate of "average" weather, so this is a bit of apples vs oranges.

One nit on your framing: NeuralGCM (https://www.nature.com/articles/s41586-024-07744-y), built by my team at Google, is currently at the top of the WeatherBench leaderboard and actually builds in lots of physics :).

We would love to metrics from your model in WeatherBench for comparison. When/if you have that, please do reach out.

> These models have little to no built-in physics and learn to forecast purely from data. Astonishingly, this approach, done correctly, produces better forecasts than traditional simulations of the physics of our atmosphere.

Haha. The old NLP saying "every time I fire a linguist, my performance goes up", now applies to the physicists....

> Silurian builds foundation models to simulate the Earth, starting with the weather.

What else do you hope to simulate, if this becomes successful?

Can we help you? We build the equivalent for land, as a non-profit. It's basically a geo Transformer MAE model (plus DINO, plus matrioska, plus ...), but largest and most trained (35 trillion pixels roughly). Most importantly fully open source and open license. I'd love to help you replace land masks with land embeddings, they should significantly help downscale the local effects (e.g. forest versus city) that afaik most weather forecast simplify with static land cover classes at most. https://github.com/Clay-foundation/model

This is awesome - how does this compare to the model that Google released last year, GraphCast?

Curious to see what other things you will simulate in the future!

Shameless plug: recently we've built a demo that allows you to search for objects in San Francisco using natural language. You can look for things like Tesla cars, dry patches, boats, and more. Link: https://demo.bluesight.ai/

We've tried using Clay embeddings but we quickly found out that they perform poorly for similarity search compared to embeddings produced by CLIP fine tuned on OSM captions (SkyScript).

Check out Climavision. They use AI to generate both hyper-local ("will there be a tornado over my town in the next 30 minutes?") and seasonal ("will there be a draught next fall?") forecasts, and they do it faster than the National Weather Service. They also operate their own private radar network to fill observational gaps.

Disclosure: I work there.

https://climavision.com/

Fascinating. I have two quick questions, if you find the time:

  …we’ve built our own foundation model, GFT (Generative Forecasting Transformer), a 1.5B parameter frontier model that simulates global weather…

I’m constantly scolding people for trying to use LLMs for non-linguistic tasks, and thus getting deceptively disappointing results. The quintessential example is arithmetic, which makes me immediately dubious of a transformer built to model physics. That said, you’ve obviously found great empirical success already, so something’s working. Can you share some of your philosophical underpinnings for this approach, if they exist beyond “it’s a natural evolution of other DL tech”? Does your transformer operate in the same rough way as LLMs, or have you radically changed the architecture to better approach this problem?

  Hence: simulate the Earth.

When I read “simulate”, I immediately think of physics simulations built around interpretable/symbolic systems of elements and forces, which I would usually put in basic opposition to unguided/connectionist ML models. Why choose the word “simulate”, given that your models are essentially black boxes? Again, a pretty philosophical question that you don’t necessarily have to have an answer to for YC reasons, lol

Best of luck, and thanks for taking the leap! Humanity will surely thank you. Hopefully one day you can claim a bit of the NWS’ $1.2B annual budget, or the US Navy’s $infinity budget — if you haven’t, definitely reach out to NRL and see if they’ll buy what you’re selling!

Oh and C) reach out if you ever find the need to contract out a naive, cheap, and annoyingly-optimistic full stack engineer/philosopher ;)

Am skeptical about the business case for this given the huge government investment in part of this.

What will your differentiators be?

Are you paying for weather data products?

Weather models are chaotic, are ML methods more numerically stable than a physics based simulation? And how do they compare in terms of compute requirements? the Aurora paper seemed to be promising, but I would love a summary of comparison better than what I get out of Claude.

Once upon a time I converted spectral-transform-shallow-water-model (STSWM or parallelized as PSTSWM) from FORTRAN to Verilog. I believe this is the spectral-transform method we have run for the last 30 years to do forecasting. The forecasting would be ~20% different results for 10-day predictions if we truncated each operation to FP64 instead of Intel's FP80.

Have you had a crack at applying this approach to the effectively unforecastable - earthquakes, for example?

> Astonishingly, this approach, done correctly, produces better forecasts than traditional simulations of the physics of our atmosphere.

It seems like this is another instance of The Bitter Lesson, no?

Does anyone predict economy/population/... by simulating individual people based on real census information? Monte carlo simulation of major events (births, death, ...) based on known statistics based on age, economic background, location, education, profession, etc.? It seems there are not that many people that this would be computationally infeasible, and states and companies have plenty of data to feed into such systems. Is it not needed because other alternatives give better results, or is it already being done?

Every weather forecasting agency in the world is pivoting to ML methods, and some of them have very deep pockets and industry partnerships. Some big tech companies are forging ahead on their own. Unless you have proprietary data, you just bought yourself a low paying job with long hours. Typical poor judgement of naive YC founders. Founding a company is more exciting than being successful.

Is the plan to expand from weather forecasting into climate simulation? Given the complexity of the finding initial conditions on the earth, a non-physical (or only implicitly-physical) model seems like it could offer a very promising alternative to physical models. The existing physical models, while often grossly correct (in terms of averages), suffer from producing unphysical configurations on a local basis.

Congratulations on splitting off to make some money! I remember reading about ClimaX a year ago and being extremely excited – especially because of the potential to lower the costs of large physical simulations like these.

Have specific industries reached out to you for your commerical potential – natural resource exploration, for example?

Are you planning on open sourcing your code and/or model weights? Aurora code and weights were recently open sourced.

Congrats to Jayesh and team! I was lucky to meet the founding CEO recently, and happy to let everyone know he's very friendly and of course super intelligent.

As a fellow deep learning modeler of Earth systems, I can also say that what they're doing really is 100% top notch. Congrats to the team and YC.

Did you explore other branches of AI, namely KRLs? It's an underrated area especially in recent years.

Using the full expressive power of a programming language to model the real world and then execute AI algorithms on highly structured and highly understood data seems like the right way to go!

This really, really looks like a nullschool clone (https://earth.nullschool.net/). Is it not?

I'm hoping the singularity will coincide with a large-scale AI achieving simulated Earth consciousness. Human intelligence is only a spec compared to all the combined intelligence of nature.

So ChatGPT has a cutoff date on the stuff it can talk about. This predicting weather sounds like ChatGPT being able to predict next week's news from which it has been trained on. I can see how it can probably predict some stuff like Argentina winning a football match scheduled for next week when played against India given India sucks at football. But can it really give any useful predictions? Like can it predict things which are not public? Like who will Joe Rogan interview in 2 weeks? Or what would be the list of companies in YCs next batch?

How accurate is the weather prediction for a city for tomorrow on average for the min and max temperature? Thanks a lot!

Exciting idea and seems like a well-proven team. Good luck to you guys here and don't mind the endemic snark in the other threads. A couple basic questions --

1. How will you handle one-off events like volcanic eruptions for instance? 2. Where do you start with this too? Do you pitch a meteorology team? Is it like a "compare and see for yourself"?

How performed your Hurricanes forecast versus the reality?

very cool! i was thinking of doing space weather simulation using vocap and a representation of signals in the spatial domain. maybe it could be added.

This is cool. What do you mean by "defense?"

I keep waiting for someone to integrate data from NEON

Wow, so excited for this.

I had a web app online in 2020-22 called Skim Day that predicted skimboarding conditions on California beaches that was mostly powered by weather APIs. The tide predictions were solid, but the weather itself was almost never right, especially wind speed. Additionally there were some missing metrics like slope of beach which changes significantly throughout the year and is very important for skimboarding.

Basically, I needed AI. And this looks incredible. Love your website and even the name and concept of "Generative Forecasting Transformer (GFT)" - very cool. I imagine the likes of Surfline, The Weather Channel, and NOAA would be interested to say the least.