Nvidia CEO says his AI chips are improving faster than Moore's Law

One of the issues with anyone discussing Moore's Law is that people don't specify which Moore's they are talking about.

The economic version of Moore's Law, e.g. cost per transistor halves every 18-24 months, died at 28nm. Dennard scaling broke down earlier.

Jensen here seems to refer to "total compute available in a given system", which is a strange metric: It's not compute-per-dollar, or compute-per-unit-of-energy, but ... as far as I can tell "compute per unit of ... volume"?

>"However, Huang claims that Nvidia’s AI chips are moving at an accelerated pace of their own; the company says its latest data center superchip is more than 30x faster for running AI inference workloads than its previous generation.

> We can build the architecture, the chip, the system, the libraries, and the algorithms all at the same time,” said Huang.

This seems only partially related to Moore's law.

Much of the last 5 years of AI improvements have been FP32 -> FP16 -> FP8 -> FP4, not any particularly impressive hardware or software improvements.

Similarly on the GPU side much of the improvements is more fake pixels and more fake frames, now we can have multiple fake frames per real frame.

> “We can build the architecture, the chip, the system, the libraries, and the

> algorithms all at the same time,” said Huang. “If you do that, then you can

> move faster than Moore’s Law, because you can innovate across the entire stack.”

So no connection at all with actual Moore's Law which states that number of transistors double each year. If you optimize your libraries to be 4x faster, no, that doesn't mean your 'AI chips' are improving faster than Moore's Law.

Statements like these annoy me enough to actually comment. Are they gaslighting us or do they actually believe this stuff ? Makes me really wonder.

I feel like every few years someone says we beat moore’s law.

It wasn’t really a “law” in the first place, right?

Gosh, I'd love to find my old Computer Architecture Professor's slides on the like 3 axes on which you can improve Computer instructions or mflops. But in general my recollection is you can do the following (please except 1 salt rock I'm going from human memory):

1. Transistor size (This is the official Moore's law) that we're improving the number of transistors per area.

2. Clockspeed (Pretty sure this just scales with power, and I'm like 99.9% certain that they're just making gas guzzlers over at nvidia that show improvements in the top-speed, because they use more power).

3. Instruction Throughput - Improving your ISA to be more efficient is something that we've generally been getting better at. It's also why microarchitectures exist, as I understand it (though someone can correct me on this).

I feel like NVIDIA is just saying "We can ramp up #2 all day, no one has really explored how much power one of these things can take" when in fact power-draw is potentially the worst way to try and improve this. (Or at least, they will likely reach a similar limit by dialing up this piece). How far turned up companies are on these dials is kind of how I would expect to evaluate how much growth a company has before they start to really feel the limits of CPU/GPU performance. But also, I would consider it concerning if companies weren't actively investing in all 3 of these, because I think it would be much more likely for them to hit the limits of 2 of these curves and stall out while they're trying to figure out how to improve performance on the other 2.