How AlphaChip transformed computer chip design

This work from Google (original Nature paper: https://www.nature.com/articles/s41586-021-03544-w) has been credibly criticized by several researchers in the EDA CAD discipline. These papers are of interest:

- A rebuttal by a researcher within Google who wrote this at the same time as the "AlphaChip" work was going on ("Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement"): http://47.190.89.225/pub/education/MLcontra.pdf

- The 2023 ISPD paper from a group at UCSD ("Assessment of Reinforcement Learning for Macro Placement"): https://vlsicad.ucsd.edu/Publications/Conferences/396/c396.p...

- A paper from Igor Markov which critically evaluates the "AlphaChip" algorithm ("The False Dawn: Reevaluating Google's Reinforcement Learning for Chip Macro Placement"): https://arxiv.org/pdf/2306.09633

In short, the Google authors did not fairly evaluate their RL macro placement algorithm against other SOTA algorithms: rather they claim to perform better than a human at macro placement, which is far short of what mixed-placement algorithms are capable of today. The RL technique also requires significantly more compute than other algorithms and ultimately is learning a surrogate function for placement iteration rather than learning any novel representation of the placement problem itself.

In full disclosure, I am quite skeptical of their work and wrote a detailed post on my website: https://vighneshiyer.com/misc/ml-for-placement/

Some interesting context on this work: 2 researchers were bullied to the point of leaving Google for Anthropic by a senior researcher (who has now been terminated himself): https://www.wired.com/story/google-brain-ai-researcher-fired...

They must feel vindicated by their work turning out to be so fruitful now.

TSMC made a point of calling out that their latest generation of software for automating chip design has features that allow you to select logic designs for TDP over raw speed. I think that’s our answer to keep Dennard scaling alive in spirit if not in body. Speed of light is still going to matter, so physical proximity of communicating components will always matter, but I wonder how many wins this will represent versus avoiding thermal throttling.

Questions for those in the know about chip design. How are they measuring the quality of a chip design? Does the metric that Google is reporting make sense? Or is it just something to make themselves look good?

Without knowing much, my guess is that “quality” of a chip design is multifaceted and heavily dependent on the use case. That is the ideal chip for a data center would look very different from those for a mobile phone camera or automobile.

So again what does “better” mean in the context of this particular problem / task.

Eurisco [1], if I remember correctly, was once used to perform placement-and-route task and was pretty good at it.

[1] https://en.wikipedia.org/wiki/Eurisko

What's more, Eurisco was then used in designing Traveler TCS' game fleet of battle spaceships. And Eurisco used symmetry-based placement learned from VLSI design in the design of the spaceships' fleet.

Can AlphaChip's heuistics be used anywhere else?

What is Google doing here? At best, the quality of their "computer chip design" work can be described as "controversial" https://spectrum.ieee.org/chip-design-controversy . What is there to gain by just making a PR now without doing anything new?

Why do they keep saying "superhuman"? Algorithms are used for these tasks, humans aren't laying out trillions of transistors by hand.

To me, there is an underlying issue: why are so many DeepX papers being sent to Nature, instead of appropriate CS forums? If you are doing better work in chip design, send it to IPSD or ISCA or whatever, and then you will get the types of reviews needed for this work. I have no idea what Nature does with a paper like this.

Chips are the limiting factor for AI, and now we have AIs making chips better than human engineers. This feels like an infinite compute cheat code, or at least a way to get us very, very quickly to the physical optimum.

I'd love a tool like this for PCB design/layout

Looks like this is only about placement. I wonder if it can be applied to routing?

How far are we from memory-based computing going from research into competitive products? I get the impression that we are already well passed the point where it makes sense to invest very aggressively to scale up experiments with things like memristors. Because they are talking about how many new nuclear reactors they are going to need just for the AI datacenters.

What occupation is there that is purely intellectual that has no chance of an AI ever progressing to a point where it can take it over?

I must be old because first thing I thought reading AlphaChip was why is deepmind talking about chips in DEC Alpha :-) https://en.wikipedia.org/wiki/DEC_Alpha.

I hope I'll still be alive when they'll announce AlephZero.

How good are TPUs in comparison with state of the art Nvidia datacenter GPUs, or Groq's ASICs? Per watt, per chip, total cost, etc.? Is there any published data?

So AI designing it's own chips. Now that is moving towards exponential growth. Like at the end of "Colossus" the movie.

Forget LLM's. What DeepMind is doing seems more like how an AI will rule, in the world. Building real world models, and applying game logic like winning.

LLM's will just be the text/voice interface to what DeepMind is building.

Seems to me the article is claiming a lot of things, but is very light on actual comparisons that matter to you and me, namely: how does one of those fabled AI-designed chop compare to their competition ?

For example, how much better are these latest gen TPU's when compared to NVidia's equivalent offering ?

A marvellous achievement from DeepMind as usual, I am quite surprised that Google acquired them for a significant discount of $400M, when I would have expected it to be in the range of $20BN, but then again Deepmind wasn’t making any money back then.

Every generation of chips is used to design next generation. That seems to be the root of exponential growth in Moore's law.

Technology singularity is around the corner as soon as the chips (mostly) design themselves. There will be a few engineers, zillions of semiskilled maintenance people making a pittance, and most of the world will be underemployed or unemployed. Technical people better understand this and unionize or they will find themselves going the way of piano tuners and Russian physicists. Slow boiling frog...

Can this be abstracted and generalized into a more generally applicable optimization method?

I'm pretty sure Cadence and Synopsys have both released reinforcement-learning-based placing and floor planning tools. How do they compare...?

Deepmind is producing science vapourware while OpenAI is changing the world

So one other designer plus Google is using alpha chip for their layouts? - not sure on that title, call me when amd and nvidia are using it

Did it, though? Google’s chips still aren’t very good compared with competitors.

There's a lot of... passionate discussion in this thread, but we shouldn't lose sight of the big picture -- Google has used AlphaChip in multiple generations of TPU, their flagship AI accelerator. This is a multi-billion dollar project that is strategically critical for the success of the company. The idea that they're secretly making TPUs worse in order to prop up a research paper is just absurd. Google has even expanded their of AlphaChip use to other chips (e.g. Axion).

Meanwhile, MediaTek built on AlphaChip and is using it widely, and announced that it was used to help design Dimensity 5G (4nm technology node size).

I can understand that, when this open-source method first came out, there were some who were skeptical, but we are way beyond that now -- the evidence is just overwhelming.

I'm going to paste here the quotes from the bottom of the blog post, as it seems like a lot of people have missed them:

“AlphaChip’s groundbreaking AI approach revolutionizes a key phase of chip design. At MediaTek, we’ve been pioneering chip design’s floorplanning and macro placement by extending this technique in combination with the industry’s best practices. This paradigm shift not only enhances design efficiency, but also sets new benchmarks for effectiveness, propelling the industry towards future breakthroughs.” --SR Tsai, Senior Vice President of MediaTek

“AlphaChip has inspired an entirely new line of research on reinforcement learning for chip design, cutting across the design flow from logic synthesis to floor planning, timing optimization and beyond. While the details vary, key ideas in the paper including pretrained agents that help guide online search and graph network based circuit representations continue to influence the field, including my own work on RL for logic synthesis. If not already, this work is poised to be one of the landmark papers in machine learning for hardware design.” --Siddharth Garg, Professor of Electrical and Computer Engineering, NYU

"AlphaChip demonstrates the remarkable transformative potential of Reinforcement Learning (RL) in tackling one of the most complex hardware optimization challenges: chip floorplanning. This research not only extends the application of RL beyond its established success in game-playing scenarios to practical, high-impact industrial challenges, but also establishes a robust baseline environment for benchmarking future advancements at the intersection of AI and full-stack chip design. The work's long-term implications are far-reaching, illustrating how hard engineering tasks can be reframed as new avenues for AI-driven optimization in semiconductor technology." --Vijay Janapa Reddi, John L. Loeb Associate Professor of Engineering and Applied Sciences, Harvard University

“Reinforcement learning has profoundly influenced electronic design automation (EDA), particularly by addressing the challenge of data scarcity in AI-driven methods. Despite obstacles including delayed rewards and limited generalization, research has proven reinforcement learning's capability in complex electronic design automation tasks such as floorplanning. This seminal paper has become a cornerstone in reinforcement learning-electronic design automation research and is frequently cited, including in my own work that received the Best Paper Award at the 2023 ACM Design Automation Conference.” --Professor Sung-Kyu Lim, Georgia Institute of Technology

"There are two major forces that are playing a pivotal role in the modern era: semiconductor chip design and AI. This research charted a new path and demonstrated ideas that enabled the electronic design automation (EDA) community to see the power of AI and reinforcement learning for IC design. It has had a seminal impact in the field of AI for chip design and has been critical in influencing our thinking and efforts around establishing a major research conference like IEEE LLM-Aided Design (LAD) for discussion of such impactful ideas." --Ruchir Puri, Chief Scientist, IBM Research; IBM Fellow

Yet, their “frontier” LLM lags all the others…

Why aren’t they using this technique to design better transformer architectures or completely novel machine learning architectures in general? Are plain or mostly plain transformers really peak? I find that hard to believe.

I understand the achievement, but can't square it with my belief that uniform systolic arrays will prove to be the best general purpose compute engine for neural networks. Those are almost trivial to route, by nature.