August 13th, 2024

China's Taichi-II Chip: First Optical AI Processor Outperforms Nvidia H100

Beijing researchers unveiled the Taichi-II chip, the first fully optical AI processor, achieving six orders of magnitude energy efficiency improvement and 40% higher accuracy than NVIDIA's H100, enhancing AI technology in China.

Read original articleLink Icon
China's Taichi-II Chip: First Optical AI Processor Outperforms Nvidia H100

Beijing researchers have introduced the Taichi-II chip, the world's first fully optical AI processor, which significantly outperforms NVIDIA's H100 in energy efficiency. This innovative chip operates entirely on light, leading to a remarkable six orders of magnitude improvement in energy efficiency, particularly in low-light imaging scenarios. Additionally, Taichi-II demonstrates a 40% increase in accuracy for classification tasks compared to the H100. The chip employs a novel training method called Fully Forward Mode (FFM) learning, which allows for parallel processing directly on the optical chip, enhancing its performance. Developed by Professors Fang Lu and Dai Qionghai from Tsinghua University, the Taichi-II chip represents a substantial advancement from its predecessor, the original Taichi chip, which had already set records for energy efficiency. The introduction of Taichi-II is strategically significant, especially in light of U.S. restrictions on China's access to advanced GPUs for AI training, providing a viable alternative for continued progress in AI technology. The chip's capabilities could play a crucial role in China's technological advancements and defense capabilities, marking a pivotal moment in the evolution of AI and optical computing.

- Taichi-II is the first fully optical AI processor, surpassing NVIDIA's H100 in energy efficiency.

- It achieves a six orders of magnitude improvement in energy efficiency in low-light scenarios.

- The chip features a 40% accuracy boost in classification tasks compared to the H100.

- Fully Forward Mode (FFM) learning enables efficient parallel processing on the optical chip.

- The development of Taichi-II offers a strategic alternative amid U.S. restrictions on advanced AI hardware for China.

Related

Sohu AI chip claimed to run models 20x faster and cheaper than Nvidia H100 GPUs

Sohu AI chip claimed to run models 20x faster and cheaper than Nvidia H100 GPUs

Etched startup introduces Sohu AI chip, specialized for transformer models, outperforming Nvidia's H100 GPUs in AI LLM inference. Sohu aims to revolutionize AI processing efficiency, potentially reshaping the industry.

AMD MI300X performance compared with Nvidia H100

AMD MI300X performance compared with Nvidia H100

The AMD MI300X AI GPU outperforms Nvidia's H100 in cache, latency, and inference benchmarks. It excels in caching performance, compute throughput, but AI inference performance varies. Real-world performance and ecosystem support are essential.

Intel CPU with Optical OCI Chiplet Demoed with 4Tbps of Bandwidth and 100M Reach

Intel CPU with Optical OCI Chiplet Demoed with 4Tbps of Bandwidth and 100M Reach

Intel unveiled a new CPU with an Optical Compute Interconnect chiplet, offering 4Tbps bandwidth and 100m reach. The chiplet merges Photonic and Electronic Integrated Circuits, featuring built-in lasers for improved server communication, especially in AI servers. Intel plans to scale the technology up to 32Tbps, focusing on power savings and enhanced bandwidth. This advancement aligns with the industry's move towards co-packaged optics for better efficiency and performance, marking a significant shift in server architecture towards power efficiency and high-speed data transfer.

China Is Closing the A.I. Gap with the United States

China Is Closing the A.I. Gap with the United States

Chinese tech companies are advancing in AI, showcasing innovations like Kuaishou's video generator. Despite U.S. trade restrictions, they leverage open-source technologies, supported by the government, to enhance development and competition.

Nvidia forecast to bounce back in China to make $12B selling GPUs

Nvidia forecast to bounce back in China to make $12B selling GPUs

Nvidia is expected to generate $12 billion in GPU sales in China this year, despite U.S. trade restrictions, with over 1 million H20 GPUs shipped, nearly doubling Huawei's sales.

Link Icon 4 comments
By @justinclift - 9 months
"... which outperforms the Nvidia H100 in energy efficiency".

Very unlike what the headline was saying.

The article doesn't seem to really state how much real world performance this thing actually has. Seems pretty dodgy?

By @creer - 9 months
This is the FFM paper, one of the papers in question (since no mention of a TAICHI here):

https://www.nature.com/articles/s41586-024-07687-4

A Tsinghua page mentions only this paper.

https://media.au.tsinghua.edu.cn/info/1016/1419.htm

And this is one from one year ago from the same group, also photonic chip.

https://www.nature.com/articles/s41586-023-06558-8

I haven't read any of this yet.

By @yogrish - 9 months
Can any expert in this domain validate the claimed performance over H100? Any pointers on how optical processors work and how mature they are over traditional processors.
By @doodlebugg - 9 months
I would be shocked if nvidia didn’t use optical interconnects in their next iteration. My understanding is that the big power loss in these racks comes from having to bus the data around.

The article only seems to mention the power savings and speed, both of which are obvious benefits over traditional electrical circuits. Optical computing isnt constrained by the speed of light or the efficiency of a fiber, it’s that components are orders of magnitude larger and have low to no manufacturing support.

Don’t get me wrong, I’m hyped for the eventual PIC revolution, i just don’t think this is it.