DeepSeek's AI breakthrough bypasses industry-standard CUDA, uses PTX

When I grew up, the media in my country kept telling us how great the geek culture in the US was, how deep down the stack the geeks were willing to go, and both adults and us kids were left in awe. The entire nation, for what I could tell, routinely reflected why we couldn't be like the US: educating and nurturing generations of geeks to be the best engineers and scientists in the world.

Well, it was quite a reverse culture shock after I moved to the US. I definitely didn't know that "teacher's pet" was a thing, or my coworker, a brilliant engineer who went to a highly reputed public school, was chased off his school bus simply because he used some poetic words, or geeks were not that respected in schools, or a mile wide and an inch deep with great leadership is what the US people revered. In the meantime, I guess other countries more or less picked up the baton of the US culture, and grew their own geeks.

Haha, what a shoddy headline. "Bypasses" and "industry-standard" have no place here.

CUDA is not an industry standard. Vulcan is an industry standard. They did not bypass CUDA... that's like saying if I use Vulcan I'm bypassing OpenGL. PTX is an alternative low level API provided by Nvidia because of how awful CUDA is for high performance code.

What DeepSeek wrote could only have either been written in PTX or Vulcan.

Any other company could have done this, and low latency traders on Wall Street that use Nvidia write their stuff in PTX for obvious reasons.

OpenAI, was, is, and always will be, absolutely incompetent when it comes to using their hardware effectively... and they're no different than any other company. Reading is not a goddamned super power! Just read the docs!

What it does show is that CUDA leaves serious performance optimization on the table despite its gigantic code base. Using compression to reduce memory bandwidth is a well known trick in quantization, and in other scenarios since forever. There has been little competitive pressure on Nvidia to go further since their software stack leaves the competition in the dust. This time, they may actually need to step up their efforts, due to customer pressure. Good times!

You know what IS ridiculous is that people are willing to go assembly but still do not use AMD gpus despite the price difference.

This is ridiculous. Since the actual training code for DeepSeek is _not_ public, this is a based only on the technical report, which mentions PTX one (1) time in §3.2.2 Efficient Implementation of Cross-Node All-to-All Communication:

> Specifically, we employ customized PTX (Parallel Thread Execution) instructions and auto-tune the communication chunk size, which significantly reduces the use of the L2 cache and the interference to other SMs.

So they have some intrinsic in some part of their training framework. That's it.

lol this is the wackiest non-news; every serious project has at least some parts of their kernels implemented in PTX/AMDGCN.

Parallel Thread Execution: https://en.wikipedia.org/wiki/Parallel_Thread_Execution

This is strange. PTX is still at the IR level ?

IIRC this is still relatively hardware agnostic. Can you actually get very far by doing this ? From a quick perusal, DeepSeek also uses Triton in the codebase.

> DeepSeek's AI breakthrough bypasses industry-standard CUDA, uses PTX

Isn't CUDA an Nvidia child ? This sounds like "Microsoft = industry standard".

i would like to assume that low level optimization is crucial part of AI training and openai, meta and others aren't wasting billions on this

Maybe I’m misunderstanding but CUDA compiles to PTX? Is the implication they wrote in a different language than CUDA and to generate the PTX?

Next article: What's SASS?

tldr: they wrote in low level code instead of using a higher level framework like their competitors have been doing so they were able to hand tune the performance.

This gives them a few months head start before meta and Google start doing the same thing.