Let’s just say what it is: devs are too constrained to jump ship right now. It’s a massive land grab and you are not going to spend time tinkering with CUDA alternatives when even a six-month delay can basically kill your company/organization. Google and Apple are two companies with enough resources to do it. Google isn’t because they’re keeping it proprietary to their cloud. Apple still have their heads stuck in sand barely capable of fixing Siri.
Yeah, ROCm focused code will always beat generic code compiled down. But this is a really difficult game to win.
For example, Deepseek R-1 released optimized for running on Nvidia HW, and needed some adaption to run as well on ROCm. This was for the exact same reasons that ROCm code will beat generic code compiled into ROCm, in the same way. Basically the Deepseek team, for their own purposes, created R-1 to fit Nvidia's way of doing things (because Nvidia is market-dominant) on their own. Once they released, someone like Elio or AMD would have to do the work of adapting the code to run best on ROCm.
For more established players who weren't out-of-left-field surprises like Deepseek, e.g. Meta's Llama series, mostly coordinate with AMD ahead of release day, but I suspect that AMD still has to pay for the engineering work themselves while Meta does the work to make it run on Nvidia themselves. This simple fact, that every researcher makes their stuff work on CUDA themselves, but AMD or someone like Elio has to do the work to move it over to get it to be as performant on ROCm, that is what keeps people in the CUDA universe.
Right now we need diversity in the ecosystem. AMD is finally getting mature and hopefully that will lead to them truly getting a second, strong, opinion into ecosystem. The friction this article talks about is needed to push new ideas.
I agree pretty strongly. A translation layer like this is making an intentional trade: Giving up performance and HW alignment for less lead time and effort to make a proper port.
Why can't it be done w/ AI? Why does it need to be maintained w/ manual programming? Take the ROCm specification, take your CUDA codebase, let one of the agentic AIs translate it all into ROCm or the AMD equivalent.
The real question is whether it will be as unprofitable to do this type of automated runtime translation from one GPU vendor to another as it is to generate Mario clips & Ghibli images.
The article is literally about how rote translation of CUDA code to AMD hardware will always give sub-par performance. Even if you wrangled an AI into doing the grunt work for you, porting heavily-NV-tuned code to not-NV-hardware would still be losing strategy.
> Take the ROCm specification, take your CUDA codebase, let one of the agentic AIs translate it all into ROCm
...sounds like asking for a 1:1 mapping to me. If you meant asking the AI to transmute the code from NV-optimal to AMD-optimal as it goes along, you could certainly try doing that, but the idea is nothing more than AI fanfic until someone shows it actually working.
Now that I have clarified the point about AI optimizing the code from CUDA to fit AMD's runtime what is your contention about the possibility of such a translation?
There are tons of test suites so if the tests pass then that provides a reasonable guarantee of correctness. Although it would be nice if there was also proof of correctness for the compilation from CUDA to AMD.
I am not saying this is impossible, but I am down voting this because this is _not an interesting discussion_.
The whole point of having an online discussion forum is to exchange and create new ideas. What you are advocating is essentially "maybe we can stop generating new ideas because we don't have to. we should just sit and wait"... Well, yes, no, maybe. but this is not what I expect to get from here.
You can do whatever you want & I didn't ask you to participate in my thread so unless you are going to address the actual points I'm making instead of telling me it is not interesting then we don't have anything to discuss further.
They keep promising that this kind of capability is right around the corner & they keep showing how awesome they are at passing math exams so why is this a more difficult problem than solving problems in abstract algebra & scheme theory on humanity's last exam or whatever is the latest & greatest benchmark for mathematical capabilities?
I don't know why you're being downvoted because even if you're Not Even Wrong, that's exactly the sort of thing that has been endlessly presented by people trying to sell AI as something that AI will absolutely do for us.
Because it doesn't work like that. TFA is an explanation of how GPU architecture dictates the featureset that is feasibly attainable at runtime. Throwing more software at the problem would not enable direct competition with CUDA.
I am assuming that is all part of the specification that the agentic AI is working with & since AGI is right around the corner I think this is a simple enough problem that can be solved with AI.
Google isn't internally, so far as we know. Google's hyperscaler products have long offered CUDA options, since the demand isn't limited to AI/tensor applications that cannibalize TPU's value prop: https://cloud.google.com/nvidia
In situations like this, I try to focus on whether the other person understood what was being communicated rather than splitting hairs. In this case, I don't think anyone would be confused.
https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-...
https://tinygrad.org/ is the only viable alternative to CUDA that I have seen popup in the past few years.
Let’s just say what it is: devs are too constrained to jump ship right now. It’s a massive land grab and you are not going to spend time tinkering with CUDA alternatives when even a six-month delay can basically kill your company/organization. Google and Apple are two companies with enough resources to do it. Google isn’t because they’re keeping it proprietary to their cloud. Apple still have their heads stuck in sand barely capable of fixing Siri.
Yeah, ROCm focused code will always beat generic code compiled down. But this is a really difficult game to win.
For example, Deepseek R-1 released optimized for running on Nvidia HW, and needed some adaption to run as well on ROCm. This was for the exact same reasons that ROCm code will beat generic code compiled into ROCm, in the same way. Basically the Deepseek team, for their own purposes, created R-1 to fit Nvidia's way of doing things (because Nvidia is market-dominant) on their own. Once they released, someone like Elio or AMD would have to do the work of adapting the code to run best on ROCm.
For more established players who weren't out-of-left-field surprises like Deepseek, e.g. Meta's Llama series, mostly coordinate with AMD ahead of release day, but I suspect that AMD still has to pay for the engineering work themselves while Meta does the work to make it run on Nvidia themselves. This simple fact, that every researcher makes their stuff work on CUDA themselves, but AMD or someone like Elio has to do the work to move it over to get it to be as performant on ROCm, that is what keeps people in the CUDA universe.
Right now we need diversity in the ecosystem. AMD is finally getting mature and hopefully that will lead to them truly getting a second, strong, opinion into ecosystem. The friction this article talks about is needed to push new ideas.
Perhaps I'm misunderstanding the market dynamics; but isn't AMDs real opp inference over research?
Training etc still happens on NVDA but inference is somewhat easy to do on vLLM et al with a true ROCm backend with little effort?
I agree pretty strongly. A translation layer like this is making an intentional trade: Giving up performance and HW alignment for less lead time and effort to make a proper port.
Why can't it be done w/ AI? Why does it need to be maintained w/ manual programming? Take the ROCm specification, take your CUDA codebase, let one of the agentic AIs translate it all into ROCm or the AMD equivalent.
The AI is too busy making Ghibli profile pictures or whatever the thing is now.
We asked it to make a plan for how to fix the situation, but it got stuck.
“Ok, I’m helping the people build an AI to translate NVIDIA codes to AMD”
“I don’t have enough resources”
“Simple, I’ll just use AMD chips to run an AI code translator, they are under-utilized. I’ll make a step by step process to do so”
“Step 1: get code kernels for the AMD chips”
And so on.
The real question is whether it will be as unprofitable to do this type of automated runtime translation from one GPU vendor to another as it is to generate Mario clips & Ghibli images.
The article is literally about how rote translation of CUDA code to AMD hardware will always give sub-par performance. Even if you wrangled an AI into doing the grunt work for you, porting heavily-NV-tuned code to not-NV-hardware would still be losing strategy.
The point of AI is that it is not a rote translation & 1:1 mapping.
> Take the ROCm specification, take your CUDA codebase, let one of the agentic AIs translate it all into ROCm
...sounds like asking for a 1:1 mapping to me. If you meant asking the AI to transmute the code from NV-optimal to AMD-optimal as it goes along, you could certainly try doing that, but the idea is nothing more than AI fanfic until someone shows it actually working.
Now that I have clarified the point about AI optimizing the code from CUDA to fit AMD's runtime what is your contention about the possibility of such a translation?
There is an old programmer's joke about writing abstractions and expecting zero-cost.
Has this been done successfully at scale?
There's a lot of handwaving in this "just use AI" approach. You have to figure out a way to guarantee correctness.
There are tons of test suites so if the tests pass then that provides a reasonable guarantee of correctness. Although it would be nice if there was also proof of correctness for the compilation from CUDA to AMD.
The same as "Why just outsourcing it to <some country >"
AI aint magic.
You need more effort to manage, test and validate that.
Isn't AGI around the corner? If it is then this is a very simple problem that should be solvable w/ existing pre-AGI capabilities.
I am not saying this is impossible, but I am down voting this because this is _not an interesting discussion_.
The whole point of having an online discussion forum is to exchange and create new ideas. What you are advocating is essentially "maybe we can stop generating new ideas because we don't have to. we should just sit and wait"... Well, yes, no, maybe. but this is not what I expect to get from here.
You can do whatever you want & I didn't ask you to participate in my thread so unless you are going to address the actual points I'm making instead of telling me it is not interesting then we don't have anything to discuss further.
So, your strategy for solving this is: Convert it to another harder problem (AGI). Now it is somebody else (AI researcher)'s problem.
This is outsourcing the task to AI researchers.
They keep promising that this kind of capability is right around the corner & they keep showing how awesome they are at passing math exams so why is this a more difficult problem than solving problems in abstract algebra & scheme theory on humanity's last exam or whatever is the latest & greatest benchmark for mathematical capabilities?
I don't know why you're being downvoted because even if you're Not Even Wrong, that's exactly the sort of thing that has been endlessly presented by people trying to sell AI as something that AI will absolutely do for us.
Let's see who else manages to catch on to the real point I'm making.
Because it doesn't work like that. TFA is an explanation of how GPU architecture dictates the featureset that is feasibly attainable at runtime. Throwing more software at the problem would not enable direct competition with CUDA.
I am assuming that is all part of the specification that the agentic AI is working with & since AGI is right around the corner I think this is a simple enough problem that can be solved with AI.
All they have to do is release air cooled 96GB GDDR7 PCIe5 boards with 4x Infinity Link, and charge $1,900 for it.
Are the hyperscalers really using CUDA? This is what really matters. We know Google isn't. Are AWS and Azure for their hosting of OpenAI models et al?
All Nvidia GPUs, which are probably >70% of the market, use CUDA.
> We know Google isn't.
Google isn't internally, so far as we know. Google's hyperscaler products have long offered CUDA options, since the demand isn't limited to AI/tensor applications that cannibalize TPU's value prop: https://cloud.google.com/nvidia
Actual article title says "won't"; wont is a word meaning habit or proclivity.
In situations like this, I try to focus on whether the other person understood what was being communicated rather than splitting hairs. In this case, I don't think anyone would be confused.