ROCm really is hit or miss depending on the use case.
Plus their consumer card support is questionable to say the least. I really wish it was a viable alternative, but swapping to CUDA really saved me some headaches and a ton or time.
Having to run MiOpen benchmarks for HIP can take forever.
m_mueller 4 hours ago [-]
Exactly the same has been said over and over again, ever since CUDA took off for scientific computing around 2010. I don’t really understand why 15 years later AMD still hasn’t been able to copy the recipy, and frankly it may be too late now with all that mindshare in NVIDIA’s software stack.
bayindirh 24 minutes ago [-]
Just remember that 4 of the top 10 of Top500 systems run on AMD Instinct cards, based on the latest June 2025 list announced at ISC Hamburg.
NVIDIA has a moat for smaller systems, but that is not true for clusters.
As long as you have a team to work with the hardware you have, performance beats mindshare.
bigyabai 3 hours ago [-]
It's just not easy. Even if AMD was willing to invest in the required software, they would need a competitive GPU architecture to make the most of it. It's a lot easier to split 'cheap raster' and 'cheap inference' into two products, despite Nvidia's success.
alecco 7 hours ago [-]
Jensen knows what he is doing with the CUDA stack and workstations. AMD needs to beat that more than thinking about bigger hardware. Most people are not going to risk years learning an arcane stack for an architecture that is used by less than 10% of the GPGPU market.
hyperbovine 4 hours ago [-]
I'm willing to bet almost nobody you know calls the CUDA API directly. What AMD needs to focus on is getting the ROCm backend going for XLA and PyTorch. That would unlock a big slice of the market right there.
They should also be dropping free AMD GPUs off helicopters, as Nvidia did a decade or so ago, in order to build up an academic userbase. Academia is getting totally squeezed by industry when it comes to AI compute. We're mostly running on hardware that's 2 or 3 generations out of date. If AMD came with a well supported GPU that cost half what an A100 sells for, voila you'd have cohort after cohort of grad students training models on AMD and then taking that know-how into industry.
bwfan123 1 hours ago [-]
Indeed. the user-facing software stack componentry - pytorch and jax/xla - are owned by meta, and google and open sourced. Further, the open-source models (llama/deepseek) are largely hw agnostic. There is really no user or eco-system lock-in. Also, clouds are highly incentivized to have multiple hardware alternatives.
pjmlp 4 hours ago [-]
Additionally when people discuss CUDA they always think about C, ignoring that has been a C++ first since CUDA 3.0, also has Fortran surpport, and NVidia always embraced having multiple languages being able to play on PTX land as well.
And as of 2025, there is a Python CUDA JIT DSL as well.
Also, even if not the very latest version, the fact that CUDA SDK works on any consumer laptop with NVidia hardware, anyone can slowly get into CUDA, even if their hardware isn't that great.
rbanffy 5 hours ago [-]
Indeed. The stories I hear about software support for their entry-level hardware aren't great. Having a good on-ramp is essential.
OTOH, by emphasizing datacenter hardware, they can cover a relatively small portfolio and maximize access to it via cloud providers.
As much as I'd love to see an entry-level MI350-A workstation, that's not something that will likely happen.
cedws 3 hours ago [-]
At this point it looks to me like something is seriously broken internally at AMD resulting in their software stack being lacklustre. They’ve had a lot of time to talk to customers about their problems and spin up new teams, but as far as I’ve heard there’s been very little progress, despite the enormous incentives.
I think Lisa Su is a great CEO but perhaps not shaking things up enough in the software department. She is from a hardware background after all.
bwfan123 53 minutes ago [-]
There used to be a time when hw vendors begudgingly put out sample driver code which contained 1 file with 5000 lines of C code - which just about barely worked. The quality of software was not really a priority, as most of the revenue was from hw sales. That reflected in the quality of hires and incentive structures.
AlexanderDhoore 6 hours ago [-]
Can someone with more knowledge give me a software overview of what AMD is offering?
Which SDKs do they offer that can do neural network inference and/or training?
I'm just asking because I looked into this a while ago and felt a bit overwhelmed by the number of options. It feels like AMD is trying many things at the same time, and I’m not sure where they’re going with all of it.
Paradigma11 2 hours ago [-]
Don't call us, we will call you when that future is the present.
numpad0 2 hours ago [-]
fyi: ROCm support status currently isn't crucial for casual AI users - standard proprietary AMD drivers include Vulkan API support going back ~10 years. It's slower, but llama.cpp supports it, and so do many oneclick automagic LLM apps like LM Studio.
What really matters is how much of "Software++: ROCm 7 Released" can I use on a regular consumer laptop, like I can with CUDA.
user____name 7 hours ago [-]
Is Bob Page leading the effort?
kombine 12 hours ago [-]
If hope AMD can produce a chip that matches H100 in training workloads.
lhl 11 hours ago [-]
Last year I had issues using MI300X for training, and when it did work, was about 20-30% slower than H100, but I'm doing some OpenRLHF (transformers/DeepSpeed-based) DPO training atm w/ latest ROCm and PyTorch and it seems to be doing OK, roughly matching GPU-hour perf w/ an H200 for small ~12h runs.
Note: previous testing I did was on a single (8x) MI300X node, currently I'm doing testing on just a single MI300X GPU, so not quite apples-to-apples, multi-GPU/multi-node training is still a question mark, just a single data point.
moralestapia 11 hours ago [-]
You mean a slower chip?
Their MI300s already beat them, 400s coming soon.
Vvector 1 hours ago [-]
Chip speed isn't as important as good software
moralestapia 1 hours ago [-]
The software is the same, AMD is not doing its own LLMs.
jjice 1 hours ago [-]
I think the software they were referring to is CUDA and the developer experience around the nvidia stack.
moralestapia 29 minutes ago [-]
???
Know any LLMs that are implemented in CUDA?
fooker 10 hours ago [-]
It gets even more jarring that H100 is about three years old now.
halJordan 14 hours ago [-]
Honestly that was a hard read. I hope that guy gets an mi355 just for writing this.
AMD deserves exactly zero of the credulity this writer heaps onto them. They just spent four months not supporting their rdna4 lineup in rocm after launch. AMD is functionally capable of day120 support. None of the benchmarks disambiguated where the performance is coming from. 100% they are lying on some level, representing their fp4 performance against fp 8/16.
jchw 11 hours ago [-]
I still find their delay with properly investing in ROCm on client to be rather shocking, but in fairness they did finally announce that they would be supporting client cards on day 1[1]. Of course, AMD has to keep the promise for it to matter, but they really do seem to, for whatever reason, finally realized just how important it is that ROCm is well-supported across their entire stack (among many other investments they've announced recently.)
It's baffling that AMD is the same company that makes both Ryzen and Radeon, but the year-to-date for Radeon has been very good, aside from the official ROCm support for RDNA4 taking far too long. I wouldn't get overly optimistic; even if AMD finally committed hard to ROCm and Radeon it doesn't mean they'll be able to compete effectively against NVIDIA, but the consumer showing wasn't so bad so far with the 9070 XT and FSR4, so I'm cautiously optimistic they've decided to try to miss some opportunities to miss opportunities. Let's see how long these promises last... Maybe longer than a Threadripper socket, if we're lucky :)
Is this day 1 support a claim about the future or something they've demonstrated? Because if it involves the future it is safer to just assume AMD will muck it up somehow when it comes to their AI chips. It isn't like their failure in the space is a weird one-off - it has been confusingly systemic for years. It'd be nice if they pull it off, but it could easily be day 1 support for a chip that turns out to crash the computer.
I dunno; I suppose they can execute on server parts. But regardless, a good plan here is to let someone else go first and report back.
jchw 4 hours ago [-]
They've been able to execute well for Ryzen, EPYC, and Radeon in the data center. I don't really think there's any reason to believe they can't or even wouldn't be able to do ROCm on client cards, but up until recently they wouldn't commit.
pclmulqdq 14 hours ago [-]
AMD doesn't care about you being able to do computing on their consumer GPUs. The datacenter GPUs have a pretty good software stack and great support.
fc417fc802 12 hours ago [-]
I'm inclined to believe it but that difference is exactly how nvidia got so far ahead of them in this space. They've consistently gone out of their way to put their GPGPU hardware and software in the hands of the average student and professional and the results speak for themselves.
tormeh 3 hours ago [-]
I wouldn't say so. Nvidia bet on machine learning a decade or so before AMD got the memo. That was a good bet on Nvidia's part. In 2015 you just had to have an Nvidia card if you wanted to do ML research. Sure, Nvidia did hand them out in some cases, but even if you bought an AMD card it just wouldn't work. It was Nvidia or go home. Even if AMD now did everything right (and they don't), there's a decade+ of momentum in Nvidia's favor.
zombiwoof 10 hours ago [-]
Just look at the disaster of rocm or you need to spend 300k on software engineers to get anything so work
stingraycharles 11 hours ago [-]
Yes but then they fail to understand a lot of “long tail” home projects, opensource stuff etc is done on consumer GPUs at home, which is tremendously important for ecosystem support.
wmf 11 hours ago [-]
What if they understand that and they don't care? Getting one hyperscaler as a customer is worth more than the entire long tail.
lhl 8 hours ago [-]
On the corp side you have FB w/ PyTorch, xformers (still pretty iffy on AMD support tbt) and MS w/ DeepSpeed. But let's see about some others:
Flash Attention: academia, 2y behind for AMD support
bitsandbytes: academia, 2y behind for AMD support
Marlin: academia, no AMD support
FlashInfer: acadedmia/startup, no AMD
ThunderKittens: academia, no AMD support
DeepGEMM, DeepEP, FlashMLA: ofc, nothing from China supports AMD
Without the long tail AMD will continue to always be in a position where they have to scramble to try to add second tier support years later themselves, while Nvidia continues to get all the latest and greatest for free.
This is just off the top of my head on the LLM side where I'm focused on, btw. Whenever I look at image/video it's even more grim.
jimmySixDOF 7 hours ago [-]
Modular says Max/Mojo will change this and make refactoring between different vendors (and different lines of the same vendor) less of a showstopper but tbd for now
stingraycharles 10 hours ago [-]
The problem is that this is short-term thinking. You need students and professionals playing around with your tools at home and/or on their work computers to drive hyperscale demand in the long term.
This is why it’s so important AMD gets their act together quickly, as the benefits of these kind of things are measured in years, not months.
selectodude 11 hours ago [-]
Then they’re fools. Every AI maestro knows CUDA because they learned it at home.
jiggawatts 10 hours ago [-]
It’s the same reason there’s orders of magnitude more code written for Linux than for mainframes.
danielheath 9 hours ago [-]
Why would a hyperscaler pick the technology that’s harder to hire for (because there’s no hobbyist-to-expert pipeline)?
moffkalast 8 hours ago [-]
Then they will stay irrelevant in the GPU space like they have been so far.
littlestymaar 7 hours ago [-]
Why should we care about them if they don't care?
I mean of they want to stay at a fraction of the market value and profit of their direct competitor, good for them.
dummydummy1234 4 hours ago [-]
I want a competitive market so I can have cheaper gpus.
It's Nvidia, AMD, and maybe Intel.
cma 11 hours ago [-]
Nvidia started removing nvlink with the 4000 series, they aren't heavily focused on it either anymore and want to sell the workstation cards for uses like training models at home.
"25 complimentary GPU hours (approximately $50 US of credit for a single MI300X GPU instance), available for 10 days. If you need additional hours, we've made it easy to request additional credits."
archerx 12 hours ago [-]
If they care about their future they should. I am a die hard AMD supporter and even I am getting over their mediocrity and what seems to be constant self sabotage in the GPU department.
zombiwoof 10 hours ago [-]
It’s the AMD management . They just are recycling 20 year VP lifers at AMD to take over key projects
archerx 3 hours ago [-]
They could have slapped 48gb of vram on their new Radeon cards and they would have instantly sold out but that would cut into cousins profit margin at nvidia so that’s obviously a no go.
booder1 12 hours ago [-]
I have had trained on both large AMD and Nvidia clusters and your right AMD support is good. I never had to talk to Nvidia support. That was better.
They should care about the availability of their hardware so large customers don't have to find and fix their bugs. Let consumers do that...
echelon 12 hours ago [-]
> AMD doesn't care about you being able to do computing on their consumer GPUs
Makes it a little hard to develop for without consumer GPU support...
fooker 10 hours ago [-]
It’s the same software stack.
shmerl 11 hours ago [-]
Aren't they addressing it with the unified UDNA architecture? That's going to be a thing in the future GPUs, making consumer and datacenter ones share the same arch.
Different architectures was probably a big reason for the above issue.
caycep 13 hours ago [-]
this is ROCm?
fooblaster 13 hours ago [-]
Yes, the mi300x/mi250 are best supported as they directly compete with data center gpus from Nvidia which actually make money. Desktop is a rounding error by comparison.
ethbr1 6 hours ago [-]
> I hope that guy gets an mi355 just for writing this. AMD deserves exactly zero of the credulity this writer heaps onto them.
Plus their consumer card support is questionable to say the least. I really wish it was a viable alternative, but swapping to CUDA really saved me some headaches and a ton or time.
Having to run MiOpen benchmarks for HIP can take forever.
NVIDIA has a moat for smaller systems, but that is not true for clusters.
As long as you have a team to work with the hardware you have, performance beats mindshare.
They should also be dropping free AMD GPUs off helicopters, as Nvidia did a decade or so ago, in order to build up an academic userbase. Academia is getting totally squeezed by industry when it comes to AI compute. We're mostly running on hardware that's 2 or 3 generations out of date. If AMD came with a well supported GPU that cost half what an A100 sells for, voila you'd have cohort after cohort of grad students training models on AMD and then taking that know-how into industry.
And as of 2025, there is a Python CUDA JIT DSL as well.
Also, even if not the very latest version, the fact that CUDA SDK works on any consumer laptop with NVidia hardware, anyone can slowly get into CUDA, even if their hardware isn't that great.
OTOH, by emphasizing datacenter hardware, they can cover a relatively small portfolio and maximize access to it via cloud providers.
As much as I'd love to see an entry-level MI350-A workstation, that's not something that will likely happen.
Which SDKs do they offer that can do neural network inference and/or training? I'm just asking because I looked into this a while ago and felt a bit overwhelmed by the number of options. It feels like AMD is trying many things at the same time, and I’m not sure where they’re going with all of it.
Note: previous testing I did was on a single (8x) MI300X node, currently I'm doing testing on just a single MI300X GPU, so not quite apples-to-apples, multi-GPU/multi-node training is still a question mark, just a single data point.
Their MI300s already beat them, 400s coming soon.
Know any LLMs that are implemented in CUDA?
AMD deserves exactly zero of the credulity this writer heaps onto them. They just spent four months not supporting their rdna4 lineup in rocm after launch. AMD is functionally capable of day120 support. None of the benchmarks disambiguated where the performance is coming from. 100% they are lying on some level, representing their fp4 performance against fp 8/16.
It's baffling that AMD is the same company that makes both Ryzen and Radeon, but the year-to-date for Radeon has been very good, aside from the official ROCm support for RDNA4 taking far too long. I wouldn't get overly optimistic; even if AMD finally committed hard to ROCm and Radeon it doesn't mean they'll be able to compete effectively against NVIDIA, but the consumer showing wasn't so bad so far with the 9070 XT and FSR4, so I'm cautiously optimistic they've decided to try to miss some opportunities to miss opportunities. Let's see how long these promises last... Maybe longer than a Threadripper socket, if we're lucky :)
[1]: https://www.phoronix.com/news/AMD-ROCm-H2-2025
I dunno; I suppose they can execute on server parts. But regardless, a good plan here is to let someone else go first and report back.
Flash Attention: academia, 2y behind for AMD support
bitsandbytes: academia, 2y behind for AMD support
Marlin: academia, no AMD support
FlashInfer: acadedmia/startup, no AMD
ThunderKittens: academia, no AMD support
DeepGEMM, DeepEP, FlashMLA: ofc, nothing from China supports AMD
Without the long tail AMD will continue to always be in a position where they have to scramble to try to add second tier support years later themselves, while Nvidia continues to get all the latest and greatest for free.
This is just off the top of my head on the LLM side where I'm focused on, btw. Whenever I look at image/video it's even more grim.
This is why it’s so important AMD gets their act together quickly, as the benefits of these kind of things are measured in years, not months.
I mean of they want to stay at a fraction of the market value and profit of their direct competitor, good for them.
It's Nvidia, AMD, and maybe Intel.
"25 complimentary GPU hours (approximately $50 US of credit for a single MI300X GPU instance), available for 10 days. If you need additional hours, we've made it easy to request additional credits."
They should care about the availability of their hardware so large customers don't have to find and fix their bugs. Let consumers do that...
Makes it a little hard to develop for without consumer GPU support...
Different architectures was probably a big reason for the above issue.
You mean Ryan Smith of late AnandTech fame?
https://www.anandtech.com/author/85/
AMD is a marketing company now