Reiner Pope and Mike Gunter ( https://matx.com/ founders and Architects for Google’s ML chips) made a great point about CUDA/Nvidia.
While CUDA is a huge moat to Nvidia, it also restricts chip design. Jensen Huang has an iron rule that every Nvidia AI chip must support CUDA stack. CUDA forces certain design choices to microarchitecture. Just having CUDA API for AMD or Intel chips does not mean much unless chip designers commit to same design choices as Nvidia to get optimal performance.
Reiner Pope and Mike Gunter ( https://matx.com/ founders and Architects for Google’s ML chips) made a great point about CUDA/Nvidia.
While CUDA is a huge moat to Nvidia, it also restricts chip design. Jensen Huang has an iron rule that every Nvidia AI chip must support CUDA stack. CUDA forces certain design choices to microarchitecture. Just having CUDA API for AMD or Intel chips does not mean much unless chip designers commit to same design choices as Nvidia to get optimal performance.