Harnessing Raw Hardware: Writing Custom Triton and CUDA 12 Kernels on Blackwell and Hopper Architectures

As AI models scale, standard matrix multiplication libraries like cuBLAS often leave significant performance on the table for specialized sparse attention patterns or custom fused activation functions.

Kernel engineers developing in Triton or raw CUDA C++ require deep visibility into hardware registers, shared memory bank conflicts, and Tensor Memory Accelerator (TMA) asynchronous transfer pipelines.

Because NovaPod instances run with direct bare-metal hardware passthrough, developers have unencumbered access to NVIDIA Nsight Systems and Nsight Compute profiling tools without hypervisor interference.

Whether optimizing warp-group matrix multiply-accumulate (WGMMA) instructions on Hopper SM90a or preparing kernels for Blackwell's second-generation Transformer Engine, NovaPod provides the cleanest low-level development environment in the cloud.

nvcc -O3 -arch=sm_90a custom_attention.cu -o custom_attention
nsys profile --stats=true ./custom_attention