Show HN: cuTile Rust: Safe, data-race-free GPU kernels in Rust
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- cuTile Rust is a new tile-based system that extends Rust's ownership discipline across the GPU launch boundary: mutable tensors are partitioned into disjoint pieces before launch, immutable tensors are shared, and generated launchers preserve ownership while GPU work is in flight
- On NVIDIA B200, cuTile Rust reaches 7 TB/s for element-wise operations (91% of peak memory bandwidth) and 2 PFlop/s for GEMM (92% of dense f16 peak), with the GEMM result described as competitive with cuBLAS
- Safe Rust persistent GEMM via cuTile Rust hits 2.07 PFlop/s at M=N=K=8192, within 0.3% of the corresponding low-level Tile IR variant — the B200 safety-overhead microbenchmarks show safety without measurable runtime overhead
- Grout, a Qwen3 inference engine built with cuTile Rust in collaboration with Hugging Face, reaches 171 tokens/s for Qwen3-4B on NVIDIA GeForce RTX 5090 and 82 tokens/s for Qwen3-32B on B200 in batch-1 decode
- The project is explicitly an early-stage research release — the authors tell users to "expect bugs, incomplete features, and API breakage" while inviting feedback to shape direction
- Requirements include CUDA 13.3, Rust 1.89+, an NVIDIA GPU with sm_80 or higher compute capability (sm_100+ needs CUDA 13.1+; sm_90 support landed in CUDA 13.3), and Linux (tested on Ubuntu 24.04)
- The accompanying paper, "Fearless Concurrency on the GPU," is authored by Melih Elibol, Jared Roesch, Isaac Gelado, Eric Buehler, and Michael Garland and is cited as arXiv:2606.15991
Why it matters: GPU programming has traditionally forced a choice between unsafe C++/CUDA and performance compromises from safer abstractions. cuTile Rust hits 92% of B200's dense f16 peak for GEMM and is within 0.3% of its own low-level Tile IR variant — meaning Rust teams building inference engines and HPC workloads can get cuBLAS-competitive throughput without leaving the borrow checker. The Grout/Qwen3 numbers (171 tok/s on a 5090, 82 tok/s on B200) show the model already works end-to-end for real LLM serving.
