10 hands-on CUDA projects · from first kernel to advanced optimization
CATEGORY
DIFFICULTY
10 results
The 'Hello World' of GPU programming. Add two arrays element-wise on the GPU, then measure memory bandwidth to understand the roofline model.
Write your first GPU kernel and understand the CUDA execution model: thread indexing, kernel launch syntax, and GPU/CPU synchronization.
Implement matrix transpose three times — naive, shared memory, and bank-conflict-free — and benchmark each version. This project exposes why memory access patterns dominate GPU performance.
Build a complete image processing pipeline: grayscale conversion → Gaussian blur → Sobel edge detection. Each stage is a separate CUDA kernel; chain them with CUDA streams and benchmark every step.
Implement the fundamental parallel scan primitive — first a work-inefficient Hillis-Steele scan, then a work-efficient Blelloch scan, then scale to multi-block arrays.
Compute an 8-bit grayscale histogram three ways: global atomics, per-warp privatization, and per-block shared-memory privatization.
Follow the classic NVIDIA reduction progression. Start with a naive implementation and apply six successive optimizations — from divergent warps to warp shuffle intrinsics.
Implement single-precision general matrix multiply (SGEMM) from scratch. Progress through naive, tiled, and register-file micro-tiled kernels. Benchmark against cuBLAS.
Simulate gravitational N-body dynamics on the GPU. Build the naive all-pairs force kernel, tile it with shared memory, implement leapfrog time integration, and track energy conservation.
Build a fully parallel LSD radix sort on the GPU. Implement digit histogram, exclusive scan for scatter addresses, and the scatter kernel.