Occupancy is the ratio of active warps to the maximum number of warps an SM can support. It determines how effectively the GPU can hide memory latency.
Global memory accesses take ~800 cycles. During those cycles the warp stalls; the warp scheduler switches to another ready warp at zero cost (all state lives in the register file). The more active warps, the more latency can be hidden.
Rule of thumb: 4–8+ active warps per SM are needed to effectively hide global memory latency. On Ampere, the max is 48 warps per SM (1,536 threads).
Whichever resource runs out first determines maximum occupancy:
Each SM has ~65,536 × 32-bit registers shared across all active threads. If a kernel uses 64 registers/thread with 256 threads/block:
registers per block = 64 × 256 = 16,384
max blocks per SM = floor(65,536 / 16,384) = 4
active warps = 4 × (256/32) = 32
occupancy = 32 / 48 = 67%
Check register usage:
nvcc --ptxas-options=-v kernel.cu
# → registers per thread = N, spills = M
If each SM has 96 KB and a block uses 32 KB, only 3 blocks fit (regardless of thread count).
int blockSize, minGridSize;
// Find the block size that maximizes occupancy for your kernel
cudaOccupancyMaxPotentialBlockSize(
&minGridSize, &blockSize,
myKernel,
0, // dynamic shared memory per block (bytes)
0 // max block size hint (0 = no limit)
);
// Query how many blocks actually fit per SM
int activeBlocks;
cudaOccupancyMaxActiveBlocksPerMultiprocessor(
&activeBlocks, myKernel, blockSize, 0);
cudaDeviceProp prop;
cudaGetDeviceProperties(&prop, 0);
float occ = (float)(activeBlocks * blockSize)
/ prop.maxThreadsPerMultiProcessor;
printf("Occupancy: %.1f%%\n", occ * 100);
Higher occupancy ≠ higher performance. A kernel at 50% occupancy that does lots of compute per memory access can outperform a 100% occupancy kernel.
The goal is enough warps to hide latency, not maximizing occupancy:
Profile with Nsight Compute to distinguish these cases.
__launch_bounds__ HintTell the compiler the maximum block size and desired minimum blocks per SM. The compiler uses this to allocate registers more aggressively:
// At most 256 threads/block, target at least 4 blocks/SM
__global__ void __launch_bounds__(256, 4) myKernel(float *data, int n) { ... }
// CODE EXAMPLE
#include <cuda_runtime.h>
#include <stdio.h>
// __launch_bounds__ guides the compiler's register allocation
__global__ void __launch_bounds__(256, 4) scaleKernel(
float * __restrict__ out,
const float * __restrict__ in,
float scale, int n)
{
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) out[i] = in[i] * scale;
}
int main() {
// Let CUDA find the optimal block size for occupancy
int blockSize, minGridSize;
cudaOccupancyMaxPotentialBlockSize(
&minGridSize, &blockSize, scaleKernel, 0, 0);
const int N = 1 << 24;
int gridSize = (N + blockSize - 1) / blockSize;
printf("Grid: %d blocks × %d threads\n", gridSize, blockSize);
// Measure theoretical occupancy
int activeBlocks;
cudaOccupancyMaxActiveBlocksPerMultiprocessor(
&activeBlocks, scaleKernel, blockSize, 0);
cudaDeviceProp prop;
cudaGetDeviceProperties(&prop, 0);
float occ = (float)(activeBlocks * blockSize)
/ prop.maxThreadsPerMultiProcessor;
printf("Occupancy: %.1f%%\n", occ * 100);
scaleKernel<<<gridSize, blockSize>>>(d_out, d_in, 2.0f, N);
cudaDeviceSynchronize();
}// TRY IT
GPU execution