Shared memory is on-chip SRAM with ~32-cycle latency vs ~800 cycles for global DRAM. It is the primary tool for data reuse within a thread block and for staging non-coalesced global memory accesses.
Static (size known at compile time):
__shared__ float tile[32][32];
Dynamic (size passed at launch):
extern __shared__ float s[];
myKernel<<<grid, block, sharedBytes>>>(args);
Shared memory is physically organized as 32 banks, each 4 bytes wide. Successive 4-byte words map to successive banks cyclically:
bank(address) = (address / 4) % 32
| Access pattern | Result | |---|---| | 32 threads → 32 different banks (any permutation) | No conflict — 1 cycle | | Multiple threads → same address | Broadcast — no conflict, 1 cycle | | N threads → different addresses in same bank | N-way conflict — N cycles (serialized) |
Stride-1 — no conflict:
tile[threadIdx.x] // thread i → bank i → all 32 banks, no conflict
Stride-2 — 2-way conflict:
tile[threadIdx.x * 2] // thread 0 → bank 0, thread 16 → bank 0 (conflict)
Stride-32 — 32-way conflict:
tile[threadIdx.x * 32] // all 32 threads → bank 0 at different addresses
Matrix transpose produces 32-way conflicts when reading tile[threadIdx.x][threadIdx.y]: all 32 elements land in the same bank (one per column).
Fix: pad the inner dimension by 1:
__shared__ float tile[32][32 + 1]; // +1 offsets each row's bank assignments
With padding, row r starts at byte offset r * 33 * 4. Element [r][c] maps to bank (r * 33 + c) % 32 — no two rows' column c align to the same bank.
Cost: 32 × 1 × 4 = 128 bytes of wasted memory per tile — negligible.
On Ampere (A100), each SM has 192 KB of on-chip SRAM, configurable as shared memory / L1 cache. Default split: 100 KB shared / 28 KB L1 (varies by architecture).
You can request more shared memory per block:
cudaFuncSetAttribute(myKernel,
cudaFuncAttributeMaxDynamicSharedMemorySize, 98304); // 96 KB
More shared memory per block reduces how many blocks fit simultaneously on an SM (occupancy tradeoff).
Use it when:
Skip it when:
// CODE EXAMPLE
// Bank-conflict-free matrix transpose: +1 padding eliminates 32-way conflicts
#define TILE 32
__global__ void transposeNoBankConflict(
const float * __restrict__ in,
float * __restrict__ out, int N)
{
// +1 padding shifts each row's bank offset, breaking the 32-way alignment
__shared__ float tile[TILE][TILE + 1];
int x = blockIdx.x * TILE + threadIdx.x;
int y = blockIdx.y * TILE + threadIdx.y;
// Coalesced read: consecutive threads read consecutive x values
if (x < N && y < N)
tile[threadIdx.y][threadIdx.x] = in[y * N + x];
__syncthreads();
// Transposed output coordinates
int ox = blockIdx.y * TILE + threadIdx.x;
int oy = blockIdx.x * TILE + threadIdx.y;
// Coalesced write; tile access tile[threadIdx.x][threadIdx.y]
// is conflict-free due to the +1 padding
if (ox < N && oy < N)
out[oy * N + ox] = tile[threadIdx.x][threadIdx.y];
}// TRY IT
GPU execution