Parallelism

Dynamic Parallelism

Dynamic Parallelism

Dynamic Parallelism (CUDA 5.0+, CC 3.5+) allows kernels to launch child kernels directly from the device, without returning control to the host. It enables adaptive algorithms where the parallel structure isn't known until runtime.

How It Works

From inside a kernel, you can use the full CUDA launch syntax:

c
__global__ void parentKernel(float *data, int n) {
    // Decide launch config based on runtime data
    if (someCondition(data, n)) {
        int childBlock = 128;
        int childGrid  = (n / 2 + childBlock - 1) / childBlock;
        childKernel<<<childGrid, childBlock>>>(data, n / 2);
    }
    // cudaDeviceSynchronize() on the device syncs child kernels
    cudaDeviceSynchronize();
}

Key constraints:

  • ▸The device-side cudaDeviceSynchronize() only waits for children launched by the calling thread block, not all blocks
  • ▸Child launches are asynchronous relative to the parent block unless explicitly synchronized
  • ▸Memory written by the parent before launch is visible to children; parent can read children's output after cudaDeviceSynchronize()

Use Cases

Adaptive Mesh Refinement

c
__global__ void refineGrid(Cell *cells, int n) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= n) return;

    if (cells[i].errorEstimate > THRESHOLD) {
        // Recursively refine only this cell
        subdivideCell<<<1, 256>>>(cells[i].data, cells[i].level + 1);
    }
}

Recursive Algorithms (e.g., Quicksort)

c
__global__ void quicksort(int *data, int lo, int hi) {
    if (lo >= hi) return;
    int pivot = partition(data, lo, hi);
    // Launch two child kernels for each partition
    if (pivot - lo > 1)
        quicksort<<<1, 1>>>(data, lo, pivot - 1);
    if (hi - pivot > 1)
        quicksort<<<1, 1>>>(data, pivot + 1, hi);
    cudaDeviceSynchronize();
}

Variable-Length Output (Compaction + Kernel-per-Cluster)

Process data of unknown density: first count elements in each region, then launch exactly as many threads as needed per region.

Compilation and Linking

Dynamic parallelism requires device-side CUDA runtime and separate compilation:

bash
nvcc -arch=sm_70 -rdc=true -o program program.cu -lcudadevrt
# -rdc=true enables relocatable device code (required)
# -lcudadevrt links the device runtime library

Performance Considerations

Dynamic parallelism has higher overhead than host-side launches:

  • ▸Each device-side launch has ~1–10 µs overhead (vs ~1–5 µs from host)
  • ▸Deep recursion creates many small kernels; use tail-call flattening when possible
  • ▸Memory for child launch configs is allocated from a device-side pool — can exhaust with very deep recursion
  • ▸Better alternatives for simple subdivision: grid-stride loops, cooperative groups grid sync, or precomputed offsets from prefix scan

Use dynamic parallelism when the work distribution genuinely can't be determined before kernel execution.

// CODE EXAMPLE

example.cu
// Dynamic parallelism: parent kernel launches children for dense regions only
#include <cuda_runtime.h>
#include <cstdio>

// Child kernel: process one dense region
__global__ void processRegion(float *data, int start, int count) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < count) data[start + i] *= 2.0f;
}

// Parent kernel: decide which regions need processing
__global__ void adaptiveParent(float *data, float *density, int nRegions, int regionSize) {
    int r = blockIdx.x * blockDim.x + threadIdx.x;
    if (r >= nRegions) return;

    if (density[r] > 0.5f) {
        // Launch a child kernel only for dense regions
        int start = r * regionSize;
        int tpb   = 128;
        int blocks = (regionSize + tpb - 1) / tpb;
        processRegion<<<blocks, tpb>>>(data, start, regionSize);
    }
    // No cudaDeviceSynchronize here: parent continues immediately
    // If parent needs child results, call cudaDeviceSynchronize() after all launches
}

int main() {
    const int REGIONS = 64, REGION_SIZE = 1024;
    const int N = REGIONS * REGION_SIZE;

    float *d_data, *d_density;
    cudaMalloc(&d_data,    N * sizeof(float));
    cudaMalloc(&d_density, REGIONS * sizeof(float));

    // Must use -rdc=true and -lcudadevrt to compile this example
    adaptiveParent<<<(REGIONS + 31) / 32, 32>>>(d_data, d_density, REGIONS, REGION_SIZE);
    cudaDeviceSynchronize();

    cudaFree(d_data);
    cudaFree(d_density);
}

// TRY IT

GPU execution
main.cu
loading editor…