ACENET Summer School - GPGPU: Glossary

A GPU (Graphics Processing Unit) is best at data-parallel, arithmetic-intense calculations

CUDA is one of several programming interfaces for general-purpose computing on GPUs

Alliance clusters have special GPU-equipped nodes, which must be requested from the scheduler

Use nvcc to compile

CUDA source files are suffixed .cu

Use salloc to get an interactive session on a GPU node for testing

The CPU (the ‘host’) and the GPU (the ‘device’) have separate memory banks

This requires explicit copying of data to and from the GPU

Threads are the lowest level of parallelization on a GPU

A kernel function replaces the code inside the loop to be parallelized

The CUDA <<<M,N>>> notation replaces the loop itself

Use nvprof to profile CUDA functions

Blocks are the batches in which a GPU handles data

Blocks are handled by streaming multiprocessors (SMs)

Each block can have up to 1024 threads (on our current GPU cards)

A typical kernel indexes data using both blocks and threads

Compute capability is a version number that represents the features supported by a GPU.

Shared memory is a small, but fast memory available for each multiprocessor that needs to be managed manually.

Since compute capability 2, each block can consist of up to 1024 threads, which can further be organized in up to three dimensions.

Active threads within a warp can only ever execute the same instructions at the same time. If some threads branch, they will be set aside for later.

Using 2D or 3D GridDefs and BlockDefs can make it easier to address multi-dimensional data.

CUDA has a special type dim3 to define multi-dimensional grid and block definitions.

GPU memory has much higher bandwidth than CPU memory, which comes at a cost of higher latency.

Time to copy data between CPU- and GPU memory can be a large fraction of total runtime.

Access of consecutive memory addresses is much faster than strided access.

Using shared memory can help when strided access cannot be avoided, but needs to be managed manually.

Many software libraries implement highly-optimized solutions for common problems.