CUDA Programming Guide

The official, comprehensive resource for developers to learn the CUDA programming model and how to write high-performance code that executes on NVIDIA GPUs. This guide covers the platform architecture, programming interface, advanced hardware features, and technical specifications.

Master the art of parallel computing with the industry-standard guide to NVIDIA CUDA.

Author: NVIDIA Corporation

Acknowledgments: Copyright © 2007-2024 NVIDIA Corporation & affiliates. All rights reserved.

1. Define the roles of the host (CPU) and device (GPU) within a heterogeneous system.
2. Explain the SIMT programming model and the hierarchical organization of threads, blocks, and grids.
3. Differentiate between PTX (Parallel Thread Execution) and binary code (cubins) and explain how Just-in-Time (JIT) compilation facilitates compatibility.
4. Develop and Compile CUDA Kernels: Write global functions, configure execution with triple-chevron notation, and manage the NVCC compilation workflow.
5. Optimize Memory and Data Movement: Distinguish between Unified, Explicit, and Mapped memory models, and implement page-locked host memory for efficient transfers.
6. Manage Parallel Execution: Utilize CUDA Streams, Events, and Cooperative Groups to manage asynchronous tasks and synchronize CPU-GPU operations.
7. Perform complex pointer arithmetic and identify architectural bottlenecks (von Neumann vs. Harvard).
8. Implement advanced CUDA execution patterns, including Programmatic Dependent Kernel Launches and Heterogeneous Batched Memory Transfers.
9. Utilize hardware-specific features like Thread Scopes, Asynchronous Proxies, and Pipelines to maximize concurrency.
10. Configure and tune Unified Memory performance using prefetching, usage hints, and page size management.