Programmation de processeurs massivement parallèles : Approche pratique
Ce cours propose une introduction complète au calcul sur GPU et à la programmation parallèle en utilisant l'environnement CUDA C. Il couvre les architectures GPU, la parallélisation des données, la gestion des threads, l'optimisation de la mémoire et les considérations avancées liées aux performances, illustrées à travers des études de cas du monde réel telles que la reconstruction d'IRM et la visualisation moléculaire.
Aperçu du cours
📚 Résumé du contenu
Ce cours offre une introduction complète au calcul sur GPU et à la programmation parallèle en utilisant l’environnement CUDA C. Il couvre les architectures GPU, la parallélisation des données, la gestion des threads, l’optimisation de la mémoire et les considérations avancées sur les performances, illustrées à travers des études de cas réels comme la reconstruction d’IRM et la visualisation moléculaire.
Maîtrisez l’art du calcul parallèle haute performance grâce à un guide pratique et appliqué à CUDA et aux architectures GPU.
Auteur : David B. Kirk, Wen-mei W. Hwu
Remerciements : Ian Buck, John Nickolls, équipe NVIDIA DevTech, Jensen Huang, David Luebke, Bill Bean, Simon Green, Mark Harris, Manju Hedge, Nadeem Mohammad, Brent Oster, Peter Shirley, Eric Young et Cyril Zeller.
🎯 Objectifs d'apprentissage
- Différencier les philosophies de conception et les trajectoires de performance des CPUs multicœurs et des GPUs many-core.
- Identifier les composants clés d'une architecture GPU moderne, notamment les Streaming Multiprocessors (SMs) et les structures de mémoire.
- Appliquer la loi d'Amdahl pour calculer le gain théorique et identifier l'impact des goulets d'étranglement séquentiels.
- Contraire les différences architecturales entre pipelines fixes et tableaux de processeurs unifiés programmables.
- Expliquer le rôle du "GPGPU" comme étape intermédiaire et les limitations des premiers modèles de programmation shader.
- Analyser comment des fonctionnalités matérielles comme les opérations atomiques, la synchronisation par barrière et le support de la double précision ont permis la transition vers un calcul généralisé évolutif.
- Identifier et exploiter la parallélisation des données dans les algorithmes de multiplication matricielle.
- Mettre en œuvre la gestion de la mémoire périphérique, incluant l'allocation, le transfert de données entre hôte et périphérique, et la libération.
- Créer et exécuter des noyaux CUDA en utilisant des index de thread et des configurations appropriées de grille/bloc.
- Concevoir des hiérarchies de threads multidimensionnelles (grilles et blocs) pour mapper des structures de données complexes sur le matériel GPU.
Leçons 共 12 课时 · 预计 36.0h
Leçons
Lesson
This lesson explores the evolution of parallel computing, highlighting the "Great Divergence" where GPUs surpassed CPUs in performance by prioritizing throughput-oriented architecture over sequential latency. Students will learn to differentiate between these processing models, understand the impact of the "Power Wall" on CPU design, and analyze how GPU transistor budgeting enables massive parallel computation.
This lesson explores the evolution of GPU architecture, focusing on the "real-time imperative" that necessitated a shift from serial CPU processing to parallel hardware acceleration. Students will learn how early innovations like SLI and the "wide and slow" design philosophy enabled the high-throughput performance required to meet strict frame-time budgets in modern computing.
This lesson explores the CUDA execution model, focusing on the architectural differences between the latency-optimized CPU (Host) and the throughput-optimized GPU (Device). Students will learn how to manage the lifecycle of a CUDA kernel, implement memory allocation using cudaMalloc and cudaMemcpy, and organize threads into grids and blocks to perform parallel computations.
This lesson explores the fundamentals of CUDA kernel execution, focusing on the transition from CPU-based iteration to data-centric GPU parallelism. Students will learn to implement the global indexing formula, manage execution configurations for transparent scalability, and apply boundary guards to ensure safe memory access across multidimensional data.
This lesson explores the "Memory Wall" in GPU computing, where computational throughput outpaces memory bandwidth, creating a significant performance bottleneck. Students will learn to mitigate these constraints by implementing shared memory tiling strategies, optimizing data reuse, and managing hardware resource limits to maximize occupancy.
This lesson explores the SIMT execution model, focusing on how hardware organizes threads into 32-thread warps and linearizes them for efficient scheduling. Students will learn to evaluate performance through warp partitioning, branch divergence analysis, and memory access patterns to optimize GPU kernel utilization.
This lesson explores how Excess Encoding (biased representation) enables high-speed hardware sorting by ensuring that bit patterns maintain a monotonic relationship with their numerical values. By replacing the sign-bit discontinuity of Two's Complement with this biased format, engineers can utilize simple, efficient unsigned comparators to perform rapid operations like Z-buffering in parallel processors.
This lesson explores the computational challenges of non-Cartesian MRI reconstruction, where spiral trajectories require iterative solvers or gridding instead of standard Fast Fourier Transforms. Students will learn how to overcome these bottlenecks by leveraging GPU-based massive parallelism, specifically focusing on voxel-to-thread mapping to optimize reconstruction speed for time-sensitive clinical applications like Sodium MRI.
This lesson explores the use of Direct Coulomb Summation (DCS) and GPU acceleration to generate electrostatic potential maps for molecular visualization. Students will learn to optimize rendering pipelines through techniques like loop unrolling and constant memory broadcasting to efficiently handle large-scale atomic data.
This lesson explores the transition from sequential processing to parallel computing, emphasizing how computational thinking helps overcome the power wall and frequency limits. Students will learn to evaluate parallel algorithm performance, manage the trade-offs between numerical precision and execution speed, and apply problem decomposition to optimize distributed systems.
This lesson introduces the OpenCL framework as a solution for managing heterogeneous computing environments, where a host CPU orchestrates tasks across diverse accelerators like GPUs and FPGAs. Students will learn to utilize the OpenCL platform layer for hardware discovery, understand the device model's hierarchy, and implement portable, efficient kernels that adapt to different architectural requirements.
This lesson explores the evolution of GPU architecture from graphics-focused designs to the compute-first Fermi generation, which introduced unified memory hierarchies and IEEE 754-2008 compliance. Students will learn how these advancements, including hardware-managed caching and improved thread scheduling, enable complex scientific computing and general-purpose programming beyond traditional 2D grid tasks.