Deep Dive into Large Language Models
This course provides a comprehensive and in-depth introduction to the development history of large language models (LLMs), their core technical architectures, training paradigms (pretraining, fine-tuning, and alignment), multimodal extensions, prompt engineering, chain-of-thought reasoning, agents, as well as frontier topics such as model safety and privacy protection.
Lessons
Lesson
This lesson explores the paradigm shift from task-specific AI to general-purpose Large Language Models, highlighting how scaling laws and the Transformer architecture enable emergent reasoning capabilities. Students will learn how the self-attention mechanism allows models to process data in parallel and why decoder-only architectures have become the standard for modern generative AI.
This lesson explores the evolution of Transformer architectures, highlighting why the industry has shifted toward Decoder-only models due to their superior scaling laws and generative capabilities. It also examines the foundational training pipeline, covering the transition from pre-training and Chinchilla optimality to modern instruction tuning and alignment techniques.
This lesson explores the evolution of prompting strategies, ranging from zero-shot instructions to few-shot demonstrations and Chain-of-Thought reasoning for complex logic. Students will learn how to improve model performance and reliability by using structural constraints, logical decomposition, and precise formatting to ensure outputs are suitable for programmatic use.
This lesson explores the evolution of reasoning in large language models, moving from linear Chain-of-Thought to advanced structured architectures like Tree-of-Thought and Graph-of-Thought. Students will learn how these frameworks, alongside techniques like Program of Thought and knowledge editing, enable models to perform deliberate planning, verification, and complex mathematical problem-solving.
This lesson explores the challenges of AI safety, focusing on the training paradox between model helpfulness and harmlessness, as well as the mechanics of adversarial jailbreak attacks. It also introduces statistical watermarking techniques, such as the KGW framework, which use vocabulary partitioning and logit bias to embed detectable, invisible signatures into AI-generated content.
This lesson explores the fundamental conflict in AI alignment between maintaining model helpfulness and ensuring harmlessness, highlighting how jailbreak attacks exploit this tension through role-playing and cognitive coercion. Students will learn how these vulnerabilities function and examine technical strategies, such as intent analysis and robust system prompting, to defend against prompt-based bypass attempts.
This lesson explores the evolution of Multi-modal Large Language Models (MLLMs) from vision-centric systems to integrated architectures capable of processing diverse sensory inputs like audio and 3D data. Students will learn how specialized encoders and projection bridges align non-textual signals into a unified semantic space, enabling the model to effectively reason across multiple modalities.
This lesson explores the development of autonomous GUI agents that use a tripartite architecture—planning, decision-making, and reflection—to interact with software interfaces. It further examines how Reinforcement Learning and RLHF enable these agents to adapt to dynamic environments while maintaining safety and reliability against potential risks.
Course Overview
📚 Content Summary
This course provides a comprehensive and in-depth introduction to the evolution of Large Language Models (LLMs), core technical architectures, training paradigms (pre-training, fine-tuning, and alignment), multimodal extensions, prompt engineering, Chain of Thought (CoT), agents, as well as frontier topics such as model safety and privacy protection.
Deep analysis of the technological evolution and safety alignment of full-stack large models, from pre-training to general agents.
🎯 Learning Objectives
- Distinguish between model architectures: Identify the structural differences and use cases for Encoder-only, Decoder-only, and Encoder-Decoder models.
- Explain the LLM Training Pipeline: Describe the transition from self-supervised pre-training to Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF).
- Analyze Model Scaling and Behavior: Explain the concepts of Scaling Laws, Emergent Abilities (In-context learning, Chain of Thought), and the phenomenon of Hallucinations.
- Analyze the structural differences between Encoder-only (BERT), Decoder-only (GPT), and Encoder-Decoder (T5) architectures.
- Explain the three-stage training process: Pre-training (Base model), Instruction Tuning (SFT), and Alignment (RLHF/PPO).
- Compare the performance, scaling laws, and architectural innovations of mainstream LLMs including GPT, Llama, Qwen, and DeepSeek.
- Implement zero-shot and few-shot prompting strategies for structured data extraction and classification.
- Calibrate model hyperparameters (Temperature, Top P, Penalties) to balance creative and deterministic outputs.
- Construct effective Chain-of-Thought (CoT) prompts using manual, automatic, and zero-shot ("Let's think step by step") methods.
- Analyze and Compare CoT Variants: Differentiate between Self-Consistency, Program of Thought (PoT), Tree-of-Thought (ToT), and Graph-of-Thought (GoT) architectures.
Lessons
Overview: This lesson explores the evolution of Artificial Intelligence from specialized small-scale models to general-purpose Large Language Models (LLMs). It details the architectural shift from Encoder-based "BERTology" to Decoder-only generative paradigms, covering the critical technical pipeline of pre-training, instruction tuning, and alignment (RLHF). Furthermore, the content examines industry-leading model families including GPT, Llama, and domestic innovations like Qwen and DeepSeek.
Learning Outcomes:
- Distinguish between model architectures: Identify the structural differences and use cases for Encoder-only, Decoder-only, and Encoder-Decoder models.
- Explain the LLM Training Pipeline: Describe the transition from self-supervised pre-training to Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF).
- Analyze Model Scaling and Behavior: Explain the concepts of Scaling Laws, Emergent Abilities (In-context learning, Chain of Thought), and the phenomenon of Hallucinations.
Overview: This lesson provides a comprehensive technical overview of Large Language Models (LLMs), tracing their evolution from basic encoder-decoder architectures to modern multimodal and agent-based systems. It details the core technical pipeline—comprising pre-training, instruction tuning, and alignment—while evaluating mainstream case studies like GPT-4, Llama 3, and DeepSeek. The module concludes with practical deployment strategies (API vs. Local) and advanced prompt engineering frameworks such as RAG and ReAct.
Learning Outcomes:
- Analyze the structural differences between Encoder-only (BERT), Decoder-only (GPT), and Encoder-Decoder (T5) architectures.
- Explain the three-stage training process: Pre-training (Base model), Instruction Tuning (SFT), and Alignment (RLHF/PPO).
- Compare the performance, scaling laws, and architectural innovations of mainstream LLMs including GPT, Llama, Qwen, and DeepSeek.
Overview: This lesson covers the advanced transition from basic zero-shot prompting to structured few-shot learning and the emergent reasoning capabilities of Chain-of-Thought (CoT). Students will analyze how to control model behavior through technical hyperparameters and structured demonstrations to solve complex logical, mathematical, and linguistic tasks. The material concludes with an exploration of process-supervised learning and automated CoT construction methods.
Learning Outcomes:
- Implement zero-shot and few-shot prompting strategies for structured data extraction and classification.
- Calibrate model hyperparameters (Temperature, Top P, Penalties) to balance creative and deterministic outputs.
- Construct effective Chain-of-Thought (CoT) prompts using manual, automatic, and zero-shot ("Let's think step by step") methods.
Overview: This lesson explores advanced structural variants of Chain-of-Thought (CoT) prompting, the technical frameworks for editing knowledge within Large Language Models (LLMs), and the specialized domain of mathematical reasoning. It details how models transition from simple linear reasoning to complex graph-based structures, how "unwanted" knowledge is corrected through internal and external editing, and the training/evaluation pipelines for state-of-the-art mathematical models.
Learning Outcomes:
- Analyze and Compare CoT Variants: Differentiate between Self-Consistency, Program of Thought (PoT), Tree-of-Thought (ToT), and Graph-of-Thought (GoT) architectures.
- Evaluate Knowledge Editing Techniques: Understand the metrics of Reliability, Locality, and Portability, and distinguish between internal (ROME) and external (SERAC) editing solutions.
- Assess Mathematical Logic Pipelines: Identify the training data (GSM8K, MATH, AIME) and distillation processes used to enhance long-form mathematical reasoning in models like DeepSeek-Math and o1.
Overview: This lesson explores the technical mechanisms for identifying LLM-generated content and the security challenges posed by adversarial attacks. It covers statistical watermarking techniques (KGW, SIR, X-SIR) designed to survive translation and re-writing, alongside an analysis of "Jailbreak" prompts (DAN, STAN) used to bypass safety guardrails. The material concludes with defensive strategies and the inherent conflict between model helpfulness and harmlessness.
Learning Outcomes:
- Analyze the mathematical foundation of KGW and SIR watermarking, including vocab partitioning and semantic invariant adjustments.
- Identify and Categorize jailbreak strategies such as "Pretending," "Privilege Escalation," and "Cipher-based" attacks.
- Evaluate defensive measures including In-Context Defense (ICD), Cautionary Warning Defense (CWD), and Red Teaming.
Overview: This lesson explores the critical balance between Large Language Model (LLM) utility and security, focusing on the mechanics of "jailbreak" attacks and the technical implementation of steganography. It covers how attackers bypass safety filters using sophisticated prompt engineering (e.g., DAN, STAN) and how LLMs can be used for covert communication by embedding data within the token generation process. Additionally, it introduces the architecture and capabilities of Multimodal Large Language Models (MLLMs) in processing and generating diverse data types like images, audio, and video.
Learning Outcomes:
- Identify and analyze common jailbreak attack patterns, including role-playing (DAN), cognitive coercion (PUA), and side-channel methods (cipher/code).
- Explain the technical mechanism of LLM steganography, specifically how bitstreams are mapped to the probability distribution (logits) of next-token predictions.
- Categorize MLLM architectures based on their ability to perceive and generate across multiple modalities (Text, Image, Audio, 3D).
Overview: This lesson explores the technical infrastructure of Multi-modal Large Language Models (MLLMs), focusing on the encoding, projection, and decoding mechanisms that enable cross-modal intelligence. It details the transition from modality-specific encoders to unified representation spaces and analyzes the diverse methods used to bridge the gap between non-textual signals and LLM semantic spaces.
Learning Outcomes:
- Identify specialized encoders and tokenization methods for non-visual modalities, including audio (HuBERT, Whisper) and 3D point clouds (Point-BERT).
- Evaluate different input-side projection techniques (Linear, Multi-layer MLP, Resamplers) used to align multimodal representations with LLM semantic spaces.
- Compare the three primary decoding-side connection strategies: discrete tokens, continuous embeddings, and codebooks for multimodal generation.
Overview: This lesson explores the evolution of Graphical User Interface (GUI) agents from static task execution to autonomous decision-making in dynamic environments. It details the technical architectures of open and closed-source agents, the integration of Reinforcement Learning from Human Feedback (RLHF) and Proximal Policy Optimization (PPO) for policy alignment, and the critical safety challenges—ranging from environmental prompt injections to model backdoors—that necessitate robust defense frameworks like GuardAgent and R-Judge.
Learning Outcomes:
- Analyze the architectural components of GUI agents, including planning, decision-making, and reflection modules in multi-agent systems.
- Explain the mechanics of Reinforcement Learning (RL) and RLHF, specifically the role of reward models and PPO in aligning agent behavior with human values.
- Evaluate safety risks and reliability issues in autonomous agents, including Out-of-Distribution (OOD) errors, jailbreak attacks, and environmental distractions.