Contributors
GPT Explained
Download PDF
Download ePub
Welcome
License
Preface
Contributors
Foundations
1
Introduction
2
Notation and Definitions
Representations
3
Tokens — Text to Numbers
4
Embeddings — Numbers to Meaning
5
Positional Encoding — Giving Order to Meaning
Transformer Core
6
Attention — Tokens Talking to Each Other
7
RoPE: Position Inside Attention
8
Multi-Head Attention — Many Conversations at Once
9
Feed-Forward Network — The Model’s Memory
10
The Transformer Block — Putting It Together
Prediction and Learning
11
Vocabulary Projection — From Vectors to Words
12
Loss — How the Model Learns
13
Training — Teaching the Model
Modern Extensions
14
Modern GPT
Appendices
A
microGPT in Python — Complete Runnable Code
Contributors
Contributors as of 71a43b5:
Ju Lin
Preface
Foundations