Everyone's talking about AI adoption. Almost nobody has the real numbers. Help us change that — and get the full report 👉 Engineers | Leaders

Understanding GPT / The Big Picture /

The Dataset and Tokenization

[*] LEARNING OBJECTIVES

[-] Explain how TinyShakespeare is structured and why it is a good training dataset
[-] Describe character-level tokenization and how encode/decode functions work
[-] Explain the relationship between block_size, batch_size, and training batches