← Tech dictionary
AI & machine learning
What is Token (in LLMs)?
A token is a piece of text — a word or part of a word — that a language model reads and generates. Context limits and API prices are measured in tokens.
Learn it properlyAI engineering tutorials — free, with diagrams, quizzes and a certificate