Nexus Learn

Log in · Create free account

Module 1: How AI language models work · 8 min

Tokens and the context window

Models do not read letters or whole words. They read tokens, which are small pieces of text. A short common word is often one token. A long or rare word may be split into several tokens. As a rough rule, 100 English words are about 130 tokens.

The context window is the maximum amount of text (your prompt, earlier messages and the reply) that the model can keep in view at once. If a chat becomes very long, the oldest parts may be forgotten or ignored.

IdeaWhat it means for you
TokenCost and limits are counted in tokens, not pages
Context windowPut the most important instructions where the model will not lose them
Long chatStart a new chat for a new topic and paste a short summary
(free account needed to save progress)

← What is a large language model?  |  Why the same question gives different answers →