Meaning
The maximum amount of text (measured in tokens) an AI language model can process and remember at one time.
Three examples from work
Model selection
DevCan we pass the entire customer contract into the prompt?
AI LeadYes, our model supports a 200k token context window, so it fits easily.
Optimization
DevWe are getting context window exhaustion in long conversational threads.
ArchitectWe need to summarize earlier exchanges and truncate redundant system logs.
Cost management
PMWhy are our API inference costs doubling every month?
DevBecause we send the full document history in every single call; we need smart retrieval.
Large context windows allow deep reasoning across books or codebases, but cost and retrieval degradation ('needle in a haystack') remain considerations.