Context Windows and Memory

A model does not remember your conversation. Each time you send a message, the whole exchange is sent back to it, and it reads all of it from scratch before responding.

That single fact explains why long conversations degrade, why costs rise, and why a tool suddenly forgets something you said earlier.

The mechanism

The context window is the maximum amount of text the model can consider at once — your messages, its replies, any documents you attached, the system instructions, and anything retrieved on your behalf.

Everything competes for the same space. A large attached document reduces room for conversation. A long conversation reduces room for documents.

When the limit is reached, something is dropped. Products handle this differently: some truncate the oldest messages, some summarise them, some warn you. The common experience of a model "forgetting" the beginning of a long session is usually this.

Sizes vary enormously between products, and the advertised figure is not always what you get on a given plan.

What this explains

Long conversations get worse. More material to attend to, older instructions further away, and eventually truncation. A conversation that has run for hours is usually worse than a fresh one with a good summary pasted in.

Instructions given early lose force. Say "always answer in British English" at message one and by message fifty it may have drifted. Repeating important constraints periodically works better than assuming they persist.

Cost scales with conversation length. Where you pay per use, every message re-sends the entire history. A long conversation costs progressively more per exchange.

Attaching a large document can crowd out everything else. A book-length attachment leaves little room for a nuanced discussion of it.

Two conversations do not share anything. Unless the product has an explicit memory feature, a new chat knows nothing about the previous one.

Long context is not the same as good use of it

Products advertise very large windows. Filling one does not mean the model uses it well.

Attention is uneven. Material at the beginning and end of a long context is used more reliably than material in the middle. Burying a crucial instruction halfway through a long document is a mistake.

Retrieval beats stuffing. Searching a document collection and supplying the relevant passages generally outperforms pasting everything and hoping.

Relevant beats comprehensive. Five pages of the right material produces better answers than five hundred pages containing it.

Working with the limit rather than against it

Start fresh when the topic changes. A new conversation with a two-paragraph summary is better than continuing a long one.

Put instructions at the end, immediately before the request, where they carry most weight. Document first, instruction last.

Summarise and restart for long projects. Ask for a summary of what has been established, open a new conversation, paste it in.

Attach only relevant sections of long documents.

Repeat constraints that matter rather than assuming they hold.

Split large tasks. Ten focused conversations beat one enormous one.

Memory features

Several products now store facts across conversations — your preferences, your projects, details you mentioned.

This is a separate mechanism from the context window: stored notes injected into future conversations.

It is worth knowing what is stored. Products that do this generally expose the list. Reviewing it occasionally is worthwhile, because it accumulates things you would not expect and it shapes answers in ways that are hard to trace.

It can be wrong. A misremembered preference persists across conversations and produces consistently odd behaviour until you find and remove it.

It has privacy implications. Anything stored is retained on the provider's systems under their policy.

The practical summary

The model reads everything, every time, and has a fixed budget. Manage that budget deliberately: relevant material, instructions near the request, fresh conversations for new topics, and periodic summaries for long work.

Most complaints about a model being forgetful or inconsistent over a long session are this and are fixable by the user.

Structuring a long input

When you must supply a lot of material, the arrangement affects the result more than people expect.

Instruction last. Put the document first and the question immediately before the model responds. Material at the end of the context carries the most weight.

Repeat the key constraint at both ends for anything long. Once at the top, once at the bottom.

Label the sections. Clear markers between documents prevent the model conflating two sources, which is a common and hard-to-notice failure.

Say what each document is. "The first is our current policy, the second is the proposed revision" prevents a great deal of confusion.

Ask for citations to the section. "Quote the part you based this on" makes it checkable and tends to improve accuracy, because it constrains the answer to something actually present.

Split when it does not fit. Several focused passes with a summary carried between them beats one pass that exceeds what the model handles well.