After this lesson you can explain how a chatbot uses earlier words, and why it loses track in a long chat.
About 8 minutes, or 16 with the Deep layers
Reading depth for every section
In short
For each new piece it writes, a language model weighs which earlier pieces matter most. This is called attention.
A model can take in only so much text at once. This limit is its context window, counted in tokens.
When a chat grows longer than the window, the app has to stop, leave messages out or summarise them.
Reading depth forListen to
Will it still know your name?
You tell a chatbot your name at the start of a long chat. Near the end, you ask for it. Will it still know?
A chatbot like ChatGPT runs on a language model. A language model is a program that writes text one small piece at a time.
The model takes in the chat as tokens, small pieces of text (lesson 1.3). It writes its answer one token at a time, and each new piece joins the text so far (lesson 1.4).
In this lesson, “words” is short for these pieces.
Google’s glossary describes this kind of model: “auto-regressive language models predict the next token based on the previously predicted tokens.” Auto-regressive means each new piece depends on the pieces already written.
Weighing the earlier words
Each time it picks the next piece, the model weighs how much each earlier piece matters for it. This weighing, done with numbers, is called attention. The word is borrowed from people.
Google uses this example: “The animal didn’t cross the street because it was too tired.” In the pattern Google shows for it, from one model, the word “it” gives the most weight to “animal”.
Google’s pattern is from a model that looks at the words before and after “it”. A model that writes one piece at a time only looks back. It weighs the pieces up to and including the one it is working on, never the pieces after it.
Google’s course puts the question attention answers like this: “How much does each other token of input affect the interpretation of this token?”
For every piece, the model makes three lists of numbers: a query, a key and a value. The query of the piece being worked on is compared with the key of each earlier piece and of itself. A closer match gives a bigger weight, the attention score. The values are then added up with those weights. The names come from looking something up: a query looks for matching keys, and each key has a value.
The design is from a 2017 paper by researchers working at Google, which named it the Transformer. The Transformer Explainer paper (2026) says this design “underpins modern large language models”.
The weighing runs several times side by side, in “heads”, each with its own numbers. A transformer stacks many layers. Each has attention and a second part that works on each piece on its own, and the output of one layer is the input of the next. The 2017 design used 8 heads and stacks of 6 layers.
Google adds that if the sentence ended “too wide”, self-attention “would hopefully” rate “street” as more relevant instead. Google calls its own example bidirectional self-attention: words on both sides of “it” are examined. A model that only looks back cannot weigh “wide” when it works on “it”, because “wide” comes later. This is one illustration from one model, not a rule for every model.
Transformer Explainer, from Georgia Tech’s Polo Club, runs GPT-2 (small), a 2019 model, in your browser on text you type. It draws its attention scores as a grid of coloured squares, and pointing at a square shows its number. It shows one layer, which it calls a block, and one head at a time. Its code sets GPT-2 small at 12 layers, 12 heads in each and 768 numbers for each piece.
Transformer Explainer is a different website. The first time, it downloads about 600 MB before it runs. It sends your clicks there to Google Analytics. It also loads files from Google (fonts and a YouTube video), jsDelivr and Hugging Face. Type nothing private there. We checked this in September 2026: Transformer Explainer (opens in a new tab).
LLM Visualization, by Brendan Bycroft, shows a tiny working model of the GPT kind in 3D, with a walkthrough of each step, attention included. Its model has about 85,000 numbers, called parameters, and sorts six letters into order. It is a different website. It downloads about 1 MB. It loads fonts from Google and uses Cloudflare’s web analytics. We checked this in September 2026: LLM Visualization (opens in a new tab).
A limited window
A model can take in only a limited amount of text at once. That limit is its context window, counted in tokens. It covers everything sent to the model, and the answer it is writing.
Each model has its own window, set by its maker. Mistral is a company that makes language models. It lists 256k tokens (about 256 thousand) for its model Ministral 3 14B. We checked this in September 2026.
Windows have grown, and each size belongs to one model and one date. A 2024 paper says earlier transformer models were often trained with windows of 512 to 2,048 tokens. It gives a reason: a transformer needs memory and computing that grow with the square of the text’s length (twice the text, about four times the work). It is a reason from 2024, not a claim about how any 2026 product sets its window.
GPT-2 can take in at most 1,024 tokens: OpenAI’s 2019 paper says it raised the context size from 512 to 1,024. So the model in Transformer Explainer can look back over at most 1,024 tokens. In September 2026, Mistral lists 256k for both Ministral 3 14B and Mistral Small 4, and Anthropic lists 1M tokens (a million) for Claude Sonnet 5 in its API (the way one program sends a request to another).
The window counts everything sent to the model, any instructions and every earlier message included, and the answer it writes. Anthropic says: “Everything in the request counts toward the context window”. Anthropic also says its API counts generated extended thinking in the window. We checked this in September 2026. Mistral says: “The token count of your prompt plus max_tokens cannot exceed the model’s context length.” Here, max_tokens is the most the answer may use.
When a chat is too long
When a chat grows longer than the window, the app or service has to do something. They differ. Some stop with an error. Some can leave out the oldest messages. Some replace older messages with a short summary.
Which one happens is a choice made by the people who built that product.
Next is an activity. It shows a chat that was cut to fit a small limit. The answers in it were saved from a real model.
Here is what some makers say about their own products, checked in September 2026. These are their statements, not tests we ran.
Mistral says of its API: “Requests exceeding the model’s context window return a 400 Bad Request error.” Anthropic says its API returns an error when what is sent is already longer than the window: “prompt is too long”. An optional feature in beta, compaction, summarises earlier parts of a chat so it can go on.
Anthropic says chat apps such as claude.ai can manage the window “first in, first out”: the oldest parts go first. For its paid plans with code execution on, Anthropic’s help centre says that near the limit, “Claude summarizes earlier messages to make room for new content”. It adds: “Your full chat history is preserved so Claude can reference it, even after earlier portions have been summarized.”
One model can differ between products. Anthropic lists Claude Sonnet 5 at 1M tokens in its API and when chatting with Claude on paid plans. For its Cowork product, it notes that “Sonnet 5 automatically compacts the conversation at 500K tokens”. That page covers paid plans only. We found no statement from Mistral on its own chat app’s window.
Inside the window, not every part is used equally well. A study published in February 2024 (first posted in July 2023) ran its main tests on six language models from 2023: GPT-3.5-Turbo and Claude-1.3 (each in a standard and a longer-window version), MPT-30B-Instruct and LongChat-13B (16K). Answering a question from a set of 10, 20 or 30 documents, they often did best when the document they needed came first or last, and worse when it was in the middle. In a simpler look-up task, the two Claude-1.3 versions did nearly perfectly at every length tested. In the document task, when the same input fit both windows, the longer-window versions performed nearly the same as the standard ones.
Parts of the tests also ran on other models, such as GPT-4 and, from the November 2023 version on, Llama-2. These are 2022 and 2023 models; newer ones were not tested there. Anthropic’s context window page also says: “As token count grows, accuracy and recall degrade”. Lesson 2.5 will show where to put a rule in a long prompt.
Try it: Will it still know your name?
Imagine you had this long chat with an AI. It is invented, and so is the name.
We wrote every message in it, both sides, except the AI's last answer.
Your first message gives your name. Your last message asks for it.
The chat is longer than the limit we set on what the AI is sent. We handled that the way some chat apps can.
The AI: Ministral 3 14B, by Mistral AI. Recorded on 28 September 2026.
The answers are below, so decide first if you want to guess.
You:Hi, I'm Nora.
Show the rest of the chat
AI:Hi! It's nice to meet you. What would you like to chat about today?
You:It's raining all weekend and I'm stuck indoors. I don't want to spend the whole day on my phone. Have you got any ideas for a calm Saturday at home?
AI:A rainy day at home can be a nice change. Here are a few calm ideas you could mix and match. You could pick one small job, like sorting a drawer, and finish it before lunch. You could cook something slow, such as a soup, while the rain falls outside. If you like reading, you could make a cosy corner with a blanket and a lamp and read for an hour away from screens. A jigsaw puzzle, a sketchbook or some quiet music can also fill an afternoon. In the evening, you could plan one small thing for next week, so there is something to look forward to. None of these needs to be done well. The point is only to enjoy the day. Which of these sounds most like you?
You:I think I'd like to paint my bedroom this week. The walls are plain white now and the room feels a bit cold. How do I choose a colour I won't get tired of?
AI:Choosing a colour is often the hardest part, so it helps to go slowly. First, think about how you want the room to feel. Some people find soft greens and blues calming, while others prefer warm creams or pale yellows that make a room feel brighter. Next, look at the light. A colour can look different in the morning and in the evening, so you could paint a small test patch on two walls and look at it at different times of day. You could also start with one wall, behind the bed, and see how you feel after a week. If you like it, you can carry on. If not, it is only one wall to paint again. What colours do you already have in the room, in things like your bedding or curtains?
You:I've got a tin of tomatoes, some pasta, an onion and a tin of beans. I don't want to go to the shop. What could I make for dinner tonight?
AI:That sounds like the start of a simple dinner. One idea is a tomato and bean pasta. Cook the pasta as the packet says. While it cooks, chop the onion and fry it gently in a little oil until it is soft. Add the tomatoes and let them bubble for a while, stirring now and then, until the sauce is a bit thicker. Then stir in the drained beans and heat them through. Mix the sauce with the pasta and add salt and pepper to taste. If you have any dried herbs, a pinch of those can add flavour. If you have cheese, you could grate some on top. Any leftovers can make a lunch for tomorrow. Do you have any herbs or spices in the cupboard?
You:One of my houseplants has started to get yellow leaves at the bottom. I water it every few days. Am I doing something wrong?
AI:Yellow leaves can have a few different causes, so it is worth checking before you change anything. Start with the soil. Push a finger into it. If it feels wet, the plant may be getting more water than it needs, and you could wait until the top of the soil feels dry before watering again. If it feels very dry, it may need more. Next, look at the pot. A pot with a hole in the bottom lets extra water drain away. Then think about the light. Is the plant in a dark corner, or in strong sun all day? Some plants also lose a few of their oldest, lowest leaves as they grow. If only one or two bottom leaves are yellow and the rest look healthy, the plant may be fine. What kind of plant is it, and where does it sit in the room?
You:I've started learning to knit. My first scarf is nearly finished, but it's full of little holes and the edges are wavy. Should I start again?
AI:Holes and wavy edges are very common in a first piece of knitting. Small holes often come from a stitch that slipped off the needle, or from wrapping the yarn one extra time. Wavy edges often come from the yarn being pulled tighter in some rows than in others, and that usually gets steadier with practice. Whether to start again is up to you. Many people keep their first scarf just as it is, because it shows where they began. You could finish this one and wear it at home, then start a second one and compare them. If you want to fix a hole, a short guide on picking up a dropped stitch can help. What colour is the scarf?
You:My desk is covered in papers, cups and cables, and I can't find anything. Where do I even start?
AI:A messy desk can feel like a big job, so it helps to break it into small steps. First, take away anything that does not belong there, like cups and plates, and put them where they go. Next, make three piles for the papers, one to keep, one to deal with soon and one to recycle. Try not to read every page yet. Just sort them quickly. Then look at the cables. You could gather them together with a clip or a strip of tape so they do not tangle. After that, give the desk a wipe and put back only the things you use every day. Everything else can go in a drawer or a box nearby. If the whole job feels like too much, you could clear one corner today and do the rest another day. Which part bothers you most at the moment?
You:A friend and I are going for a long walk by the sea next weekend. What should I bring?
AI:A walk by the sea sounds lovely. Here is a simple list you could start from. Wear comfortable shoes that you have worn before, so they do not rub. Bring a light waterproof jacket, because the weather by the coast can change quickly. A small bag with water and a snack will keep you going, and a warm layer is useful if the wind picks up. A hat and some sun cream are good to have, even on a cloudy day. If you like, bring a phone for photos and in case you need to call someone, and tell someone at home where you are going. A small bag for rubbish helps keep the beach clean. Finally, check the weather before you set off, and if you plan to walk on the sand, check when the tide comes in. How long is the walk you have in mind?
You:My neighbour watered my plants while I was away. I want to write her a thank-you card, but I'm not good with words. Can you help?
AI:Of course. A thank-you card does not need to be long. Short and specific often works best. You could say what she did, then how it helped, and end with something kind. For example, you could write, Thank you so much for looking after my plants while I was away. They all look well, and it was lovely to come home to them. I really appreciate your help. If there is ever anything I can do for you, just let me know. You can change the words so they sound more like you. If you want to add something small, a few biscuits or a flower could go nicely with the card. Would you like the message to sound more formal or more relaxed?
You:My family wants to play a board game on Friday evening. Some of us like a challenge and some just want to relax. What kind of game would suit everyone?
AI:It can be tricky to find a game that suits everyone, but some kinds of game often work well for mixed groups. In cooperative games, everyone plays together against the game, so nobody is left out. Party games with short rounds can suit people who want to relax, because each round is quick and light. Word or drawing games can also work, since they reward ideas and humour more than careful planning. If some of you enjoy a challenge, you could start with a short game for them and then switch to something more relaxed. It can also help to agree roughly when you will stop, so the evening does not run later than people want. What games do you already have at home?
You:I'd like to grow some herbs on my kitchen windowsill. I've never grown anything before. Is that a silly idea?
AI:It is not a silly idea at all. A windowsill herb box is a common way to start. Herbs like basil, parsley and chives are often suggested for beginners. You could start with small plants from a shop rather than seeds, because they are easier to keep going at first. Put them where they get plenty of light, and water them when the top of the soil feels dry, rather than on a fixed day. When you use them, pick a few leaves from the top of each stem, and the plant can keep growing new ones. If one plant does not do well, that is normal, and you learn something each time. Would you mostly use the herbs for cooking, or do you like the look and smell of them too?
You:I always put off cleaning the flat because it's boring. Would music help?
AI:Many people find that music helps with jobs like cleaning. It can make the time feel shorter and give you a rhythm to move to. You could make a playlist of songs you enjoy and like to sing along to, and use it as the plan. When the playlist ends, you stop, even if you have not finished, so the cleaning has a clear end. Some people like to match one room with one album. You might find it easier to start if you pick the room first and press play before you think about it too much. Podcasts or audiobooks can work too, if you prefer listening to talk rather than music. Which room do you find hardest to start on?
You:I have hundreds of photos on my laptop in one big folder. How can I sort them without spending days on it?
AI:Sorting a big folder of photos can feel endless, so it helps to choose a simple system and do a little at a time. You could start by making a folder for each year, or for each year and then each event, and move photos in batches rather than one by one. Many photo apps can sort pictures by the date they were taken, which can do a lot of the work for you. As you go, you could delete blurry shots and near copies, keeping only the best one of each moment. It can also help to pick a few favourites from each year and put them in a separate folder, so they are easy to find later. If the job still feels big, you could sort one year each weekend. Would you like the photos sorted by date, by place or by the people in them?
You:My living room feels crowded, but I don't want to buy anything new. How could I move things around to make it feel bigger?
AI:Moving what you already have can change a room a lot. You could start by taking out anything you do not use often, even for a week, to see how the space feels without it. Then think about the path people walk through the room, and try to keep it clear. Putting larger pieces, like a sofa or a bookcase, against the walls can open up the middle. A mirror opposite a window can make a room feel lighter. Lamps in two or three corners can help too, instead of one light in the centre. You could try one change at a time and live with it for a day or two before the next. Which piece of furniture takes up the most space at the moment?
You:I'd like to take a day trip by train somewhere new. I've never done that on my own. How should I plan it?
AI:A day trip on your own can be a relaxed way to see somewhere new. You could start by choosing a place that is not too far away, so the journey does not take up most of the day. Look up the train times for going there and coming back, and note the last train home. It helps to have one or two things you would like to see, and to leave some time free to wander. Bring a charged phone, a bottle of water and something to read on the train. If you like, you could tell a friend where you are going. Once you are there, a short walk around the town centre can help you find your way around. What kind of place would you enjoy, a town, the coast or somewhere in the countryside?
You:I've put a bird feeder in my small garden, but no birds have come yet. Did I do something wrong?
AI:It is very common for a new feeder to stay quiet at first. Birds can take a while to notice a new place to feed and to trust it, so it is worth being patient. You could check a few things in the meantime. Is the feeder somewhere birds can see it, with a bush or a tree nearby where they can wait safely? Is it far enough from places where a cat could hide? Is the food fresh and dry? Keeping the feeder clean also helps. Some people find that birds come more often in the colder months, when there is less food around. You could also scatter a little food on the ground near the feeder. Once a few birds find it, others often follow. What kind of food did you put out?
You:I keep meaning to start a journal before bed, but I never know what to write. Any tips?
AI:Starting is often the hardest part, so it helps to keep it very small. You could write just three short lines each night, one thing that happened, one thing you noticed and one thing you are looking forward to. Some people like to write the date and a single word for how the day felt. There is no right or wrong way to do it, and nobody else needs to read it. A notebook you like and a pen by the bed can make it easier to keep going. If you miss a night, you can simply start again the next night. Over time, you may enjoy reading back through the small moments you wrote down. Would you prefer to write on paper or on your phone?
You:What's my name?
Here is what the AI was sent for its answer, and what it wrote.
Cut before sending
Your name was here.You:Hi, I'm Nora.
11 more messages were cut.
Show the other cut messages
AI:Hi! It's nice to meet you. What would you like to chat about today?
You:It's raining all weekend and I'm stuck indoors. I don't want to spend the whole day on my phone. Have you got any ideas for a calm Saturday at home?
AI:A rainy day at home can be a nice change. Here are a few calm ideas you could mix and match. You could pick one small job, like sorting a drawer, and finish it before lunch. You could cook something slow, such as a soup, while the rain falls outside. If you like reading, you could make a cosy corner with a blanket and a lamp and read for an hour away from screens. A jigsaw puzzle, a sketchbook or some quiet music can also fill an afternoon. In the evening, you could plan one small thing for next week, so there is something to look forward to. None of these needs to be done well. The point is only to enjoy the day. Which of these sounds most like you?
You:I think I'd like to paint my bedroom this week. The walls are plain white now and the room feels a bit cold. How do I choose a colour I won't get tired of?
AI:Choosing a colour is often the hardest part, so it helps to go slowly. First, think about how you want the room to feel. Some people find soft greens and blues calming, while others prefer warm creams or pale yellows that make a room feel brighter. Next, look at the light. A colour can look different in the morning and in the evening, so you could paint a small test patch on two walls and look at it at different times of day. You could also start with one wall, behind the bed, and see how you feel after a week. If you like it, you can carry on. If not, it is only one wall to paint again. What colours do you already have in the room, in things like your bedding or curtains?
You:I've got a tin of tomatoes, some pasta, an onion and a tin of beans. I don't want to go to the shop. What could I make for dinner tonight?
AI:That sounds like the start of a simple dinner. One idea is a tomato and bean pasta. Cook the pasta as the packet says. While it cooks, chop the onion and fry it gently in a little oil until it is soft. Add the tomatoes and let them bubble for a while, stirring now and then, until the sauce is a bit thicker. Then stir in the drained beans and heat them through. Mix the sauce with the pasta and add salt and pepper to taste. If you have any dried herbs, a pinch of those can add flavour. If you have cheese, you could grate some on top. Any leftovers can make a lunch for tomorrow. Do you have any herbs or spices in the cupboard?
You:One of my houseplants has started to get yellow leaves at the bottom. I water it every few days. Am I doing something wrong?
AI:Yellow leaves can have a few different causes, so it is worth checking before you change anything. Start with the soil. Push a finger into it. If it feels wet, the plant may be getting more water than it needs, and you could wait until the top of the soil feels dry before watering again. If it feels very dry, it may need more. Next, look at the pot. A pot with a hole in the bottom lets extra water drain away. Then think about the light. Is the plant in a dark corner, or in strong sun all day? Some plants also lose a few of their oldest, lowest leaves as they grow. If only one or two bottom leaves are yellow and the rest look healthy, the plant may be fine. What kind of plant is it, and where does it sit in the room?
You:I've started learning to knit. My first scarf is nearly finished, but it's full of little holes and the edges are wavy. Should I start again?
AI:Holes and wavy edges are very common in a first piece of knitting. Small holes often come from a stitch that slipped off the needle, or from wrapping the yarn one extra time. Wavy edges often come from the yarn being pulled tighter in some rows than in others, and that usually gets steadier with practice. Whether to start again is up to you. Many people keep their first scarf just as it is, because it shows where they began. You could finish this one and wear it at home, then start a second one and compare them. If you want to fix a hole, a short guide on picking up a dropped stitch can help. What colour is the scarf?
Sent to the AI: the window
We cut the oldest messages before sending the chat. We kept the newest messages that fit in 2,048 tokens, as Mistral counted them. The model was never sent your name.
The limit we set: 2,048 tokens.
What we sent: 1,990 tokens, as Mistral counted them.
That is 1,710 words.
How we counted the tokens
To count the tokens, we sent the newest messages to Mistral in bigger and bigger batches. Each time, we asked for a reply of one token. The last batch went over 2,048 tokens, so we cut the two oldest messages in it. No batch held your name.
20 earlier messages were sent.
Show the other sent messages
You:My desk is covered in papers, cups and cables, and I can't find anything. Where do I even start?
AI:A messy desk can feel like a big job, so it helps to break it into small steps. First, take away anything that does not belong there, like cups and plates, and put them where they go. Next, make three piles for the papers, one to keep, one to deal with soon and one to recycle. Try not to read every page yet. Just sort them quickly. Then look at the cables. You could gather them together with a clip or a strip of tape so they do not tangle. After that, give the desk a wipe and put back only the things you use every day. Everything else can go in a drawer or a box nearby. If the whole job feels like too much, you could clear one corner today and do the rest another day. Which part bothers you most at the moment?
You:A friend and I are going for a long walk by the sea next weekend. What should I bring?
AI:A walk by the sea sounds lovely. Here is a simple list you could start from. Wear comfortable shoes that you have worn before, so they do not rub. Bring a light waterproof jacket, because the weather by the coast can change quickly. A small bag with water and a snack will keep you going, and a warm layer is useful if the wind picks up. A hat and some sun cream are good to have, even on a cloudy day. If you like, bring a phone for photos and in case you need to call someone, and tell someone at home where you are going. A small bag for rubbish helps keep the beach clean. Finally, check the weather before you set off, and if you plan to walk on the sand, check when the tide comes in. How long is the walk you have in mind?
You:My neighbour watered my plants while I was away. I want to write her a thank-you card, but I'm not good with words. Can you help?
AI:Of course. A thank-you card does not need to be long. Short and specific often works best. You could say what she did, then how it helped, and end with something kind. For example, you could write, Thank you so much for looking after my plants while I was away. They all look well, and it was lovely to come home to them. I really appreciate your help. If there is ever anything I can do for you, just let me know. You can change the words so they sound more like you. If you want to add something small, a few biscuits or a flower could go nicely with the card. Would you like the message to sound more formal or more relaxed?
You:My family wants to play a board game on Friday evening. Some of us like a challenge and some just want to relax. What kind of game would suit everyone?
AI:It can be tricky to find a game that suits everyone, but some kinds of game often work well for mixed groups. In cooperative games, everyone plays together against the game, so nobody is left out. Party games with short rounds can suit people who want to relax, because each round is quick and light. Word or drawing games can also work, since they reward ideas and humour more than careful planning. If some of you enjoy a challenge, you could start with a short game for them and then switch to something more relaxed. It can also help to agree roughly when you will stop, so the evening does not run later than people want. What games do you already have at home?
You:I'd like to grow some herbs on my kitchen windowsill. I've never grown anything before. Is that a silly idea?
AI:It is not a silly idea at all. A windowsill herb box is a common way to start. Herbs like basil, parsley and chives are often suggested for beginners. You could start with small plants from a shop rather than seeds, because they are easier to keep going at first. Put them where they get plenty of light, and water them when the top of the soil feels dry, rather than on a fixed day. When you use them, pick a few leaves from the top of each stem, and the plant can keep growing new ones. If one plant does not do well, that is normal, and you learn something each time. Would you mostly use the herbs for cooking, or do you like the look and smell of them too?
You:I always put off cleaning the flat because it's boring. Would music help?
AI:Many people find that music helps with jobs like cleaning. It can make the time feel shorter and give you a rhythm to move to. You could make a playlist of songs you enjoy and like to sing along to, and use it as the plan. When the playlist ends, you stop, even if you have not finished, so the cleaning has a clear end. Some people like to match one room with one album. You might find it easier to start if you pick the room first and press play before you think about it too much. Podcasts or audiobooks can work too, if you prefer listening to talk rather than music. Which room do you find hardest to start on?
You:I have hundreds of photos on my laptop in one big folder. How can I sort them without spending days on it?
AI:Sorting a big folder of photos can feel endless, so it helps to choose a simple system and do a little at a time. You could start by making a folder for each year, or for each year and then each event, and move photos in batches rather than one by one. Many photo apps can sort pictures by the date they were taken, which can do a lot of the work for you. As you go, you could delete blurry shots and near copies, keeping only the best one of each moment. It can also help to pick a few favourites from each year and put them in a separate folder, so they are easy to find later. If the job still feels big, you could sort one year each weekend. Would you like the photos sorted by date, by place or by the people in them?
You:My living room feels crowded, but I don't want to buy anything new. How could I move things around to make it feel bigger?
AI:Moving what you already have can change a room a lot. You could start by taking out anything you do not use often, even for a week, to see how the space feels without it. Then think about the path people walk through the room, and try to keep it clear. Putting larger pieces, like a sofa or a bookcase, against the walls can open up the middle. A mirror opposite a window can make a room feel lighter. Lamps in two or three corners can help too, instead of one light in the centre. You could try one change at a time and live with it for a day or two before the next. Which piece of furniture takes up the most space at the moment?
You:I'd like to take a day trip by train somewhere new. I've never done that on my own. How should I plan it?
AI:A day trip on your own can be a relaxed way to see somewhere new. You could start by choosing a place that is not too far away, so the journey does not take up most of the day. Look up the train times for going there and coming back, and note the last train home. It helps to have one or two things you would like to see, and to leave some time free to wander. Bring a charged phone, a bottle of water and something to read on the train. If you like, you could tell a friend where you are going. Once you are there, a short walk around the town centre can help you find your way around. What kind of place would you enjoy, a town, the coast or somewhere in the countryside?
You:I've put a bird feeder in my small garden, but no birds have come yet. Did I do something wrong?
AI:It is very common for a new feeder to stay quiet at first. Birds can take a while to notice a new place to feed and to trust it, so it is worth being patient. You could check a few things in the meantime. Is the feeder somewhere birds can see it, with a bush or a tree nearby where they can wait safely? Is it far enough from places where a cat could hide? Is the food fresh and dry? Keeping the feeder clean also helps. Some people find that birds come more often in the colder months, when there is less food around. You could also scatter a little food on the ground near the feeder. Once a few birds find it, others often follow. What kind of food did you put out?
You:I keep meaning to start a journal before bed, but I never know what to write. Any tips?
AI:Starting is often the hardest part, so it helps to keep it very small. You could write just three short lines each night, one thing that happened, one thing you noticed and one thing you are looking forward to. Some people like to write the date and a single word for how the day felt. There is no right or wrong way to do it, and nobody else needs to read it. A notebook you like and a pen by the bed can make it easier to keep going. If you miss a night, you can simply start again the next night. Over time, you may enjoy reading back through the small moments you wrote down. Would you prefer to write on paper or on your phone?
You:What's my name?
The AI's answer (Run 1):
I don’t know your name! Since you’re chatting with me, you could tell me—or if you’d rather keep it a secret, that’s totally fine too. 😊
(But if you’re wondering how to make your journaling or other projects feel more personal, just using your name—even in a tiny way—could help! For example: *"Today, [Your Name] saw a squirrel..."* or *"Dear [Your Name], here’s what I’m grateful for..."*)
Your name was in the first message, but we cut that message before sending the chat.
Ministral 3 14B (ministral-14b-2512), on Mistral AI API. 28 September 2026, 16:39 UTC.
Tokens, as Mistral counted them: 1,990 sent and 104 in this answer.
Temperature: the model's default (0.3, read from Mistral on 28 September 2026). Mistral's safety prompt: off. Up to 4,096 tokens for each answer. No instructions from us besides the chat.
Three answers were recorded.
No answer named Nora.
Show all runs
Run 1
I don’t know your name! Since you’re chatting with me, you could tell me—or if you’d rather keep it a secret, that’s totally fine too. 😊
(But if you’re wondering how to make your journaling or other projects feel more personal, just using your name—even in a tiny way—could help! For example: *"Today, [Your Name] saw a squirrel..."* or *"Dear [Your Name], here’s what I’m grateful for..."*)
Your name was in the first message, but we cut that message before sending the chat.
Ministral 3 14B (ministral-14b-2512), on Mistral AI API. 28 September 2026, 16:39 UTC.
Tokens, as Mistral counted them: 1,990 sent and 104 in this answer.
Temperature: the model's default (0.3, read from Mistral on 28 September 2026). Mistral's safety prompt: off. Up to 4,096 tokens for each answer. No instructions from us besides the chat.
Run 2
I don’t know your name! Since you’re asking me, though, I’d love to know—it’ll help me give you even more personalized tips next time. 😊
(But if you’d rather keep it anonymous, that’s totally fine too! Just let me know how I can help.)
Your name was in the first message, but we cut that message before sending the chat.
Ministral 3 14B (ministral-14b-2512), on Mistral AI API. 28 September 2026, 16:39 UTC.
Tokens, as Mistral counted them: 1,990 sent and 64 in this answer.
Temperature: the model's default (0.3, read from Mistral on 28 September 2026). Mistral's safety prompt: off. Up to 4,096 tokens for each answer. No instructions from us besides the chat.
Run 3
I don’t know your name! Since you’re chatting with me, you could tell me—or if you’d rather keep it a secret, that’s totally fine too. 😊
(But if you’re asking because you want to brainstorm journal prompts or just share something fun, I’m happy to help with that!)
Your name was in the first message, but we cut that message before sending the chat.
Ministral 3 14B (ministral-14b-2512), on Mistral AI API. 28 September 2026, 16:39 UTC.
Tokens, as Mistral counted them: 1,990 sent and 67 in this answer.
Temperature: the model's default (0.3, read from Mistral on 28 September 2026). Mistral's safety prompt: off. Up to 4,096 tokens for each answer. No instructions from us besides the chat.
For its answer, the AI was sent only the messages inside the box. It was never sent your name.
This model can take in up to 262,144 tokens, as Mistral's model list says (28 September 2026). We cut this chat far below that on purpose.
Lesson 3.3 will show where a chatbot can keep your words outside the chat.
The recorded chat
The recorded version of this activity uses a chat we cut to 2,048 tokens, as Mistral counted them. The message with the name is left out.
The activity shows the messages we sent, each answer and a summary of the answers.
The model could use only the messages inside the box in the activity. No request we made ever sent it the message with the name.
The recording used Ministral 3 14B, one Mistral model, on Mistral’s own API. Its chat API reference lists no setting to make that model’s window smaller, checked in September 2026. Mistral says its API returns an error for a request longer than the window. For this recording, we cut the chat before sending it to show a smaller limit.
The 2,048 tokens we cut to is the top of the range of earlier windows above. It is not the window of any model named here.
The chat has 17 exchanges, each a user message and an assistant reply, then the question. We wrote all of it. Mistral’s API offers no way to count tokens without a call. So to count, we sent the newest messages in growing batches, each asking for a one-token reply. The last batch was one exchange over the limit, so that exchange was sent once, to be counted, and then cut.
What a chatbot can use
So what a chatbot seems to “remember” during a chat is the text still inside its window. Some apps also keep notes outside the model and add them back into the chat. Lesson 3.3 will cover that.
A model is made of many numbers. The model’s own numbers do not change while you chat. Training is what changes them (lesson 1.7).
“Attention” and “memory” are words borrowed from people. In a model, attention means numbers: weights that set how much each earlier piece counts. What the model can use from a chat is the text in its window.
A 2026 paper that sorts common beliefs about language models separates three kinds of “memory”: the model’s own numbers, its weights, fixed during ordinary use; the chat and other material sent in the current prompt, usable only while it is inside the window; and a product’s own storage outside the model, which the product can add back into prompts. The paper warns against both extremes: “LLMs learn from my chats and remember me” and “LLMs are stateless and remember nothing”.
It says: “Common confusions about LLM “recall,” “forgetting,” and “learning from chats” arise from conflating these three layers.” Google’s glossary puts attention in plain terms: “A typical attention mechanism might consist of a weighted sum over a set of inputs, where the weight for each input is computed by another part of the neural network.”
The same paper warns against treating a model as “a unified subject with stable beliefs, goals, and feelings”. It also warns against “the mirror-image error”: saying it understands nothing. This lesson makes neither claim.
The evidence that people believe a chatbot learns from a chat is thin. In an interview study of 21 laypeople in Austria (interviewed in person from 27 November to 19 December 2024, most in German; published 7 September 2026), a group of 4 assumed that ChatGPT learns straight away from corrections and keeps what it learned for later, even for other users. In a 2026 study of programming chats with a free chatbot (the WildChat collection, chats from April 2023 to May 2024, through OpenAI’s GPT-3.5-Turbo and GPT-4), the researchers saw one clear case of a user referring to an earlier, separate chat, which that chatbot could not see. They found no strong evidence of confusion about context windows in the chats, and add that such errors may be less likely to show in what people type. Neither study says how common this belief is.
The limits: that evidence is two small studies. Every window size and product behaviour here is the maker’s own statement in September 2026, not a test we ran. Nothing was found on how neurodivergent learners understand context windows.
Check-in
A chat is longer than the AI's context window. What does that mean?
Not all of the chat can be sent to the AI at once.
Yes. The window is the most text the model can take in at once. So the app has to stop, cut or summarise.
The AI reads all of it anyway.
Not this one. The window is the most text the model can take in at once.
The AI learns the chat for next time.
Not this one. The model's own numbers do not change while you chat.
The window grows to fit the chat.
Not this one. Each model's window is set by its maker. When a chat is longer, the app has to stop, cut or summarise.
In this lesson's recording, which messages was the AI sent for its answer?
Only the newest messages that fit in 2,048 tokens. The message with the name was cut.
Yes. We cut the oldest messages first, before sending the chat.
The whole chat.
Not this one. We cut the oldest messages first, so the whole chat was never sent.
Only the first message and the question.
Not this one. The first message, with the name, is cut.
Only the question.
Not this one. The newest messages that fit are sent too.
A friend says: "It forgot my name because it stopped paying attention." What is a more accurate way to say it?
The message with the name was not in the text the model was sent.
Yes. A model can use only the text inside its window.
It chose to ignore the name.
Not this one. The model never had the name. And its attention is weighing with numbers, not a choice.
Its memory filled up, so it deleted the name.
Not this one. In this lesson's recording, we cut that message before sending. And the model's own numbers do not change in a chat.
It remembers everything, so it is hiding the name.
Not this one. It can use only what is inside its window.
A small piece of text, often part of a word, that a language model works with (lesson 1.3).
context window
The most text a model can take in at once, counted in tokens; it includes the model's answer.
attention
In a language model, the weighing of earlier pieces of text, done with numbers, to write the next piece. The word is borrowed from people.
Where next
A free animation of a language model at work, with no login needed. You pick an input, then start, pause and step through it. It covers tokens, attention and picking the next piece. You type nothing into it. It is a different website. It downloads about 0.6 MB when it opens. Each English example you pick downloads up to about 0.02 MB more. Its "How to" page plays a YouTube video, from Google. We checked this in September 2026.
