The next word

What word comes next? “I put the kettle on and made a cup of …”

Keep your guess in mind. A chatbot like ChatGPT meets this kind of question again and again, each time it writes. This lesson shows how it answers.

Chatbots like ChatGPT run on large language models. Google’s machine learning glossary puts it this way: “A chatbot is an application of a large language model.” The chatbot is the app you type into. The language model inside it works out the answer. It is a program trained on a huge amount of text.

A chatbot like ChatGPT writes its answer one small piece at a time. A piece is called a token. It is often a whole word. It can also be part of a word, or a single mark.

A chance says how likely something is. For each piece, the chatbot’s language model works out a chance for every piece that could come next.

Then one piece is picked and added. The same steps run again, and again. This goes on until an end marker is picked, or the answer reaches a length limit.

A chance is a probability: a number from 0 (never) to 1 (certain). The chances for all the possible next pieces add up to 1.

A textbook on language processing (Jurafsky and Martin, draft of 19 August 2026) describes the loop. The model gives each possible next token a probability. Then “we randomly select a token to generate according to its probability in context as defined by the model, generate it, and iterate”. Generation stops at a set length, or when the end-of-sentence token is picked.

Lesson 1.3 shows how text is cut into tokens.

The chances come from training. A language model is trained on a very large amount of text. This text is its training data.

From it, the model learns to give a big chance to pieces that fit what came before. It does not keep a list of answers to look up. It writes each answer piece by piece.

Some chatbot apps also search a set of documents. Lesson 1.5 shows how to tell.

A 2026 research paper describes what training produces: “not a table of canned completions but a high-dimensional approximation to the conditional distribution of language given context”. Put simply, the model holds patterns of which pieces tend to follow which, not stored answers. Lesson 1.7 shows how training works.

Many people picture something else. In an August 2025 online survey of 2,301 US adults (published December 2025), almost half (45%) said an AI like ChatGPT is “looking up the exact answer in a database”. About 1 in 5 (21%) said it is “following a script of prewritten responses”. Just over 1 in 4 (28%) chose “guessing what words should come next based on patterns it learned”. This is one online survey, from the US only.

The pick is usually made at random, but not evenly. A piece with a big chance is picked often. A piece with a small chance is picked sometimes.

So if you ask the same question twice, you can get two different answers.

Many language models have a setting called temperature. It has nothing to do with heat. A low temperature means the most likely piece is picked more often. A high one evens out the chances.

Always taking the most likely piece is called greedy decoding. The textbook says greedy decoding “is so predictable that it is deterministic”: the same input gives the same text. That text tends to be “generic and often quite repetitive”. So the most common method in large language models is sampling: a random pick, weighted by the chances.

Temperature changes how even the chances are before the pick. Below 1, the likely pieces get likelier; close to 0, it becomes greedy decoding. Above 1, the chances are evened out, so unlikely pieces are picked more often. One way to do it: raise each chance to the power 1 over the temperature, then rescale them so they add up to 1. This gives the same result as the textbook’s rule, which divides the model’s raw scores by the temperature.

This is a trade-off, not a truth switch. The textbook reports that ways of picking that favour the most probable words give text people rate as “more accurate, more coherent, and more factual, but also more boring and more repetitive”. A low temperature makes the pick steadier. It checks nothing: if the most likely word is the wrong one, it is picked more often at a low setting too.

People set the temperature, and services differ. Mistral’s API documentation recommends a range of values. Scaleway’s API uses its own value unless told otherwise. Mistral’s page for its Mistral Small 3.2 model recommends a lower one. All three were checked in September 2026.

Even at temperature 0, a post about language-model APIs (Thinking Machines, September 2025) found that “LLM APIs are still not deterministic in practice”. Mistral’s API documentation says that a fixed “seed” makes repeated calls give the same result (checked in September 2026).

There are other ways to trim the unlikely pieces before the pick, such as top-p, which keeps only the most likely pieces that together hold a set share of the chance.

This lesson uses a small toy model. It counts which word came next after the last two words in one book. The book is Dubliners by James Joyce (1914).

The toy’s pieces are whole words and punctuation marks. A chatbot’s language model learns from far more text, and works differently. It takes in the whole text so far, up to a limit.

The toy is a trigram model: it looks only at the last two pieces. Google’s course on language models puts the limit plainly: “the only context a 3-gram provides is the first two words.” Language models like the ones behind chatbots are neural networks that take in the whole text so far, up to a limit that lesson 1.8 explains.

The toy’s chances are counts. For each pair of pieces in the book, it counts which piece came next. A piece’s chance is how often it came after that pair, divided by how often the pair appeared. In the book, “the old” appears 73 times, and the next word is “man” 35 of those times. The textbook calls this ratio a relative frequency.

A word the toy never saw after a pair gets no chance at all, and most pairs are rare: about 3 in 4 of the pairs the toy saw appear only once. A setting like temperature changes chances. It cannot add a word the toy never saw.

The toy’s chances reflect its one book: stories set in Dublin and published in 1914. The textbook says such probabilities “often encode specific facts about a given training corpus”.

The toy learned from Dubliners by James Joyce (1914), public domain in Ireland and the US, from Project Gutenberg. It reads the book in lower case, and keeps runs of letters and six punctuation marks. It drops digits and every other mark.

A likely piece fits the text so far. Nothing in these steps checks the facts. So a chatbot can write fluent text that sounds sure and is wrong.

The toy shows this. In the whole book, “cup of” comes once, in “his cup of beef-tea”. So after “cup of”, only “beef” is ever picked, and “tea” never is.

The toy is sure, and it is not what you expected. The toy’s chances are counts from one book, and likely is not the same as true.

In a study published in January 2025, people judged answers from GPT-3.5, PaLM2 and GPT-4o. With the models’ usual explanations, they were only slightly better than guessing at telling right from wrong.

How sure a reply sounds is not proof. Check anything that matters.

Google’s glossary calls wrong but fluent output a hallucination: “The production of plausible-seeming but factually incorrect output by a generative AI model that purports to be making an assertion about the real world.” Lesson 1.6 shows why a chatbot can make things up.

In the January 2025 study, 301 people living in the US read answers and explanations written by GPT-3.5, PaLM2 and GPT-4o, and judged how likely each answer was to be right. With the models’ usual explanations, they were only slightly better than random guessing at telling right answers from wrong ones. The models’ own chances did much better. Longer explanations made people more confident, without making them any better at it.

The limits: one study, with people in the US, and three named models, the newest GPT-4o. Newer models were not tested.

How sure a text sounds comes from which pieces were likely, not from a check. That is not a reason to distrust every reply: the textbook reports that favouring likely words gives text people rate as more accurate. It is a reason to check what matters.

The same goes for tone. A caring reply is also written one likely piece at a time, and a caring tone is not proof of caring (lesson 1.1). In July 2023 (published April 2024), 2 in 3 (67%) of 300 US adults did not rule out that ChatGPT has experiences such as feelings; 1 in 3 (33%) said it definitely does not. In January 2025, 4 in 10 (40%) of 500 US chatbot users said the one they use most acts like it understands them, at least some of the time. That question asks how it seems, not what people believe.

The opposite mistake is to say that it only predicts words, so it understands nothing. A 2026 paper calls that “the mirror-image error”.

Every survey and the study here are from the US. None was found for Ireland or for neurodivergent learners.