03 Lesson 3 of 15
Why it never gives quite the same answer twice
One answer is a sample, not a verdict. This is easy to show, and it changes how much you should trust any single reply.
What it said is what it thinks. You asked, it answered, so that is its position. A poor answer means a poor tool. A good answer means a good tool. Either way, you now know where it stands.
Both answers came from the same system in the same state. Neither is its position.
See it in the lessonYou do not. At each step there were many candidates, and one got picked. Ask again and a different one may get picked, and the whole answer can head somewhere else. Both answers came from the same system in the same state. Neither is its position.
Press the button below a few times. It runs the prompt from the last lesson again and again. When two runs pick the same word, they share a block, so you can see where the routes meet and split.
Read this diagram as text
Each press picks three words in a row, using the weights from the last lesson. The log under the buttons lists every run in full. The count above it shows how many different endings have come up. Two runs that pick the same word in the same place pass through the same point.
What to do with this
This gives you a habit. When an answer matters, ask twice. Do not rephrase, and do not ask in the same chat: open a fresh chat and ask the same question. If the two answers agree, that is a weak sign the answer is stable. If they disagree, you have learned something important for almost no effort: it is guessing on this one. If your tool remembers past chats, use a temporary or private chat for the second try, so it starts truly fresh.
This also clears up two opposite mistakes. One bad answer does not prove the tool is useless. One impressive answer does not prove it is reliable. You saw one draw out of many possible ones. Take two.
Some tools let you turn the variation down, often with a setting called temperature. Turning it down makes answers more repeatable. It does not make them more likely to be true. The system just picks the top candidate more often, and the top candidate is the most typical one, not the most correct one.
One analogy. It is closer to asking a crowd than asking one person. Ask the same crowd twice and you get two answers, both typical of that crowd.
Where the analogy breaks: a crowd has people who really know, and you can ask them how sure they are. Here the variation is not experts disagreeing. It is the same process picking differently.
Take a question you have already asked a chatbot. Open a brand new chat and ask it again, word for word. Compare the two. Whatever you find is worth knowing before you rely on either answer.