A model you can hand work to

What it includes

Four parts. The first two are the minimum that makes the rest worth anything.

• 01

  Rules written down once

  What is allowed, what is not, and what a correct result looks like — in a file the model reads before every task. While the rules live in your head, every new text is a lottery.

• 02

  Three rings of checking

  The finished result passes three independent checks, and each one catches a different kind of error. A single check waves through exactly what it cannot see.

• 03

  A set of tasks with known answers

  Questions whose correct answer is known in advance. After every edit to the rules the whole set runs again, and you can see whether it got better or worse.

• 04

  Permission to say «I don't know»

  Refusal has to be an allowed answer, or the model will invent something rather than stay silent. This is the cheapest and the most overlooked setting of all.

Three rings of checking

The rings run in order and work differently. That is the point: each one sees what the one before it lets through.

• First ring: scripts

  Ordinary programs check what can be checked unambiguously: a banned word, a broken format, a figure that is not in the source material. A machine does not get tired by the thirtieth text and never talks itself into «close enough».

• Second ring: a second reader

  A separate agent that did not write this text. An author defends their own decision; an outsider does not. This is not a quirk of neural networks, it is the same reason a newsroom keeps a proofreader.

• Third ring: an outside model

  The text goes to a model built by somebody else. It does not know who wrote it and says how likely the statement is to be true. Two models fail in different places, and that is their main use to each other.

How the work goes

Four steps. The first changes nothing and exists so the other three are not done blind.

1. 1

   Looking at what comes out now

   I take your real tasks and read the real answers. The point is to find which errors repeat: invented facts, the wrong voice, a broken format. The treatment depends on the diagnosis.

2. 2

   Rules

   Written so that a result can be checked against them. A rule nobody can check is not a rule, it is a wish.

3. 3

   Checking in three rings

   Scripts, a second reader, an outside model. Built as part of the work, not as a read-through before delivery.

4. 4

   Measurement

   The set of tasks with known answers, plus a sample read by a person. From this step on you have a number instead of an impression.

What stops an invention appearing

The three rings check finished work. These habits act earlier and remove a share of the errors before they are written.

• Tied to a source

  Every statement has to carry a quote from your own material. A statement without one is struck out, however right it looks. This single habit does more than any other measure.

• Finding and writing kept apart

  One step only pulls quotes; the next writes and may use nothing else. Inventions appear where one step does both at once.

• A strict shape for the answer

  The answer arrives as fields, and the source field is required. Without it the program refuses the answer, so the model cannot pass it on.

• No speaking for the client

  The model never names prices, deadlines or commitments, even when it knows them. A wrongly quoted price costs more than every other error put together.

How you know it improved

Without measurement three rings are a belief. They become engineering once there is a number.

• The same question several times

  Ask it five times. Where the answers agree, the fact holds. Where one diverges, the model most likely invented it, and that is the place to look.

• Trap questions

  Ask about something the material does not contain. The correct answer is «there is no such data». If the model answers on the merits, the checking is not working, and you see it at once.

• A run after every edit

  A rule is changed, the whole set runs again. If anything got worse, the edit does not go into the work.

• A sample read by a person

  Someone reads part of the answers and writes down how many errors they found. Otherwise the checking is grading itself, which is not grading.

What I do not promise

You will be offered these three promises, and none of them can be kept.

• That nothing will be invented

  No check catches everything. Three rings cut the errors that reach you sharply and give you a number to watch. Promising zero means promising something this technology does not have yet.

• That it gets cheaper

  Checking spends more than a single unchecked answer. You are paying for a result you can pass on without reading every line, not for a saving.

• That the rules are written once

  They age: your tasks change and new models come out. Rules live like a manual that gets corrected, not like a document signed once.