Wine That Pleased Its Own Maker
King Suchi of Avanti was suspicious of his own kitchen. The royal vintner brought a new wine each season, declaring it the finest yet. The court applauded. The king sipped. He could not tell whether it was better than last year's, worse, or simply different. "How," he asked his minister, "do I know if my own wine is good, when the only people I ask are the people who made it?" The minister had no answer. The king resolved to find one.
“The maker is the worst judge of the making.”
The Cellar of Old Vintages
First, the king commanded that a cellar be set aside — fifty bottles from each of the last twenty years, sealed and labelled, not to be opened except for testing. "When the new wine comes, we shall pour it beside the old. Without a baseline we are guessing. With a baseline we are measuring." The vintner protested that this was a waste of good wine. The king replied that wine drunk in ignorance was the greater waste.
“You cannot know whether you are improving without yesterday's bottle to pour beside today's.”
The Council With the Rubric
Next the king summoned nine tasters — none of them in the vintner's pay. He gave each a written rubric: clarity, nose, body, finish, aftertaste; each scored from one to five. The tasters did not see each other's scores until after they had written. Where they agreed, the king trusted the verdict. Where they disagreed, he ordered another tasting next week — disagreement was a signal that the rubric was vague, not that the tasters were stupid.
“A vague rubric makes wise tasters look foolish. Sharpen the rubric, not the tongue.”
The Mirror-Vintner Judges the Pair
The king then did something stranger. He hired a second wine-maker — a rival from a distant kingdom — and asked him not to make wine but to judge. "Pour him my wine and a competitor's, side by side, in unmarked cups. Ask him which is better." The mirror-vintner, knowing wine the way only a maker knows it, gave swift verdicts that the council's rubric had missed. Where his verdict agreed with the council's, the king was doubly confident. Where it disagreed, the king investigated.
“A judge of the same craft sees what a checklist misses — but a checklist sees what a judge's mood obscures. Use both.”
The Inn at the Crossroads
Yet the king knew that the palace cellar was not the world. He sent flagons of his wine to the inn at the crossroads, where merchants of every kingdom stopped. The innkeeper kept a quiet ledger: how many cups poured, how many returned half-drunk, how many travellers asked for a second. None of these were judgements. All of them were truths. When the new vintage was sent and the second-cup count fell, the king knew — before any council could meet — that something was wrong.
“In the real inn, the silent leaving of half-drunk cups is the only verdict that matters.”
The Vintner Who Learned to Test First
In time the royal vintner stopped resisting and started using the testing himself. Before any new wine reached the court, he would pour a small batch against the cellar baseline, take it to a quiet corner of the council, listen to the rubric scores, send a trial flagon to the inn. Only then would he scale to the royal table. Disasters became rare. When a new vintage failed, it failed cheaply, in a corner, where no king ever drank from it. The kitchen had stopped fearing judgement and started using it.
🪔 Deepak — the lamp of meaning · what this fable means in code
The royal vintner's self-praise is the trap of "vibes-based" evaluation — the maker certifying the make. The cellar of old vintages is a golden dataset, the immovable baseline that lets you say "better than last week" with evidence. The nine tasters with a written rubric are graders running a structured eval; their disagreement is a signal that the rubric needs sharpening, not that the panel is wrong. The mirror-vintner is LLM-as-judge — a same-craft model evaluating outputs in pairs, fast and cheap, with known biases. The inn at the crossroads is online evaluation: implicit signals from real users (retry, half-cup, second-cup) that no offline rubric can replace. The closing scene — testing before serving — is the final discipline: bake evals into CI, run them on every prompt change, and let no untested vintage reach the king.