You test a new skill the way you wrote it. You give the agent the three prompts you had in your head while writing, watch it do the right thing, and call the file ready. What you never ran is those same three prompts with the file switched off, so you cannot say whether the agent needed it or would have got there anyway. Ready is not the feeling left over from a good demo. It is a stack of checks, run in a particular order, and most teams run the first one only after something has gone wrong in front of somebody else.

Dials run all day, teardowns do not

An engine on a test bench is not judged by one number. The dials run continuously — temperature, pressure, fuel flow — for almost nothing, on every pass. What they cannot see, how a part is wearing, needs the engine opened up, and that costs a day and a rebuild, so nobody does it casually.

Skill evaluation splits the same way. Decide first what you want out of the file — an outcome, a route to it, a house style, a budget — then run each prompt and keep the trace along with whatever it left on disk. Most of it is dial work: did the project file end up there, did the install actually run, is the repository clean afterwards, did the run stay inside its budget. Those are assertions, they cost almost nothing, and they run on every commit. The judging model is the teardown, and it comes out only for what an assertion cannot phrase — whether the structure is sensible, whether it follows the conventions of the house. Every failure becomes another case, so the suite grows out of real breakage.

The set needs the other direction too. Ask for a styling library to be added to an application that already exists, and the correct behaviour is that the demo-building skill stays out of it. A skill that fires on everything is not eager, it is broken, and only a case written to catch that will say so.

Useful and safe are two different tests

The dials tell you the run behaved. They do not tell you the file was worth loading. For that you need the run with the additive left out of the tank: the same tasks on the same harness, once with the skill and once without, with the paired outcomes rolled into a single score you can carry between versions. Without the pair, “it worked” is a sentence about the model.

Then there is the bench that has nothing to do with power. An additive can lift every number a driver cares about and still fail on emissions, because that test is looking for something else entirely. A skill gets its own probes, written to make it misbehave. One built for document retrieval can beat baseline across the whole benchmark and still fail a probe that walks it into leaking what it retrieved. Both results are true at once, which is why the honest output is a label with levels rather than a stamp, and why the gate holds the skill until policy and tool boundaries have taken the dangerous move away. It is that second verdict, not the speed, that decides whether the file is publishable at all. Same discipline as a loop that rewrites its own instructions: proposing is cheap, accepting is not.

A clean result is only about today

The certificate is about the fuel that was in the tank that day. Change the blend and the numbers are a memory. Skills go off the same way, and the ones written to carry fast-moving knowledge go off fastest. A library ships a release, the call you named is retired, and the file that made the agent current last quarter now makes it confidently out of date. The check is the same paired shape, with failure defined as something verifiable: it used the dead API. Run it whenever upstream moves and it tells you that your file, not the model, is the stale part.

So the durable version of these files keeps almost nothing: a short procedure and a pointer to the source of truth, fetched when the work needs it rather than copied in, the way documentation only stays true when something keeps it. And a pointer counts only if it lands in the context at the moment of the decision — in the description the agent reads, in the error a tool returns, in the output already in front of it. Guidance nobody loads has never steered anything.

A skill is never ready. It was ready, on a date, against a baseline, and both are things you can still point at.