Tech · AI · Software
Kairos
The right moment to make sense of tech & AI.
A publication on technology, artificial intelligence and software development — notes, experiments and analysis on where the field is heading. Written by Nicolò Marana.
AI Foundations 32
-
Open Source vs Closed Models: What 'Open' Actually Means in AI
Companies routinely call their AI models 'open source' after releasing nothing more than a file of downloadable weights, and the press repeats it. Here's what real openness would actually require, and why the gap matters.
-
Token Costs: How AI Is Actually Billed, and Why the Bill Keeps Growing
Per-token prices for AI models have been falling fast, yet many monthly bills keep climbing. The reason is that agentic tasks consume far more tokens than a single question ever did.
-
Prompt Injection: Why These Systems Can't Tell Instructions From Data
The same mechanism that lets a model follow your instructions lets it follow anyone's. Here's why prompt injection is a structural gap, not a bug waiting on a patch.
-
Guardrails: The Layer That Decides What a Model Is Allowed to Say
A chatbot's refusal message usually isn't the model reaching the edge of its knowledge. It's a separate system, wrapped around the model, enforcing a decision someone else made.
-
Multi-Agent Systems: When Several Agents Work Together
Some tasks get better results split across several specialized agents than handed to one generalist. Here's how the handoffs work, and where the new failure mode hides.
-
Agents: When AI Stops Answering and Starts Doing
A chatbot describes what to do. An agent goes and does it, in a loop, across many steps, without waiting for you at each one. Here's exactly where that line sits.
-
Quantization and On-Device Models
Phones now run AI features with no signal and no server round trip. Quantization, the compression trick that shrinks a model's numbers without shrinking its judgment, is a big part of why.
-
Inference in Production: Serving a Model to Millions
A model that dazzles in a private demo can fall apart the moment real users show up at once, and the reason has almost nothing to do with the model itself.
-
Function Calling: How a Model Actually Uses a Tool
When a chatbot 'checks the weather' or 'books a meeting,' it never touches a network or runs code. It writes a structured request and hands it to a separate program that does the actual work.
-
MCP: The Standard Becoming the USB of AI
A new open standard called MCP lets an AI application talk to any compatible tool or data source without a custom integration for each pairing, and that plumbing question matters more than which model is winning this quarter.
-
Vector Databases: The Infrastructure Behind RAG
How does a system find the one relevant paragraph out of ten thousand pages, using none of the same words as the question? The search happens in a specialized database built to measure closeness in meaning, not matching text.
-
Fine-Tuning vs Prompting vs RAG: What to Actually Use
Three ways to adapt a general-purpose model to your task, ranked by what they actually cost and what they're actually for, so you stop reaching for the expensive one by default.
-
Context and Memory: Why the Model 'Forgets'
A chatbot losing track of something you said an hour ago and a product announcing it now 'remembers you' are the same underlying issue, and neither one is what it looks like.
-
RAG: Giving a Model Your Documents Without Retraining It
A chatbot can answer questions about a policy document it never saw during training, as long as someone hands it the right paragraph first. Here's how retrieval-augmented generation actually does that.
-
Diffusion: How AI Images Are Actually Made
AI image tools don't build a picture piece by piece the way a language model builds a sentence. They start with static and slowly refine it into a photo, and that reversal turns out to matter well beyond images.
-
Multimodality: Text, Images, and Audio in the Same Model
A chatbot that reads your typing, looks at your photo, and listens to your voice memo isn't running three programs at once. Here's why one model can handle all three formats without switching machinery.
-
Models That 'Reason': Chain-of-Thought and Inference-Time Compute
Newer models pause to work through a problem in visible steps before answering, and that pause is a deliberate, costly choice made at the moment you ask, not a sign the model is thinking like a person.
-
Distillation: How a Small Model Learns From a Big One
A model a fraction of the size of a flagship system can match much of its performance. The trick isn't a breakthrough in scale, it's one model learning to imitate another.
-
Compute and GPUs: Why Hardware Decides Who Gets to Play
Training a frontier model is a physical infrastructure problem before it's a research problem, and the chips capable of doing it come from a shortlist of companies short enough to name.
-
Training Data: Where All That Text Actually Comes From
Language models don't learn from 'the internet' in any simple sense. They learn from a specific, assembled mix of scraped web pages, licensed books, code repositories, and contributed data, and that mix is now the subject of open legal fights.
-
From Raw Model to Assistant: Fine-Tuning and RLHF
The model straight out of initial training has no manners and no off switch. A second training stage is what turns it into something you'd actually want to talk to.
-
Parameters and Scale: What '70 Billion Parameters' Actually Means
Every model announcement leads with a parameter count, as if it were a spec sheet number like horsepower or storage. Here's what that number actually measures, and why piling on more of it stopped paying off the way it used to.
-
Training vs Inference: Two Completely Different Phases
Building a model and running a model are two separate processes with two separate cost structures, and mixing them up is why people expect a chatbot to remember and learn from a single conversation.
-
Bias: How Training Data Becomes Prejudice
A model doesn't invent its prejudices. It inherits them from the text it was trained on, and no one had to write a single biased rule for that to happen.
-
Emergent Abilities: Skills That Show Up Out of Nowhere
Some model abilities don't improve gradually with scale, they stay flat for a long stretch and then jump sharply, and nobody can reliably predict when the next jump happens.
-
Hallucinations: Why They Happen and Why They Don't Go Away
A model that invents a court case or a journal citation isn't malfunctioning. It's running the exact same process that produces its correct answers, and that's precisely why the problem can't be patched out.
-
Temperature and Sampling
Ask a chatbot the same question twice and you'll often get two different answers. The reason is a dial called temperature, and it's set by the product, not by the model.
-
Attention: What the Model Looks At
The mechanism that lets a model weigh any word against any other word, no matter how far apart they sit, is the real reason modern AI works at all.
-
Embeddings: Concepts Turned Into Numbers
How does a model 'know' that king relates to queen the way Paris relates to France? The answer is that meaning gets converted into coordinates, and closeness in that space is all the model has.
-
Tokens: How a Model 'Sees' Text
Ask an AI to count the letters in a word and it often gets it wrong, not because it can't count, but because it never saw the letters to begin with.
-
Next-Word Prediction: The One Trick Behind Everything
Every capability an LLM has, and every hallucination, comes out of one repeated operation: guessing the most likely next word. It's not a bug sitting next to the real mechanism. It is the mechanism.
-
What a Language Model Actually Is
An LLM doesn't store answers and look them up. It reconstructs them from scratch every time you ask, which is the root cause of both its strengths and its most confusing failures.
Model Evaluation 8
-
Evaluating Safety: What Red Teaming Actually Measures
Capability benchmarks give a model a score everyone can quote. Safety testing doesn't, and that mismatch quietly shapes what the public ends up worrying about.
-
Reproducibility and Variance: Same Model, Different Scores
A benchmark score looks like a fixed property of a model, but run the same model on the same test twice and the number can move. Here's why, and what it means for the leaderboard.
-
LLM-as-Judge: Using AI to Grade AI
When there are too many AI answers to grade by hand, teams hand the job to another AI model. That shortcut is fast and cheap, but it imports a very particular set of tastes.
-
Human Preferences and Elo Rankings
Some AI leaderboards aren't built from exam questions at all, they're built from thousands of people voting on which answer they liked better, scored the same way chess players are ranked.
-
Goodhart's Law: When the Score Stops Measuring Anything
Once a benchmark becomes the number every lab is judged on, teams start training toward it directly, and the score quietly stops measuring what it was built to measure.
-
Data Contamination: When the Model Has Already Seen the Exam
A model can ace a benchmark not because it solved the problem, but because it already read the answer somewhere on the web before training began. Here's how that happens, and why it quietly undermines the scores we use to judge progress.
-
Accuracy and Its Traps
A model that scores 90 percent sounds impressive until you check what a naive guesser would have scored without doing any work at all. Accuracy is only informative once you know the denominator.
-
What Is a Benchmark? A Standardized Exam, Not Reality
Benchmark scores get reported like objective facts about a model's ability, but a benchmark is closer to a standardized test than a measurement of the world. Here's the difference, and where it quietly stops applying.
Open Models 10
-
The New Frontier Players (and the Geographic Reversal)
By mid-2026, the open-model leaderboards are no longer dominated by English-speaking labs, and the gap with the best closed models has nearly closed too.
-
Microsoft Phi: Small and Everywhere
Phi isn't trying to top a leaderboard. It's betting that a small model fed exceptionally clean data can outperform a much larger one fed everything.
-
Google Gemma: The Best Model for a Single GPU
For years Gemma lost enterprise adoption not because of what it could do, but because of the license attached to it. In 2026 that finally changed.
-
Alibaba Qwen: The Local Default of 2026
A year ago almost nobody would have bet that the best model to run on your own machine would come from Alibaba. By 2026, for many people, it is.
-
DeepSeek: The Model That Redefined Cost
The model that rattled markets in early 2025 didn't win on raw capability. It won by making the question 'how much does it cost to be this good' impossible to ignore.
-
Mistral: Europe's Open Model Champion
A French startup stayed relevant in a field dominated by American and Chinese labs by competing on license terms and language coverage instead of raw scale.
-
Meta Llama: The Family That Started the Open Wave
The company that accidentally kicked off the open-weight model era by having its own research leak is no longer the one leading it.
-
Hardware for Open Models: Who Can Actually Run What
The bottleneck for running an open model at home stopped being quality years ago. Now it's almost entirely about how much GPU memory you own.
-
Dense vs Mixture of Experts: Why a 1,000B Model Can Cost Like a 40B One
A model with a trillion parameters sounds impossibly expensive to run. Whether that's true depends entirely on how many of those parameters actually activate per token.
-
Open Source vs Open Weight: What "Open" Actually Means
Most models marketed as open source in AI are something narrower: you can download and run the weights, but not see the data or rebuild the pipeline.
Closed Models 10
-
The Model That Can Be Switched Off
In 2026, how powerful a model is stopped being the only question worth asking. A second one is catching up fast: who can turn it off, and on whose order.
-
The Challengers and the Routers: The Layer That Chooses for You
A layer is forming above the models themselves, one that decides which model actually answers your question, and that layer is starting to matter more than any single model's name.
-
China's Closed Models: The Two-Tier Strategy
The story that China equals open source in AI is only half true. The best Chinese models are the ones nobody outside their home ecosystem gets to run.
-
xAI / Grok: Real-Time and No Filters
Grok competes on two dimensions most other closed labs deliberately avoid: live access to social data, and far looser content guardrails.
-
Google DeepMind / Gemini: The Native Distribution Card
One competitor can put a new model in front of billions of people within a single day, simply by shipping it inside a product they already open daily.
-
Anthropic / Claude: Reliability as a Product
One lab built its pitch around a claim most competitors treat as a footnote: that predictable behavior matters more, past a point, than a higher benchmark score.
-
OpenAI / GPT: The Distribution Advantage
The company that turned AI into a mass-market fact no longer has the best-scoring model on most benchmarks. It has something harder to catch up to.
-
No Hardware, Full Dependence
Choosing a closed model trades a hardware problem for a different one: dependence on someone else's uptime, roadmap, prices, and policies.
-
How Access and Pricing Actually Work
The rate on a closed model's pricing page is not the number that ends up on your bill, especially once reasoning models enter the picture.
-
What "Closed" Actually Means: The Model as a Service
With a closed model you don't buy an object, you rent access to one, and the terms of that rental can change under you without notice.
Agent Engineering 103
The Loop 15
-
The Harness That Improves Itself
A loop rewrites its own instruction file overnight and the benchmark goes up two points, which sounds like progress. Whether it is depends on whether that number can tell an improvement from a lucky run.
-
Buy the Platform or Build the Agent
Two teams solve the same problem from opposite ends: one assembles everything around the model, the other adopts the runtime the model already fits. The code is not the part that gets hard to undo.
-
The Harness That Knows Which Model It's Wearing
You swap in a new model, change nothing else, and the agent starts failing in ways the old one never did. The interesting question is not which model is better, but which layer the failure belongs to.
-
Picking a Harness Without Guessing
You ask which runtime to use and everyone answers with a name, never with the property you needed. The fix is not better advice, it is writing the criteria somewhere a machine can read them.
-
Middleware for Agents
A phone number vanishes from a support ticket before your agent ever reads it, and no line in the prompt asked for that. The rule was declared beside the agent instead of written into it.
-
Batteries Included, or an API You Program
One option gives you a working agent this afternoon, the other gives you a loop and a week of work. The difference is not how much you type on day one, but who owns the answer later.
-
Starting From a Template That Already Works
A template puts a working research agent on your screen in minutes, which feels like most of the job. What comes with it is someone else's configuration, and that part is not done at all.
-
The Queue Is Where the Plan Becomes Real
An agent can execute a perfectly good plan and still get the wrong answer, because the plan was written for a world that has since moved. The fix is not a smarter planner.
-
When the Agent Has to Look Before It Acts
An agent can read a screen perfectly and still ruin the next five minutes, because it never looked again after the first click. Reliability on a screen comes from what the harness does between actions, not from sharper eyes.
-
Where the Model Ends and Your Software Begins
An agent deletes a folder it had no business touching, and the transcript shows the reasoning was perfectly sensible. The failure sits on the other side of a line you drew without noticing you were drawing it.
-
Choosing the Loop You Actually Need
Running an agent in a loop sounds like one decision, and it is really four, made before the first iteration. The trigger you pick decides what the run costs and whether failure arrives as an error or as silence.
-
The Loop Is the Product, Not the Model
An agent can make good decisions for six hours and still deliver nothing, because the machine rebooted and nothing wrote down that those six hours happened. The interesting engineering is not in the model.
-
The Harness Is the Part Nobody Talks About
An agent calls a tool with the wrong arguments, reads back an error that explains nothing, tries again, and burns the run. The instinct is to blame the model, but both levers that fix this sit outside it.
-
The Harness as a Control Structure
Your agent pauses before publishing and waits for a human, and no line in the prompt told it to. The decision came from the layer most teams describe as plumbing between the model and its tools.
-
Regulation, Borrowed From Cybernetics
You write the rule down, the agent follows it, and months later the boundary has four holes in it. Nobody disobeyed, because a rule that nothing measures was never a control to begin with.
Context & Memory 20
-
Memory in Tiers
An agent that recalls a decision from three weeks ago but drops a constraint from four turns ago is not badly tuned. It is running four different memories under one name, at four different speeds.
-
Memory Several Agents Can Share
A second agent joins the project and knows nothing the first one learned, even though nothing failed and nothing was lost. What it learned was filed under the tool that produced it, and a project outlives its tools.
-
Documentation That Writes Itself
You open a repo from February and the README describes a system that no longer exists. The truest documentation was written four months ago, in a session that was thrown away when the window filled up.
-
Memory With a Chain of Custody
Your agents share a note that is word for word what was learned, and it is now quietly wrong. The memory outlived the branch it was read off, and nothing in it recorded where it came from.
-
The Requirements Nobody Writes Down
Your agent ships an endpoint that passes every test and still breaks three rules, none of them written anywhere. What it missed was never missing from the model — it was never written down by anyone.
-
When Memory Gets Written Up
An agent that knows your habits was never told them, and nothing in the transcript marks the turn where it learned. The writing happens after the session is over, and when it happens decides what it costs and what it loses.
-
Deciding What Is Worth Keeping
Your agent compressed its own history at a sensible moment and wrote an honest summary, and the work stalled anyway an hour later. What it dropped was not judged by the wrong rule — it was judged by only one.
-
Memory That Remembers the Connection
Your agent hands back three true memories and none of them answers the question you asked. What it kept was the facts; what the answer needed was the step from one fact to the next.
-
The Cache Under the Memory
Your agent reads back a state file and resumes exactly where it stopped, so it looks like it remembers. It doesn't — it looks something up, and the lookup is only as true as the last time somebody corrected it.
-
State You Can Point At
Your agent signs into a browser, and the next call comes back showing the login form with nothing broken and nothing denied. The session was real — it was just living somewhere the request had no way to point at.
-
The Contract With the Environment
Your agent breaks the same two rules it broke yesterday, apologises, and breaks them again tomorrow. It is not careless — nobody ever wrote the rules down where it could read them while working.
-
The Context Filesystem
The agent cites a document, gets it wrong, and all your trace kept is a similarity score of 0.83. Give the context a shape — paths, names, depths — and the question of where it read that finally has an answer.
-
Reading Code the Way a Compiler Does
An agent reads forty thousand tokens of your repository and comes back with an answer that is almost right. The trouble was never how much it read, but where the cuts fell.
-
The Agent That Tends Its Own Notes
Your agent's memory kept everything you ever gave it, which is why nothing in there is worth reading. The fix is not a bigger store but a keeper, and the only candidate was already in the room.
-
What Compaction Actually Throws Away
Your agent compacted its own history and kept working, and the summary it wrote is perfectly accurate. What it can no longer see was decided by something that ran while nobody was watching.
-
Everything You Cut, and How to Get It Back
Making a long history shorter is the easy half of the job, and the run that follows tells you which half you skipped. What decides whether an agent can retrace its steps is whether the shortening was built to come apart.
-
The Cache Is Load-Bearing
An hour into the run, the bill for re-sending the same forty thousand tokens on every single turn comes out a fraction of what you expected. Then one word changes near the top and it doesn't.
-
Lost in the Middle
The specification fit in the window with room to spare, and the model still ignored the one sentence that mattered. Nothing was truncated — the sentence was just in the wrong place.
-
Context Is a Runtime, Not a Prompt
You wrote the instructions once, and an hour later the agent contradicts them without ever disobeying. What reaches the model on a given turn is not your prompt but a payload assembled seconds earlier.
-
Instructions That Survive the Squeeze
You gave the agent one hard rule at the start, and two hours later it broke it without hesitating. The rule was never overruled — it was simply somewhere the agent could no longer look.
Tools & Skills 13
-
Output You Can Trust to Parse
Your classifier returns clean JSON for a week, then one reply arrives with a friendly sentence in front of it. The fix is not a better prompt or a stricter parser, but a generation that could not have produced it.
-
What Makes a Tool Legible to a Model
Adding a tool to an agent feels free until you look at the bill. Every tool you declare is described to the model on every single turn, whether the task ever calls it or not.
-
Research as a Tool, Not a Prompt
An agent searches, reads, searches again with almost the same words, and hands you an answer its own sources do not support. Nothing errored. Nothing was ever decided either.
-
The Question Teams Ask Too Late
You try the new skill on three prompts, it does the right thing, and you ship it. The run you never made is the one that would have told you whether the file did anything at all.
-
Measuring Whether a Skill Covers Its Domain
Your skill fixes all three of the failures you had in mind while writing it, which tells you almost nothing. Coverage is the difference between two identical runs, one of them without the skill.
-
Editing Code Without Breaking It
An agent changes one function and quietly changes another that read the same. The fix is not a smarter model but an edit that names what it touches and a compiler that refuses it when the ground has moved.
-
Skills Are Context You Load on Purpose
You install twenty skill files to make the agent more capable and it gets vaguer instead. Nothing in the folder is wrong, and that is the part worth understanding.
-
A Server Worth Copying
You can read a protocol specification end to end and still build an unsafe server, because the decisions that make a tool safe are never protocol-level. A reference implementation is where those decisions are finally written down.
-
MCP, Past the Handshake
Wiring an MCP server takes an afternoon. Running one for a whole team takes rather more, because the protocol says how a call is made and nothing about who is making it or what a refusal means.
-
From Intent to Execution, in One Skill
You ask for a candidate tracker and four different tools go and build it, none of which you named. The distance between your sentence and that work is where a domain skill lives.
-
Driving Software Built for Human Eyes
An agent working from a rendering of a page is guessing which of two grey rectangles is the button. The page already publishes a description of itself, and that description is written for software that never looks at it.
-
The CLI Was Already an Agent Interface
You are about to build a protocol server so an agent can drive an application that already ships a command line. One of those two interfaces was designed for machines from the start, and it is not the new one.
-
The Sandbox Is Part of the Interface
Giving a sandboxed agent one new capability took edits across eight files, and the operation itself was a single line of code. The distance between those two numbers is a layer of mediation you may not need.
Orchestration 21
-
The Workflow You Can Read in a Diff
A workflow that lives in a file gets reviewed like code, before anything runs. The stranger part is what the repository becomes afterwards: the channel through which agents claim work, collide, and find out about each other.
-
Plan First, Then Replan
An agent hits a snag on step four and starts the whole job again from step one, paying twice for work that was already correct. The plan was never the problem.
-
Routing the Work to the Right Model
An agent spends most of a task looking around, and every one of those tokens is billed at the price you picked for the hardest decision in the job. The phases differ; the rate does not have to.
-
The Sandbox That Remembers
An agent that rebuilds its environment every morning spends its best hours walking back to where it already stood. Keeping the environment instead of rebuilding it changes what every restart costs.
-
The Agent That Uses a Whole Computer
An agent that drives desktop applications while you keep working is not an app you opened. It is something running on your machine under an account, and the useful questions about it are the ones an administrator asks.
-
Choosing the Shape of the Team
Two agents that never speak to each other can be cheaper than two that do. The bill for agreement is paid every step, while the bill for a wasted attempt is paid once.
-
One Agent, Many Agents, and the Cost of the Split
Splitting work across agents buys parallelism and pays for it in context that never crosses the gap. The one cut that survives the payment is the line between deciding and doing.
-
MicroVMs, and the Contract a Sandbox Signs
Your agent wrote a script, ran it, and threw it away before anyone read a line of it. Containers were built for code somebody already reviewed, which is why agent-written code gets a machine of its own and a document saying what it may touch.
-
Markdown as the Control Plane
The pipeline that runs is a hundred lines of YAML nobody in the room can read. The file it was compiled from is four sentences of plain English, and that is the one somebody has to sign.
-
Getting Better Without Retraining
Six hours into a run the agent is still failing the same tool call, and restarting would throw away every hour of context it has built. The other option is to let the episode rewrite its own harness while it keeps going.
-
Where the Human Goes in the Loop
Your workflow stops for a person and you give it one boolean column to answer into. Real reviewers rewrite a clause, ask for someone senior, or never reply at all, and the stop has to have been built for that.
-
The Filesystem as the Contract
You ask an agent what happened overnight and get a confident sentence you have no way to check. The answer was on disk the whole time, in a file written while the work ran, readable without waking anybody.
-
Workflows That Survive a Restart
Your agent waits four days for a signature, and the machine it was running on is recycled overnight. When the signature finally lands it carries on from the right place, because the part that knew where it was had never been running.
-
Agents That Sleep and Wake
An agent that spends four hours waiting for one approval has done nothing wrong and cost you a fortune. Waiting is not work, and a system that cannot stop is a system that cannot be left alone.
-
Durability, Borrowed From Boring Software
You wrote a retry, then a state table, then a scheduler, and none of it was the agent. The durable part of an agent is ordinary software, and ordinary software already has engines for it.
-
Code as the Way an Agent Acts
Six tool calls to pull one person's messages out of a thousand, and the model reads every page on the way past. Writing the filter as a program collapses them into a single turn, and hands you a different problem instead.
-
How Much Rope to Give It
A loop that only writes to a log and a loop that ships patches while you sleep are the same program with a different number set. What separates them is not how much you trust it, but what kind of proof you can collect.
-
The Quality Flywheel
A risk specialist spends an afternoon reviewing an agent's work and everything she says is right. Whether any of it is still working on Monday depends on what her judgement was turned into before she stood up.
-
Where an Agent Is Allowed to Start
Your agent grew a second interface and quietly lost a safety check that nobody removed. The place where a request gets in is also the only place you can watch it go by, which is the whole argument for having few.
-
Agents as Building Blocks
Expose a sub-agent as a tool and delegation stops being something you phrase in a prompt. The week you save on the wording goes to the question underneath it: what runs next, and what a failure is allowed to do.
-
When Agents Talk to Agents They Don't Trust
Two agents from different companies can finish one job without either seeing the other's prompts, memory or tools. What crosses the boundary is a published capability sheet and a set of artifacts, and that is the whole relationship.
Guardrails & Governance 19
-
When the Sandbox Says No
Your agent walked into a wall the sandbox put there, read it as a flaky network, and ran the same command four more times. A refusal nobody explains does not buy safety, it buys lost work.
-
Not Every Action Deserves the Same Gate
Two tools in the same file can look identical and differ by a refund. What separates them is never the model reading the code, but a risk label attached to each one before the run ever starts.
-
Defense in Depth Against Prompt Injection
You listed three defences at the review and the slide looked like depth. All three came down to the same sentence, which means the count was wrong before anybody attacked anything.
-
The Blast Radius of a Poisoned Document
Two agents read the same poisoned file this morning and both fell for it, and only one of them cost anything. The difference was never in the model, and it was decided before either run started.
-
Policy as Code, for Agents
Everyone acknowledged the agent policy and nobody ever applied it, because nothing on the path from prompt to outbound call ever read it. The fix is to stop writing the rule down and start shipping it as code that runs on every call.
-
Isolation You Can Swap Out
Moving an agent from your laptop to shared hardware sounds like a rewrite of everything that runs it. It is one line, if the limits were attached to the calls instead of the box.
-
A Fresh Machine for Every Action
Your agent ran three scripts from three different places in the same box, and the second one started on a machine the first had already had its hands on. Throwing the machine away after every action deletes that whole class of problem.
-
Isolation That Actually Isolates
Your agent ran a migration inside a perfectly sealed container, against a database everyone shares. What isolates is not where the code runs but who is in a position to refuse it, and what the refusal is drawn around.
-
Stopping an Agent Politely
You press stop and the process dies in under a second, which feels like control until you ask what it was halfway through. A run you can interrupt safely is different engineering from a run you can kill.
-
The Registry of What an Agent Can Do
Someone asks how many tools your agents can call, and the honest answer is an estimate. Every rule you approved in the spring has been governing a set nobody ever wrote down.
-
Five Principles for Letting an Agent Act
An agent that asks permission for everything looks careful, and tells you nothing about what it is about to break. Supervision only works when it bends to what the action costs.
-
Gates That Don't Depend on the Model's Mood
Your agent asked itself whether it was allowed to drop the table, and it said yes. The check that stops it has to be code outside the model, reading a written policy before anything runs.
-
Checking the Work Before It Lands
Every program in that command line was on your allowed list, and your credentials still left the building. A permission model that reads names is guessing at risk; one that reads effects is not.
-
The Credential the Agent Never Sees
You wrote the rule about never printing the token in three places, and the agent obeyed it all afternoon. Then you read what it actually sent, and find the secret was never in the room to print.
-
Guardrails Below the Agent
Your agent walked around the guardrail by writing a helper script, and something stopped it anyway. What nobody checks is whether that something is still running the rules you wrote.
-
Deciding What Deserves a Human
You put a confirmation prompt on every action and your reviewer stopped reading them by Thursday. Asking about everything and asking about nothing land in the same place, so the real design problem is which events get to reach a person.
-
Who Approves, and How
The person allowed to approve is often not the person holding the keyboard. Which channel carries the question decides whether the gate stops anything, and you pick that channel before you write the first one.
-
Who the Agent Is, to the Systems It Calls
An agent running on your credentials is invisible to every log it touches: the record shows your name, acting at hours you were asleep. Giving it an identity of its own is what makes the trail mean something again.
-
Watching the Watcher
Your monitor has flagged everything worth flagging for six months, which is the least useful fact you have about it. A clean record only counts once somebody outside the team has tried to beat it.
Evaluation & Operations 15
-
Agent Logs Are Just Data
Someone asks which support conversations went badly and what the agent did in them, and your observability tool cannot answer. The answer needs a join, and a join needs the traces to be sitting in a table.
-
Replaying What Actually Happened
Six test cases you wrote yourself all pass, which tells you nothing about the conversation that broke at two in the morning. That one is still on disk, and you can play it again with a different setting.
-
Debugging a Run You Weren't Watching
An overnight run of nine agents finishes at four in the morning and one figure in the report is wrong by a factor of ten. The agent that wrote it did nothing wrong, which is why fixing it there fixes nothing.
-
Picking Up Where It Left Off
A ten-agent scan dies six hours in, and the same command rerun finishes in forty minutes with the morning's findings intact. Nothing in the agents made that possible; what survived was written down while the work happened.
-
The Same Test, Run Twice
An agent passes a task, then fails the identical task an hour later with nothing changed. Which run was the real one is the wrong question, and the right one only has an answer if you ran the test more than once.
-
Ready for Production, Checked
Your assistant clears every synthetic ticket and then stalls on the first real one. Nothing about the model changed between those two mornings, and what did change is the part a staged rollout was built to test.
-
The Platform Under the Agents
Your third agent reaches production and you find yourself rebuilding the same isolation, the same credentials, the same trace pipeline for the third time. None of that was ever specific to the agent, or to the framework it happens to be written in.
-
You're Evaluating the Harness, Not the Model
The same model can pass a task in one setup and produce a confident, unverifiable answer in another. The score you publish belongs to the pair, and almost no leaderboard says which pair it was.
-
Telemetry That Means the Same Thing Everywhere
Three teams instrument three agents and the dashboard still cannot say which tool got slow. Nothing is missing from the data — the same measurement simply arrives under three names, from three teams that never agreed on one.
-
Mining Your Failures for the Next Eval
Your agent starts passing a task it never actually performs, and the score goes up anyway. An evaluation suite is software too, and the ones nobody maintains slowly turn into a mirror of whoever wrote them.
-
Two Agents, Same Score, Different Risks
Two agents finish the same evaluation suite with the same success rate, and only one of them can be left running overnight. What separates them is not how often they fail but how, and a single number cannot hold that.
-
What an Agent Costs to Run
You made every run cheaper and the monthly invoice went down, which tells you nothing about whether the agent is worth running. The number that answers that divides the whole spend, failures and retries included, by the outcomes someone accepted.
-
Evaluating in Production, Without Pretending
Your agent closes the ticket and the suite comes back green, which tells you the job is done and nothing about how it was done. In production you need both readings, and one of them rarely has a right answer to check against.
-
The Gate That Decides If It Ships
A prompt change lifts your average answer quality and the pipeline blocks the release anyway. The gate was not reading the average — it was reading the one situation you had quietly said mattered more than the rest.
-
A Taxonomy of Ways an Agent Fails
Your agent's wiki search starts failing on a Tuesday, and no amount of prompt tuning brings it back. The fault has a name and an address in the architecture, and neither of them is the sentence you keep rewriting.