PrimetimeGeek independent technology journalism · relaunched 2026
How to evaluate an AI agent before you hand it the keys
Agents are the most tempting and the most dangerous shape in AI right now. Here is a buyer's evaluation frame: task scope, tool boundary, observability, failure cost, and the evidence to demand before an agent is allowed to act on your systems or data.
Every month a new crop of AI tools promises to act on your behalf. They will write the email, book the meeting, update the CRM, file the ticket, review the code, or run the campaign. The marketing usually calls them agents. The word is doing a lot of work, because the gap between a helpful assistant and a piece of software that takes irreversible action in your systems is enormous. A buyer who treats them as the same thing is buying the demo, not the risk model.
This is an evaluation checklist for American technology leaders, security teams and product managers who are being asked to approve, budget for or build with an AI agent. It does not tell you which product to buy. It tells you what has to be true before an agent should be trusted with anything that matters. The frame comes from the security and AI-safety literature, from the small number of production deployments that have been documented, and from the recurring failure modes that show up whenever software is given both a goal and a tool. For a companion on reading launch claims, see our earlier guide on how to read a technology product announcement.
What an agent actually is
For this evaluation, an agent is a system that can choose a sequence of actions, use external tools, and keep working toward a goal without a human confirming each step. That definition matters because it separates agents from three close relatives that are much safer:
- A chatbot answers questions. It does not change state outside the conversation.
- A copilot suggests and drafts. It waits for a human to accept, edit or reject.
- A scripted automation follows a fixed recipe. It does not improvise when the recipe breaks.
An agent, by this definition, makes consequential choices on its own. It may decide to send the email, refund the invoice, or push the code. The question is not whether it is impressive; the question is whether it is safe enough to be useful.
The four questions to answer before granting access
1. What is the exact scope of the task?
Vague scopes are where agents cause the most damage. "Manage my inbox" is not a task. "For every incoming support email that matches one of these seven templates, draft a reply and leave it in the pending folder for a human to send" is. A useful scope statement names the inputs the agent is allowed to read, the actions it is allowed to take, the conditions under which it is allowed to take them, and the output or state change it is allowed to produce.
The hard part is not writing the allowed list; it is writing the disallowed list. Every agent should have a short, explicit inventory of things it must never do. The inventory should be enforced by permissions, not by prompting. Relying on a system prompt that says "do not delete anything" is like relying on a sign at the edge of a cliff.
2. What tools can it reach, and who owns them?
An agent is only as safe as the least safe tool it can invoke. The evaluation should list every API, database, inbox, repository, calendar, billing system and cloud account the agent can touch, then ask three things for each:
- Can the agent's credentials be scoped to the minimum necessary action?
- Can the tool itself enforce limits the agent cannot override?
- Is there a log of what the agent did that survives even if the agent is compromised?
If the answer to any of those is no, the tool should be off limits until it is yes. Agents that inherit a user's full session privileges are particularly dangerous, because a confused or jailbroken agent inherits the user's power without the user's judgment.
3. Can you observe, interrupt and reverse it?
Production agents need three observability properties that most demos skip over. First, state: you can see what the agent believes it is doing right now. Second, interruptibility: a human can stop the agent at any point without needing a code change. Third, reversibility: any action the agent takes can be undone in a known, tested way.
The absence of any of these is a veto-level problem. An agent that operates silently, cannot be paused, or leaves a trail of irreversible changes is not ready for a live environment, no matter how well it scores on a benchmark. The gold standard is not autonomy; it is controllable autonomy.
4. What happens when it is wrong?
Agents will be wrong. The question is whether the wrong answer is a harmless mistake or a liability. A wrong support reply is embarrassing; a wrong refund is expensive; a wrong code push can be a security incident. The evaluation should classify every possible action by failure cost and require human approval for anything above a low, pre-agreed threshold.
Cost is not just money. Reputation, regulatory exposure, customer trust and data leakage all count. If an agent has access to sensitive data, its failure mode must include containment: it should not be able to exfiltrate, summarize or share what it saw just because the user asked nicely.
A minimal 30-day test plan
Do not deploy an agent to production on the strength of a vendor demo. The minimum responsible test looks like this:
- Week 1: shadow mode. The agent observes real work and proposes actions, but a human executes every proposal. Measure precision and recall against the actual decisions the team made.
- Week 2: sandboxed action. The agent acts, but only on synthetic data and a copy of the real environment. Test the disallowed list, the interrupt switch and the rollback procedure under failure.
- Week 3: narrow live pilot. One real task, one bounded scope, one tool, with a human in the loop and a daily review of every action.
- Week 4: measure and gate. Compare outcomes to the human baseline. Decide whether to expand, constrain or stop. Write down the conditions under which the agent would be automatically suspended.
If a vendor cannot support this sequence, or if the commercial model punishes slow rollout, that is a signal about who the product is really built for.
Red flags in vendor claims
Some claims are reliably associated with immature agent products. Treat them as prompts for more questions rather than reasons to buy.
- "Autonomous end-to-end workflows." Ask for the complete list of actions the workflow can take, and the conditions that trigger each one.
- "Human in the loop when needed." "When needed" is not a control. Ask how the system decides what needs approval and how that decision is audited.
- "Learns from your data." Ask where that data goes, whether it trains the vendor's model, and how you delete it.
- "Works with all your tools." All usually means the most common integrations; the one you actually need may be missing, or may require the agent to inherit excessive permissions.
- "State of the art benchmarks." Benchmarks measure narrow capability, not bounded behavior in your environment. See our earlier guide on how to read a technology product announcement.
The honest limits
No evaluation framework can prove an agent is safe. Formal verification of agent behavior at scale is still largely unsolved, and the interaction between a large language model, external APIs and an organization's actual data is too complex to model completely. The test plan above reduces risk; it does not eliminate it.
There is also no independent body certifying agent safety for business use, and vendor-provided security documentation is often thin on the specific failure modes that matter most. Buyers should ask for references from organizations running similar tasks at similar scale, and should prefer vendors that publish detailed incident-response and data-handling policies over vendors that publish only benchmark scores.
What is not in doubt is the direction. Agents will move from demos to production faster than many organizations are ready for. The ones that succeed will be the ones deployed with narrow scope, minimum permissions, strong observability and a clear plan for when things go wrong.