Start at the customer’s end of the work
Choose a result someone outside the agent system would recognise. A customer receives an accurate answer they can act on. A new user completes their first useful task. An order reaches the person who needs it, with the right information. These are clearer starting points than the number of agents running or the number of drafts produced.
For an existing company, ask the people closest to customers which part of the experience needs attention. For a new business, identify the first useful outcome that would help someone decide whether your offer matters. The measurement should follow that choice. It should not start with whichever number is easiest to display on a dashboard.
Separate activity, delivery and value
There are three different questions. Did the system do the work? Did the intended recipient receive a usable result? Did that result help the business or customer? Keep those questions separate. Sending a message proves a different thing from receiving a useful reply. Publishing a feature proves a different thing from someone successfully using it.
A useful scorecard can contain all three, with honest labels. Activity helps explain what is happening. Delivery shows whether the work reached its intended destination. Value tells you whether to keep investing. Missing evidence belongs in the unknown column. It should not quietly become zero, and it should certainly not become success.
Compare the whole job, including the checking
Imagine a service business preparing customer requests for a specialist. This is an illustrative example, not a reported client result. An agent gathers approved information and prepares a brief. Timing only the brief generation would leave out much of the work: finding missing details, reviewing the draft, correcting errors and answering the customer.
Compare the existing process with the proposed one from the same starting point to the same finish. Record elapsed time, human attention, material corrections and operating cost. Include waiting for review as well as the minutes spent reviewing. A fast draft that sits untouched until tomorrow may not improve the customer’s experience.
Use comparable requests and keep unusual cases visible. A trial made entirely of straightforward requests tells you little about how the system handles conflicting information or a change in scope. You do not need a perfect research programme to begin, but you do need to know what your comparison can and cannot show.
Agree what must stay good
Faster work is useful only within the standards the business needs. Define those standards with the people who understand the work. What would make an answer unusable? Which missing detail would create a return trip? Which commitment requires an experienced person to decide?
Build those boundaries into the workflow. Agents can prepare, organise and flag gaps within their agreed access. The appropriate person retains consequential decisions and checks exceptions. Record the corrections that matter so they can improve the process. A check that repeatedly catches the same problem is a reason to fix the system, not simply add more checking forever.
Give recovered capacity a destination
Even a measured reduction in effort is not automatically growth. Someone has to decide what the organisation will do with that capacity. A team might spend more time understanding customers, shorten a queue, improve an existing service or test a new offer. Each choice needs a responsible owner and a way to see whether it helped.
This is a management and people question as much as a building question. Talk with the team about what changes in their role, what support they need and which priorities take precedence. Do not quietly fill every saved minute with unrelated responsibilities. The aim is work people can do well and a business that knows where it is going.
Make a decision before expanding
At the end of a bounded trial, decide whether to expand, adjust or stop. Look at the customer result alongside the effort, quality and cost of achieving it. A promising result may justify testing a broader range of cases. A weak result may mean the process is poorly chosen, the inputs are incomplete or the business needs a different intervention.
My role is to connect that decision with what we build: shape the business outcome, work through responsibilities and governance with your people, then implement the support and test it in practice. Agents are part of the means. A useful new business, or a better way for an existing company to serve its customers, is the point.
Before starting your next AI initiative, write down one customer outcome, the current baseline and what evidence would convince you to continue. That is a practical foundation for a conversation about what to build.