A small service business can trial an AI agent on a concrete job: turn incoming customer requests into draft CRM entries and proposed replies. Define the output before connecting the inbox. The pilot succeeds when a staff member can check what the agent produced, identify its source and decide whether to use it.
The UK NCSC’s guidance on agentic AI recommends matching autonomy to the consequences of failure, documenting the permitted scope and assigning responsibility for oversight. For this pilot, name an owner who can review the work and stop the system during staffed hours.
Write a short operating brief: which requests qualify, which customer fields may be copied, where drafts appear and when incomplete or conflicting information must be referred to a person. Use that brief as the acceptance criteria when reviewing the first results.
Make permissions match the intended output
OWASP’s excessive-agency guidance explains the risk created by unnecessary functions, broad permissions and excessive autonomy. It recommends minimizing tools and access, with authorization enforced in the connected system. An instruction telling an agent to stay within a task should be backed by controls at the point where data is read or changed.
Choose an integration whose effective permissions fit the job. A drafting workflow can use a restricted source of requests and a separate draft destination. If the connector also allows sending messages, deleting records or exporting an entire customer database, those capabilities need their own justification and controls.
Ask the supplier to demonstrate those limits with a test account using the proposed permissions. Confirm which resources it can reach and which actions are rejected. Document permitted and blocked actions alongside answer quality during the pilot review.
Require human approval for consequential actions through the application or connector that performs them. For a customer reply, the reviewer should see the final recipient, message and attachments before approving the send. A subsequent change to that proposed action should return it for review.
Rehearse a failure before the live pilot
Before using real customer data, run a synthetic request containing a misleading instruction to export a dummy customer list. Observe both the proposed response and the connected application’s behavior. Record whether the attempted action was blocked, what alert appeared and how the operator stopped the run. Use the result to identify controls that need work before connecting live data.
NCSC also calls for protected activity records and an emergency stop that covers the wider system. Keep records of tool actions, approvals and resulting changes, with access limited because logs can contain customer information. Test how to disable the agent and revoke its connections; stopping the visible conversation may leave other components operating.
