AI Agents for Business: Where They Work, the Risks and How to Run a Pilot

ApexlyTech Team 10 min read
AI Agents for Business: Where They Work, the Risks and How to Run a Pilot

TL;DR: An AI agent is a system in which a large language model (LLM) decides which steps to take and which tools to call to reach a goal, rather than following a fixed script. Agents work today on bounded, checkable tasks: triaging tickets, drafting replies from a knowledge base, reconciling records, researching across internal documents. Their main risks are confident wrong answers, leaking data through prompts and tools, and costs that grow with every loop. The OWASP Top 10 for LLM Applications names these risks, and its advice is consistent: give agents the fewest tools and permissions they need, enforce authorisation outside the model, and keep a human approval step for anything with real-world impact. Start with a narrow pilot that has a measurable baseline, a budget cap and a kill switch.

What an AI agent actually is

"Agent" is used loosely, so it helps to pin it down. Anthropic's engineering guide Building effective agents draws a useful line between two kinds of "agentic system":

  • Workflows are systems where LLMs and tools are orchestrated through predefined code paths. You decide the steps; the model fills in each one (classify this email, then draft a reply, then check the draft against a policy).
  • Agents are systems where the LLM dynamically directs its own process and tool use. You give it a goal and a set of tools, and it decides what to call, in what order, and when it is finished.

The same guide recommends finding the simplest solution possible and only adding complexity when it is needed, noting that for many applications a single well-prompted model call with retrieval and examples is enough. That is the most practical advice in this field. Many projects described as "agents" are better built as workflows: they are cheaper, faster and far easier to test, because every path through them is known in advance.

The building blocks

Whatever you call it, an agentic system for business usually has the same parts:

  • A model that reads the goal and the context and decides the next step.
  • Tools: functions the model may call, such as "search orders", "read a contract", "create a draft refund". Each tool is ordinary code with its own inputs, outputs and permissions.
  • Context: the documents, records and conversation history the model sees, often retrieved from a search index (retrieval-augmented generation, or RAG).
  • A loop and a stop condition: the agent acts, observes the result and decides again, until it reaches the goal, a step limit or a human checkpoint.
  • Logs of every prompt, tool call and output, so you can audit what happened and why.

Connecting tools used to mean custom glue for every model and every system. The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems such as files, databases and other tools, and it is now supported by a range of AI assistants and developer tools. It standardises the plumbing; it does not decide what an agent should be allowed to do. That remains your design decision.

Where agents work today

The tasks that suit agents share three traits: the inputs are mostly text or structured records, the number of steps varies from case to case, and a person or a rule can check the result cheaply. Some patterns that fit:

  • Support triage and drafting. Read an incoming ticket, look up the customer's orders and the relevant help articles, classify the issue and draft a reply for an agent to approve.
  • Internal knowledge search. Answer staff questions from policies, contracts and wikis, with citations to the source passages so the answer can be checked.
  • Back-office reconciliation. Compare invoices against purchase orders and delivery notes, flag mismatches and prepare a summary for finance.
  • Sales and research preparation. Gather public information and internal CRM history into a briefing before a call.
  • Software and operations tasks. Investigate a failing job by reading logs, suggest a fix and open a change for review (see our guide to governing AI coding assistants).

Where they fit poorly: decisions with legal or safety consequences that nobody reviews, tasks with no ground truth to check against, and high-volume, low-margin steps where a deterministic rule would do the same job for a fraction of the cost.

The risks, named precisely

The OWASP Top 10 for LLM Applications (2025) is the most widely referenced catalogue of what goes wrong. Four entries matter most for agents.

Hallucination and misinformation (LLM09)

Models produce fluent text that can be wrong, including invented facts, figures and references. In a chat window, a person may notice. Inside an agent, a wrong intermediate result can be passed to the next tool call and compound. Anthropic's guide makes the same point: autonomy brings higher costs and the potential for compounding errors. Mitigations: ground answers in retrieved sources and require citations, validate outputs against schemas and business rules, and design checkpoints where a person or a deterministic check confirms the result before it is acted on.

Prompt injection and data leakage (LLM01, LLM02)

Prompt injection is when input changes the model's behaviour in ways you did not intend. OWASP distinguishes direct injection, where a user's prompt alters the behaviour, from indirect injection, where content the model reads from external sources such as websites or files does it. For agents, the indirect form is the dangerous one: an email, a PDF or a web page the agent reads can contain instructions. OWASP states that it is unclear whether there are fool-proof methods of preventing prompt injection, so the design has to assume it will sometimes succeed.

Sensitive information disclosure is the related risk: personal data, credentials or confidential documents ending up in a prompt, a log, a third-party API or an answer shown to the wrong user. The fix is architectural: retrieve only what the current user is allowed to see, keep secrets out of prompts, and check what your model provider does with submitted data under your contract.

Excessive agency (LLM06)

OWASP identifies three root causes of excessive agency: excessive functionality, excessive permissions and excessive autonomy. In plain terms, the agent has tools it does not need, the tools can do more than the task requires, or the agent can take high-impact actions without anyone checking. Combine that with prompt injection and an attacker can make your agent act on their behalf with your permissions.

Cost (LLM10, unbounded consumption)

Every loop of an agent is another model call, and long contexts are billed per token. OWASP lists unbounded consumption as a risk in its own right. Anthropic's guide notes that agentic systems often trade latency and cost for better task performance. A runaway loop, an oversized context or a public endpoint without rate limits can turn into an unpleasant invoice. Set per-task step limits, per-user and per-day budgets, and alerts, from the first day of the pilot.

Guardrails that work in practice

The OWASP prevention guidance for excessive agency and prompt injection translates into a short list of engineering rules:

  1. Minimise tools and their functions. Give the agent only the tools the task needs, and prefer narrow tools ("get order status") over open-ended ones (a shell, raw SQL, "send any email").
  2. Least privilege, in the user's context. Tools should run with the identity and permissions of the person the agent is working for, not a powerful service account.
  3. Complete mediation. Enforce authorisation in the downstream system, not by asking the model to behave. The API that issues refunds should check limits whatever the model says.
  4. Human approval for high-impact actions. Payments, deletions, external emails and changes to records should be drafted by the agent and approved by a person.
  5. Segregate untrusted content. Mark retrieved documents and web content as data, and never let them silently change tool permissions.
  6. Validate outputs. Define expected formats, check them in code, and treat model output as untrusted input to the next system.
  7. Log, monitor and rate-limit. These do not prevent problems, as OWASP notes, but they limit damage and make incidents investigable.
  8. Test adversarially. Before launch, try to break it: injected instructions in documents, requests for data the user should not see, loops that never end.

For governance beyond the engineering, the NIST AI Risk Management Framework (AI RMF 1.0, January 2023) organises the work into four functions, Govern, Map, Measure and Manage, and NIST's Generative AI Profile (NIST AI 600-1, July 2024) applies it to generative systems. It is voluntary, but it gives a pilot a shared vocabulary for who owns the risk and how it is measured.

How to start a pilot

A good pilot is small, measurable and reversible. A sequence that works:

1. Pick one task with a baseline

Choose a task you already do by hand, where you can measure today's time, error rate or backlog. Without a baseline you cannot say whether the agent helped. Good candidates are repetitive, text-heavy and currently reviewed by someone anyway.

2. Start as a workflow, then loosen

Build the fixed-step version first. Only give the model freedom to choose steps where the fixed version demonstrably falls short. This also gives you a comparison point.

3. Keep the human in the loop

In the first phase the agent drafts and a person approves. Track how often drafts are accepted unchanged, edited or rejected. That ratio is the most honest measure of quality you will get.

4. Build the evaluation set early

Collect real examples (anonymised where necessary) with known good answers, including awkward edge cases and a few adversarial ones. Re-run them on every prompt, model or tool change, the same way you would run regression tests on code.

5. Put limits in code

Step limits, budget caps, timeouts, rate limits and a switch that turns the agent off without a deployment. Decide in advance what happens when a limit is hit: hand the case to a person, not retry forever.

6. Review data handling before go-live

Map what data the agent can read, where prompts and logs are stored, how long they are kept, and what your model provider's terms allow. If personal data is involved, involve whoever owns data protection in your organisation.

7. Decide what "success" means before you start

Write down the threshold that justifies expanding the pilot (for example, a target acceptance rate with no serious errors over an agreed period) and the conditions that would stop it.

Build, buy or both

Off-the-shelf assistants inside tools you already use are the fastest way to learn, and often enough for personal productivity. A custom agent makes sense when the value depends on your own systems and data: your order database, your contract archive, your approval rules. In practice most organisations end up with both. If you build, keep the model behind an interface you control, so you can switch providers, run models privately where data requires it, and keep logs in your own environment. Our Apex AI platform is built on that principle for teams that need private deployment.

How ApexlyTech can help

We design and build AI features and agents around existing business systems, with the guardrails above as part of the architecture rather than an afterthought: scoped tools, approval steps, evaluation sets and cost controls. See our AI and automation and custom software services, or contact us to talk through a pilot.

Frequently asked questions

What is the difference between an AI agent and a chatbot?

A chatbot answers in conversation. An agent can also take steps toward a goal by calling tools, such as searching records or drafting a change, and decides which steps to take. Many useful systems sit in between, as fixed workflows that use a model for individual steps.

Can prompt injection be fully prevented?

Not reliably. OWASP states it is unclear whether fool-proof prevention methods exist. Design on the assumption that it will sometimes succeed: limit what the agent's tools can do, enforce permissions outside the model, and require human approval for high-impact actions.

How do we stop an agent from running up large costs?

Set a maximum number of steps per task, budgets per user and per day, timeouts and rate limits, and alert on unusual usage. OWASP lists unbounded consumption as a top-ten risk because these controls are often missing.

Should an AI agent be allowed to send emails or make payments on its own?

Not at first, and for many organisations not at all. Have the agent prepare the action and a person approve it. OWASP's guidance on excessive agency recommends human approval for high-impact actions and authorisation checks in the downstream system.

Which framework should we use to govern AI risk?

The NIST AI Risk Management Framework and its Generative AI Profile are a practical, voluntary starting point. Pair them with the OWASP Top 10 for LLM Applications for the engineering controls, and with whatever legal obligations apply in the places you operate.

Sources