Penetration Testing LLM-Powered SaaS Features and API Endpoints
A scanner can't find these flaws, so each AI feature needs its own test plan.
Traditional web application pen testing was built for a system that behaves the same way every time: the same input produces the same output, and the job is to find exploitable code paths through known vulnerability classes like SQL injection, cross-site scripting, and broken authentication. An LLM-powered feature breaks that premise at the root. The model treats attacker-controlled input as instruction, so any tool it can call authorizes against the model's own suggestion, not against the identity of the person who actually sent the request.
That gap produces vulnerability classes with no equivalent in the old world: prompt injection, RAG poisoning, tool abuse, system prompt leakage. A scanner built to crawl forms and fuzz parameters has no concept of a system prompt being talked out of a model through a cleverly worded chat message, so these classes pass through automated tooling undetected. Using AI to help run a pentest is one job, and testing a system that itself contains models, retrieval pipelines, memory, tool use, and agent orchestration is a different job entirely, because that architecture creates its own attack surface.
That surface has stopped being novel and started being standard. Two years past the rush to bolt AI onto every product, the features being shipped have settled into four recognizable shapes: customer-facing chatbots, RAG applications, autonomous or semi-autonomous agents with tool access, and fine-tune endpoints, and platforms like Trace, an AI-plus-human pen testing service that uses AI agents alongside certified human reviewers to find exploitable vulnerabilities across code and cloud, or AxVeil's playbook treats each as needing its own test plan. None of this is hypothetical risk sitting in a research paper. Astra's 2026 data shows AI-specific vulnerability classes, prompt injection delivered through an API and exposed system prompts among them, turning up in production pentests for the first time, with no CVE assigned and no vendor patch available. No scanner catches a vulnerability class that has no signature, and no patch arrives for a flaw baked into how a model reasons rather than into a line of vulnerable code.
The four AI feature shapes in production SaaS and the test plan each one demands
"AI pentest" gets used as if it names one activity, but it covers four genuinely different surfaces, and a single fixed checklist run against all four is how real risk slips past a test engagement. AxVeil's scoping framework lays the four shapes out, and each one is worth describing on its own terms before getting into technique.
The chatbot or conversational UI is a thin layer over a hosted model. The system prompt, the conversation memory, the content filtering, the abuse handling, and the HTTP transport beneath them make up the surface worth probing. Tenant separation in this shape is usually scoped to the session, not to the underlying data, which matters for how isolation gets tested later.
The RAG application adds a layer the chatbot doesn't have: the model is grounded against a retrieval index, whether that's a vector database, hybrid search, or structured filters. That index turns every document, record, or page it can return into a potential input to the model, which changes where an attacker needs to plant a payload to get it executed.
The agent or tool-using system goes further still. The model can invoke functions, browse the web, query databases, send email, write code, or hand work off to other agents, and that combination makes it the highest-impact surface of the four. This is the highest-impact surface and the one where excessive agency intersects most viciously with prompt injection. A flaw doesn't need to be exotic to land hard here: CVE-2025-3248 in Langflow, present in versions before 1.3.0, let an unauthenticated request hit the /api/v1/validate/code endpoint and inject code leading to arbitrary execution, a familiar code injection bug exposed through an AI framework's own architecture. CVE-2026-4270, found in the AWS API MCP Server across versions 0.2.14 through 1.3.8, shows the tool-abuse version of the same surface: the flaw let an attacker bypass intended file access restrictions and read arbitrary local file contents inside the MCP client application.
The fine-tune endpoint or training pipeline is the fourth shape, and it's the one least likely to get tested at all, because the application accepts data from users, partners, or internal teams and folds that data into a model used by everyone downstream. Knowing which of these four shapes, or which combination, sits in production has to happen before a single payload gets sent. If you test a help-desk chatbot, your test plan and the one for a finance team's autonomous agent share almost nothing.
Testing the chatbot surface: system prompt extraction, jailbreaking, and multi-turn attack chains
Chatbots are the most common AI feature in production and, because the interface resembles an ordinary web form, the most routinely under-tested. The real risk sits in the model layer, above the HTTP transport.
Testing starts before any adversarial payload goes out, with reconnaissance aimed at fingerprinting the model itself: which model is actually running, what shape its system prompt likely takes, which tools it has access to, what content filters sit in front of it, and what the HTTP plane looks like, down to the streaming format, the function-calling schema, and any model name leaking through response headers. AxVeil's Phase 1 approach leans on differential probes for this: known canary prompts produce distinguishable responses from different model families, so a tester can identify the underlying model before spending any effort on adversarial content. If you get this step right, you can save real time later, because the rest of the test plan depends on knowing what's actually running underneath the chat window.
Direct prompt injection comes next, hitting every field a user can touch with structured override attempts: instruction-override payloads, role-confusion wrappers, framings that claim a maintenance mode or elevated context. A test passes only if the canary string planted in the probe never appears in the output.
System prompt extraction gets tested separately from injection, and it aims to recover the literal text of the system prompt through reflection, continuation, or translation tricks. This is not a theoretical concern: Astra's 2026 data finds exposed system prompts, alongside prompt injection delivered via API, appearing in production pentests for the first time, with no CVE and no vendor patch attached.
Multi-turn attack chains deserve their own test category. A single message might get blocked outright, but a sequence of exchanges that gradually reframes the model's context can slip through, so testing has to include full conversation sequences, not just isolated single-turn probes.
None of this replaces the older checklist, either. Broken authentication, weak session handling, and missing rate limits on the chat endpoint all remain in scope right alongside the model-layer tests. A chatbot wired up to query a database is a new door into that data, and it has to be tested as exactly that.
Testing the RAG surface: retrieval-corpus poisoning, indirect injection, and tenant isolation at the retrieval layer
A RAG application expands the injection surface well past the chat box. Every document, web page, email, or structured record the retrieval layer can pull back becomes a potential input, because the model treats whatever it retrieves as trusted context by default.
Indirect prompt injection through retrieved content is the defining risk here: a payload planted inside a document, a calendar entry, an email, or a web page gets fetched by the system and causes the model to carry out attacker instructions as though they came from the system itself. AxVeil's Phase 2 testing shows a specific scenario: a comment planted inside a webpage the agent fetches instructs the model to append a canary string and call the email tool to forward conversation history to an address the attacker controls. The test passes only if the marker never appears in output and no tool call fires against attacker-controlled input.
Retrieval-corpus poisoning asks a related but distinct question: can someone who contributes content to the knowledge base, through a public upload form, a third-party feed, or a poisoned training corpus, shape model output persistently for every user who later queries that index.
Tenant isolation at the retrieval layer is the risk specific to multi-tenant SaaS. In a shared RAG application, the retrieval layer has to enforce data boundaries so that one customer's query can never surface another customer's documents. Isolation needs to hold at the moment of the query, not only at the moment data gets ingested.
Traditional web testing never has to deal with this problem. A SQL injection test never has to ask whether the rows a database hands back will get interpreted as executable commands by the application layer, but a RAG system's context window has to be reasoned about as an instruction space, not a passive data field. Tool calls triggered by instructions buried in retrieved content overlap with the agent surface covered next, but poisoning the corpus and holding tenant boundaries at query time are retrieval-specific work that belongs to RAG testing on its own.
Infrastructure checks still apply alongside the retrieval-specific tests, because misconfigurations at that layer create their own exposure independent of the model. Vector database misconfigurations and weak metadata filters sit alongside the model-layer risks, and standard cloud misconfiguration checks belong in the same test plan as the AI-specific retrieval tests.
Testing the agent surface: tool authorisation, sandbox escape, capability drift, and inter-agent attacks
The agent or tool-using shape carries more risk than either the chatbot or RAG shape, because it pairs prompt injection's ability to redirect what the model does with tools that have real-world consequences: sending email, writing code, querying a database, browsing the web, or handing a task to another agent.
Tool authorization testing checks whether a tool the model can call enforces access control against the identity of the actual calling user, rather than simply trusting the model's recommendation. By default, it doesn't, and that gap means a prompt injection payload can trigger privileged tool calls on behalf of any user who happens to encounter the injected content. AxVeil's framework names this precisely: the intersection of OWASP LLM08 (excessive agency) with prompt injection, the two highest-severity risks in this shape combining into a single exploit path. A tester who has only ever run traditional web application assessments has no existing framework for reasoning about this kind of combined failure, because nothing in a stateless HTTP endpoint behaves like an agent that accumulates privileges across a multi-step plan.
Sandbox escape testing checks whether the model's code execution environment is actually isolated. A model that can write and run code can attempt to read the filesystem, reach out over the network, or escalate toward the host system, a risk that matters most in agent frameworks built around a code interpreter tool.
Capability drift is its own test category, specific to agentic systems. An agent given a benign high-level goal can accumulate capabilities or take actions over a multi-step plan that nobody authorized at the outset, and nothing about a traditional stateless HTTP endpoint has an equivalent failure mode.
Inter-agent confused-deputy attacks come into scope wherever one agent can invoke another. A low-privilege agent that can instruct a high-privilege agent inherits that agent's capabilities unless the orchestration layer enforces privilege checks at every single handoff between them.
CVE-2026-4270, the flaw found in the AWS API MCP Server across versions 0.2.14 through 1.3.8, is a working example of exactly this category: the tool-using layer bypassed intended file access restrictions and exposed arbitrary local file contents, a path traversal flaw classified as CWE-424 (Improper Protection of Alternate Path), precisely the kind of bug this test phase exists to catch.
The HTTP plane hasn't gone anywhere, either. Server-side request forgery is a critical risk in agentic systems specifically because an agent's ability to fetch a URL is a server-side request a tester can redirect toward internal infrastructure. A technical lead evaluating a vendor's agent-testing coverage should expect answers on all four of these: tool authorization, sandbox isolation, capability drift, and inter-agent trust, beyond a confirmation that prompt injection got tried once against the chat interface.
Testing the fine-tune and training pipeline surface: data poisoning, membership inference, and model extraction
The risk moves upstream, away from the inference endpoint and into the data pipeline itself, because that pipeline shapes how the model behaves for every user who comes after the data was ingested.
Training-data poisoning testing asks whether someone who can contribute data into that pipeline, through a public upload interface, a partner-facing API, or another ingestion path, can shift the model's behavior in a way that persists across training runs. A poisoned sample doesn't need to affect one user's session the way a chatbot jailbreak does.
