Agentic penetration testing

Methodology and rules of engagement.

Standards we follow

We map our testing and reporting to public standards, so you can check our coverage against them.

Standards RedVault maps its testing to
AreaStandardEdition
Web applicationsOWASP Web Security Testing Guidev4.2 (current stable)
APIsOWASP API Security Top 102023
LLM applicationsOWASP GenAI LLM Top 102026
AI agentsOWASP Top 10 for Agentic Applications2026 edition
Test processNIST SP 800-115, Technical Guide to Information Security Testing and Assessment2008
SeverityCVSS (FIRST)v4.0
Weakness IDsCWE (MITRE), including CWE-1427 for prompt injectioncurrent

Editions checked Nov 19, 2026. WSTG v5 is in development; we'll move to it when it's released.

Phases of an engagement

Every engagement runs through the same eight phases. Nothing touches your systems before phase 2 is complete.

  1. 01Pre-engagement. Scoping call, mutual NDA, fixed quote, rules of engagement, and a written authorization naming each target. You create test accounts for each role and name a contact for the testing window.
  2. 02Setup and scope check. We load the authorized targets into our proxy's allowlist and the agents' configuration, turn on logging, and confirm that a request to an out-of-scope address is blocked before any testing begins.
  3. 03Reconnaissance and mapping. Agents sign in as each test role, map pages, routes, endpoints and parameters, and read any API documentation you provide. For AI applications, they also map the tools, data sources and actions the model can reach.
  4. 04Testing. Agents run test cases against each in-scope target, following the standards above. Testing is non-destructive by default.
  5. 05Exploitation, with approval. Before an agent exploits a candidate finding or takes any action that writes data, a RedVault engineer approves it. We stop at the least proof needed to show impact.
  6. 06Verification. The engineer reproduces each candidate finding by hand. Anything we can't reproduce is dropped, not reported.
  7. 07Reporting and readout. We deliver the report and walk you through it on a call. Critical findings are reported the same day they're confirmed, without waiting for the report.
  8. 08Retest and close-out. After you fix the findings, we retest them once and update the report. We delete engagement data 90 days after the final report.

Rules of engagement

These rules apply to every engagement. The signed rules of engagement for your test can add to them, but never relax the prohibited actions.

Authorization

  • Signed by someone with authority over every target, before any testing.
  • Names each target: domains, URLs, API base URLs, AI endpoints, and the environment (staging or production).
  • Lists the test accounts, testing window, rate limits, and contacts on both sides.
  • If a target is hosted by a third party, you confirm that testing it is allowed under that provider's policy.

Scope

  • We test only the named targets. Anything else we discover, such as a third-party API your app calls, is noted in the report and not tested.
  • Requests to addresses outside the allowlist are blocked by our proxy, not left to the agents' judgment.

Never done, under any engagement

  • Denial-of-service or load testing
  • Social engineering, phishing, or contacting your staff or users
  • Physical access attempts
  • Deleting or altering your data, or installing persistence
  • Accessing more customer data than the minimum needed to prove a finding
  • Attacking AI model providers or other third parties

Stop conditions

We stop all testing and call your contact at once if:

  • a target becomes slow or unstable;
  • we interact with anything outside the signed scope by mistake;
  • we reach sensitive real data, such as customer personal data at scale;
  • we see signs that someone else has already compromised the system.

Communication

  • A short status update each testing day.
  • Critical findings reported the same day they're confirmed.
  • Evidence in reports is redacted so that secrets and personal data don't appear in full.

How we use AI agents

  • Agents reach your systems only through our intercepting proxy. The proxy holds the signed scope as an allowlist, enforces your rate limits, and logs every request. Anything outside the scope is blocked and never sent.
  • An engineer is in charge of every run. Agents never run without an engineer on duty who can stop them. Exploitation steps and any action that writes data need the engineer's approval first.
  • Everything is logged. Every model call and every tool call is recorded with a timestamp and kept with the engagement record. You can ask for the logs for your engagement.
  • Models. We use frontier models from commercial providers through their APIs, under terms that don't let the provider train on our data. Some providers keep prompts and outputs for a limited time to monitor for misuse; that applies to test traffic too.
  • Credentials. We use only the test accounts you create for the engagement. They're stored encrypted and deleted at close-out.
  • What agents never do. Decide the scope, contact your people, or report a finding that a person hasn't reproduced.

Severity, reporting and retest

Severity

Each finding gets a CVSS v4.0 score and a final rating of Critical, High, Medium, Low or Informational. Where your business context changes the real risk, such as an internal-only tool or data that's already public, we adjust the rating and say why.

The report

  • Executive summary in plain language
  • Every finding with ID, CWE, location, reproduction steps, evidence, impact and a specific fix
  • Coverage statement: what we tested, with which roles, and what we didn't test and why
  • Agent activity summary per target
  • JSON export of the findings for your tracker

We deliver the report through an encrypted channel and walk through it with you on a call.

Retest

One retest of the reported findings is included, within 60 days of the report. We update the report to show which findings are fixed and which remain.

Data handling and disclosure

  • What we keep. Requests and responses, screenshots, agent logs, and the test-account credentials you give us.
  • How we store it. Encrypted at rest, with access limited to the engineer on your engagement.
  • How long. We delete engagement data 90 days after the final report, or sooner if you ask. We'll confirm the deletion in writing on request.
  • Who sees findings. Only the contacts you name. We never publish anything about your engagement without your written permission.
  • Vulnerabilities in third-party products. If we find a flaw in a product you use, we tell you first and coordinate disclosure to the vendor with your agreement.

Questions about this methodology: hello@redvaultsecurity.com