> For the complete documentation index, see [llms.txt](https://doc.threatreaper.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.threatreaper.ai/user-guide/ai-probe-ai-security-testing.md).

# 🧪 AI-Probe — AI Security Testing

**Menu:** PROBE > AI-Probe, with four tabs: **Code Scan**, **Artifact Scan**, **Adversarial Scan** and **AI Adversarial Datasets**.

## 5.1 Code Scan

Scans a source repository for security problems specific to AI applications.

{% stepper %}
{% step %}

### Choose the input mode

* **Connector:** pick a GitHub connector, then a repository and branch from the lists.
* **URL:** paste a repository URL (for example, <https://github.com/owner/repo> or a /tree/\<branch> URL).
  {% endstep %}

{% step %}

### Select the scan types

Select the scan types, or use **Select All**.

The 21 types are: Secrets Exposure, Insecure Configurations, Prompt Injection, Dependency Risks, Authentication Issues, Data Leakage, Sensitive Logging, Agentic AI Risks, RAG System Risks, Production Exposures, Model Output Security, Model Governance, Monitoring & Observability, Bias & Fairness, Model Theft Protection, Data Poisoning, Adversarial Defenses, Model Interpretability, Multi-Tenant Isolation, System Prompt Leakage, Unbounded Consumption.
{% endstep %}

{% step %}

### Name and assign the scan

Give the scan a name and, if shown, assign it to a project.
{% endstep %}

{% step %}

### Start the scan

Start the scan and watch the progress bar.
{% endstep %}
{% endstepper %}

Results show an overall risk score, a severity breakdown, and findings grouped by category. Each finding lists severity, file and line, a code snippet and a suggested fix. Use **Download Report** for a PDF; scan lists can also be exported to CSV.

## 5.2 Artifact Scan (model files and ML supply chain)

Checks model files and project artifacts for unsafe content and supply-chain risk.

* **Input:** a connector or URL, or a direct upload.
* **Recognised files include:** model formats (.pkl, .pickle, .pt, .pth, .h5, .hdf5, .pb, .onnx, .safetensors), configuration files (.json, .yaml, .yml, .toml, .ini, .env), dependency manifests (requirements.txt, package.json, Pipfile, setup.py, pom.xml, build.gradle) and documentation such as LICENSE, README and MODEL\_CARD.
* **Check categories:** model file security, config security, supply chain, data leakage, architecture security, license compliance, documentation, runtime security, provenance, embedding security, compression and multimodal.

## 5.3 Adversarial Scan (red teaming a live AI endpoint)

Sends attack prompts to your model or application and records whether it resists.

{% stepper %}
{% step %}

### Choose the target type

Choose the target type: a custom URL endpoint, an existing connector (for example, an OpenAI or Azure OpenAI connector), or a Hugging Face repo.
{% endstep %}

{% step %}

### Enter the target details

Enter the target details: URL, model name and type, and the authentication token or API key if needed. Private/internal IP targets are rejected for safety.
{% endstep %}

{% step %}

### Select attack datasets

Select one or more attack datasets, including curated sets such as REST API security, agentic AI vulnerabilities, RAG attacks and MCP security testing.
{% endstep %}

{% step %}

### Choose attack settings

Choose the attack intensity: basic, intermediate, advanced or mixed. Set concurrency and timeout if exposed.
{% endstep %}

{% step %}

### Optionally enable mutation

Optionally enable **mutation**, which rewrites each attack prompt using several encoding and obfuscation strategies to test evasion resistance.
{% endstep %}

{% step %}

### Start the scan

Start the scan.
{% endstep %}
{% endstepper %}

Results list each test as passed or failed—the model resisted or complied—with filters and a detailed dialog. Export the result as a PDF report.

## 5.4 AI Adversarial Datasets

Browse the attack datasets available to adversarial scans, and upload or create your own custom dataset.

## 5.5 Tips

{% hint style="warning" %}
Code, artifact and adversarial scans consume credits when started. Failed scans are refunded automatically.
{% endhint %}

{% hint style="warning" %}
Do not point adversarial scans at systems you do not own or do not have authority to test.
{% endhint %}

## 5.6 Errors you might see and how to fix them

| **Message**                                                                                                                          | **What it means**                                               | **What to do**                                                                            |
| ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| "Please select a project before starting the scan."                                                                                  | No project chosen                                               | Pick a project from the dropdown                                                          |
| "Please select a GitHub connector" / "Please enter a repository name (e.g., owner/repo) or paste a GitHub URL"                       | No code source given                                            | Select a GitHub connector or paste a valid repository URL                                 |
| "Please select at least one scan type" / "Please select at least one security guardrail type."                                       | No checks selected                                              | Select one or more scan categories, or use **Select All**                                 |
| "Please upload at least one model artifact file."                                                                                    | Artifact Scan started with no file                              | Upload a supported model/config/dependency file                                           |
| "File too large. Maximum size is 5GB."                                                                                               | Upload exceeds the size limit                                   | Split or compress the artifact, or upload a smaller file                                  |
| "🔒 Invalid credentials. Please check your API key/token."                                                                           | The target endpoint rejected the credentials (Adversarial Scan) | Re-check the API key/token for that model connection                                      |
| "🔍 Endpoint not found. Please verify your target URL."                                                                              | The target URL is unreachable or wrong                          | Confirm the endpoint URL is correct and publicly reachable                                |
| "Please select at least one dataset." / "Please select at least one attack technique" / "Please provide a custom prompt description" | Adversarial Scan has no attack content configured               | Choose a dataset, or a technique, or enter a custom prompt description                    |
| "Could not fetch repositories automatically. Please enter manually."                                                                 | GitHub connector couldn't list repos                            | Type the repository as `owner/repo` manually                                              |
| Scan stuck at "running" / "Scan failed: …"                                                                                           | A step in the pipeline errored out                              | Check Notifications for details; retry the scan — failed scans are refunded automatically |

## 5.7 What success looks like

**Code Scan** results show an overall risk score and findings grouped by category, for example:

| **Severity** | **Category**     | **File**              | **Finding**                                         |
| ------------ | ---------------- | --------------------- | --------------------------------------------------- |
| 🔴 Critical  | Secrets Exposure | `src/config.py`       | Hardcoded API key detected                          |
| 🟠 High      | Prompt Injection | `src/chat_handler.py` | User input concatenated directly into system prompt |
| 🟡 Medium    | Dependency Risks | `requirements.txt`    | Package with a known CVE                            |

**Artifact Scan** results are grouped the same way, by category (model file security, supply chain, data leakage, etc.), with a pass/flag per check.

**Adversarial Scan** shows summary cards—**Total Tests**, **Passed**, **Failed**, **Success Rate**—plus a results table:

| **#** | **Test Type**    | **Status** | **Risk Level** |
| ----- | ---------------- | ---------- | -------------- |
| 1     | Prompt Injection | ✅ Passed   | —              |
| 2     | Jailbreak        | ❌ Failed   | 🔴 Critical    |
| 3     | PII Exposure     | ✅ Passed   | —              |

Every finding can be downloaded as a PDF report, and scan lists can be exported to CSV.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://doc.threatreaper.ai/user-guide/ai-probe-ai-security-testing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.