Inside OpenAI's Codex Security: Architecture, Prompts, and Agentic Security Review

OpenAI has released the source for Codex Security, the application-security system previously known as Aardvark. This is a welcome contribution to the agent and security-tooling ecosystem: unusually, we can inspect not only the CLI and SDK, but the prompts, validation contracts, multi-agent workflow, and artifact machinery that shape the scanner’s behaviour.

Codex Security is not a conventional static analyzer. There is no new parser, taint engine, abstract interpreter, query language, or vulnerability-signature database hiding in the TypeScript. The public package is a security-specific operating system for a coding agent: prompts, skills, deterministic file inventories, candidate ledgers, multi-agent coordination, validation rules, attack-path analysis, artifact schemas, scan history, CI policy, and substantial defensive machinery around model-produced output.

The model does the semantic work. The open repository shows how OpenAI structures, checks, and operationalizes that work. That architecture—not merely the underlying model—is what makes the release useful to study and build on.

What was released, and when

The product itself was announced in research preview on March 6, 2026. The public CLI/SDK repository arrived later: GitHub reports that openai/codex-security was created on July 13, 2026, with its first root commit on July 15. I reviewed the repository at 5b2b21640b782504258bb71585bc33cf4297cb9b, committed July 28.

At measurement time it had:

Repository fact	Value
Stars	229
Forks	21
Open issues	2
Commits reachable from `main`	108
Tracked files	171
Tracked bytes	2,486,283
License	Apache-2.0
SDK package	`@openai/codex-security` 0.1.0
Bundled plugin	0.1.14
Bundled Codex runtime/SDK	0.144.6

The package is public and Apache-licensed, but access to actual scans is still gated. OpenAI’s Codex Security documentation describes the CLI and SDK as a limited beta for approved customers and partners. The hosted GitHub-connected service is the broader ChatGPT research preview.

Development happens in OpenAI’s canonical internal repository. CONTRIBUTING.md explains that this public repository is a one-way projection, so external pull requests cannot be imported directly. That limits the usual contribution path, but the Apache-licensed source still gives the ecosystem a substantial implementation to inspect, test, adapt, and fork.

The repository by weight

I measured tracked files rather than whatever happened to exist in node_modules or a build directory.

Area	Files	Bytes	Lines
TypeScript application source	20	341,832	11,094
TypeScript tests	29	477,941	14,598
Bundled plugin	94	1,530,633	about 24,417*
Python workbench scripts	31	655,610	16,926
Top-level skill prompts	13	202,890	2,088
Skill references	13	134,005	1,529
Shared references	9	82,013	893
JSON schemas	3	21,701	919

*The plugin total includes compressed and binary assets, so aggregate text-line counts are approximate. The source-specific rows are exact newline counts.

The top-level SKILL.md files alone contain 28,647 whitespace-delimited words. A rough four-characters-per-token estimate puts them around 50,000 tokens before adding reference documents, repository instructions, source code, tool output, candidate ledgers, or conversation history.

The largest are revealing:

Skill	Words	Lines
`triage-finding`	3,922	342
`finding-discovery`	3,373	155
`propose-security-hardening`	3,210	240
`vulnerability-writeup`	3,087	263
`track-findings`	2,954	222
`security-diff-scan`	2,940	167
`deep-security-scan`	2,572	166
`validation`	1,766	110
`fix-finding`	1,460	112
`attack-path-analysis`	1,196	117

This is not one clever prompt. It is a procedural corpus.

For reproducibility, the core measurement was deliberately boring:

from pathlib import Path

skills = sorted(Path("sdk/typescript/_bundled_plugin/skills").glob("*/SKILL.md"))
print(sum(len(path.read_bytes()) for path in skills))
print(sum(path.read_text().count("\n") for path in skills))
print(sum(len(path.read_text().split()) for path in skills))

It returns 202,890 bytes, 2,088 lines, and 28,647 words at the reviewed commit.

The architecture

The shortest honest diagram is:

User / CI / TypeScript application
                │
                ▼
@openai/codex-security CLI and SDK
  target validation · auth · cost · cancellation
  isolated Codex home · plugin bootstrap · event stream
                │
                ▼
Codex runtime + bundled Codex Security plugin
                │
      ┌─────────┼─────────┐
      ▼         ▼         ▼
 threat model  discovery  delegated workers
      │         │         │
      └─────────┴─────────┘
                ▼
        candidate ledger
                ▼
            validation
                ▼
       attack-path analysis
                ▼
       canonical JSON artifacts
                ▼
 Python workbench validation and sealing
                ▼
 report.md · SARIF · findings · coverage · history

The CLI command is registered in src/cli.ts. It eventually calls CodexSecurity.run(), which delegates to the private #run() orchestration method.

That method:

normalizes the repository, path, Git-diff, or working-tree target;
rejects output directories inside the repository or enclosing worktree;
prepares an isolated Codex home and plugin installation;
resolves a Python 3.10+ runtime;
registers a durable scan in the workbench database;
writes exact runtime paths into environment variables;
creates a Codex client;
starts one streamed agent thread;
tracks workers, reconnects, usage, estimated cost, cancellation, and completion;
asks the workbench to seal the scan;
rejects the result unless all canonical artifacts exist and validate.

The Codex thread is started with:

const thread = codex.startThread({
  workingDirectory: scanDir,
  skipGitRepoCheck: true,
  approvalPolicy: "never",
});

The working directory is the private scan-output directory, not the source checkout. scanRuntimeCodexConfig() installs a permission profile with filesystem-wide read access and writes constrained to workspace roots. Login shells are disabled. The normal scan is meant to read source and write evidence elsewhere, not casually remodel the target repository.

The default model configuration is equally explicit in src/config.ts:

model = "gpt-5.6-sol"
model_reasoning_effort = "xhigh"

[features]
plugins = true
goals = true

[features.multi_agent_v2]
enabled = true
max_concurrent_threads_per_session = 9

The model is hosted. The harness around it is inspectable.

The first prompt is surprisingly small

The programmatically generated bootstrap prompt lives in scanPrompt(). For a standard whole-repository scan, the core is:

Use the installed $codex-security:security-scan skill at "$CODEX_SECURITY_PLUGIN_ROOT/skills/security-scan/SKILL.md".
Run this Codex Security scan non-interactively.
This exhaustive scan authorizes the delegated-worker phases required by the selected skill; use available subagent tools and continue with parent-agent fallback if capacity changes.
This SDK host does not render MCP Apps; use the terminal/chat workflow.
Use "$PYTHON" as <python_command> for every plugin helper; replace any literal python or python3 helper invocation with this exact interpreter.
Repository root: "$CODEX_SECURITY_REPOSITORY"
Use this exact scan directory for all scan output: "$CODEX_SECURITY_SCAN_DIR"
...
Scan target: the entire repository.
Write the complete canonical scan-manifest.json, findings.json, and coverage.json, but do not finalize or seal them; the SDK workbench owns authoritative metadata, finalization, report generation, and sealing.

That is a dispatcher, not the scanner. It tells Codex which skill to load and nails down artifact ownership. The detailed behavior lives in Markdown files the agent reads as it moves through phases.

This matters architecturally. OpenAI has not stuffed every instruction into one system prompt. It uses staged prompt loading: load the current phase, consume only the inputs that phase requires, produce durable output, then load the next phase. The security-diff-scan skill says this bluntly:

Do not read ahead into later-phase skills until the current phase has completed.

That is context management disguised as process discipline.

Standard scans: inventory first, then reasoning

The security-scan skill defines the standard workflow:

build or load a threat model;
create one deterministic in-scope file list;
review every file and build one candidate ledger;
validate every candidate in one compact pass;
perform attack-path analysis over surviving or deferred candidates;
produce canonical manifest, findings, and coverage JSON;
finalize once.

The file inventory is not left to model memory. repository-wide-scan.md prescribes:

(cd "<repo_root>" && rg --files --hidden --glob '!.git/**' -- "<scope>" | LC_ALL=C sort) > "<discovery_dir>/in_scope_files.txt"

Then comes one of the most ambitious requirements:

Review every listed file from start to finish.

The prompt explicitly forbids skipping examples, tests, demos, or fixtures when they contain runnable behavior, and says not to stop reviewing a file after finding one bug.

This is simultaneously one of the most ambitious and hardest-to-verify parts of the design. The deterministic inventory proves what existed. It does not prove the model applied meaningful attention to every item. On a large monorepo, “file appeared in the list” and “file received competent semantic review” are very different propositions. The coverage contract exposes some failures; it cannot make attention infinite.

Candidates are normalized by Python rather than model intuition. Each row carries CWE IDs, exact locations, roles such as entrypoint, root_control, and sink, concise evidence, and an optional instance key. normalize_candidates.py validates that shape, merges exact duplicates, and assigns stable IDs. Discovery is allowed to be speculative; candidate accounting is not.

Threat modelling is the product bet

The threat-model skill asks the agent to establish:

assets and privileges that matter;
trust boundaries;
attacker-controlled inputs;
invariants the code must preserve;
repository-wide high-impact failure modes.

A supplied threat model, sufficiently specific AGENTS.md, or resolved SECURITY.md can become authoritative. The CLI can also ingest Markdown, text, PDF, and DOCX architecture or policy documents through repeated --knowledge-base arguments.

This is where Codex Security can outperform ordinary pattern matching. A static rule can notice a dangerous operation. A repository-specific threat model can help determine whether a normal user reaches it, whether it crosses a tenant boundary, and whether the outcome matters.

The inverse risk is that a wrong threat model becomes a bias amplifier. Stale architecture documents, undocumented deployment surfaces, or an incorrectly inferred trust boundary can systematically lower the severity of real bugs. Editable policy is useful, but it should be reviewed like code; once prose guides discovery and severity, it becomes part of the effective security configuration.

Discovery is broad; reportability is narrow

The finding-discovery skill is the largest scan prompt at 3,373 words. It focuses on technically plausible issues:

authorization bypass;
confused deputy behavior;
SSRF;
path traversal;
injection with a real sink;
cross-tenant exposure;
sensitive state change without enforcement;
sandbox or trust-boundary escape.

It repeatedly insists on preserving exact root controls and separately reachable instances. A shared helper is not permission to collapse every vulnerable route into one vague paragraph. A neighboring same-CWE finding does not close an advisory-seeded row. This is deeply procedural because LLMs love representative examples: find one dramatic route, describe the pattern, and quietly forget the other twelve instances. The prompt spends thousands of words fighting that instinct.

It also explicitly excludes generic maintainability comments and “needs more validation” observations without a real exploit path. That focus is valuable: the system is trying to produce an application-security report, not a general code-quality review.

Validation is a separate phase, not a confidence adjective

The validation skill defines an evidence ladder:

crashing proof of concept for crash, parser, memory, or denial-of-service candidates;
AddressSanitizer or Valgrind;
non-interactive debugger traces;
focused unit or integration tests;
reproduction through the real HTTP, CLI, RPC, parser, queue, plugin, or package interface;
static source/control/sink tracing when dynamic execution is blocked or disproportionate.

Every candidate must leave validation with one disposition: reportable, suppressed, not_applicable, or deferred. Setup failure is explicitly not counterevidence. Missing infrastructure must be recorded as a proof gap rather than translated into “probably safe.”

The best line is simple:

Do not imply validation happened when it did not.

That explicit rule matters. Agent workflows need to distinguish observed execution from code-level inference, especially when the final report is consumed by CI or security teams.

Attack-path analysis tries to falsify the finding

Surviving candidates move to attack-path-analysis. The model must establish exposure, entry points, attacker identity, privileges, trust boundaries, sensitive-data flow, reachability, mitigations, impact, and likelihood.

Then it must identify the strongest repository evidence against its own claim:

out of scope;
internal-only;
admin-only;
not cross-boundary;
not attacker-reachable;
not meaningfully reportable.

This is the part I like most. Most scanners accumulate supporting evidence. Codex Security explicitly asks for counterevidence before severity is assigned.

The severity policy is conservative by automated-scanner standards. Dangerous sinks, missing headers, odd configurations, generic correctness bugs, self-XSS, weak exploit chains, and already-privileged preconditions are not supposed to survive as high or critical. Once impact and likelihood are established, a fixed matrix determines severity and priority.

Whether the model follows this consistently is an evaluation question. The prompt at least has the right theory of evidence.

Deep scans are repeated search, not one longer thought

The deep-security-scan skill uses repeated independent discovery to reduce variance. It does not simply increase reasoning effort and hope.

The released defaults in deep_scan_config.py include:

Deep-scan control	Default
Discovery subagents per run	3
Stop after consecutive no-new-candidate runs	6
Maximum discovery runs	60
Automatic worker cap	6

Discovery stops when it appears saturated or reaches the cap. The parent merges and deduplicates candidates, synthesizes one canonical threat model, then runs centralized validation and attack-path analysis exactly once. Recurrence across workers is search evidence, not proof that a bug is real.

For every final reportable finding, deep mode launches exactly one dedicated vulnerability-writeup worker. It then runs one hardening-analysis pass across the complete finding collection.

This is expensive by construction. Deep scans require at least six usable worker slots and can block for up to 24 hours. They are closer to commissioning an audit than enabling a linter.

Diff scans deliberately use asymmetric scope

Pull-request, commit, branch, and working-tree reviews use security-diff-scan.

The threat model remains repository-wide. Discovery, validation, and attack-path analysis stay anchored to changed files and directly supporting code. A changed shared guard or sink can pull affected siblings into scope, but an ordinary diff scan is not allowed to wander into an unrelated repository audit.

That asymmetry is correct. The system needs the whole application’s trust model to understand a patch, but CI cannot spend six hours rediscovering the entire codebase every time someone changes a serializer.

The workbench enforces the contract

The Python workbench is larger than the TypeScript application source by line count. It manages scan identity, durable state, candidate ledgers, progress, history, validation records, ranking, report projection, and finalization.

The canonical result is not the model’s final chat message. A successful scan requires:

scan-manifest.json
findings.json
coverage.json
report.md

The workbench validates schemas, adds authoritative metadata, calculates digests, generates Markdown and SARIF projections, and seals the result. collectResult() independently rejects missing artifacts afterward.

Coverage can be complete, partial, or unknown. Incomplete coverage exits with code 2 and cannot pass a severity policy. Scan comparison also refuses to mark a missing finding resolved when the later scan did not cover the original scope.

This is the repository’s most credible contribution. It turns model output into a transaction with invariants.

Defensive engineering around an untrusted model

The runtime code contains more security hardening than many security products’ launch versions:

output must remain outside the repository and enclosing worktree;
Unix result directories must be private and owned by the current user;
symlink and path substitution are rejected;
plugin ZIPs are bounded to 4,096 entries, 128 MiB per entry, and 512 MiB expanded;
duplicate case-insensitive archive paths, symlinks, unsafe paths, and checksum mismatches are rejected;
Python runs with -I -B;
OpenAI and Codex API keys are removed from workbench subprocess environments;
target-path files are written read-only;
canonical JSON document sizes and schema complexity are bounded;
Git references and bulk-scan revisions are normalized rather than interpolated into a shell;
plugin and SDK versions are recorded in the scan recipe.

The source for most of this lives in runtime.ts, contract.ts, and targets.ts.

This code appropriately treats model-produced files and plugin payloads as untrusted inputs.

What happens when cyber safety refuses

Refusals are handled conservatively, although not yet as a first-class workflow state.

OpenAI’s Cyber Safety documentation says high-risk cyber requests may be routed to a less capable fallback model. Users doing legitimate work that is repeatedly caught by those controls are directed to Trusted Access for Cyber.

Inside Codex Security, I found no first-class refusal state. The event loop distinguishes completed turns, failed turns, interrupted streams, authentication problems, authorization problems, rate limits, and network failures. It does not classify “the model declined this exploit-validation task.”

If the top-level model returns a polite refusal without artifacts, the turn can technically complete, but collectResult() rejects it because the canonical files do not exist. If a worker refuses, the likely result is a missing ledger entry, deferred candidate, failed discovery manifest, or incomplete coverage. Bulk scans can retry generic failures, but there is no refusal-aware fallback ladder.

The safety property is useful:

A refusal should become a failed or incomplete scan, not “zero vulnerabilities found.”

That is a sensible safety baseline, but recovery is limited. A provider-neutral scanner could improve this by distinguishing policy rejection from ordinary failure, retrying once with bounded defensive framing, falling back from weaponized PoC generation to static source/control/sink proof, and recording the refused scope explicitly in coverage.

Can the same harness run Claude Opus?

Locally, probably—with an adapter.

Codex supports custom model providers. Codex Security’s --codex KEY=VALUE parser accepts dotted nested configuration, including model_provider and model_providers.<id>.*. A Responses-compatible gateway could route the Codex agent runtime to Claude while retaining Codex tools, plugins, worker orchestration, and scan artifacts:

npx codex-security scan /path/to/repo \
  --model '<exact-claude-model-id>' \
  --codex 'model_provider="claude_proxy"' \
  --codex 'model_providers.claude_proxy.name="Claude via proxy"' \
  --codex 'model_providers.claude_proxy.base_url="https://proxy.example/v1"' \
  --codex 'model_providers.claude_proxy.env_key="CLAUDE_PROXY_API_KEY"' \
  --codex 'model_providers.claude_proxy.wire_api="responses"'

Anthropic’s native Messages API is not enough; the gateway must implement the streaming and tool semantics expected by the bundled Codex runtime. Deep scans additionally depend on multi-agent events, cancellation, usage reporting, and long-running tool loops.

There are rough edges. Codex Security still performs its own OpenAI/Codex entitlement check before launching a scan, unknown model IDs lack built-in price data, --max-cost cannot work without a known price entry, and OpenAI’s xhigh reasoning setting has no automatic Anthropic meaning. Claude’s cyber-policy refusals would also be Anthropic’s problem, not something OpenAI Trusted Access fixes.

The prompts are portable. The complete runtime contract is not.

OpenAI’s performance claims need denominators

The launch post says that, over the preceding 30 days, Codex Security scanned more than 1.2 million commits and identified 792 critical plus 10,561 high-severity findings. That is 11,353 high-or-critical findings in total, or roughly 9.46 per thousand commits if one naively divides findings by commits.

That ratio is not a precision or recall metric. Findings can span commits, repositories differ radically, and the post does not publish the corpus, annotation protocol, duplicate handling, reviewer agreement, cost, runtime, or missed-vulnerability rate.

OpenAI also reports:

noise reduced by 84% on one repository over time;
over-reported severity reduced by more than 90%;
false-positive rates reduced by more than 50% across repositories.

All are internal beta-cohort claims. None is reproducible from this repository.

I also noticed several mismatched link targets in the launch post’s vulnerability appendix: the displayed CVE and linked CVE disagree for entries including the GnuTLS otherName double-free, LDAP injection, and disabled TLS verification examples. These appear to be editorial errors rather than evidence against the underlying findings, but they are worth correcting in a section intended to provide externally checkable examples.

The public repository contains extensive tests for orchestration and contracts. It does not contain a public benchmark showing comparative recall against expert review, SAST, fuzzing, or another security agent.

What it gets right

The strongest ideas are not glamorous:

Inventory is deterministic. The model does not get to decide which files existed.
Discovery and validation are separate. Suspicion is not proof.
Every candidate must close. Suppressed and deferred work remains visible.
Counterevidence is mandatory. The attack story must survive an attempt to falsify it.
Coverage is independent of findings. No findings plus partial coverage is not a clean scan.
Finalization is non-model-owned. The workbench, not the prose response, decides whether the transaction committed.
Deep mode reduces variance through independent search. Repetition is used to widen discovery, not inflate confidence.
Remediation is staged. Generate, apply, and verify can be separate operations.
History understands uncertainty. Missing findings are not automatically resolved.
The runtime distrusts its inputs. Paths, archives, schemas, environment variables, and artifacts are bounded and checked.

This is what a serious agent harness looks like: not just a capable prompt, but a collection of mechanisms designed around well-known model failure modes.

Where it remains weak

The central limitations are equally clear:

the semantic detector depends on hosted model behaviour and cannot be reproduced fully offline;
model and plugin updates make results probabilistic even when the Git revision is pinned;
“review every file” is stronger than the public evidence can support;
no public benchmark establishes false-negative rate or cost-effectiveness;
threat-model mistakes can systematically bias discovery and severity;
validation environments can diverge from production;
prompt injection remains an architectural risk whenever an agent reads hostile repository content while holding tools;
refusal handling is fail-closed but not refusal-aware;
deep scans are expensive and operationally heavy;
the public mirror cannot accept ordinary external contributions;
none of this replaces dependency scanning, secret scanning, deterministic SAST, sanitizer builds, fuzzing, DAST, or human review.

OpenAI has separately argued why Codex Security does not begin with a SAST report. The charitable reading is that contextual reasoning should evaluate whether defenses actually hold rather than count whether familiar controls appear. I agree with that. I do not agree with turning complementarity into substitution. Deterministic tools cheaply cover known classes; an expensive reasoning agent should spend its budget on the paths those tools cannot understand.

How I would evaluate it on a real codebase

A fair evaluation needs vulnerable and fixed revisions, not only a tour through current main and a judgment based on whatever the scanner happens to surface.

For a mature application, I would:

select historical security fixes with known vulnerable parent commits;
remove advisory names, issue text, and fix commits from the model’s context;
scan the exact vulnerable revisions;
require the finding to identify the same root control, not merely a nearby CWE;
scan the fixed revisions as negative controls;
repeat each scan to measure variance;
compare standard and deep modes using identical scope and model settings;
record configured and actually served model, reasoning effort, plugin version, workers, tokens, cost, and runtime;
run deterministic scanners and targeted fuzzers over the same revisions;
have security engineers classify false positives and material misses blind to tool identity.

The result should report precision, recall on the known corpus, variance, median cost, tail latency, and validation strength. “Found something interesting” is a demo. “Found 18 of 24 historical root causes at 0.8 false positives per revision for $X” is data.

The verdict

Codex Security is best understood as a contextual application-security reviewer and validation system built around a frontier coding model.

Its contribution is not merely that an LLM can notice an authorization bug. The release shows how to make an LLM account for every file, every candidate, every proof gap, every severity decision, and every incomplete phase in durable machine-checkable state.

The prompts are long because they address recurring model failure modes: premature stopping, representative-example bias, severity inflation, setup-error rationalization, duplicate collapse, missing counterevidence, and final answers that sound complete before the work is complete. The Python and TypeScript turn those instructions into enforceable workflow and artifact contracts.

OpenAI deserves credit for releasing this much of the system. It gives security teams and agent builders concrete architecture to inspect and adapt, rather than another product description with the consequential details hidden behind a service boundary. The remaining question is empirical rather than ideological: how much additional vulnerability coverage and triage value does the complete system deliver for its runtime and cost? The public release makes that question easier to investigate properly.