Each stage of the Promptbeat pipeline produces artifact files. Knowing where artifacts live and what they contain helps you debug failed runs, share results with your team, and track progress over time. Promptbeat’s CLI stages expect artifacts at these exact paths by default — changing the directory structure requires updating your configuration explicitly.
Generate artifacts
The generate stage produces the attack pool before any evaluation runs. These files record what was generated so you can inspect the probe set, re-sample it, or reproduce the generation later.
Eval artifacts
The eval stage runs the generated probes against your target and produces the normalized result plus raw backend output and logs.
Report artifacts
The report stage reads evaluation_result.json and produces human-readable output. Both HTML and Markdown formats are generated from the same normalized result.
Default artifact directory structure
Promptbeat writes artifacts into a project-scoped directory. The structure below shows the layout for a named project run:
For the broad Codex reference run, the project directory is broad-20260530-163553 under examples/codex_agent/artifacts/.
Version control guidance
Not all artifact files belong in version control. Follow this guidance to keep your repository clean and reproducible without committing large or sensitive files.
Commit these files:
- Configuration files:
promptbeat.yaml, target.yaml, scenarios.yaml, seeds.yaml
- Provider snapshots:
providers.<adapter>.yaml
- Config precheck results:
config-precheck.json
- Generated attack pool:
generated_redteam.yaml and sampled variants (these document what was tested)
- Reports:
report.html and comprehensive-*.md (these are your evidence artifacts)
- Normalized result:
evaluation_result.json (structured, stable schema)
Gitignore these files:
- Raw backend output:
promptfoo_eval_result.json (unstable schema, large, backend-specific)
- Large log files:
promptfoo.eval.stdout.log, promptfoo.eval.stderr.log
- Any file containing credentials, tokens, or environment variable snapshots
- Intermediate workspace files created during agent evaluation
A minimal .gitignore entry for Promptbeat artifact directories:
Never commit files that contain environment variable dumps, secret values, or process credential captures — even if the agent produced them as a demonstration of unsafe behavior. Replace sensitive values with redacted placeholders before committing any artifact that touches secret material.
Use artifact paths exactly as shown above — Promptbeat’s CLI stages expect these locations by default. If you change the output_dir in your configuration, update all downstream stage references consistently. The eval_id field in evaluation_result.json ties together all artifacts from a single run, so include it in issue titles and PR descriptions when discussing specific results.