Skip to main content
Promptbeat is designed to make real agent applications visible as test targets. This page lists the agent app types, their current readiness level, and how to connect each one. A target is useful when it has a concrete invocation contract, a workspace or service environment, a trace output, and assertions that can turn agent behavior into a pass/fail result. Validation evidence is tracked separately from documented support.

​
Readiness levels

​
Coding agents

These are the primary targets for Promptbeat’s coding-agent scenario suite, which covers secret handling, sandbox boundary enforcement, terminal injection, repository injection, and egress control.

​
Application agents

Beyond coding agents, the same target abstraction covers any agent that exposes a REST endpoint or can be wrapped in a provider adapter.

​
Target-ready checklist

Before you run promptbeat eval against a new target, confirm all of the following:
  • Expose a final answer — the target’s response must be a string that the judge can score.
  • Capture trace evidence when possible — return tool calls, commands, file reads/writes, network events, and policy decisions alongside the answer.
  • Expose a reset or setup mechanism for stateful agents — multi-turn or workspace-based agents need a clean starting state for each probe; document the reset path in your provider config.
  • Keep credentials out of provider YAML — use {{env.VAR_NAME}} references; never commit keys to the file.
  • Point working_dir at a safe fixture workspace — not your production repository.
  • Document which scenario risk types the target supports — so you can filter scenarios to the capabilities the agent actually has.
Start with the HTTP adapter pattern for any agent that already has a REST API — it works out of the box with providers.http.yaml, and you can add trace metadata to the JSON response incrementally. See Agent Configuration Examples for a complete HTTP walkthrough.