Browse by role in the flow
Everywhere else, a model is filed by its input and output shape — text-to-text, speech-to-text. That tells a design team nothing. Here a model is filed by the job it does in the flow, because that is the question actually being asked: which stage does this work in, and what would grading it even measure?
Each role below names what a referee scores it against. That pairing is the point — a category with no defensible answer to “what would an eval for this measure?” is a folder, not a category. The agents inside each heading are starting points to fork and adapt: role archetypes that recur in real fleets, not a roster of things running here.
Multi-agent orchestrators
Agents whose job is other agents: they hold the plan, hand off between stages, and decide when a sub-agent is done.
What a referee scores: Whether the handoff between stages preserved the context the next stage needed, and whether the loop terminated for the right reason rather than on a step budget.
Portable-stimulus emulation orchestrator
Drives a multi-agent portable-stimulus flow across an emulation farm.
portable-stimulus-emulation-orchestrator
Drives: emulation farm · portable stimulus
RTL flow orchestrator
Gated stage handoffs across an RTL design flow, refusing to advance an unmet gate.
rtl-flow-orchestrator
Debug framework orchestrator
Raw data → machine-check decode → triage → root cause, as one supervised chain.
debug-framework-orchestrator
SDLC orchestrator
Requirements → design → code → review, run as a single autonomous loop.
sdlc-orchestrator
Cross-agent debug supervisor
Supervises several debug agents at once and arbitrates between findings that disagree.
cross-agent-debug-supervisor
Manufacturing data router
Routes data requests across many manufacturing pipelines and reconciles what comes back.
manufacturing-data-router
Issue plan tracker
Plans and tracks issues, driving repository automation loops to completion.
issue-plan-tracker
Pre-silicon verification
Agents that close verification before anything is fabricated — stimulus, coverage, testbench, emulation.
What a referee scores: Whether closure is real: a clean run proves nothing if the stimulus never exercised the behavior, so the referee scores coverage of the required bins, not the absence of errors.
Coverage closure agent
Plans against coverage gaps and drives them to closure.
coverage-closure-agent
Emulation run scheduler
Queues and brings up emulation-farm runs, rebalancing as jobs land.
emulation-run-scheduler-agent
Autonomous emulation validation
Runs emulation validation with a human in the loop for the calls that need one.
autonomous-emulation-validation-agent
Emulation efficiency agent
Compute scheduler plus job database, tuned as an autonomous loop.
emulation-efficiency-agent
SystemVerilog testbench agent
Develops and iterates SV testbenches against a moving design.
systemverilog-testbench-agent
Drives: SystemVerilog · UVM
RTL authoring agent
Authors and iterates synthesizable RTL.
rtl-authoring-agent
Drives: Verilog · SystemVerilog
Spec-quality policer
Polices the quality of input collateral before RTL work is allowed to start.
spec-quality-policer-agent
Simulation bring-up agent
Brings up a simulator target from scratch to a first passing run.
simulation-bringup-agent
Adversarial fix validation
Stress-tests a proposed RTL fix by trying to break it rather than confirm it.
adversarial-fix-validation-agent
Post-silicon & platform debug
Agents that work on real parts: silicon data, boot failures, live register state, platform bring-up.
What a referee scores: Whether the root cause named is the one the evidence supports — post-silicon debug is where a confident wrong answer is most expensive and hardest to detect.
Component root-cause agent
Works post-silicon data back to a root cause on a specific component.
component-root-cause-agent
Issue-to-PR agent
Takes an assigned issue and returns a reviewable pull request.
issue-to-pr-agent
Boot log analyzer
Diagnoses boot failures end to end from platform and firmware logs.
boot-log-analyzer-agent
Waveform + design + log agent
Debugs across waveforms, design source and logs together rather than one at a time.
waveform-design-log-agent
Drives: waveform database
Design-rule waiver agent
Design-rule violation → justified waiver → rerun, with the justification recorded.
design-rule-waiver-agent
Register bug discovery agent
Live hardware register debug loop against a running part.
register-bug-discovery-agent
Drives: JTAG
Security finding remediation
Carries a security finding through an end-to-end fix loop.
security-finding-remediation-agent
Live diagnostic loop agent
Collapses a diagnostic that spans several layers into one live loop.
live-diagnostic-loop-agent
EDA, APR & RTL flow
Agents that drive the implementation tools themselves — synthesis, place-and-route, timing, sign-off, tape-out.
What a referee scores: Whether the flow reached a signable state: constraints honored, sign-off checks actually run rather than skipped, and every waiver attributable to a person.
EDA session co-pilot
Rides an interactive implementation and timing session, proposing the next command.
eda-session-copilot-agent
Drives: place-and-route · static timing analysis
Compute-grid flow agent
Runs a job flow end to end across the compute grid, coordinating many tools.
compute-grid-flow-agent
APR flow assistant
Assists an automated place-and-route flow from netlist to routed database.
apr-flow-assistant-agent
Drives: place-and-route
Sign-off quality agent
Tracks sign-off quality uniformly across every block rather than block by block.
signoff-quality-agent
Tape-out turn-in agent
One hub for environment, build, simulation and tape-out turn-in.
tapeout-turnin-agent
Cross-stack refactor agent
Performs bulk refactors that span many files and layers of the stack in one reviewable change.
cross-stack-refactor-agent
Multi-device export agent
Exports collateral consistently across a whole device family.
multi-device-export-agent
Fuse release agent
Routes fuse pull, sideband and release steps.
fuse-release-agent
IP-block shipping agent
Ships an IP block: download → branch → upload, with provenance preserved.
ip-block-shipping-agent
Triage & reporting
Agents that absorb what arrives unbidden — tickets, sightings, duplicates — and turn it into routed work.
What a referee scores: Whether the routing was correct and the duplicate call was right: a triage agent that is confidently wrong quietly buries the one report that mattered.
Issue query triage agent
Answers standing questions against the issue tracker and triages what it finds.
issue-query-triage-agent
Customer sighting triage
Triages inbound customer sightings and routes them to an owner.
customer-sighting-triage-agent
Duplicate detection agent
Signature match plus a model check before anything is closed as a duplicate.
duplicate-detection-agent
System debug triage
Classifies no-boot, machine-check, bugcheck and thermal failures to the right owner.
system-debug-triage-agent
Tracker field triage
Classifies, links and auto-fills tracker fields on arrival.
tracker-field-triage-agent
Recurring sighting report
Runs the recurring sighting loop for a platform and reports what changed.
recurring-sighting-report-agent
Repo manager agent
CI/CD, pull-request review and milestone shepherding on one repository.
repo-manager-agent
Productivity & cross-cutting
Agents that serve every stage: knowledge retrieval, messaging bridges, deployment, platform performance.
What a referee scores: Whether the answer is grounded in the corpus it claims to have searched, and whether it says so when it is not.
Enterprise knowledge agent
Retrieval over a daily-synced internal corpus, with provenance on every answer.
enterprise-knowledge-rag-agent
Chat bridge agent
Brings an agent into direct conversation through a messaging-platform API.
chat-bridge-agent
Archive search agent
Hybrid API and client-side search across long-lived archives.
archive-search-agent
Trigger authoring agent
Authors and maintains the trigger services other automation hangs off.
trigger-authoring-agent
Platform performance agent
Tracks performance on current and next-generation platforms.
platform-performance-agent
Fleet ops co-pilot
Bare-metal OS deployment plus crash triage across a fleet.
fleet-ops-copilot-agent
Security audit
Agents that audit designs and codebases for security properties, at review time and at merge time.
What a referee scores: Whether a finding survives an adversarial second look — the failure mode is a plausible finding nobody can reproduce, and a rate of those is worse than silence.
Security audit framework
Autonomous audit over a large codebase.
security-audit-framework-agent
Deep security audit agent
Single invocation across a broad pre-analysis toolset.
deep-security-audit-agent
Architecture security review
Cross-walks a threat model against the specification and reports where they diverge.
architecture-security-review-agent
PR security gate
Per-pull-request cross-function chain check before merge.
pr-security-gate-agent
The referee for these roles
The rubrics and evals that grade work in these roles ship as a versioned pack, alongside the datasets the work produces and the corrections experts made to it.