Back to lattice
Stable node URL

Evals Engineer

The discipline role that keeps agent quality measurable and honest.

hybrid
pilot
hybrid
Senior
Aliases
4
Sources
1
Neighbors
3

What this page is for

A stable inspection page for one graph node.

• The path identifies this node directly.

• Compare state stays in the query string.

• Add interest saves this node on your profile for a later aligned path.

Compare with a neighbor
Advanced

Personality-aligned path

Use saved Role Lattice interests plus your Workplace Archetype Quiz result.

Node profile

Humans set thresholds. Agents draft and classify tests, but humans own the final quality line.

Role frame
Design reliable evaluation systems that reveal whether agentic behavior is good enough.
Obligations
Measure quality
Detect regressions
Make failure visible
Tasks
Build eval harnesses
Label edge cases
Track regressions
Compare model outputs
Skills
Testing
Experiment design
Metric definition
Data analysis
Tools
Eval dashboards
Test runners
Prompt logs
Annotation tools
Prerequisites
Engineering discipline
Product judgment
Agent eligible
Draft test cases
Cluster failure patterns
Summarize eval runs
Human gated
Pass/fail criteria
Quality threshold changes
Release sign-off
Comparison

Evals Engineer vs Invoice Reconciliation Agent

A first-class non-human node that keeps repetitive reconciliation work structured and auditable.

Kind
hybrid vs agent
Seniority
Senior vs Operational
Demand
256 vs 666
Space
pilot-space:agentops → pilot-space:agentops
Neighbors in common
1
Shared tools
0
Compare is meant to answer one question: what changes when I move from one node to the other?
Evals Engineer only skills
Testing
Experiment design
Metric definition
Data analysis
Invoice Reconciliation Agent only skills
Pattern matching
Rule execution
Exception detection
Role Lattice — Career Node | Archetype AI