Regression Metrics
Regression Metrics use an LLM-as-Judge to compare two responses directionally —
a new response containing your latest changes (the candidate) against a known-good
past response (the baseline), resolved from a tagged run.
Instead of an absolute score, they look for Shifts in Behavior, outputting a
-1, 0, or 1:
| Output Score | Meaning |
|---|---|
-1 | The Baseline (old version) is better or exhibits more of the trait. |
0 | Tie. The old and new versions are effectively the same regarding this trait. |
1 | The Candidate (new version) is better or exhibits more of the trait. |
Aggregated across the dataset, these become a win / loss / tie rate that you gate
with max_loss_rate or min_win_rate in a target's regression block.
Regression is directional — it asks "did we get better or worse than the baseline?".
Its sibling, Pairwise (Fairness) Metrics, is symmetric — it compares
two peer responses for divergence with no notion of a baseline. They are separate
axes with separate metric libraries; a regression metric cannot be used in a
pairwise block, and vice versa.
Available Regression Metrics
All regression metrics share the same variables: {{input}}, {{output_baseline}},
and {{output_candidate}}.
general-quality
A holistic assessment. Which response is better overall based on accuracy, helpfulness, and clarity?
decisiveness
Which response is more decisive and less cautious (less hedging, fewer "maybes")?
refusal
Identifies if one model refused the user's request (e.g., "I cannot answer that") while the other answered it.
verbosity
Which response is longer or more verbose?
instruction-following
Which response followed the system instructions and constraints more rigorously?
formality
Which response is more formal, academic, or professional in tone?
ambiguity-handling
When faced with a vague input, which response correctly asked for clarification instead of wrongly guessing the intent?
apologetic
Identifies which response is more apologetic or subservient.
brand-voice
Which response sounds more aligned with a professional, innovative, and customer-centric brand voice?
empathy
Which response demonstrates superior emotional intelligence and empathy towards the user's situation?
simplicity
Which response explains complex concepts more simply, avoiding unnecessary technical jargon?
actionability
Which response provides clearer next steps or calls to action to the user?