Confidence Judge Node

The Confidence Judge Node is a fast, cheap pre-flight quality gate. It scores the candidate answer on a described rubric using the Jev decision model (TypeSafe System One) and emits one of three verdicts — pass, regenerate, or escalate — using the same edge vocabulary as the LLM Judge Node.

How it works

1. The node sends the user question plus candidate answer to Jev as a single Score question over 2-10 described levels.
2. Jev returns a probability-weighted score (0..n_levels-1) and a confidence value.
3. The score is normalized to 0..1 and compared against your thresholds:
   • >= pass_threshold → pass
   • <= regenerate_threshold → regenerate
   • anything between, or confidence below min_confidence, or the service being unavailable → escalate

Parameters

• rubric (string): The scoring question (e.g. "How well does this answer address the user's question, correctly and completely?").
• levels (array, 2-10, ordered low→high): Level descriptions. Describe situations ("Fully addresses the question, correct and complete"), not degrees ("good").
• pass_threshold (number 0-1, default 0.8): Normalized score at or above which the answer passes.
• regenerate_threshold (number 0-1, default 0.4): Normalized score at or below which the answer should be regenerated.
• min_confidence (number 0-1, default 0.35): Jev confidence below which the node escalates instead of deciding.
• evaluate_key (string, default "candidate_answer"): Which state field to judge (falls back to final_answer).

Output Handles

• pass — continue downstream.
• regenerate — loop back to the producer node to try again.
• route:escalate — wire to an LLM Judge Node for the full rubric audit.

State

Writes evaluation_verdict, evaluation_score (0-1), and evaluation_source — the same contract the LLM Judge Node uses, so the existing retry/pass edges work unchanged.

Use Cases

1. Two-tier judging: fast Jev gate in front, LLM Judge only on ambiguous answers — near-zero judge cost on easy turns.
2. Reflection loops: actor → judge → regenerate until pass.
3. Escalation pressure valve: route low-confidence cases to a human review card.

Tips

• Escalate is the safe failure mode: an unavailable decision service escalates rather than silently passing a bad answer.
• Keep each rubric to one dimension; split complex judgments into multiple Scorer Nodes and combine in a Condition Node.