LLM-as-a-Judge

What Is LLM-as-a-Judge?

LLM-as-a-judge is an evaluation technique where one language model grades another model's output against a written rubric, scoring qualities like correctness, helpfulness, or faithfulness to a source. It makes qualitative evaluation cheap and fast enough to run on every change, at the cost of inheriting the judge model's own biases and blind spots. The technique earned its credibility in 2023, when Zheng et al. showed GPT-4 as a judge matched human preferences with over 80 percent agreement on MT-Bench and Chatbot Arena, the same level of agreement humans reach with each other [1].

Key Takeaways

• The judge replaces human review for the middle tier of evaluation: qualities too fuzzy for string matching but too high-volume for people to grade by hand.
• The rubric is the product. A judge with a vague prompt ("rate quality 1-10") produces noise; a judge with explicit criteria, scale anchors, and examples produces usable signal.
• Judges have known biases: they favor longer answers, prefer outputs matching their own style, drift by position when comparing pairs, and grade their own model family generously.
• Calibrate against humans before trusting the numbers. A judge is only valid to the degree it agrees with expert human labels on a sample.
• Use it inside LLM evals pipelines and production monitoring, not as a substitute for deterministic checks that code can do for free.

How It Works

A judge setup has three parts: the output to grade, a rubric, and a grading prompt that binds them. The rubric defines each criterion concretely, what a 5 looks like versus a 3, which failures are automatic zeros, and includes worked examples of scored outputs. The judge model receives the original input, the output under test, any reference material (a gold answer, a source document), and returns a structured verdict, usually a score plus a written justification. Asking for the reasoning before the score improves reliability and gives humans something to audit when a grade looks off.

Two grading modes dominate. Pointwise grading scores a single output against the rubric, which suits regression testing: run the suite, track the mean score per commit. Pairwise grading shows the judge two outputs and asks which is better, which suits comparisons like prompt A versus prompt B or model shootouts, and is generally more reliable because relative judgment is easier than absolute scoring. Pairwise setups must swap the order of candidates across runs, because judges measurably favor whichever answer appears first. The effect is not subtle: in a 2023 study, researchers exploited position bias by simply reordering candidate answers and made Vicuna-13B beat ChatGPT on 66 of 80 test queries with ChatGPT as the evaluator [2].

The bias problem is managed, not solved. Standard mitigations: use a judge from a different model family than the system under test, cap the influence of answer length in the rubric, run each grade multiple times and take the majority when stakes are high, and periodically re-check judge-human agreement. Teams typically re-calibrate whenever they change the judge model or the rubric, holding out a set of human-labeled examples as the reference.

Example

A team ships a RAG assistant that answers questions from their product documentation. Their eval suite has 200 real user questions, and the quality that matters most is faithfulness: does the answer stick to what the docs say, or does it improvise. String matching cannot measure that, and human review of 200 answers per prompt change is a nonstarter. They write a judge prompt that hands a judge model the question, the retrieved doc passages, and the answer, with a three-point rubric: fully supported, partially supported, contains unsupported claims. Before relying on it, they have two engineers hand-label 100 answers and find the judge agrees with humans 91 percent of the time, disagreements clustering on partially-supported cases, which they tighten in the rubric. The judge now runs in CI on every retrieval or prompt change and nightly on a sample of production traffic, and a faithfulness drop of more than three points pages the team.

What People Get Wrong

The core misconception is treating judge scores as ground truth. A judge is a measurement instrument with error bars, and an uncalibrated one can be confidently, systematically wrong, happily awarding high marks to fluent answers that contain an AI hallucination the rubric never taught it to catch. Teams see a dashboard trending at 4.6 out of 5 and assume quality is high, when the honest claim is only that the judge likes the outputs. Until you have measured agreement between the judge and expert humans on your own data, the scores are decoration.

FAQ

Which model should be the judge? A model at least as capable as the one being evaluated, and ideally from a different provider or family to dodge self-preference bias. Frontier models are the default choice because grading is lower-volume than generation, so the cost premium is tolerable. Some teams fine-tune a smaller dedicated judge once they have enough human-labeled data, trading generality for consistency and price.

Can the judge grade agents, not just responses? Yes, and it is standard practice in agent evaluation: the judge reads a full trajectory, the plan, tool calls, and results, and scores process qualities like efficiency and safety that outcome checks miss. Traces are long, so judges benefit from a summarized or structured view rather than raw logs.

Is LLM-as-a-judge reliable enough to replace human review? For ranking options and catching regressions, usually yes once calibrated. For final sign-off on high-stakes output, no. The stable pattern keeps a human in the loop at the top of the pyramid: deterministic checks run everywhere, judges handle scale, and humans audit samples and adjudicate the cases judges flag as uncertain.

Sources

1. Zheng et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena: GPT-4 judge reached over 80% agreement with human preferences." https://arxiv.org/abs/2306.05685. Accessed August 2026.
2. Wang et al. "Large Language Models are not Fair Evaluators: reordering answers made Vicuna-13B beat ChatGPT on 66 of 80 queries." https://arxiv.org/abs/2305.17926. Accessed August 2026.