Create Evaluator

Build custom evaluators with @arizeai/phoenix-evals

Use createEvaluator when your evaluation logic is plain TypeScript and you want a reusable evaluator object with consistent metadata and telemetry.

Relevant Source Files

• src/helpers/createEvaluator.ts

Example

import { createEvaluator } from "@arizeai/phoenix-evals";

const exactMatch = createEvaluator(
  ({ output, expected }) => ({
    score: output === expected ? 1 : 0,
    label: output === expected ? "match" : "mismatch",
  }),
  {
    name: "exact-match",
    kind: "CODE",
  }
);

const result = await exactMatch.evaluate({
  output: "Paris",
  expected: "Paris",
});

What You Get

• an evaluator name
• evaluator kind such as CODE or LLM
• optimization direction metadata
• optional OpenTelemetry spans around execution

When To Use Code Evaluators

Code evaluators are the right fit when the scoring logic should stay deterministic, cheap, and fully under your control:

• regex or exact-match checks
• JSON structure validation
• latency and cost thresholds
• post-processing checks on existing model output

import { createEvaluator } from "@arizeai/phoenix-evals";

const lengthCheck = createEvaluator(
  ({ output }) => {
    const score = typeof output === "string" && output.length < 280 ? 1 : 0;
    return {
      score,
      label: score ? "fits-limit" : "too-long",
    };
  },
  {
    name: "response-length",
    kind: "CODE",
  }
);

Related Helpers

• asEvaluatorFn
• toEvaluationResult

Source Map

• src/helpers/createEvaluator.ts
• src/helpers/asEvaluatorFn.ts
• src/helpers/toEvaluationResult.ts
• src/core/FunctionEvaluator.ts
• src/core/EvaluatorBase.ts
• src/types/evals.ts