Kousigan A

Tag

3 posts

LLM Evaluation

Posts about LLM Evaluation by Kousigan A, filed under AI Coding. Each one links the official docs it relies on.

  1. Deep dive13 minAI Coding

    LLM feedback loop: turn production failures into eval cases

    Run an LLM feedback loop: collect user feedback, read failing traces, name failure types, fix one layer, and keep each fixed failure as an eval case.

  2. Deep dive12 minAI Coding

    LLM model routing: test on evals, pin it, fall back safely

    Choose an LLM per task with your own eval set, route by task type, classifier or failed check, keep fallbacks on the same contract, and pin versions.

  3. Deep dive13 minAI Coding

    Unit tests miss LLM output that varies: build an eval suite

    Build an eval suite for an LLM feature: real and edge-case inputs, a grader per property, several runs per case, and a rerun on every prompt or model change.

All topics and tags