Skills101
← All practice quizzes

Quiz: Evaluating LLM Apps

5 questionsNo time limitPass: 60%+10 XP on first pass

Stop judging AI features by "looks good": build a test set from real questions, score with rules and reference answers, use an LLM as a judge carefully, compare versions before shipping, and keep evaluating in production.

Log in to take the quiz

Free account · see your score, answers and explanations, earn XP

  1. Question 1

    Why isn't trying a few questions by hand enough?

  2. Question 2

    Which cases belong in a good test set?

  3. Question 3

    What should you do before trusting an LLM-as-judge?

  4. Question 4

    You change the prompt and the model at the same time and the score drops. What's the problem?

  5. Question 5

    In the harness above, what was the most useful output?