← All topics
Quiz: Evaluating LLM Apps: questions with answers
10 questions, each with the answer and a short explanation.
Take the timed quiz + get a certificate →- Q1. Why isn't trying a few questions by hand enough?
- Q2. Which cases belong in a good test set?
- Q3. What should you do before trusting an LLM-as-judge?
- Q4. You change the prompt and the model at the same time and the score drops. What's the problem?
- Q5. In the harness above, what was the most useful output?
- Q6. What does this print? (code)
- Q7. Which scoring method fits "the answer must be valid JSON and under 80 words"?
- Q8. Why does the lesson prefer yes/no rubric checks over 1–10 scores for an LLM judge?
- Q9. A bug reaches production. What should happen to the test set?
- Q10. The provider releases a new model version. What should you do before switching?