🎬 Curso evaluacion LLM con Promptfoo — episodio 1
Video Details
Channel: La Hora Maker
URL: https://www.youtube.com/watch?v=nGaHoH9HHu0
Relevance: ⭐⭐⭐⭐⭐
Summary
First episode of La Hora Maker’s Promptfoo course: installing Promptfoo, configuring providers (OpenAI, Anthropic, Ollama), defining test cases in YAML format, running evaluations with multiple assertions, and interpreting the HTML report. Uses a classification task as the example — directly analogous to PUMA’s triage classification.
PUMA Relevance
The Ollama provider configuration shown here is directly applicable to PUMA: configure Promptfoo to call Ollama’s Llama 3.2 8B with the four PUMA prompt strategies as separate providers, run 200 test cases (Jira SR stratified sample), and compare F1-macro across conditions. The YAML test case format allows defining PUMA’s ground truth labels for automatic evaluation.