PUMA Vault
Search
Buscar
Modo oscuro
Modo claro
Explorador
Etiqueta: engine
83 artículos con esta etiqueta.
15 jun 2026
LN-Tools — The puma CLI (Typer-based) exposes ~15 commands and 5 groups including…
literature-tools
puma
engine
wilcoxon
dataset
baseline
bias
15 jun 2026
LN-Tools — The Streamlit dashboard offers read-only views (Overview, Model…
literature-tools
puma
engine
sustainability
fairness
community
15 jun 2026
LN-Tools — Reproducibility is enforced by puma validate-baseline, which re-runs a…
literature-tools
puma
engine
mae
reproducibility
baseline
release
15 jun 2026
LN-Tools — puma-cp v4.0.0 is published to PyPI (released 2026-05-31), a Python >=3.11…
literature-tools
puma
engine
docker
release
15 jun 2026
LN-Tools — PUMA ships 15 hardware profiles (CPU-only, GPU, Apple Silicon M3/M4/M5)…
literature-tools
puma
engine
15 jun 2026
PN — PUMA is a local-first benchmarking platform that evaluates open-weight…
permanent-note
puma
engine
ollama
benchmark
15 jun 2026
PN — H1 (triage): qwen2.5:3b + contextual-anchoring (F1 0.5894) statistically…
permanent-note
puma
engine
triage
wilcoxon
contextual-anchoring
baseline
15 jun 2026
PN — H2 (estimation): best config qwen2.5:7b cuts MAE only 0.17 SP vs a…
permanent-note
puma
engine
estimation
mae
baseline
15 jun 2026
PN — Model scaling is non-monotonic on these PMO tasks: in 4 of 6 (family x…
permanent-note
puma
engine
triage
local-llm
15 jun 2026
PN — PUMA's sustainability measurement is explicitly bounded: it covers the…
permanent-note
puma
engine
sustainability
15 jun 2026
PN — The +/-0.01 baseline tolerance does not absorb seed variance (zero at…
permanent-note
puma
engine
ollama
baseline
15 jun 2026
PN — All model execution is delegated to a local Ollama daemon over HTTP at the…
permanent-note
puma
engine
ollama
15 jun 2026
PN — A benchmark run is fully spec-driven: a RunSpec YAML validated with…
permanent-note
puma
engine
benchmark
15 jun 2026
PN — Failed response parses are recorded as None, excluded from metric…
permanent-note
puma
engine
15 jun 2026
PN — A full Phase B sweep ran 9 models x 3 scenarios = 27 runs (2,700…
permanent-note
puma
engine
sweep
15 jun 2026
PN — PUMA reports a multi-dimensional metric suite
permanent-note
puma
engine
f1-macro
mae
calibration
latency
15 jun 2026
PN — Calibration metrics (ECE over 10 bins, MCE, Brier) are derived from…
permanent-note
puma
engine
calibration
ollama
metrics
15 jun 2026
PN — Sustainability is reported per run via CodeCarbon (CO2 kg, energy kWh,…
permanent-note
puma
engine
mae
sustainability
carbon
codecarbon
15 jun 2026
PN — PUMA treats inference compute cost as a first-class reported metric…
permanent-note
puma
engine
mae
carbon
15 jun 2026
PN — CodeCarbon runs in tracking_mode='machine' so whole-machine accounting…
permanent-note
puma
engine
carbon
codecarbon
architecture
15 jun 2026
PN — Submission integrity uses a SHA-256 predictions_summary_hash over a…
permanent-note
puma
engine
json-schema
integrity
15 jun 2026
PN — PUMA's security posture rests on local inference (no SaaS path/token to…
permanent-note
puma
engine
benchmark
integrity
15 jun 2026
PN — CI security tooling wires pip-audit (CVE scanning), bandit (SAST),…
permanent-note
puma
engine
15 jun 2026
PN — An optional inference cache stores (model, prompt_hash, temperature, seed)…
permanent-note
puma
engine
ollama
sqlite
15 jun 2026
PN — Prompt templates use Jinja2, one .jinja file per (scenario x strategy) pair
permanent-note
puma
engine
15 jun 2026
PN — PUMA supports multi-strategy prompting
permanent-note
puma
engine
prompting
contextual-anchoring
few-shot
zero-shot
15 jun 2026
RES — Reproducibility is achieved by design with seed=42 and temperature=0.0…
result-note
puma
engine
reproducibility
metrics
15 jun 2026
RES — The canonical triage baseline is F1-macro = 0.5867 (qwen2.5:3b,…
result-note
puma
engine
triage
f1-macro
contextual-anchoring
zero-shot
15 jun 2026
RES — The canonical estimation baseline is MAE = 5.7150 SP (qwen2.5:3b,…
result-note
puma
engine
estimation
mae
zero-shot
baseline
15 jun 2026
RES — The best reproducible estimation configuration observed is qwen2.5:7b at…
result-note
puma
engine
estimation
story-points
mae
sweep
15 jun 2026
RES — A three-seed validation {42,123,456} of the triage baseline produced…
result-note
puma
engine
triage
f1-macro
baseline
metrics
15 jun 2026
RES — No single model wins across all PMO tasks
result-note
puma
engine
triage
estimation
mae
15 jun 2026
RES — A parser x hardware incompatibility can waste more compute than the useful…
result-note
puma
engine
sweep
15 jun 2026
RES — On the reference gpu-entry host a 200-instance canonical run costs a…
result-note
puma
engine
triage
estimation
baseline
15 jun 2026
RES — Cross-architecture (x86_64 vs arm64 Apple Silicon) bit-exactness of F1/MAE…
result-note
puma
engine
mae
architecture
15 jun 2026
RES — Gender-prefix robustness testing on triage found qwen2.5:3b shifts…
result-note
puma
engine
triage
15 jun 2026
RES — The most consequential cost-effectiveness finding: qwen2.5:1.5b
result-note
puma
engine
triage
estimation
carbon
15 jun 2026
DEC — Metrics are stored in a single flat (run_id, metric_name, value) table…
decision-note
puma
engine
metrics
15 jun 2026
DEC — SQLite was chosen over Postgres for zero-infrastructure single-file storage
decision-note
puma
engine
sqlite
integrity
15 jun 2026
DEC — Carbon conversion defaults to country_iso=ESP (Spain) for reproducibility…
decision-note
puma
engine
reproducibility
carbon
15 jun 2026
DEC — A triage run between an Ollama restart and an estimation validation…
decision-note
puma
engine
triage
estimation
mae
ollama
15 jun 2026
DEC — PUMA Community is a separate GitHub repo (single source of truth) with…
decision-note
puma
engine
dataset
baseline
community
15 jun 2026
DEC — puma share-results is dual-mode: --dry-run validates locally, runs a PII…
decision-note
puma
engine
sweep
integrity
15 jun 2026
REL — PUMA follows semantic versioning
release-note
puma
engine
baseline
community
release
15 jun 2026
DEBT — The published H1 triage test deviates from pre-registration: the intended…
decision-note
puma
engine
triage
baseline
15 jun 2026
DEBT — The published H2 estimation test is data-limited (debt D44)
decision-note
puma
engine
estimation
baseline
15 jun 2026
DEBT — PUMA maintains a transparent known-debt register
decision-note
puma
engine
estimation
verifier
determinism
15 jun 2026
DEBT — Reasoning-style output is handled by a strip_reasoning helper that removes…
decision-note
puma
engine
triage
ollama
15 jun 2026
ARCH — PUMA exercises two production scenarios end to end
artifact-note
puma
engine
triage
estimation
15 jun 2026
ARCH — The triage_jira scenario is a 4-class severity classification…
artifact-note
puma
engine
triage
f1-macro
jira
15 jun 2026
ARCH — The estimation_tawos scenario predicts story points as a Fibonacci value…
artifact-note
puma
engine
estimation
story-points
mae
tawos
15 jun 2026
ARCH — The prioritization_jira scenario is a binary pairwise-ranking task (which…
artifact-note
puma
engine
jira
15 jun 2026
ARCH — Results persist to a local SQLite database with a bi-temporal schema…
artifact-note
puma
engine
sqlite
metrics
15 jun 2026
ARCH — The triage scenario uses a balanced 200-row dataset (50 per class) derived…
artifact-note
puma
engine
triage
dataset
jira
15 jun 2026
ARCH — The estimation scenario uses a cleaned 9,020-row TAWOS dataset (SOLAR…
artifact-note
puma
engine
estimation
story-points
mae
dataset
15 jun 2026
ARCH — The first official PUMA Community submission (2026-05-31) recorded…
artifact-note
puma
engine
triage
f1-macro
community
15 jun 2026
RES — gemma3:12b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — gemma3:12b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — gemma3:12b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — gemma3:1b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — gemma3:1b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — gemma3:1b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — gemma3:4b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — gemma3:4b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — gemma3:4b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — gemma4:e2b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — gemma4:e2b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — gemma4:e2b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — llama3.1:8b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — llama3.1:8b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — llama3.1:8b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — mistral:7b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — mistral:7b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — mistral:7b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — qwen2.5:1.5b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — qwen2.5:1.5b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — qwen2.5:1.5b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — qwen2.5:3b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — qwen2.5:3b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — qwen2.5:3b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine
15 jun 2026
RES — qwen2.5:7b · estimation (Phase-B cell)
result-note
puma
benchmark
estimation
engine
15 jun 2026
RES — qwen2.5:7b · prioritization (Phase-B cell)
result-note
puma
benchmark
prioritization
engine
15 jun 2026
RES — qwen2.5:7b · triage (Phase-B cell)
result-note
puma
benchmark
triage
engine