Translate UI strings to Spanish keeping placeholders intact

← All tasks

Translate a JSON/PO file of English UI strings to Spanish without altering interpolation placeholders or markup.

Category: Language

🟡 Local with a check Evaluated July 2, 2026 · 12 cases · Ollama (local, Apple Silicon)

Each model was asked to translate 12 English UI strings to Spanish via Ollama (temperature 0, greedy); each output was checked for exact, count-matching preservation of every interpolation placeholder and for not simply echoing the source. Infra-error cases are excluded from pass rates.

Measured results

Each model actually ran; these are real pass counts against the verifier, best first.

Model (Ollama tag)	Cases passed	Pass rate
gemma4:26b-mlx best 21c59a2eae30	12/12	100%
qwen3.5:27b 7653528ba5cb	12/12	100%
qwen3-coder:30b 06c1097efce0	12/12	100%
devstral-small-2:latest 24277f07f62d	11/12	92%

verifier For each listed placeholder token, the verifier counts its exact substring occurrences in the model output and requires the count to equal that token's count in the English source; it also rejects output that is identical to the source. It does NOT judge translation quality, grammar, or even that the output is actually Spanish.

Caveats

• Small-sample datapoint: 12 objective cases, a single greedy run per model. Best model 95% CI: 76%–100%.
• Placeholder preservation and non-echo are verified; translation QUALITY is not judged by this eval — a clumsy but placeholder-faithful translation still passes.
• The verifier does not confirm the output is Spanish, so an English paraphrase that keeps every token would pass; treat pass rates as a floor on formatting discipline, not translation skill.
• English→Spanish UI strings are heavily represented in training data, so absolute pass rates flatter the models; relative rankings are the sturdier read.

Preserved raw run (every prompt and output): eval/results/translate-ui-strings-spanish-keep-placeholders-r20260702-101257.json in the repo.