Rewrite a paragraph to a 6th-grade reading level
Simplify a passage to a target U.S. grade-6 reading level while preserving all factual content.
Category: Language
Local with a check
Each model was asked to rewrite 8 original dense passages for a 6th-grade reader via Ollama (temperature 0, greedy); each rewrite was checked for Flesch-Kincaid grade <= 7, a minimum word count, and required fact tokens. Infra-error cases are excluded from pass rates.
Measured results
Each model actually ran; these are real pass counts against the verifier, best first.
| Model (Ollama tag) | Cases passed | Pass rate |
|---|---|---|
| devstral-small-2:latest best 24277f07f62d | 7/8 | |
| qwen3-coder:30b 06c1097efce0 | 6/8 | |
| gemma4:26b-mlx 21c59a2eae30 | 5/8 | |
| qwen3.5:27b 7653528ba5cb | 3/8 |
verifier Each rewrite is scored with the Flesch-Kincaid grade formula (heuristic vowel-group syllable counting) and must come in at grade 7 or below, keep at least a minimum word count, and still contain every required fact token (case-insensitive substring). It does not judge fluency or verify facts beyond those listed tokens.
Caveats
- Small-sample datapoint: 8 objective cases, a single greedy run per model. Best model 95% CI: 53%–98%.
- The verifier checks a readability formula, a length floor, and 4-5 fact tokens per case — a rewrite could pass while distorting or dropping facts outside the token list.
- Flesch-Kincaid syllable counting is heuristic, so a rewrite near the 7.0 line can be mis-scored by roughly a grade in either direction; the reference rewrites sit at grade 2-5 to stay clear of that noise.
- Text simplification is heavily represented in instruction-tuning data, so absolute pass rates flatter the models; relative rankings are the sturdier read.
Preserved raw run (every prompt and output): eval/results/rewrite-paragraph-to-grade-six-reading-level-r20260702-102224.json in the repo.
Built by Sam Carlton