M2M100 418M failed the quality gate on real articles. Measured on a 420-word
post: "frijoles" came back as "Beeren" (berries), "rompe la idea" as "breitet
die Idee" (spreads it — the opposite), "así sabe mi barrio" as "so weiß mein
Viertel" (knows, not tastes), and the opening sentence was not grammatical
German. Invented non-words throughout ("verforscht", "Treffenraum").
The pipeline itself was never at fault — masking held, structure survived,
zero placeholder failures across 14 files. The model was.
OPUS-MT is stronger on these specific pairs, and fits the existing CX22 with
no server upgrade, which was the constraint. Licence moves from MIT to
CC-BY-4.0, so attribution now ships with the platform.
Routes are verified against Hugging Face rather than assumed — the earlier
research could not reach HF, and half the names it guessed do not exist:
es -> de opus-mt-es-de (small, ~74M)
de -> es opus-mt-tc-big-de-es (tc-big, ~237M)
es <-> pt-br opus-mt-tc-big-itc-itc (>>pob<< / >>spa<<)
de <-> pt-br no model in either direction — pivots through Spanish
Models now live at /srv/mt-models and are mounted read-only rather than baked
into the image. Model choice has needed iteration, and a directory swap beats
a 1GB image rebuild each time. It also keeps model conversion out of the image
build, which matters on a box that OOM-killed the last conversion.
MT_PROVIDER=m2m100 still selects the old engine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
The DeepL free tier is metered (it failed in production with HTTP 456
Quota exceeded) and would require every deployment of this platform to
carry its own API account. Translation now runs on M2M100 418M (MIT) via
CTranslate2, shipped inside the pipeline image: no key, no quota, and no
content or visitor data leaving the server.
Also restores the multi-source behaviour of the original WordPress plugin,
which the Python port had narrowed to Spanish-only. Any of the three site
languages can now be the authored original.
Because any language can be a source, loop prevention is no longer
structural and is now explicit: generated siblings carry `translated_from`
and are never treated as sources, and the bot's own [skip-translate]
commits are skipped outright (that marker was already being written but
never read).
Markup protection moves in-process now that DeepL's tag_handling=html is
gone. Code blocks and raw HTML pass through untouched; link targets,
inline code and protected community terms are masked with placeholders
that are verified to survive the round trip, failing the pipeline rather
than shipping corrupted text.
Two fixes along the way:
- Generated siblings no longer inherit the source's `slug`. They did,
which meant the first retranslation of a WordPress-migrated post moved
/de/<german-slug>/ onto /de/<spanish-slug>/ and destroyed the inbound
link preservation wp-to-hugo.py exists for.
- `manual_translation` now works from the CMS. Decap only ever exposed it
on the source while the script read it on the target, so the toggle did
nothing. It now means "hands off" on both sides.
wp-to-hugo.py marks migrated Polylang siblings frozen, since those are
human translations and regenerating them would replace them with weaker
machine output.
Adds --backfill for sources missing siblings, which also fixes the
existing 404s on /de/page/acerca/ and /pt-br/page/contacto/.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
