Go to file
Claude 986c066ae3
Replace DeepL with a self-hosted translation engine
The DeepL free tier is metered (it failed in production with HTTP 456
Quota exceeded) and would require every deployment of this platform to
carry its own API account. Translation now runs on M2M100 418M (MIT) via
CTranslate2, shipped inside the pipeline image: no key, no quota, and no
content or visitor data leaving the server.

Also restores the multi-source behaviour of the original WordPress plugin,
which the Python port had narrowed to Spanish-only. Any of the three site
languages can now be the authored original.

Because any language can be a source, loop prevention is no longer
structural and is now explicit: generated siblings carry `translated_from`
and are never treated as sources, and the bot's own [skip-translate]
commits are skipped outright (that marker was already being written but
never read).

Markup protection moves in-process now that DeepL's tag_handling=html is
gone. Code blocks and raw HTML pass through untouched; link targets,
inline code and protected community terms are masked with placeholders
that are verified to survive the round trip, failing the pipeline rather
than shipping corrupted text.

Two fixes along the way:

- Generated siblings no longer inherit the source's `slug`. They did,
  which meant the first retranslation of a WordPress-migrated post moved
  /de/<german-slug>/ onto /de/<spanish-slug>/ and destroyed the inbound
  link preservation wp-to-hugo.py exists for.
- `manual_translation` now works from the CMS. Decap only ever exposed it
  on the source while the script read it on the target, so the toggle did
  nothing. It now means "hands off" on both sides.

wp-to-hugo.py marks migrated Polylang siblings frozen, since those are
human translations and regenerating them would replace them with weaker
machine output.

Adds --backfill for sources missing siblings, which also fixes the
existing 404s on /de/page/acerca/ and /pt-br/page/contacto/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
2026-09-14 16:34:25 +00:00
content Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
docker/translate Replace DeepL with a self-hosted translation engine 2026-09-14 16:34:25 +00:00
docs Replace DeepL with a self-hosted translation engine 2026-09-14 16:34:25 +00:00
infra Document uid-1000 ownership for Woodpecker v3 data volume 2026-07-31 17:55:53 +00:00
scripts Replace DeepL with a self-hosted translation engine 2026-09-14 16:34:25 +00:00
static Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
themes/vienalatina Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
.gitignore Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
.woodpecker.yml Replace DeepL with a self-hosted translation engine 2026-09-14 16:34:25 +00:00
config.yaml Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
README.md Replace DeepL with a self-hosted translation engine 2026-09-14 16:34:25 +00:00

Viena Latina

Static, self-hosted, trilingual blog for the Latin American community in Vienna — Hugo + Decap CMS + Gitea + Woodpecker CI on a single Hetzner CX22 (Nuremberg, DE). Replaces the previous WordPress + Polylang + synchronous DeepL stack.

Translation is self-hosted too: M2M100 418M (MIT) runs on CPU via CTranslate2 inside the pipeline image. No API key, no quota, and no visitor or content data leaving the server.

Pablo ──► Decap CMS (/admin) ──commit──► Gitea ──webhook──► Woodpecker CI
                                                              │
                                  translate (M2M100, async) ──┤
                                  build (hugo)                ─┤
                                  deploy (rsync)              ─┘
                                                              ▼
                                            Caddy 2 serves /var/www/vienalatina.com

Languages

Spanish, German (/de/) and Brazilian Portuguese (/pt-br/). Any of the three can be the authored original; the other two are generated by CI as reviewable git commits. Siblings pair by filename basename:

content/post/mi-articulo.es.md      ← authored (no `translated_from`)
content/post/mi-articulo.de.md      ← written by scripts/translate.py
content/post/mi-articulo.pt-br.md   ← written by scripts/translate.py

A generated file carries translated_from, which is what stops CI translating its own output back into a loop. Machine output is never treated as a source.

manual_translation: true means "hands off", on both sides:

  • on an authored original — don't generate siblings for this post at all
  • on a generated sibling — never overwrite it again

Frontmatter contract

---
title: "Mi artículo"
date: 2026-07-31
lang: es
manual_translation: false
categories: [Gastronomía]      # taxonomy terms stay Spanish in every language
---

Generated siblings additionally carry translated_from: es.

Translation flow

  1. Publish in Decap → one git commit.
  2. Gitea webhook fires Woodpecker.
  3. scripts/translate.py diffs the push, finds changed authored files in any language, generates the missing siblings, and pushes them back as a bot commit (translations@vienalatina.com) marked [skip-translate].
  4. hugo --minify builds, rsync --delete deploys, Caddy serves.

Markup never reaches the model: code blocks and raw HTML pass through untouched, and link targets, inline code and protected community terms (Grätzl, Naschmarkt, …) are masked and verified to survive the round trip. A mask that doesn't come back fails the pipeline rather than shipping corrupted text — no half-translated sets ever ship.

Backfill anything missing siblings (after the WP migration, or for pages that predate the pipeline):

python scripts/translate.py --backfill

Translation engine

Built once on the server, and again only when changing models:

docker build -t vienalatina/translate:1 docker/translate

The image bakes in a pre-converted CTranslate2 build of M2M100 418M plus its tokenizer, so a publish makes no network calls. Tunable via MT_MODEL_DIR, MT_TOKENIZER, MT_COMPUTE_TYPE and MT_THREADS.

Swapping engines means writing one class against scripts/translation/provider.py — nothing in the pipeline changes. Two upgrades worth knowing about:

  • M2M100 1.2B — same MIT licence and same code path, materially better output, but ~2–2.5GB peak RAM. Needs an 8GB box (Hetzner CX32), not the CX22.
  • opus-mt-tc-big — better still for these specific language pairs and the only permissive option that handles Brazilian Portuguese distinctly (>>pob<<), at the cost of CC-BY-4.0 attribution and one model per directed pair.

Do not build on NLLB-200 (CC-BY-NC, non-commercial) or LibreTranslate (AGPL-3.0) if this stack is ever to be sold or offered as a service.

Server setup

Full command-by-command walkthrough: docs/server-setup.md. The Caddyfile and Docker Compose files it uses live in infra/.

Local development

hugo server          # http://localhost:1313

CI secrets (Woodpecker → repo settings → secrets)

Secret Value
gitea_push_token Gitea token for the translations bot user, repo write access

Translation needs no secret — the model is local.

The repo must be marked trusted in Woodpecker so the deploy step can mount /var/www/vienalatina.com.

Decap CMS

static/admin/config.yml uses the Gitea backend. Register an OAuth app in Gitea admin (redirect URI https://vienalatina.com/admin/) and put its client ID in app_id. The Decap JS bundle is downloaded at build time into static/admin/decap-cms.js (gitignored) — zero third-party requests at runtime.

GEO/SEO surfaces

  • hreflang + og:locale(:alternate) + JSON-LD BlogPosting/Blog with inLanguage — themes/vienalatina/layouts/partials/seo-head.html
  • robots.txt AI-crawler allowlist — static/robots.txt
  • llms.txt — generated at build time from layouts/index.llms.txt
  • sitemap.xml — Hugo native, multilingual

One-shot content migration

pip install requests html2text
python scripts/wp-to-hugo.py https://<old-wp-site>

Converts every WP post (with Polylang siblings) to content/post/<slug>.<lang>.md, downloads images into static/uploads/, and preserves existing slugs. Spot-check ~10 articles before committing. Add Caddy 301s for WP URL patterns that don't map cleanly (?p=123, /categoria/... → /categories/...).