986c066ae3
The DeepL free tier is metered (it failed in production with HTTP 456 Quota exceeded) and would require every deployment of this platform to carry its own API account. Translation now runs on M2M100 418M (MIT) via CTranslate2, shipped inside the pipeline image: no key, no quota, and no content or visitor data leaving the server. Also restores the multi-source behaviour of the original WordPress plugin, which the Python port had narrowed to Spanish-only. Any of the three site languages can now be the authored original. Because any language can be a source, loop prevention is no longer structural and is now explicit: generated siblings carry `translated_from` and are never treated as sources, and the bot's own [skip-translate] commits are skipped outright (that marker was already being written but never read). Markup protection moves in-process now that DeepL's tag_handling=html is gone. Code blocks and raw HTML pass through untouched; link targets, inline code and protected community terms are masked with placeholders that are verified to survive the round trip, failing the pipeline rather than shipping corrupted text. Two fixes along the way: - Generated siblings no longer inherit the source's `slug`. They did, which meant the first retranslation of a WordPress-migrated post moved /de/<german-slug>/ onto /de/<spanish-slug>/ and destroyed the inbound link preservation wp-to-hugo.py exists for. - `manual_translation` now works from the CMS. Decap only ever exposed it on the source while the script read it on the target, so the toggle did nothing. It now means "hands off" on both sides. wp-to-hugo.py marks migrated Polylang siblings frozen, since those are human translations and regenerating them would replace them with weaker machine output. Adds --backfill for sources missing siblings, which also fixes the existing 404s on /de/page/acerca/ and /pt-br/page/contacto/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
153 lines
5.5 KiB
Markdown
153 lines
5.5 KiB
Markdown
# Viena Latina
|
||
|
||
Static, self-hosted, trilingual blog for the Latin American community in
|
||
Vienna — Hugo + Decap CMS + Gitea + Woodpecker CI on a single Hetzner CX22
|
||
(Nuremberg, DE). Replaces the previous WordPress + Polylang + synchronous
|
||
DeepL stack.
|
||
|
||
Translation is self-hosted too: M2M100 418M (MIT) runs on CPU via CTranslate2
|
||
inside the pipeline image. No API key, no quota, and no visitor or content data
|
||
leaving the server.
|
||
|
||
```
|
||
Pablo ──► Decap CMS (/admin) ──commit──► Gitea ──webhook──► Woodpecker CI
|
||
│
|
||
translate (M2M100, async) ──┤
|
||
build (hugo) ─┤
|
||
deploy (rsync) ─┘
|
||
▼
|
||
Caddy 2 serves /var/www/vienalatina.com
|
||
```
|
||
|
||
## Languages
|
||
|
||
Spanish, German (`/de/`) and Brazilian Portuguese (`/pt-br/`). **Any of the
|
||
three can be the authored original**; the other two are generated by CI as
|
||
reviewable git commits. Siblings pair by filename basename:
|
||
|
||
```
|
||
content/post/mi-articulo.es.md ← authored (no `translated_from`)
|
||
content/post/mi-articulo.de.md ← written by scripts/translate.py
|
||
content/post/mi-articulo.pt-br.md ← written by scripts/translate.py
|
||
```
|
||
|
||
A generated file carries `translated_from`, which is what stops CI translating
|
||
its own output back into a loop. Machine output is never treated as a source.
|
||
|
||
`manual_translation: true` means "hands off", on both sides:
|
||
|
||
- on an **authored original** — don't generate siblings for this post at all
|
||
- on a **generated sibling** — never overwrite it again
|
||
|
||
### Frontmatter contract
|
||
|
||
```yaml
|
||
---
|
||
title: "Mi artículo"
|
||
date: 2026-07-31
|
||
lang: es
|
||
manual_translation: false
|
||
categories: [Gastronomía] # taxonomy terms stay Spanish in every language
|
||
---
|
||
```
|
||
|
||
Generated siblings additionally carry `translated_from: es`.
|
||
|
||
## Translation flow
|
||
|
||
1. Publish in Decap → one git commit.
|
||
2. Gitea webhook fires Woodpecker.
|
||
3. `scripts/translate.py` diffs the push, finds changed authored files in any
|
||
language, generates the missing siblings, and pushes them back as a bot
|
||
commit (`translations@vienalatina.com`) marked `[skip-translate]`.
|
||
4. `hugo --minify` builds, `rsync --delete` deploys, Caddy serves.
|
||
|
||
Markup never reaches the model: code blocks and raw HTML pass through
|
||
untouched, and link targets, inline code and protected community terms
|
||
(`Grätzl`, `Naschmarkt`, …) are masked and verified to survive the round trip.
|
||
A mask that doesn't come back fails the pipeline rather than shipping corrupted
|
||
text — no half-translated sets ever ship.
|
||
|
||
Backfill anything missing siblings (after the WP migration, or for pages that
|
||
predate the pipeline):
|
||
|
||
```sh
|
||
python scripts/translate.py --backfill
|
||
```
|
||
|
||
## Translation engine
|
||
|
||
Built once on the server, and again only when changing models:
|
||
|
||
```sh
|
||
docker build -t vienalatina/translate:1 docker/translate
|
||
```
|
||
|
||
The image bakes in a pre-converted CTranslate2 build of M2M100 418M plus its
|
||
tokenizer, so a publish makes no network calls. Tunable via `MT_MODEL_DIR`,
|
||
`MT_TOKENIZER`, `MT_COMPUTE_TYPE` and `MT_THREADS`.
|
||
|
||
Swapping engines means writing one class against
|
||
`scripts/translation/provider.py` — nothing in the pipeline changes. Two
|
||
upgrades worth knowing about:
|
||
|
||
- **M2M100 1.2B** — same MIT licence and same code path, materially better
|
||
output, but ~2–2.5GB peak RAM. Needs an 8GB box (Hetzner CX32), not the CX22.
|
||
- **opus-mt-tc-big** — better still for these specific language pairs and the
|
||
only permissive option that handles Brazilian Portuguese distinctly (`>>pob<<`),
|
||
at the cost of CC-BY-4.0 attribution and one model per directed pair.
|
||
|
||
Do **not** build on NLLB-200 (CC-BY-NC, non-commercial) or LibreTranslate
|
||
(AGPL-3.0) if this stack is ever to be sold or offered as a service.
|
||
|
||
## Server setup
|
||
|
||
Full command-by-command walkthrough: **[docs/server-setup.md](docs/server-setup.md)**.
|
||
The Caddyfile and Docker Compose files it uses live in [infra/](infra/).
|
||
|
||
## Local development
|
||
|
||
```sh
|
||
hugo server # http://localhost:1313
|
||
```
|
||
|
||
## CI secrets (Woodpecker → repo settings → secrets)
|
||
|
||
| Secret | Value |
|
||
|---|---|
|
||
| `gitea_push_token` | Gitea token for the translations bot user, repo write access |
|
||
|
||
Translation needs no secret — the model is local.
|
||
|
||
The repo must be marked **trusted** in Woodpecker so the deploy step can
|
||
mount `/var/www/vienalatina.com`.
|
||
|
||
## Decap CMS
|
||
|
||
`static/admin/config.yml` uses the Gitea backend. Register an OAuth app in
|
||
Gitea admin (redirect URI `https://vienalatina.com/admin/`) and put its
|
||
client ID in `app_id`. The Decap JS bundle is downloaded at build time into
|
||
`static/admin/decap-cms.js` (gitignored) — zero third-party requests at
|
||
runtime.
|
||
|
||
## GEO/SEO surfaces
|
||
|
||
- `hreflang` + `og:locale(:alternate)` + JSON-LD `BlogPosting`/`Blog` with
|
||
`inLanguage` — `themes/vienalatina/layouts/partials/seo-head.html`
|
||
- `robots.txt` AI-crawler allowlist — `static/robots.txt`
|
||
- `llms.txt` — generated at build time from `layouts/index.llms.txt`
|
||
- `sitemap.xml` — Hugo native, multilingual
|
||
|
||
## One-shot content migration
|
||
|
||
```sh
|
||
pip install requests html2text
|
||
python scripts/wp-to-hugo.py https://<old-wp-site>
|
||
```
|
||
|
||
Converts every WP post (with Polylang siblings) to
|
||
`content/post/<slug>.<lang>.md`, downloads images into `static/uploads/`,
|
||
and preserves existing slugs. Spot-check ~10 articles before committing.
|
||
Add Caddy 301s for WP URL patterns that don't map cleanly
|
||
(`?p=123`, `/categoria/...` → `/categories/...`).
|