cc32cf49b5
First real pipeline run died with: PlaceholderError: masked span 'Viena Latina' came back 0 times M2M100 drops placeholder tokens often enough that failing the pipeline on mismatch would block the whole site deploy over a single proper noun. The verification itself was right — it stopped a literal ｟0｠ reaching a reader — but the policy was too blunt. Masks are now tiered by how much they actually matter. Markup must survive; terminology is a preference. So: try markup + terms, and on a lost term retry guarding only markup, accepting the term may come back translated. Only if markup itself is lost does the segment stay in the source language. A stray placeholder or mangled URL still never reaches a reader, but one awkward proper noun no longer blocks a publish. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
162 lines
5.9 KiB
Markdown
162 lines
5.9 KiB
Markdown
# Viena Latina
|
||
|
||
Static, self-hosted, trilingual blog for the Latin American community in
|
||
Vienna — Hugo + Decap CMS + Gitea + Woodpecker CI on a single Hetzner CX22
|
||
(Nuremberg, DE). Replaces the previous WordPress + Polylang + synchronous
|
||
DeepL stack.
|
||
|
||
Translation is self-hosted too: M2M100 418M (MIT) runs on CPU via CTranslate2
|
||
inside the pipeline image. No API key, no quota, and no visitor or content data
|
||
leaving the server.
|
||
|
||
```
|
||
Pablo ──► Decap CMS (/admin) ──commit──► Gitea ──webhook──► Woodpecker CI
|
||
│
|
||
translate (M2M100, async) ──┤
|
||
build (hugo) ─┤
|
||
deploy (rsync) ─┘
|
||
▼
|
||
Caddy 2 serves /var/www/vienalatina.com
|
||
```
|
||
|
||
## Languages
|
||
|
||
Spanish, German (`/de/`) and Brazilian Portuguese (`/pt-br/`). **Any of the
|
||
three can be the authored original**; the other two are generated by CI as
|
||
reviewable git commits. Siblings pair by filename basename:
|
||
|
||
```
|
||
content/post/mi-articulo.es.md ← authored (no `translated_from`)
|
||
content/post/mi-articulo.de.md ← written by scripts/translate.py
|
||
content/post/mi-articulo.pt-br.md ← written by scripts/translate.py
|
||
```
|
||
|
||
A generated file carries `translated_from`, which is what stops CI translating
|
||
its own output back into a loop. Machine output is never treated as a source.
|
||
|
||
`manual_translation: true` means "hands off", on both sides:
|
||
|
||
- on an **authored original** — don't generate siblings for this post at all
|
||
- on a **generated sibling** — never overwrite it again
|
||
|
||
### Frontmatter contract
|
||
|
||
```yaml
|
||
---
|
||
title: "Mi artículo"
|
||
date: 2026-07-31
|
||
lang: es
|
||
manual_translation: false
|
||
categories: [Gastronomía] # taxonomy terms stay Spanish in every language
|
||
---
|
||
```
|
||
|
||
Generated siblings additionally carry `translated_from: es`.
|
||
|
||
## Translation flow
|
||
|
||
1. Publish in Decap → one git commit.
|
||
2. Gitea webhook fires Woodpecker.
|
||
3. `scripts/translate.py` diffs the push, finds changed authored files in any
|
||
language, generates the missing siblings, and pushes them back as a bot
|
||
commit (`translations@vienalatina.com`) marked `[skip-translate]`.
|
||
4. `hugo --minify` builds, `rsync --delete` deploys, Caddy serves.
|
||
|
||
Markup never reaches the model: code blocks and raw HTML pass through
|
||
untouched, and link targets, inline code and protected community terms
|
||
(`Grätzl`, `Naschmarkt`, …) are masked and verified to survive the round trip.
|
||
|
||
Small models drop those masks occasionally, so a failed round trip degrades in
|
||
steps rather than failing the publish:
|
||
|
||
1. mask markup **and** protected terms — the normal path;
|
||
2. if a term is lost, retry guarding only markup, and log that the term may now
|
||
be translated;
|
||
3. if markup itself is lost, leave that segment in the source language and warn.
|
||
|
||
A stray `｟0｠` or a mangled URL therefore never reaches a reader, and one
|
||
awkward proper noun never blocks a deploy.
|
||
|
||
Backfill anything missing siblings (after the WP migration, or for pages that
|
||
predate the pipeline):
|
||
|
||
```sh
|
||
python scripts/translate.py --backfill
|
||
```
|
||
|
||
## Translation engine
|
||
|
||
Built once on the server, and again only when changing models:
|
||
|
||
```sh
|
||
docker build -t vienalatina/translate:1 docker/translate
|
||
```
|
||
|
||
The image bakes in a pre-converted CTranslate2 build of M2M100 418M plus its
|
||
tokenizer, so a publish makes no network calls. Tunable via `MT_MODEL_DIR`,
|
||
`MT_TOKENIZER`, `MT_COMPUTE_TYPE` and `MT_THREADS`.
|
||
|
||
Swapping engines means writing one class against
|
||
`scripts/translation/provider.py` — nothing in the pipeline changes. Two
|
||
upgrades worth knowing about:
|
||
|
||
- **M2M100 1.2B** — same MIT licence and same code path, materially better
|
||
output, but ~2–2.5GB peak RAM. Needs an 8GB box (Hetzner CX32), not the CX22.
|
||
- **opus-mt-tc-big** — better still for these specific language pairs and the
|
||
only permissive option that handles Brazilian Portuguese distinctly (`>>pob<<`),
|
||
at the cost of CC-BY-4.0 attribution and one model per directed pair.
|
||
|
||
Do **not** build on NLLB-200 (CC-BY-NC, non-commercial) or LibreTranslate
|
||
(AGPL-3.0) if this stack is ever to be sold or offered as a service.
|
||
|
||
## Server setup
|
||
|
||
Full command-by-command walkthrough: **[docs/server-setup.md](docs/server-setup.md)**.
|
||
The Caddyfile and Docker Compose files it uses live in [infra/](infra/).
|
||
|
||
## Local development
|
||
|
||
```sh
|
||
hugo server # http://localhost:1313
|
||
```
|
||
|
||
## CI secrets (Woodpecker → repo settings → secrets)
|
||
|
||
| Secret | Value |
|
||
|---|---|
|
||
| `gitea_push_token` | Gitea token for the translations bot user, repo write access |
|
||
|
||
Translation needs no secret — the model is local.
|
||
|
||
The repo must be marked **trusted** in Woodpecker so the deploy step can
|
||
mount `/var/www/vienalatina.com`.
|
||
|
||
## Decap CMS
|
||
|
||
`static/admin/config.yml` uses the Gitea backend. Register an OAuth app in
|
||
Gitea admin (redirect URI `https://vienalatina.com/admin/`) and put its
|
||
client ID in `app_id`. The Decap JS bundle is downloaded at build time into
|
||
`static/admin/decap-cms.js` (gitignored) — zero third-party requests at
|
||
runtime.
|
||
|
||
## GEO/SEO surfaces
|
||
|
||
- `hreflang` + `og:locale(:alternate)` + JSON-LD `BlogPosting`/`Blog` with
|
||
`inLanguage` — `themes/vienalatina/layouts/partials/seo-head.html`
|
||
- `robots.txt` AI-crawler allowlist — `static/robots.txt`
|
||
- `llms.txt` — generated at build time from `layouts/index.llms.txt`
|
||
- `sitemap.xml` — Hugo native, multilingual
|
||
|
||
## One-shot content migration
|
||
|
||
```sh
|
||
pip install requests html2text
|
||
python scripts/wp-to-hugo.py https://<old-wp-site>
|
||
```
|
||
|
||
Converts every WP post (with Polylang siblings) to
|
||
`content/post/<slug>.<lang>.md`, downloads images into `static/uploads/`,
|
||
and preserves existing slugs. Spot-check ~10 articles before committing.
|
||
Add Caddy 301s for WP URL patterns that don't map cleanly
|
||
(`?p=123`, `/categoria/...` → `/categories/...`).
|