635372d7a2
Decap looks like a different product because it cannot be made to look like this one: its maintainer's answer is override the CSS and accept that class names move between releases, or fork it and carry that forever. Neither is worth building on, so editing moves into the app that already has the site's design and this project's tests. Posts are files in git, so the editor is a form that commits a file through Gitea's contents API. No working copy, no queue, no new state, and no way for the pipeline to tell which editor wrote the file — which is what makes running both at once safe. Decap keeps working; nothing here removes it, and the fallback if something is missing is switching tabs. Filenames and frontmatter are not this module's to invent. They are the contract scripts/translate.py reads: `<basename>.es.md` is what split_lang() parses, the date prefix is what keeps two posts with one title off a single URL, and the absence of `translated_from` is what marks a file as something a person wrote. Several tests import translate.py and run its own parser over what the editor produced, because a file it cannot parse publishes in Spanish and is never translated, with nothing reported anywhere. Commits carry the writer's own Gitea account rather than a bot's, so history says who wrote each post and Gitea's permissions apply unchanged. That needs their access token, which lives in the database and never in a cookie: Flask signs cookies but does not encrypt them, and a token is enough to commit as its owner. Gitea expires tokens after about an hour, so they refresh ahead of expiry and retry once on rejection — without that, saving would start failing partway through an afternoon for no reason the writer could see. Publishing is admin-only. Posting to the internal board and publishing to the public site are different permissions, and the second is the larger grant. Listing caches frontmatter against the git blob sha, because the contents API returns names without bodies: that turns one request per post on every page load into one request in total, and needs no invalidation, since a sha changes only when the file does. Verified: 97 checks. The filename translate.py parses, an authored source that is_generated() rejects, the freeze toggle round-tripping, a stale sha refused with the other edit intact, six filename shapes that must 404 including a generated sibling, plain members refused, an SVG rejected as an image, uploads not colliding, the cache reading each file once and refreshing when it changes, and a preview that renders markdown, escapes script tags and publishes nothing. Smoke-tested live: routes register and gate correctly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
224 lines
9.0 KiB
Markdown
224 lines
9.0 KiB
Markdown
# Viena Latina
|
||
|
||
Self-hosted, trilingual site for the Latin American community in Vienna —
|
||
Hugo + Decap CMS + Gitea + Woodpecker CI on a single Hetzner CX22 (Falkenstein,
|
||
DE). Replaces the previous WordPress + Polylang + synchronous DeepL stack.
|
||
|
||
Translation is self-hosted too: OPUS-MT (CC-BY-4.0) runs on CPU via CTranslate2
|
||
in the pipeline image, with the models mounted from the host. No API key, no
|
||
quota, and no visitor or content data leaving the server.
|
||
|
||
```
|
||
Pablo ──► Decap CMS (/admin) ──commit──► Gitea ──webhook──► Woodpecker CI
|
||
▲ │
|
||
│ translate ──────┤
|
||
OAuth2 build (hugo) ─┤
|
||
│ deploy (rsync) ─┘
|
||
│ ▼
|
||
Members ──► /comunidad/ ────────────────┘ Caddy 2 serves /var/www/vienalatina.com
|
||
Flask + SQLite …and proxies /comunidad/ to Flask
|
||
```
|
||
|
||
Everything except `/comunidad/` is a static file built from git. The members
|
||
area is the one component that runs code to answer a request, and the one whose
|
||
data is not reproducible from the repository — see **Members area** below.
|
||
|
||
## Languages
|
||
|
||
Spanish, German (`/de/`) and Brazilian Portuguese (`/pt-br/`). **Any of the
|
||
three can be the authored original**; the other two are generated by CI as
|
||
reviewable git commits. Siblings pair by filename basename:
|
||
|
||
```
|
||
content/post/mi-articulo.es.md ← authored (no `translated_from`)
|
||
content/post/mi-articulo.de.md ← written by scripts/translate.py
|
||
content/post/mi-articulo.pt-br.md ← written by scripts/translate.py
|
||
```
|
||
|
||
A generated file carries `translated_from`, which is what stops CI translating
|
||
its own output back into a loop. Machine output is never treated as a source.
|
||
|
||
`manual_translation: true` means "hands off", on both sides:
|
||
|
||
- on an **authored original** — don't generate siblings for this post at all
|
||
- on a **generated sibling** — never overwrite it again
|
||
|
||
### Frontmatter contract
|
||
|
||
```yaml
|
||
---
|
||
title: "Mi artículo"
|
||
date: 2026-07-31
|
||
lang: es
|
||
manual_translation: false
|
||
categories: [Gastronomía] # taxonomy terms stay Spanish in every language
|
||
---
|
||
```
|
||
|
||
Generated siblings additionally carry `translated_from: es`.
|
||
|
||
## Translation flow
|
||
|
||
1. Publish in Decap → one git commit.
|
||
2. Gitea webhook fires Woodpecker.
|
||
3. `scripts/translate.py` diffs the push, finds changed authored files in any
|
||
language, generates the missing siblings, and pushes them back as a bot
|
||
commit (`translations@vienalatina.com`) marked `[skip-translate]`.
|
||
4. `hugo --minify` builds, `rsync --delete` deploys, Caddy serves.
|
||
|
||
Markup never reaches the model: code blocks and raw HTML pass through
|
||
untouched, and link targets, inline code and protected community terms
|
||
(`Grätzl`, `Naschmarkt`, …) are masked and verified to survive the round trip.
|
||
|
||
Small models drop those masks occasionally, so a failed round trip degrades in
|
||
steps rather than failing the publish:
|
||
|
||
1. mask markup **and** protected terms — the normal path;
|
||
2. if a term is lost, retry guarding only markup, and log that the term may now
|
||
be translated;
|
||
3. if markup itself is lost, leave that segment in the source language and warn.
|
||
|
||
A stray `｟0｠` or a mangled URL therefore never reaches a reader, and one
|
||
awkward proper noun never blocks a deploy.
|
||
|
||
Backfill anything missing siblings (after the WP migration, or for pages that
|
||
predate the pipeline):
|
||
|
||
```sh
|
||
python scripts/translate.py --backfill
|
||
```
|
||
|
||
## Translation engine
|
||
|
||
Built once on the server, and again only when changing models:
|
||
|
||
```sh
|
||
docker build -t vienalatina/translate:1 docker/translate
|
||
```
|
||
|
||
The image bakes in a pre-converted CTranslate2 build of M2M100 418M plus its
|
||
tokenizer, so a publish makes no network calls. Tunable via `MT_MODEL_DIR`,
|
||
`MT_TOKENIZER`, `MT_COMPUTE_TYPE` and `MT_THREADS`.
|
||
|
||
Swapping engines means writing one class against
|
||
`scripts/translation/provider.py` — nothing in the pipeline changes. Two
|
||
upgrades worth knowing about:
|
||
|
||
- **M2M100 1.2B** — same MIT licence and same code path, materially better
|
||
output, but ~2–2.5GB peak RAM. Needs an 8GB box (Hetzner CX32), not the CX22.
|
||
- **opus-mt-tc-big** — better still for these specific language pairs and the
|
||
only permissive option that handles Brazilian Portuguese distinctly (`>>pob<<`),
|
||
at the cost of CC-BY-4.0 attribution and one model per directed pair.
|
||
|
||
Do **not** build on NLLB-200 (CC-BY-NC, non-commercial) or LibreTranslate
|
||
(AGPL-3.0) if this stack is ever to be sold or offered as a service.
|
||
|
||
## Server setup
|
||
|
||
Full command-by-command walkthrough: **[docs/server-setup.md](docs/server-setup.md)**.
|
||
The Caddyfile and Docker Compose files it uses live in [infra/](infra/).
|
||
|
||
## Local development
|
||
|
||
```sh
|
||
hugo server # http://localhost:1313
|
||
```
|
||
|
||
## CI secrets (Woodpecker → repo settings → secrets)
|
||
|
||
| Secret | Value |
|
||
|---|---|
|
||
| `gitea_push_token` | Gitea token for the translations bot user, repo write access |
|
||
|
||
Translation needs no secret — the model is local.
|
||
|
||
The repo must be marked **trusted** in Woodpecker so the deploy step can
|
||
mount `/var/www/vienalatina.com`.
|
||
|
||
## Decap CMS
|
||
|
||
`static/admin/config.yml` uses the Gitea backend. Register an OAuth app in
|
||
Gitea admin (redirect URI `https://vienalatina.com/admin/`) and put its
|
||
client ID in `app_id`. The Decap JS bundle is downloaded at build time into
|
||
`static/admin/decap-cms.js` (gitignored) — zero third-party requests at
|
||
runtime.
|
||
|
||
## Members area
|
||
|
||
`apps/board/` — a small Flask app at `/comunidad/`, behind Caddy, holding roles
|
||
and an internal message board. Signed-in members only.
|
||
|
||
```
|
||
apps/board/
|
||
app.py factory, config, the CSRF and noindex hooks
|
||
auth.py Gitea OAuth2 (confidential client) and the membership gate
|
||
members.py roles, provisioning, ownership transfer, GDPR erasure/export
|
||
board.py threads and comments
|
||
content.py the editor — writes posts to Gitea's contents API
|
||
gitea.py the only module that talks to Gitea
|
||
tokens.py per-member access tokens, refreshed before they expire
|
||
render.py markdown with raw HTML disabled
|
||
schema.sql the tables, including the one-owner index
|
||
tests/ pytest, 97 checks — `python3 -m pytest apps/board/tests`
|
||
```
|
||
|
||
Three things about it are load-bearing and easy to undo by accident:
|
||
|
||
- **A Gitea account is not a membership.** Login succeeds only for an active row
|
||
in `members`. Drop that check and every account on the instance gets in,
|
||
starting with the translations bot.
|
||
- **One owner, enforced by a partial unique index**, not by application code.
|
||
The owner cannot be suspended or demoted by anyone; stepping down means
|
||
transferring ownership to an admin.
|
||
- **`html=False` in `render.py`** is the entire XSS defence, and it works
|
||
because markdown-it then emits only its own tags. Turning it on means owning a
|
||
sanitiser allowlist forever.
|
||
|
||
Admins can delete anyone's post; **nobody can edit anyone else's**, admins
|
||
included. Removing a post is visible to its author, quietly rewriting it is not.
|
||
|
||
Its SQLite database is the only state on the server that git does not hold.
|
||
`scripts/backup-board.sh` takes a consistent snapshot nightly — see
|
||
`docs/server-setup.md` §11.
|
||
|
||
### Writing posts
|
||
|
||
`/comunidad/contenido/` replaces Decap CMS for admins: a form that commits a
|
||
file through Gitea's contents API, so Woodpecker sees an ordinary push and
|
||
translate → build → deploy runs unchanged. Commits carry the author's own
|
||
account, not a bot's.
|
||
|
||
It exists because Decap has no supported way to be themed — its maintainer's
|
||
answer is override its CSS and accept that class names move, or fork it — so
|
||
`/admin/` would always look like a different product bolted on.
|
||
|
||
**Decap is still running.** Both editors write the same files, and the pipeline
|
||
cannot tell them apart, so there is no cutover to get wrong. `docs/server-setup.md`
|
||
§11.8 covers removing Decap when the new editor has earned it.
|
||
|
||
Filenames are the contract, not a detail: `YYYY-MM-DD-slug.es.md` is what
|
||
`split_lang()` parses and what keeps two posts off one URL. The tests import
|
||
`scripts/translate.py` and run its parser over what the editor writes, because
|
||
a file it cannot parse publishes in Spanish and is never translated, silently.
|
||
|
||
## GEO/SEO surfaces
|
||
|
||
- `hreflang` + `og:locale(:alternate)` + JSON-LD `BlogPosting`/`Blog` with
|
||
`inLanguage` — `themes/vienalatina/layouts/partials/seo-head.html`
|
||
- `robots.txt` AI-crawler allowlist — `static/robots.txt`
|
||
- `llms.txt` — generated at build time from `layouts/index.llms.txt`
|
||
- `sitemap.xml` — Hugo native, multilingual
|
||
|
||
## One-shot content migration
|
||
|
||
```sh
|
||
pip install requests html2text
|
||
python scripts/wp-to-hugo.py https://<old-wp-site>
|
||
```
|
||
|
||
Converts every WP post (with Polylang siblings) to
|
||
`content/post/<slug>.<lang>.md`, downloads images into `static/uploads/`,
|
||
and preserves existing slugs. Spot-check ~10 articles before committing.
|
||
Add Caddy 301s for WP URL patterns that don't map cleanly
|
||
(`?p=123`, `/categoria/...` → `/categories/...`).
|