Go to file
Claude 9355121b7c
Let members post pictures, and back them up
Phase B. Images on threads and comments, stored in /data/uploads —
inside the volume that already holds board.db, so there is one directory
to back up rather than two and no second mount to remember.

Not in the site repository, which is the distinction that matters: the
content editor uploads by committing, and a private photo committed
there would go through the build pipeline and out onto vienalatina.com.

The type is decided by the first bytes, not the filename. content.py
trusts the extension, which is tolerable where Hugo serves the result;
here we serve it back, so an HTML file called gato.png would be a script
running on our own origin. Five magic-number checks, no new dependency.
The uploaded name is kept only as text to show a person; the name on
disk is generated. Staging is separate from saving so a refused picture
cannot leave a half-made thread behind.

Threads and comments are soft-deleted, so the serving route checks the
parent is still live. Without it, taking a post down leaves its photo
readable by anyone who noted the URL.

And the hole this phase existed to close: backup-board.sh archived
board.db and nothing else, so the first upload would have made the
nightly backup silently incomplete while still reporting success. It now
archives the uploads directory too, verifies the archive, and names the
directory when it is empty rather than passing over it in silence.
Tested by restoring: destroyed the directory, restored from the archive,
compared bytes.

IMAGE_EXTENSIONS moves to uploads.py and content.py imports it, so the
editor and the board cannot drift apart about what counts as a picture.

46 new tests, 176 in total.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
2026-09-28 09:02:18 +00:00
apps/board Let members post pictures, and back them up 2026-09-28 09:02:18 +00:00
content Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
docker Add a members area: roles and an internal board 2026-09-22 10:57:18 +00:00
docs Let members post pictures, and back them up 2026-09-28 09:02:18 +00:00
infra Stop showing members the name of the software behind the login 2026-09-25 19:30:06 +00:00
scripts Let members post pictures, and back them up 2026-09-28 09:02:18 +00:00
static Add a members area: roles and an internal board 2026-09-22 10:57:18 +00:00
themes/vienalatina Scaffold Hugo + Decap trilingual site from WordPress migration plan 2026-07-31 11:17:32 +00:00
.gitignore Add a members area: roles and an internal board 2026-09-22 10:57:18 +00:00
.woodpecker.yml Stop two posts with the same title from sharing one URL 2026-09-22 07:49:46 +00:00
config.yaml Scope post URLs to the day, not the month 2026-09-22 07:50:51 +00:00
README.md Write the site from the members area instead of Decap 2026-09-25 15:07:31 +00:00

Viena Latina

Self-hosted, trilingual site for the Latin American community in Vienna — Hugo + Decap CMS + Gitea + Woodpecker CI on a single Hetzner CX22 (Falkenstein, DE). Replaces the previous WordPress + Polylang + synchronous DeepL stack.

Translation is self-hosted too: OPUS-MT (CC-BY-4.0) runs on CPU via CTranslate2 in the pipeline image, with the models mounted from the host. No API key, no quota, and no visitor or content data leaving the server.

Pablo ──► Decap CMS (/admin) ──commit──► Gitea ──webhook──► Woodpecker CI
                                           ▲                   │
                                           │   translate ──────┤
                                        OAuth2   build (hugo) ─┤
                                           │   deploy (rsync) ─┘
                                           │                   ▼
   Members ──► /comunidad/ ────────────────┘   Caddy 2 serves /var/www/vienalatina.com
               Flask + SQLite                  …and proxies /comunidad/ to Flask

Everything except /comunidad/ is a static file built from git. The members area is the one component that runs code to answer a request, and the one whose data is not reproducible from the repository — see Members area below.

Languages

Spanish, German (/de/) and Brazilian Portuguese (/pt-br/). Any of the three can be the authored original; the other two are generated by CI as reviewable git commits. Siblings pair by filename basename:

content/post/mi-articulo.es.md      ← authored (no `translated_from`)
content/post/mi-articulo.de.md      ← written by scripts/translate.py
content/post/mi-articulo.pt-br.md   ← written by scripts/translate.py

A generated file carries translated_from, which is what stops CI translating its own output back into a loop. Machine output is never treated as a source.

manual_translation: true means "hands off", on both sides:

  • on an authored original — don't generate siblings for this post at all
  • on a generated sibling — never overwrite it again

Frontmatter contract

---
title: "Mi artículo"
date: 2026-07-31
lang: es
manual_translation: false
categories: [Gastronomía]      # taxonomy terms stay Spanish in every language
---

Generated siblings additionally carry translated_from: es.

Translation flow

  1. Publish in Decap → one git commit.
  2. Gitea webhook fires Woodpecker.
  3. scripts/translate.py diffs the push, finds changed authored files in any language, generates the missing siblings, and pushes them back as a bot commit (translations@vienalatina.com) marked [skip-translate].
  4. hugo --minify builds, rsync --delete deploys, Caddy serves.

Markup never reaches the model: code blocks and raw HTML pass through untouched, and link targets, inline code and protected community terms (Grätzl, Naschmarkt, …) are masked and verified to survive the round trip.

Small models drop those masks occasionally, so a failed round trip degrades in steps rather than failing the publish:

  1. mask markup and protected terms — the normal path;
  2. if a term is lost, retry guarding only markup, and log that the term may now be translated;
  3. if markup itself is lost, leave that segment in the source language and warn.

A stray ⦅0⦆ or a mangled URL therefore never reaches a reader, and one awkward proper noun never blocks a deploy.

Backfill anything missing siblings (after the WP migration, or for pages that predate the pipeline):

python scripts/translate.py --backfill

Translation engine

Built once on the server, and again only when changing models:

docker build -t vienalatina/translate:1 docker/translate

The image bakes in a pre-converted CTranslate2 build of M2M100 418M plus its tokenizer, so a publish makes no network calls. Tunable via MT_MODEL_DIR, MT_TOKENIZER, MT_COMPUTE_TYPE and MT_THREADS.

Swapping engines means writing one class against scripts/translation/provider.py — nothing in the pipeline changes. Two upgrades worth knowing about:

  • M2M100 1.2B — same MIT licence and same code path, materially better output, but ~2–2.5GB peak RAM. Needs an 8GB box (Hetzner CX32), not the CX22.
  • opus-mt-tc-big — better still for these specific language pairs and the only permissive option that handles Brazilian Portuguese distinctly (>>pob<<), at the cost of CC-BY-4.0 attribution and one model per directed pair.

Do not build on NLLB-200 (CC-BY-NC, non-commercial) or LibreTranslate (AGPL-3.0) if this stack is ever to be sold or offered as a service.

Server setup

Full command-by-command walkthrough: docs/server-setup.md. The Caddyfile and Docker Compose files it uses live in infra/.

Local development

hugo server          # http://localhost:1313

CI secrets (Woodpecker → repo settings → secrets)

Secret Value
gitea_push_token Gitea token for the translations bot user, repo write access

Translation needs no secret — the model is local.

The repo must be marked trusted in Woodpecker so the deploy step can mount /var/www/vienalatina.com.

Decap CMS

static/admin/config.yml uses the Gitea backend. Register an OAuth app in Gitea admin (redirect URI https://vienalatina.com/admin/) and put its client ID in app_id. The Decap JS bundle is downloaded at build time into static/admin/decap-cms.js (gitignored) — zero third-party requests at runtime.

Members area

apps/board/ — a small Flask app at /comunidad/, behind Caddy, holding roles and an internal message board. Signed-in members only.

apps/board/
  app.py        factory, config, the CSRF and noindex hooks
  auth.py       Gitea OAuth2 (confidential client) and the membership gate
  members.py    roles, provisioning, ownership transfer, GDPR erasure/export
  board.py      threads and comments
  content.py    the editor — writes posts to Gitea's contents API
  gitea.py      the only module that talks to Gitea
  tokens.py     per-member access tokens, refreshed before they expire
  render.py     markdown with raw HTML disabled
  schema.sql    the tables, including the one-owner index
  tests/        pytest, 97 checks — `python3 -m pytest apps/board/tests`

Three things about it are load-bearing and easy to undo by accident:

  • A Gitea account is not a membership. Login succeeds only for an active row in members. Drop that check and every account on the instance gets in, starting with the translations bot.
  • One owner, enforced by a partial unique index, not by application code. The owner cannot be suspended or demoted by anyone; stepping down means transferring ownership to an admin.
  • html=False in render.py is the entire XSS defence, and it works because markdown-it then emits only its own tags. Turning it on means owning a sanitiser allowlist forever.

Admins can delete anyone's post; nobody can edit anyone else's, admins included. Removing a post is visible to its author, quietly rewriting it is not.

Its SQLite database is the only state on the server that git does not hold. scripts/backup-board.sh takes a consistent snapshot nightly — see docs/server-setup.md §11.

Writing posts

/comunidad/contenido/ replaces Decap CMS for admins: a form that commits a file through Gitea's contents API, so Woodpecker sees an ordinary push and translate → build → deploy runs unchanged. Commits carry the author's own account, not a bot's.

It exists because Decap has no supported way to be themed — its maintainer's answer is override its CSS and accept that class names move, or fork it — so /admin/ would always look like a different product bolted on.

Decap is still running. Both editors write the same files, and the pipeline cannot tell them apart, so there is no cutover to get wrong. docs/server-setup.md §11.8 covers removing Decap when the new editor has earned it.

Filenames are the contract, not a detail: YYYY-MM-DD-slug.es.md is what split_lang() parses and what keeps two posts off one URL. The tests import scripts/translate.py and run its parser over what the editor writes, because a file it cannot parse publishes in Spanish and is never translated, silently.

GEO/SEO surfaces

  • hreflang + og:locale(:alternate) + JSON-LD BlogPosting/Blog with inLanguage — themes/vienalatina/layouts/partials/seo-head.html
  • robots.txt AI-crawler allowlist — static/robots.txt
  • llms.txt — generated at build time from layouts/index.llms.txt
  • sitemap.xml — Hugo native, multilingual

One-shot content migration

pip install requests html2text
python scripts/wp-to-hugo.py https://<old-wp-site>

Converts every WP post (with Polylang siblings) to content/post/<slug>.<lang>.md, downloads images into static/uploads/, and preserves existing slugs. Spot-check ~10 articles before committing. Add Caddy 301s for WP URL patterns that don't map cleanly (?p=123, /categoria/... → /categories/...).