vienalatina/docs/server-setup.md
Claude 986c066ae3
Replace DeepL with a self-hosted translation engine
The DeepL free tier is metered (it failed in production with HTTP 456
Quota exceeded) and would require every deployment of this platform to
carry its own API account. Translation now runs on M2M100 418M (MIT) via
CTranslate2, shipped inside the pipeline image: no key, no quota, and no
content or visitor data leaving the server.

Also restores the multi-source behaviour of the original WordPress plugin,
which the Python port had narrowed to Spanish-only. Any of the three site
languages can now be the authored original.

Because any language can be a source, loop prevention is no longer
structural and is now explicit: generated siblings carry `translated_from`
and are never treated as sources, and the bot's own [skip-translate]
commits are skipped outright (that marker was already being written but
never read).

Markup protection moves in-process now that DeepL's tag_handling=html is
gone. Code blocks and raw HTML pass through untouched; link targets,
inline code and protected community terms are masked with placeholders
that are verified to survive the round trip, failing the pipeline rather
than shipping corrupted text.

Two fixes along the way:

- Generated siblings no longer inherit the source's `slug`. They did,
  which meant the first retranslation of a WordPress-migrated post moved
  /de/<german-slug>/ onto /de/<spanish-slug>/ and destroyed the inbound
  link preservation wp-to-hugo.py exists for.
- `manual_translation` now works from the CMS. Decap only ever exposed it
  on the source while the script read it on the target, so the toggle did
  nothing. It now means "hands off" on both sides.

wp-to-hugo.py marks migrated Polylang siblings frozen, since those are
human translations and regenerating them would replace them with weaker
machine output.

Adds --backfill for sources missing siblings, which also fixes the
existing 404s on /de/page/acerca/ and /pt-br/page/contacto/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
2026-09-14 16:34:25 +00:00

9.8 KiB
Raw Blame History

Server setup — step by step (Hetzner CX22, Ubuntu 24.04)

Every command you need, in order. Commands prefixed local$ run on your own computer; everything else runs on the server over SSH. Budget ~2–3 hours.

Where you see pablo, <SERVER-IP>, or a password placeholder, substitute your own values.


0. Order the server

At console.hetzner.com → Create Server:

  • Location: Nuremberg
  • Image: Ubuntu 24.04
  • Type: Shared vCPU → CX22 (2 vCPU, 4 GB RAM, €4.51/mo)
  • SSH key: add your public key (local$ cat ~/.ssh/id_ed25519.pub — if you don't have one: local$ ssh-keygen -t ed25519). Adding it here means root login works by key from the start, no password emails.

Note the server's IP address — that's <SERVER-IP> everywhere below.

1. DNS (do this first — it needs time to propagate)

At your domain registrar, add two A records:

Name Type Value TTL
git A <SERVER-IP> 300
ci A <SERVER-IP> 300

Do NOT touch the record for vienalatina.com itself — the live WordPress site keeps running until cutover day.

Check propagation (repeat until it prints the server IP):

local$ dig +short git.vienalatina.com

2. First login + basic hardening

local$ ssh root@<SERVER-IP>

Update and create your user:

apt update && apt -y upgrade

adduser pablo                 # pick a strong password, skip the questions
usermod -aG sudo pablo

# give your user the same SSH key root has
rsync --archive --chown=pablo:pablo ~/.ssh /home/pablo

Lock SSH down to keys only:

sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
systemctl restart ssh

Before closing this terminal, open a second one and confirm you can get in:

local$ ssh pablo@<SERVER-IP>

From here on, work as pablo and prefix privileged commands with sudo (or run sudo -i once).

Firewall + brute-force protection:

sudo apt install -y ufw fail2ban
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw --force enable
sudo systemctl enable --now fail2ban

3. Install Caddy

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
  | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
  | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install -y caddy

4. Install Docker

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker pablo

Log out and back in (exit, then ssh pablo@<SERVER-IP>) so the docker group takes effect. Verify: docker ps should print an empty table, not a permission error.

5. Get this repo's infra files onto the server

sudo mkdir -p /srv /var/www/vienalatina.com
cd ~
git clone https://github.com/pablovolenski/vienalatina.git
sudo cp -r vienalatina/infra/gitea /srv/gitea
sudo cp -r vienalatina/infra/woodpecker /srv/woodpecker
sudo cp vienalatina/infra/caddy/Caddyfile /etc/caddy/Caddyfile
sudo systemctl reload caddy

6. Bring up Gitea

cd /srv/gitea
sudo docker compose up -d

Wait ~30 s, then open https://git.vienalatina.com in your browser (the certificate is fetched automatically; if you get an error, wait a minute for DNS/certificate and reload). You'll see Gitea's install page:

  • Database: SQLite3 (fine at this scale)
  • Server domain / base URL: leave as pre-filled (git.vienalatina.com)
  • Administrator account (bottom of the page — expand it): username pablo, your email, a strong password. Create it now; the first account is the admin.
  • Click Install Gitea.

Then create the two OAuth apps and the bot user, all in the Gitea web UI:

  1. Woodpecker OAuth app: profile icon → Site Administration → Integrations → Applications → Create new OAuth2 application
    • Name: woodpecker
    • Redirect URI: https://ci.vienalatina.com/authorize
    • Save the Client ID and Client Secret — needed in step 7.
  2. Decap OAuth app: same screen, second application
    • Name: decap-cms
    • Redirect URI: https://vienalatina.com/admin/
    • Save the Client ID — it goes into static/admin/config.yml (step 9).
  3. Translations bot: Site Administration → Identity & Access → User Accounts → Create User Account
    • Username: translations, email: translations@vienalatina.com, any strong password.
    • Log in as the bot user once (private window), go to Settings → Applications → Generate New Token, scopes: repository (write). Save the token — it becomes the gitea_push_token secret in step 8.

7. Bring up Woodpecker

cd /srv/woodpecker
sudo cp .env.example .env
openssl rand -hex 32        # copy the output
sudo nano .env              # paste Woodpecker OAuth client ID + secret + the random hex
sudo mkdir -p data && sudo chown -R 1000:1000 data   # v3 images run as uid 1000
sudo docker compose up -d

Open https://ci.vienalatina.com → Login → it bounces you to Gitea → Authorize. You're in, as admin (the WOODPECKER_ADMIN=pablo line in the compose file — edit it if your Gitea username differs).

8. Create the site repo and wire the pipeline

Create the repo in Gitea: + (top right) → New Repository → name vienalatina, owner pablo, not initialized with anything → Create.

Add the translations bot as collaborator: repo → Settings → Collaborators → add translations with Write access.

Push the code into Gitea (from the server clone you made in step 5, or from your laptop):

cd ~/vienalatina
git remote add gitea https://git.vienalatina.com/pablo/vienalatina.git
git push gitea main

(It will ask for your Gitea username/password.)

In Woodpecker (https://ci.vienalatina.com):

  1. Repositories → Add repository → enable pablo/vienalatina (this auto-creates the push webhook in Gitea).
  2. Repo → Settings → Project settings → check Trusted (needed so the deploy step may mount /var/www/vienalatina.com).
  3. Repo → Settings → Secrets → add gitea_push_token, the bot token from step 6.3. That is the only secret — translation runs locally and needs no key.

Build the translation image before the first run (~5 minutes; it downloads about 1GB of model):

cd ~/vienalatina
docker build -t vienalatina/translate:1 docker/translate

The push in the step above has already triggered a first pipeline — it ran before the image and secret existed, so push a new commit rather than using Restart. Restart replays the old commit, and a restart's empty diff range makes the translate step find nothing to do. All three steps (translate → build → deploy) should go green, and /var/www/vienalatina.com/ on the server now contains the built site:

ls /var/www/vienalatina.com     # index.html, de/, pt-br/, robots.txt, llms.txt …

9. Point Decap at Gitea

On your working copy: edit static/admin/config.yml, replace REPLACE_WITH_GITEA_OAUTH_CLIENT_ID with the Decap OAuth Client ID from step 6.2, commit, push to Gitea. (You can't log into /admin until the main domain is live — that's expected.)

10. Test the translation loop end-to-end

cd ~/vienalatina
cat > content/post/mi-test.es.md <<'EOF'
---
title: "Artículo de prueba"
date: 2026-08-01
lang: es
manual_translation: false
categories: [Comunidad]
---

Esto es una prueba del flujo de traducción automática en el Grätzl.
EOF
git add . && git commit -m "test: pipeline round-trip" && git push gitea main

Within ~60 s the Woodpecker pipeline should finish and content/post/mi-test.de.md + content/post/mi-test.pt-br.md appear in the Gitea repo as commits by translations. Note "Grätzl" survives untranslated (protected term). Delete all three test files with another commit when done.

Saturday complete. 🎉


Cutover day (Sunday evening)

  1. Run the content migration and push (see README, "One-shot content migration"). Migrated Polylang siblings arrive frozen (manual_translation: true) because they are human translations — the machine engine must never overwrite them. Then python scripts/translate.py --backfill fills in any set that WordPress had no translation for. Spot-check the built site with grep on /var/www/vienalatina.com/index.html; the curl -H "Host: vienalatina.com" http://127.0.0.1/ trick only works after step 3, since Caddy has no matching site block until then.
  2. At the registrar: lower the vienalatina.com A record TTL to 300, wait for the old TTL to expire, then change the A record to <SERVER-IP> (and www too, as CNAME to vienalatina.com or A to the same IP).
  3. On the server: uncomment the vienalatina.com blocks in /etc/caddy/Caddyfile, then sudo systemctl reload caddy. Caddy fetches the certificate as soon as DNS resolves to this server.
  4. Verify: the checklist in the migration plan (hreflang tags, robots.txt, llms.txt, Lighthouse, manual_translation: true freeze test, and the loop-prevention test — after the bot pushes siblings, the pipeline it triggers must report "nothing to translate" rather than translating the siblings back).
  5. Keep the WP host untouched for 30 days as fallback; watch Google Search Console and add Caddy 301s for any 404s it reports.

Optional: nightly backups (restic → Hetzner Storage Box)

sudo apt install -y restic
sudo restic -r sftp:uXXXXXX@uXXXXXX.your-storagebox.de:backups init
# then a root cron entry, e.g.:
# 0 3 * * * restic -r sftp:... backup /srv /var/www --password-file /root/.restic-pw