vienalatina/docs/server-setup.md
Claude 635372d7a2
Write the site from the members area instead of Decap
Decap looks like a different product because it cannot be made to look like
this one: its maintainer's answer is override the CSS and accept that class
names move between releases, or fork it and carry that forever. Neither is
worth building on, so editing moves into the app that already has the site's
design and this project's tests.

Posts are files in git, so the editor is a form that commits a file through
Gitea's contents API. No working copy, no queue, no new state, and no way for
the pipeline to tell which editor wrote the file — which is what makes running
both at once safe. Decap keeps working; nothing here removes it, and the
fallback if something is missing is switching tabs.

Filenames and frontmatter are not this module's to invent. They are the
contract scripts/translate.py reads: `<basename>.es.md` is what split_lang()
parses, the date prefix is what keeps two posts with one title off a single
URL, and the absence of `translated_from` is what marks a file as something a
person wrote. Several tests import translate.py and run its own parser over
what the editor produced, because a file it cannot parse publishes in Spanish
and is never translated, with nothing reported anywhere.

Commits carry the writer's own Gitea account rather than a bot's, so history
says who wrote each post and Gitea's permissions apply unchanged. That needs
their access token, which lives in the database and never in a cookie: Flask
signs cookies but does not encrypt them, and a token is enough to commit as its
owner. Gitea expires tokens after about an hour, so they refresh ahead of
expiry and retry once on rejection — without that, saving would start failing
partway through an afternoon for no reason the writer could see.

Publishing is admin-only. Posting to the internal board and publishing to the
public site are different permissions, and the second is the larger grant.

Listing caches frontmatter against the git blob sha, because the contents API
returns names without bodies: that turns one request per post on every page
load into one request in total, and needs no invalidation, since a sha changes
only when the file does.

Verified: 97 checks. The filename translate.py parses, an authored source that
is_generated() rejects, the freeze toggle round-tripping, a stale sha refused
with the other edit intact, six filename shapes that must 404 including a
generated sibling, plain members refused, an SVG rejected as an image, uploads
not colliding, the cache reading each file once and refreshing when it changes,
and a preview that renders markdown, escapes script tags and publishes nothing.
Smoke-tested live: routes register and gate correctly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NizVpJ2dwzCbjCrTLCjeHn
2026-09-25 15:07:31 +00:00

504 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Server setup — step by step (Hetzner CX22, Ubuntu 24.04)
Every command you need, in order. Commands prefixed `local$` run on your own
computer; everything else runs on the server over SSH. Budget ~2–3 hours.
Where you see `pablo`, `<SERVER-IP>`, or a password placeholder, substitute
your own values.
---
## 0. Order the server
At [console.hetzner.com](https://console.hetzner.com) → Create Server:
- **Location:** Nuremberg
- **Image:** Ubuntu 24.04
- **Type:** Shared vCPU → **CX22** (2 vCPU, 4 GB RAM, €4.51/mo)
- **SSH key:** add your public key (`local$ cat ~/.ssh/id_ed25519.pub` — if you
don't have one: `local$ ssh-keygen -t ed25519`). Adding it here means root
login works by key from the start, no password emails.
Note the server's IP address — that's `<SERVER-IP>` everywhere below.
## 1. DNS (do this first — it needs time to propagate)
At your domain registrar, add two **A records**:
| Name | Type | Value | TTL |
|---|---|---|---|
| `git` | A | `<SERVER-IP>` | 300 |
| `ci` | A | `<SERVER-IP>` | 300 |
**Do NOT touch the record for `vienalatina.com` itself** — the live WordPress
site keeps running until cutover day.
Check propagation (repeat until it prints the server IP):
```sh
local$ dig +short git.vienalatina.com
```
## 2. First login + basic hardening
```sh
local$ ssh root@<SERVER-IP>
```
Update and create your user:
```sh
apt update && apt -y upgrade
adduser pablo # pick a strong password, skip the questions
usermod -aG sudo pablo
# give your user the same SSH key root has
rsync --archive --chown=pablo:pablo ~/.ssh /home/pablo
```
Lock SSH down to keys only:
```sh
sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
systemctl restart ssh
```
**Before closing this terminal**, open a second one and confirm you can get in:
```sh
local$ ssh pablo@<SERVER-IP>
```
From here on, work as `pablo` and prefix privileged commands with `sudo`
(or run `sudo -i` once).
Firewall + brute-force protection:
```sh
sudo apt install -y ufw fail2ban
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw --force enable
sudo systemctl enable --now fail2ban
```
## 3. Install Caddy
```sh
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
| sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
| sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install -y caddy
```
## 4. Install Docker
```sh
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker pablo
```
Log out and back in (`exit`, then `ssh pablo@<SERVER-IP>`) so the `docker`
group takes effect. Verify: `docker ps` should print an empty table, not a
permission error.
## 5. Get this repo's infra files onto the server
```sh
sudo mkdir -p /srv /var/www/vienalatina.com
cd ~
git clone https://github.com/pablovolenski/vienalatina.git
sudo cp -r vienalatina/infra/gitea /srv/gitea
sudo cp -r vienalatina/infra/woodpecker /srv/woodpecker
sudo cp vienalatina/infra/caddy/Caddyfile /etc/caddy/Caddyfile
sudo systemctl reload caddy
```
## 6. Bring up Gitea
```sh
cd /srv/gitea
sudo docker compose up -d
```
Wait ~30 s, then open **https://git.vienalatina.com** in your browser (the
certificate is fetched automatically; if you get an error, wait a minute for
DNS/certificate and reload). You'll see Gitea's install page:
- Database: **SQLite3** (fine at this scale)
- Server domain / base URL: leave as pre-filled (`git.vienalatina.com`)
- **Administrator account** (bottom of the page — expand it): username
`pablo`, your email, a strong password. Create it now; the first account
is the admin.
- Click **Install Gitea**.
Then create the two OAuth apps and the bot user, all in the Gitea web UI:
1. **Woodpecker OAuth app:** profile icon → Site Administration →
Integrations → Applications → *Create new OAuth2 application*
- Name: `woodpecker`
- Redirect URI: `https://ci.vienalatina.com/authorize`
- Save the **Client ID** and **Client Secret** — needed in step 7.
2. **Decap OAuth app:** same screen, second application
- Name: `decap-cms`
- Redirect URI: `https://vienalatina.com/admin/` (exactly, trailing slash
included — Gitea matches it literally)
- **Untick "Confidential Client".** Decap runs in the browser and
authenticates with PKCE; a confidential app makes Gitea demand a client
secret that a browser cannot keep, and the login fails *after* you
authorize, which makes it look like a Decap bug.
- Save the **Client ID** — it goes into `static/admin/config.yml` (step 9).
There is no secret to save, and the Client ID is not one either: it is
published in the site's JavaScript by design.
3. **Translations bot:** Site Administration → Identity & Access →
User Accounts → *Create User Account*
- Username: `translations`, email: `translations@vienalatina.com`,
any strong password.
- Log in **as the bot user** once (private window), go to Settings →
Applications → *Generate New Token*, scopes: **repository (write)**.
Save the token — it becomes the `gitea_push_token` secret in step 8.
## 7. Bring up Woodpecker
```sh
cd /srv/woodpecker
sudo cp .env.example .env
openssl rand -hex 32 # copy the output
sudo nano .env # paste Woodpecker OAuth client ID + secret + the random hex
sudo mkdir -p data && sudo chown -R 1000:1000 data # v3 images run as uid 1000
sudo docker compose up -d
```
Open **https://ci.vienalatina.com** → *Login* → it bounces you to Gitea →
*Authorize*. You're in, as admin (the `WOODPECKER_ADMIN=pablo` line in the
compose file — edit it if your Gitea username differs).
## 8. Create the site repo and wire the pipeline
Create the repo in Gitea: **+** (top right) → New Repository → name
`vienalatina`, owner `pablo`, **not** initialized with anything → Create.
Add the `translations` bot as collaborator: repo → Settings →
Collaborators → add `translations` with **Write** access.
Push the code into Gitea (from the server clone you made in step 5, or from
your laptop):
```sh
cd ~/vienalatina
git remote add gitea https://git.vienalatina.com/pablo/vienalatina.git
git push gitea main
```
(It will ask for your Gitea username/password.)
In Woodpecker (**https://ci.vienalatina.com**):
1. *Repositories* → *Add repository* → enable `pablo/vienalatina`
(this auto-creates the push webhook in Gitea).
2. Repo → Settings → *Project settings* → check **Trusted** (needed so the
deploy step may mount `/var/www/vienalatina.com`).
3. Repo → Settings → *Secrets* → add `gitea_push_token`, the bot token from
step 6.3. That is the only secret — translation runs locally and needs no key.
Build the translation image before the first run (~5 minutes; it downloads
about 1GB of model):
```sh
cd ~/vienalatina
docker build -t vienalatina/translate:1 docker/translate
```
The push in the step above has already triggered a first pipeline — it ran
before the image and secret existed, so **push a new commit rather than using
Restart**. Restart replays the old commit, and a restart's empty diff range
makes the translate step find nothing to do. All three steps (translate →
build → deploy) should go green, and `/var/www/vienalatina.com/` on the server
now contains the built site:
```sh
ls /var/www/vienalatina.com # index.html, de/, pt-br/, robots.txt, llms.txt …
```
## 9. Point Decap at Gitea
On your working copy: edit `static/admin/config.yml`, replace
`REPLACE_WITH_GITEA_OAUTH_CLIENT_ID` with the Decap OAuth Client ID from
step 6.2, commit, push to Gitea. (You can't log into `/admin` until the main
domain is live — that's expected.)
```sh
cd ~/vienalatina
sed -i 's/REPLACE_WITH_GITEA_OAUTH_CLIENT_ID/<client id>/' static/admin/config.yml
grep app_id static/admin/config.yml
git commit -am "Wire Decap to the Gitea OAuth app" && git push gitea main
```
This value is per-deployment: the placeholder is what belongs in the repo, so
leave it in place in any copy of this platform that is not this server.
If `/admin/` still shows *Client ID not registered* afterwards, the page is
serving a cached `config.yml` — hard-reload it. If it fails *after* the Gitea
authorize screen instead, the app was created as a confidential client; delete
it and recreate it with that box unticked.
## 10. Test the translation loop end-to-end
```sh
cd ~/vienalatina
cat > content/post/mi-test.es.md <<'EOF'
---
title: "Artículo de prueba"
date: 2026-08-01
lang: es
manual_translation: false
categories: [Comunidad]
---
Esto es una prueba del flujo de traducción automática en el Grätzl.
EOF
git add . && git commit -m "test: pipeline round-trip" && git push gitea main
```
Within ~60 s the Woodpecker pipeline should finish and
`content/post/mi-test.de.md` + `content/post/mi-test.pt-br.md` appear in the
Gitea repo as commits by `translations`. Note "Grätzl" survives untranslated
(protected term). Delete all three test files with another commit when done.
Saturday complete. 🎉
---
## Cutover day (Sunday evening)
1. Run the content migration and push (see README, "One-shot content
migration"). Migrated Polylang siblings arrive frozen
(`manual_translation: true`) because they are *human* translations — the
machine engine must never overwrite them. Then
`python scripts/translate.py --backfill` fills in any set that WordPress
had no translation for. Spot-check the built site with
`grep` on `/var/www/vienalatina.com/index.html`; the
`curl -H "Host: vienalatina.com" http://127.0.0.1/` trick only works after
step 3, since Caddy has no matching site block until then.
2. At the registrar: lower the `vienalatina.com` A record TTL to 300, wait
for the old TTL to expire, then change the A record to `<SERVER-IP>`
(and `www` too, as CNAME to `vienalatina.com` or A to the same IP).
3. On the server: uncomment the `vienalatina.com` blocks in
`/etc/caddy/Caddyfile`, then `sudo systemctl reload caddy`. Caddy fetches
the certificate as soon as DNS resolves to this server.
4. Verify: the checklist in the migration plan (hreflang tags, robots.txt,
llms.txt, Lighthouse, `manual_translation: true` freeze test, and the
loop-prevention test — after the bot pushes siblings, the pipeline it
triggers must report "nothing to translate" rather than translating the
siblings back).
5. Keep the WP host untouched for 30 days as fallback; watch Google Search
Console and add Caddy 301s for any 404s it reports.
## Optional: nightly backups (restic → Hetzner Storage Box)
```sh
sudo apt install -y restic
sudo restic -r sftp:uXXXXXX@uXXXXXX.your-storagebox.de:backups init
# then a root cron entry, e.g.:
# 0 3 * * * restic -r sftp:... backup /srv /var/www --password-file /root/.restic-pw
```
## 11. Members area (`/comunidad/`)
The private area: roles and an internal board. It is the only part of the site
that runs code to answer a request, and the only data on the server that is not
already in git.
### 11.1 Register the OAuth application
Gitea → **Site Administration → Integrations → Applications** →
*Create new OAuth2 application*:
- Name: `vienalatina-board`
- Redirect URI: `https://vienalatina.com/comunidad/auth/callback`
- **Leave "Confidential Client" TICKED.**
That last point is the opposite of the Decap application in step 6.2, and the
difference is worth understanding rather than memorising. Decap runs in the
visitor's browser, where any secret would be readable by the visitor, so it has
to be a public client using PKCE. The board runs on the server, so it can hold
a secret and should — a confidential client is the stronger of the two.
Save the **Client ID** and the **Client Secret**.
### 11.2 Optional: a token for creating accounts
Without it, admins can add people who already have a Gitea login, and nothing
else changes. With it, they can create the Gitea account from inside the members
area and hand over a one-time password.
Log in as a Gitea **site administrator** → Settings → Applications → *Generate
New Token* → scope **admin (write)**.
Understand what this token is before you create it: it can create and modify any
account on the instance, including administrators. Anything that can read the
board's environment — the compose file, `docker inspect`, a shell in the
container — can use it. If you would rather not have that on the box, leave
`GITEA_ADMIN_TOKEN` empty and create accounts in Gitea by hand.
### 11.3 Build and run
```sh
cd ~/vienalatina
docker build -t vienalatina/board:1 -f docker/board/Dockerfile .
sudo mkdir -p /srv/board/data
sudo cp -r infra/board/. /srv/board/
cd /srv/board
sudo cp .env.example .env
openssl rand -hex 32 # paste as BOARD_SECRET_KEY
sudo nano .env # client id, secret, BOARD_OWNER, optional admin token
sudo chown -R 1000:1000 /srv/board/data
sudo docker compose up -d
```
`BOARD_OWNER` is applied once, to an empty database, and ignored from then on.
It cannot be used to take ownership later: that is deliberate, because otherwise
editing a file on disk would be a quieter route to the top than asking for it.
Ownership moves only through *Transferir titularidad* inside the app.
### 11.4 Route it through Caddy
Add to the `vienalatina.com` block in `/etc/caddy/Caddyfile` (already present in
`infra/caddy/Caddyfile`):
```
@board path /comunidad /comunidad/*
reverse_proxy @board 127.0.0.1:8080
```
Then `sudo caddy validate --config /etc/caddy/Caddyfile && sudo systemctl reload caddy`.
Both paths are matched on purpose: Flask redirects `/comunidad` to
`/comunidad/`, and matching only the trailing-slash form lets the bare path fall
through to the static site and 404.
### 11.5 Back it up — this part is not optional
Everything else on this server is reproducible from the repository. The board's
threads, comments and membership exist in exactly one place.
```sh
sudo apt install -y sqlite3
crontab -e
# 15 4 * * * /home/pablo/vienalatina/scripts/backup-board.sh >> /home/pablo/board-backup.log 2>&1
```
The script uses SQLite's `.backup` rather than copying the file, because the
database is live and in WAL mode — a plain `cp` can capture it missing its most
recent commits. Test a restore before you rely on it: stop the container, gunzip
a backup over `/srv/board/data/board.db`, start it again.
### 11.6 Who can do what
| | Owner | Admin | User |
|---|---|---|---|
| Post, comment, edit own | ✓ | ✓ | ✓ |
| Delete any post | ✓ | ✓ | — |
| Edit someone else's post | — | — | — |
| Pin and close threads | ✓ | ✓ | — |
| Create users | ✓ | ✓ | — |
| Create admins | ✓ | — | — |
| Suspend a user | ✓ | ✓ | — |
| Suspend an admin | ✓ | — | — |
| Transfer ownership | ✓ | — | — |
Nobody edits anyone else's words, administrators included. Taking a post down is
visible to the person who wrote it; rewriting it is not, and an admin who could
do that could leave a sentence attributed to a member who never wrote it.
There is exactly one owner, and the database enforces it with a unique index
rather than trusting the application to remember. The owner cannot be suspended
or demoted by anyone, themselves included — to step down, transfer ownership to
an admin.
### 11.7 Personal data
Members' names, emails and writing are personal data under GDPR.
- **Erasure:** the owner's *Eliminar* removes the member row entirely and
reassigns their threads and comments to a tombstone shown as "Miembro
eliminado", so conversations other people took part in stay readable.
- **Access:** any member can download everything they have written from
*Descargar mis datos*.
- **Retention:** soft-deleted posts stay in the database until removed by hand.
If you want a real retention limit, that is a `DELETE ... WHERE deleted_at <`
in this same cron slot — and a decision to take deliberately, not by default.
### 11.8 The content editor (`/comunidad/contenido/`)
Admins and the owner can write, edit and delete posts and pages from inside the
members area, instead of Decap at `/admin/`.
**Decap is still there and still works.** Nothing was removed. Use the new
editor for a few real posts first; if something turns out to be missing, switch
tabs. Removing Decap is a separate decision — see below.
Nothing extra to install or configure: it runs in the container already serving
`/comunidad/`, and commits through the Gitea OAuth application registered in
§11.1. Two settings exist if the repository is ever renamed:
```
CONTENT_REPO=pablo/vienalatina # owner/repo inside Gitea
CONTENT_BRANCH=main
```
**How publishing works.** The editor is a form that commits a file through
Gitea's contents API. Gitea's webhook fires Woodpecker, and translate → build →
deploy runs exactly as it does for a Decap commit — the pipeline cannot tell
which editor wrote the file, which is what makes running both at once safe.
**Commits are made with your own account**, not a bot's, so `git log` shows who
wrote each post and Gitea's permissions apply unchanged. Your access token is
stored in the members-area database (never in a cookie) and refreshed
automatically; Gitea expires them after about an hour, and without refreshing,
saving would start failing mid-afternoon for no visible reason.
**Filenames follow the same rules Decap used**, because `scripts/translate.py`
reads them: `YYYY-MM-DD-slug.es.md` for posts, `slug.es.md` for pages. A file
whose name breaks that contract publishes in Spanish and is never translated,
with nothing reported anywhere — which is why the tests import translate.py and
run its parser over what the editor writes.
**Two people editing one post** is a visible conflict, not a silent overwrite:
the form carries the file's git sha and Gitea rejects a write whose sha has
moved on. You are asked to reopen the post rather than losing the other edit.
**Images** are committed as a second, separate commit before the post itself,
so publishing with a picture produces two pipeline runs. Harmless, and the
alternative — batching both into one commit via the git trees API — is
considerably more code for something nobody sees.
**What it deliberately does not do:** rich-text editing (markdown with a
preview button instead), a media library, drafts, or editing the generated
German and Portuguese files. Those stay the pipeline's, and a hand-written
translation is still frozen with `manual_translation: true`.
#### Worth tightening later
The OAuth application requests no explicit scope, so Gitea grants the default —
full access to the account, which is more than the editor needs. Narrowing it to
`read:user write:repository` is a one-line change in `apps/board/gitea.py`'s
`authorize_url()`, but it invalidates existing authorisations: everyone has to
approve the app again. Worth doing while the member list is short, and worth
testing on a throwaway account first, since a wrong scope string breaks sign-in
for everybody.
#### Removing Decap, once you are confident
Not urgent — leaving it costs a folder and one `wget` in the pipeline:
1. `rm -rf static/admin/`
2. Delete the `wget … decap-cms.js` line from `.woodpecker.yml`
3. Delete the `decap-cms` OAuth application in Gitea