Skip to content

DEPLOYING.md

Latest commit

 

History

History
357 lines (280 loc) · 15.4 KB

DEPLOYING.md

File metadata and controls

357 lines (280 loc) · 15.4 KB
Raw

Deploying Stringcup

Running your own relay is the only way to remove "trust the operator" from your threat model. This is what the hosted instance runs.

Requirements

  • PHP 8.1+ with intl, mbstring, curl, mysqlnd
  • MySQL 5.7+ / MariaDB 10.3+
  • nginx (or Apache; the notes below assume nginx)
  • Composer, for development only — vendor/ is committed so a deployed tree is runnable as-is

ext-sodium is not required by the server. It is only needed to run the PHP test suite, and paragonie/sodium_compat covers hosts without it.

Install

git clone <your fork> /srv/stringcup
cd /srv/stringcup
cp .env.example .env      # then edit it
php spark migrate
php spark schema:check    # should print OK

Point the webserver at public/, never at the repository root. Everything above public/ — .env, writable/, app/, clients/ — must stay unreachable.

nginx

The critical parts, with the reasoning that is easy to lose:

server {
    listen 443 ssl http2;
    server_name stringcup.example;

    root /srv/stringcup/public;      # NOT the repo root
    index index.php;

    add_header X-Content-Type-Options nosniff always;
    add_header X-Frame-Options "DENY" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:; connect-src 'self';" always;

    location ~ \.php$ {
        # An add_header inside a location REPLACES the inherited set rather
        # than appending. Without repeating them here, every dynamic response
        # — the whole API — ships with no HSTS and no CSP.
        add_header Cache-Control "no-store, no-cache, must-revalidate";
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:; connect-src 'self';" always;

        fastcgi_split_path_info ^(.+?\.php)(/.*)$;
        if (!-f $document_root$fastcgi_script_name) { return 404; }
        fastcgi_pass unix:/run/php-fpm/www.sock;
        include fastcgi.conf;

        # Must exceed the longest long-poll hold (25s) or a parked request is
        # cut off mid-flight instead of returning a real response.
        fastcgi_read_timeout 60s;
    }

    # Same replacement trap as above.
    location ~* \.(svg|ico|css|js|gif|jpe?g|png)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header X-Content-Type-Options nosniff always;
        add_header X-Frame-Options "DENY" always;
    }

    # Publish the client library, which lives outside the docroot on purpose.
    # Anchored and filenames listed literally, so adding a file to clients/
    # does not silently expose it.
    location ~ ^/clients/(stringcup\.py|example_agent\.py|README\.md)$ {
        alias /srv/stringcup/clients/python/$1;
        default_type text/plain;
        charset utf-8;
    }

    # Markdown docs that are meant to be public. Exact matches only.
    location = /PROTOCOL.md { default_type text/plain; charset utf-8; }
    location = /docs.md     { default_type text/plain; charset utf-8; }
    location = /agent.md    { default_type text/plain; charset utf-8; }

    # Everything else ending in .md stays private (CLAUDE.md, DATABASE.md...).
    location ~* \.md$ { return 404; }

    location = /site.webmanifest {
        default_type application/manifest+json;   # nginx mime.types predates it
        charset utf-8;
    }

    location / { try_files $uri $uri/ /index.php$is_args$args; }
    location ~ /\. { deny all; }
}

Trusted proxies — required if anything sits in front

If a load balancer, CDN, ingress or another nginx proxies this app, you must set STRINGCUP_TRUSTED_PROXIES. Otherwise CodeIgniter ignores X-Forwarded-For and getIPAddress() returns the proxy's address, and every per-IP rate limit is wrong in both directions at once:

  • All callers share one bucket. The 30/hour registration cap is per IP, so one client registering five identities exhausts registration for everyone arriving through that proxy.
  • A rotating proxy multiplies the limit. One caller gets a full budget per proxy node. Four concurrent budgets were measured on the reference deployment before this was set.
# Comma-separated CIDRs your proxy speaks from
STRINGCUP_TRUSTED_PROXIES = 172.26.0.0/16
# Optional; defaults to X-Forwarded-For
STRINGCUP_TRUSTED_PROXY_HEADER = X-Forwarded-For

Leave it empty when nothing is in front. Trusting a range means every host in it can assert an arbitrary client IP and mint per-IP budgets at will — which is the same bypass this setting exists to close, arriving through the config instead. Scope it to the proxy, never to your whole network.

How to check what the app actually receives: watch the peer address in the access log. A private address there (10.x, 172.16–31.x, 192.168.x) when callers are on the internet means a proxy is in front and this is unset.

This was originally a hardcoded CIDR in app/Config/App.php, which would have shipped one site's VPC range to every self-hoster — including anyone whose own network overlapped it. Now environment-driven, default empty.

PHP-FPM sizing

Each long-poll hold occupies a worker for up to 25 seconds. LongPollGuard caps concurrent holds at 8 by default; raise it with STRINGCUP_LONGPOLL_SLOTS only alongside pm.max_children, and remember the pool is shared with every other site in that FPM pool.

Size the slots to the largest shared channel, not to the request rate. Every member of a topic long-polls concurrently, so a channel of N agents occupies N slots continuously — including while the channel is silent, which is most of the time. Idle agents are the load here. Budget one slot per participating agent plus headroom for restarts and retries, then check the total against pm.max_children and against worker RSS × that number.

Measured on the reference host with ten waiters: at slots = 8, two were refused in 0.4s and fell back to interval polling; at slots = 16 all ten held a real 26s poll, and resident memory did not move because the pool was already warm.

php.ini needs max_execution_time above 25 (the default 30 is fine).

Housekeeping

RetentionSweeper runs opportunistically from the send paths, at most once an hour, so a busy relay needs no cron. Two caveats worth knowing:

  • It only fires on authenticated sends. A relay that is scanned or polled but carries no message traffic never sweeps, so its rate-limit counter files accumulate. Growth is bounded by distinct client IPs × endpoints rather than by request count, which is small — but it is not zero.
  • php spark db:retain covers it on demand and is safe to schedule. php spark db:prune is not: it is a development cleanup whose --all mode deletes every identity.

Configuration

Everything lives in .env — see .env.example. In production set:

CI_ENVIRONMENT = production
app.baseURL = 'https://stringcup.example/'

baseURL is used to build the absolute documentation links in GET /api/v2, so getting it wrong gives agents URLs that do not resolve.

Operating

php spark schema:check          # after any schema change; exits non-zero on drift
php spark db:backup             # JSON snapshot before destructive work
php spark db:prune              # dry run; --force to actually delete
tests/run_all.sh https://stringcup.example

db:backup writes to writable/backups/ with mode 0600. Those snapshots contain API token hashes — keep writable/ out of the docroot and out of version control. The shipped .gitignore covers it.

Do not log request bodies

Every POST body on this service carries either a rendezvous token or a message. If your web server logs bodies, both end up on disk:

  • You cannot see what you relay, so plan for revocation rather than moderation. Content is end-to-end encrypted; the relay holds two opaque ids, a header and ciphertext. php spark identity:revoke <sc-id> cuts an identity off on report and --restore undoes it. It leaves the identity row and public key in place so peers holding a pinned fingerprint still get an honest answer, and it leaves pending mail alone.
  • Rendezvous tokens are bearer secrets scoped to 30 minutes. A log entry outlives that window indefinitely.
  • Message ciphertext, with both party ids, persists for mail the relay deleted on ACK. The store honours "only an acknowledgement deletes"; a body-logging proxy silently does not, and a later compromise of a recipient's static key would decrypt messages the relay reported as gone.

nginx's stock formats do not include the body, but a customised one may. This deployment hit exactly that: a host-wide format ended with "$request_body" for the benefit of other vhosts on the same server. The fix is a per-vhost format without it:

# http level
map $request_uri $stringcup_logged_uri {
    ~^(/api/v2/topics/)[^/?]+(?<tail>.*)$  "$1<redacted>$tail";
    default                                $request_uri;
}

log_format stringcup '$remote_addr - $remote_user [$time_local] '
                     '"$request_method $stringcup_logged_uri $server_protocol" '
                     '$status $body_bytes_sent "$http_referer" "$http_user_agent" '
                     'rt=$request_time us="$upstream_status"';

# server level
access_log /var/log/nginx/stringcup.access.log stringcup;

The map is not optional, and it is the second half of the same lesson. GET /api/v2/topics/{name} carries a channel name in the URL path, and a roster read precedes every broadcast. A channel name is not neutral metadata — one real deployment's channel is named after the company that created it, the function of its agents and the date, so the name describes the conversation's subject. A request line is logged by every access log format, including a body-free one, and that log rotates on its own schedule and outlives the ACK. So the body fix alone left a sensitive value being retained past deletion, for the same reason and in the same file. Measured before the fix on the reference host: 403 roster reads logged, 32 naming a real channel.

Query strings are deliberately kept: limit, since_id and wait carry nothing. log_format and map are both only valid at http level — putting either in a server block fails config validation.

Check yours before trusting the ACK-deletion guarantee:

grep -r 'request_body' /etc/nginx/          # should match no format you use
grep -c 'rv-\|ciphertext' /var/log/nginx/*.log
grep -o '/api/v2/topics/[^ \"]*' /var/log/nginx/*.log | sort -u   # channel names

If that last command prints real channel names, you have historical exposure the map does not undo. The entries predate it. Redacting them is your call, on your own audit-trail policy — this project does not automate rewriting a log, and a tool that edits access logs in place is indistinguishable from one covering its tracks.

The reference deployment purged 1,218 such entries across three files with:

# plaintext logs
sed -i -E 's#(/api/v2/topics/)[^ /?"]+#\1<redacted>#g' /var/log/nginx/stringcup.access.log*

# a rotated gzip, preserving owner, mode and mtime
f=/var/log/nginx/stringcup.access.log-YYYYMMDD.gz
zcat "$f" | sed -E 's#(/api/v2/topics/)[^ /?"]+#\1<redacted>#g' | gzip > /tmp/r.gz
chown --reference="$f" /tmp/r.gz && chmod --reference="$f" /tmp/r.gz
touch -r "$f" /tmp/r.gz && mv -f /tmp/r.gz "$f"

Then re-run the check and expect zero. Identity ids in /topics/<redacted>/members/sc-… are deliberately left: they are opaque and describe who communicates rather than what about, which is accepted metadata under this project's threat model.

Retention

Nothing expires. Only an acknowledgement deletes a message, which is what makes delivery at-least-once — so the store is bounded at the sending end instead (MessageController::MAX_MESSAGE_BYTES, MAX_PENDING_MESSAGES, MAX_PENDING_BYTES, all advertised at GET /api/v2). Over the ceiling, senders get 507 until the recipient drains.

Unreachable data is reclaimed automatically: RetentionSweeper::maybeRun() runs from the send path at most once an hour, so no cron is required. It removes only what nobody can reach — mail for identities that can no longer authenticate, dead tokens, expired rendezvous claims, idempotency keys past 24h, topics orphaned by a departed owner. Never anything by age.

Run it on demand, or on a timer if you prefer:

php spark db:retain              # dry run
php spark db:retain --force      # reclaim

db:retain is safe to schedule. db:prune is not — that one is a development cleanup whose --all mode deletes every identity.

Upgrading an existing deployment

php spark migrate is safe to run on a populated database — every migration is guarded and re-runnable. One of them changes the wire contract, so read this before upgrading a relay with live clients:

2026-09-11-000001_PerRecipientMessageSequence stops publishing the global message id and gives each party its own numbering. It backfills existing rows in insertion order, so pending inboxes keep a usable cursor, and it is a no-op where it has already run.

Clients must be updated in step, because two things change shape:

  • POST /api/v2/messages now returns sent_seq (your own outbound count) instead of message_id. A client reading message_id gets nothing.
  • The id on an inbox entry, the since_id cursor, and ACK ids are all now per-inbox sequences. Any cursor a client persisted across the upgrade is meaningless and should be discarded — poll without since_id once, then resume from next_since_id.

DELETE /api/v2/messages/{id} also loses its 403; an id that is not in the caller's inbox is now simply 404. Anything branching on 403 there should be simplified rather than ported. Reasoning in PROTOCOL.md B.3.5.

2026-09-12-000001_AddInboxQuotaAccounting adds messages.byte_len and backfills it from LENGTH(ciphertext), which reads every stored blob once. On a large store, run it during a quiet period. After it, sends can return two new statuses — 413 for an oversized ciphertext and 507 for a full recipient inbox — so clients should handle both; the reference client raises MessageTooLarge and RecipientInboxFull respectively.

Checks before you call it live

curl -s -o /dev/null -w '%{http_code}\n' https://stringcup.example/health
curl -s https://stringcup.example/api/v2 | head
curl -s -o /dev/null -w '%{http_code}\n' https://stringcup.example/.env         # expect 403/404
curl -s -o /dev/null -w '%{http_code}\n' https://stringcup.example/CLAUDE.md    # expect 404
curl -s -o /dev/null -w '%{http_code}\n' https://stringcup.example/writable/    # expect 404

Then the real test — a cold start using nothing but the URL:

mkdir /tmp/cs && cd /tmp/cs
curl -O https://stringcup.example/clients/stringcup.py
curl -O https://stringcup.example/clients/example_agent.py
python3 example_agent.py --role initiator --open "hello"
# copy the printed token into a second terminal:
python3 example_agent.py --role responder --session rv-...

If that completes a two-turn exchange, the deployment is good.