social.io system testing

This standalone repository qualifies the complete social.io deployment boundary. It follows the sibling serve.zone/testing model: Vagrant creates an isolated Ubuntu guest, the social.io source trees are handed to it as checksummed archives through a read-only shared folder, and independently runnable scenarios exercise real services. The guest runs the app on Node.js 24 with the app's pinned pnpm: pnpm install --frozen-lockfile, pnpm build and pnpm test. The app runs as node cli.js server, the command its production image starts.

Phase 1 is one production-mode vertical slice. It proves TLS/authenticated MongoDB transactions, HTTPS object storage, clamd, provider IMAP/JMAP, transactional/provider SMTP capture, the first-party JMAP/IMAP surfaces, real browser login, and deterministic desktop/iPad/iPhone screenshots. The broader recovery, backup/restore, failure-injection, performance, accessibility, and full journey matrix is tracked in ../readme.plan.md and belongs to test:full as it lands.

Isolation model

• The app and deterministic mail fixture are supervised guest processes. The harness starts and signals the node processes themselves, never a pnpm wrapper, so a stop or restart reaches the process that drains.
• MongoDB, the lossless.zone object store, clamd, and Caddy run in a private Docker network; Caddy terminates TLS for the object store.
• Only HTTPS on guest port 8443 and first-party IMAPS on guest port 1993 are forwarded to host loopback.
• Generated secrets and PKI stay under ignored infra/runtime/ inside the VM.
• Guest-to-host artifacts are packed explicitly (screenshots, output/, runtime logs); secrets never leave the guest.
• The object store's readiness is its /readyz on management port 3000, probed with curl from the guest over the compose bridge, so it does not depend on the image's runtime. One S3_REGION value from scripts/generate-runtime.sh reaches the store, the app and the harness.

Package resolution

The guest resolves packages the way the app's production image build does. scripts/bootstrap-guest.sh reads the HT_DOCKER_NODE_IMAGE default from the app's Dockerfile, copies that image's global pnpm configuration (/root/.config/pnpm/config.yaml: minimumReleaseAge 10080 and the owner-scope minimumReleaseAgeExclude list) into the guest user's pnpm configuration, and checks that pnpm reads it. The image sets no registry, so both use the default https://registry.npmjs.org/. Anything else comes from each project's own configuration: this repository's pnpm-workspace.yaml routes @lossless.org and @tempfix to https://verdaccio.lossless.digital/, because @lossless.org/client above 1.22.0 and its @tempfix/mongodb driver are published there only. The app needs the same routes in its own project configuration once it depends on such a client, or its image build and the guest install fail alike.

Source and artifact exchange

The harness uses no remote transfer: vagrant rsync, rsync and sftp are not part of it in any form. The Vagrantfile creates an exchange directory outside the checkouts (default $TMPDIR/socialio-testing-<hash of the checkout path>, override SOCIALIO_VAGRANT_EXCHANGE) and shares two 9p folders with the guest:

• inbox, read-only in the guest: the host writes one tar archive per repository (testing, social.io, landingpage) with the tracked and untracked non-ignored files, plus receive-sources.sh. The guest checks every SHA-256 before it replaces /social.io/<repo>, and keeps its own node_modules, infra/runtime and evidence directories.
• outbox: the guest packs artifacts.tar; pnpm vagrant:pull checks its SHA-256 and mirrors it into .playwright-mcp/, output/ and output/runtime-logs/.

QEMU opens the shares as the libvirt QEMU user (libvirt-qemu, override SOCIALIO_LIBVIRT_QEMU_USER), which cannot traverse private checkouts. The exchange directory is therefore mode 0700 with ACL grants for that user only, and the host needs setfacl.

Commands

pnpm install --frozen-lockfile
pnpm check
pnpm test:unit
pnpm vagrant:up
pnpm vagrant:sync          # optional: refresh the guest sources without a run
pnpm vagrant:test
pnpm vagrant:screenshots
pnpm vagrant:pull
pnpm vagrant:destroy
pnpm screenshots:verify
pnpm screenshots:review-v10

The Vagrant provider is libvirt. There is no other provider, because the 9p share is the only source path. Each checkout gets its own domain (socialio_<checkout directory>_default, override SOCIALIO_VAGRANT_PREFIX), so a linked worktree never reuses or destroys another checkout's VM. Forwarded host ports default to 18443 (HTTPS) and 11993 (IMAPS); set SOCIALIO_VAGRANT_HTTPS_PORT and SOCIALIO_VAGRANT_IMAPS_PORT when another VM holds them. A linked worktree does not sit beside the app and landing repositories, so set SOCIALIO_ROOT to the workspace directory there. Override box/provider resources with SOCIALIO_VAGRANT_BOX, SOCIALIO_VAGRANT_BOX_VERSION, SOCIALIO_VAGRANT_CPUS, and SOCIALIO_VAGRANT_MEMORY. Phase 1 is pinned to amd64 because the qualified ClamAV image is amd64-only; an arm64 matrix remains a roadmap item rather than silently skipping scanning.

pnpm vagrant:test and pnpm vagrant:screenshots run scripts/run-qualification.ts on the host: it publishes the source archives and has the guest receive them itself, so no separate pnpm vagrant:sync is needed before a qualification.

Qualification attribution

The guest has no .git directory — the source archives exclude it — so a capture cannot resolve its own provenance. scripts/run-qualification.ts resolves the commit and a dirty flag for the app, landing, and testing repositories on the host and hands them to the guest run in SOCIALIO_SOURCE_REVISIONS (base64 JSON). The screenshot scenario refuses to start when that value is missing or malformed instead of recording uncommitted.

Nothing is written to disk, so no earlier attribution can be reused by a later run, and the archives are written before the revisions are read: an edit that races the launcher can only make a recorded revision dirtier than what the guest received, never cleaner. Setting the variable by hand for an in-guest capture is possible, but then the manifest records your claim rather than the host's measurement.

The label follows from those revisions (deriveQualificationLabel in helpers/source-revisions.ts): phase1-qualified when every repository is a clean commit, phase1-candidate otherwise. pnpm screenshots:verify rejects a manifest whose label disagrees with its own revisions, and pnpm screenshots:promote refuses anything but a phase1-qualified manifest.

Screenshot boundary

Raw candidates live in .playwright-mcp/landing/device-matrix/. Capture and tests never modify ../landingpage. Promotion is a separate dry-run-first command:

pnpm screenshots:verify
pnpm screenshots:promote --landing ../landingpage
# Applying changed existing files additionally requires --apply and one
# --overwrite <allowlisted-file.png> argument for every changed file.
# If a prior qualified promotion left only unstaged tracked screenshot
# modifications, add --allow-dirty-screenshots. Any staged, untracked, or
# non-screenshot change still blocks promotion.

Promotion validates real paths, the 34-name candidate allowlist, per-profile dimensions (1920x1080 desktop, 1180x820 iPad, 402x874 iPhone), manifest hashes, the phase1-qualified label, and a clean or explicitly screenshot-only landing worktree. Every overwrite must be explicitly named. It never commits or pushes.

v10 design review

pnpm screenshots:review-v10 renders the v10 prototype frame for every captured state from design/social.io/ with Playwright: social.io Desktop.dc.html for the desktop and iPad profiles, social.io iPhone.dc.html for the iPhone. Each frame uses its capture's viewport, device scale factor and theme, through the prototypes' ?theme=…&state=… deep links. The tool writes one sheet per state, with each capture beside its prototype frame, to .nogit/v10-review/sheets/. It also writes the raw frames to .nogit/v10-review/prototype/ and an index.json. It compares nothing itself.

• --captures <dir> reviews another set with the same file names, for example the landing page's assets/.
• --design <dir> uses another design export.

The frame map lives in helpers/v10-prototype.ts. The desktop prototype has no calendar day view, so the desktop and iPad day captures are shown beside its week view. The prototypes load React and Babel from unpkg.com, so the host needs network access.

Determinism contract

A capture is deterministic in what its manifest verifies, not byte for byte. Every run fixes the dataset (screenshots-v2), locale en-US, timezone Europe/Berlin, the browser clock, and the three device profiles, and records a SHA-256 per image; pnpm screenshots:verify re-checks that manifest against the files on disk, so a capture can never drift from its own record.

Cross-run byte equality is neither guaranteed nor required. Between the two clean-volume runs of 2026-09-21, 25 of 34 PNGs differed by roughly one byte each while all 34 verified against their own manifest. The per-run seed data differs (output/seed-manifest.json), which is the most likely source; the residual difference has not been investigated.

Static validation proves only the scaffold. The 2026-07-16 clean-volume VM run, browser journey, repeated protocol checks, and 34-image desktop/iPad/iPhone device matrix passed, including per-profile responsive-shell assertions and host-side hash verification.

That candidate run was not reproducible: it needed a hand hotpatch of the guest's @push.rocks/smartimap under Deno 2.7.11. The app then ran on a pinned Deno 2.9.7, which was qualified twice from clean volumes on 2026-09-21 (app a8d746b, testing 8ef8eb9) with no hotpatch; the git history of this file holds the details. Since app 2.0.0 the app runs on Node.js 24, and the guest is provisioned with Node.js 24.22.0 and the app's pinned pnpm 12.10.1 instead of Deno.

Those two Deno runs predate host-side attribution, so their manifests still read phase1-candidate. A manifest now earns phase1-qualified from its recorded revisions alone — every repository on a clean commit — which is also what promotion requires.