social.io system testing

This standalone repository qualifies the complete social.io deployment boundary. It follows the sibling serve.zone/testing model: Vagrant creates an isolated Ubuntu guest, the whole social.io workspace is copied into it with one-way rsync, and independently runnable scenarios exercise real services.

Phase 1 is one production-mode vertical slice. It proves TLS/authenticated MongoDB transactions, HTTPS object storage, clamd, provider IMAP/JMAP, transactional/provider SMTP capture, the first-party JMAP/IMAP surfaces, real browser login, and deterministic desktop/iPad/iPhone screenshots. The broader recovery, backup/restore, failure-injection, performance, accessibility, and full journey matrix is tracked in ../readme.plan.md and belongs to test:full as it lands.

Isolation model

• The app and deterministic mail fixture are supervised guest processes.
• MongoDB, MinIO, clamd, and Caddy run in a private Docker network.
• Only HTTPS on guest port 8443 and first-party IMAPS on guest port 1993 are forwarded to host loopback.
• Generated secrets and PKI stay under ignored infra/runtime/ inside the VM.
• Guest-to-host artifacts are copied explicitly; rsync never returns secrets.

Commands

pnpm install --frozen-lockfile
pnpm check
pnpm test:unit
pnpm vagrant:up
pnpm vagrant:test
pnpm vagrant:screenshots
pnpm vagrant:pull
pnpm vagrant:destroy

The default Vagrant provider is libvirt. Override box/provider resources with SOCIALIO_VAGRANT_BOX, SOCIALIO_VAGRANT_BOX_VERSION, SOCIALIO_VAGRANT_CPUS, and SOCIALIO_VAGRANT_MEMORY. Phase 1 is pinned to amd64 because the qualified ClamAV image is amd64-only; an arm64 matrix remains a roadmap item rather than silently skipping scanning.

pnpm vagrant:test and pnpm vagrant:screenshots run scripts/run-qualification.ts on the host: it rsyncs the workspace into the guest itself, so no separate pnpm vagrant:sync is needed before a qualification.

Qualification attribution

The guest has no .git directory — the Vagrant rsync excludes it — so a capture cannot resolve its own provenance. scripts/run-qualification.ts resolves the commit and a dirty flag for the app, landing, and testing repositories on the host and hands them to the guest run in SOCIALIO_SOURCE_REVISIONS (base64 JSON). The screenshot scenario refuses to start when that value is missing or malformed instead of recording uncommitted.

Nothing is written to disk, so no earlier attribution can be reused by a later run, and the sync happens before the revisions are read: an edit that races the launcher can only make a recorded revision dirtier than what the guest received, never cleaner. Setting the variable by hand for an in-guest capture is possible, but then the manifest records your claim rather than the host's measurement.

The label follows from those revisions (deriveQualificationLabel in helpers/source-revisions.ts): phase1-qualified when every repository is a clean commit, phase1-candidate otherwise. pnpm screenshots:verify rejects a manifest whose label disagrees with its own revisions, and pnpm screenshots:promote refuses anything but a phase1-qualified manifest.

Screenshot boundary

Raw candidates live in .playwright-mcp/landing/device-matrix/. Capture and tests never modify ../landingpage. Promotion is a separate dry-run-first command:

pnpm screenshots:verify
pnpm screenshots:promote --landing ../landingpage
# Applying changed existing files additionally requires --apply and one
# --overwrite <allowlisted-file.png> argument for every changed file.
# If a prior qualified promotion left only unstaged tracked screenshot
# modifications, add --allow-dirty-screenshots. Any staged, untracked, or
# non-screenshot change still blocks promotion.

Promotion validates real paths, the 34-name candidate allowlist, per-profile dimensions (1920x1080 desktop, 1180x820 iPad, 402x874 iPhone), manifest hashes, the phase1-qualified label, and a clean or explicitly screenshot-only landing worktree. Every overwrite must be explicitly named. It never commits or pushes.

Determinism contract

A capture is deterministic in what its manifest verifies, not byte for byte. Every run fixes the dataset (screenshots-v2), locale en-US, timezone Europe/Berlin, the browser clock, and the three device profiles, and records a SHA-256 per image; pnpm screenshots:verify re-checks that manifest against the files on disk, so a capture can never drift from its own record.

Cross-run byte equality is neither guaranteed nor required. Between the two clean-volume runs of 2026-09-21, 25 of 34 PNGs differed by roughly one byte each while all 34 verified against their own manifest. The per-run seed data differs (output/seed-manifest.json), which is the most likely source; the residual difference has not been investigated.

Static validation proves only the scaffold. The 2026-07-16 clean-volume VM run, browser journey, repeated protocol checks, and 34-image desktop/iPad/iPhone device matrix passed, including per-profile responsive-shell assertions and host-side hash verification.

That candidate run was not reproducible. The guest ran Deno 2.7.11, whose node:tls server is not a net.Server: it accepts only listen(port, callback) and never emits listening, so the first-party IMAP listener could not start — start(port, host) threw callback?.call is not a function and start(port) hung forever. The run only completed because the guest's installed @push.rocks/smartimap was hotpatched by hand, and that patch was never kept. Deno ported Node's _tls_wrap in 2.9, which fixes the listener without any application or library change, so scripts/provision-vm.sh pins that runtime and no hotpatch is needed any more.

Both failure modes were reproduced on 2026-09-21 against the unpatched @push.rocks/smartimap@2.1.0 on Deno 2.7.11, and both bind forms were confirmed working on Deno 2.9.4 (the app repo carries the probe and a permanent spec).

The pinned 2.9.7 was then qualified twice from clean volumes on 2026-09-21, with no guest hotpatch: app a8d746b, testing 8ef8eb9, installed @push.rocks/smartimap@2.1.0 with dist_ts/classes.imapserver.js at sha256 1be5af98a6fcf856db02a52dde5e717c7deb19764e7916b9d950e972ef267789 and its server.listen(port, host) line unmodified. Both runs passed all ten stack-health checks including the first-party IMAP and JMAP surfaces, the phase-1 browser journey, and 34/34 screenshot verification.

Those two runs predate host-side attribution, so their manifests still read phase1-candidate. A manifest now earns phase1-qualified from its recorded revisions alone — every repository on a clean commit — which is also what promotion requires.