objectstorage/readme.plan.md

# Improvement Plan

## Goal

Make this repository a solid product wrapper for `@push.rocks/smartstorage`: reliable Docker image, predictable runtime behavior, durable admin configuration, and a management surface that scales past toy datasets.

## Current Position

The repository already has a clear structure and decent coverage for auth, buckets, objects, policies, credentials, status/config, lifecycle, and S3 compatibility.

The main gaps are product hardening and dependency boundaries:

- the Docker build still references `npmextra.json` even though the repo migrated to `.smartconfig.json`
- some admin/runtime behavior is still in-memory only
- status and bucket summaries are computed by rescanning storage through S3 calls
- the product surface does not yet expose real cluster and drive health
- the UI error model is still mostly `console.error()` + stale state retention

## Repo-Owned Work

### 1. Fix Packaging Regressions First

- Update `Dockerfile` to stop copying `npmextra.json` and use the current build config layout.
- Add a Docker smoke test that proves the image builds and the container starts with both ports exposed.

Reason:

- This repo is explicitly about shipping a usable container. A broken or stale Docker build path is a product failure, not a nice-to-have cleanup.

### 2. Make Admin Configuration Durable

- Decide which runtime-managed settings must survive restart.
- Credentials are the biggest gap today: add/remove currently mutates runtime config only.
- Align credential durability with the existing named-policy persistence model.
- Add restart-persistence tests.

Reason:

- A product container should not pretend to manage credentials if those changes disappear on restart.

### 3. Tighten Management API Behavior

- Fix bucket/policy cleanup ordering so deleting a bucket does not trigger follow-up work against a missing bucket.
- Return cleaner typed errors from handlers where failure is expected.
- Surface actionable errors in the UI instead of only logging and keeping stale state.

Reason:

- The current tests pass, but post-test output already shows avoidable `NoSuchBucket` noise during policy cleanup.

### 4. Reduce Large-Object and Large-Dataset Pain

- Keep inline editing and preview for small objects.
- Add explicit size thresholds for inline/base64 flows.
- Prefer direct object URLs or streaming paths for large-object download/preview paths.
- Stop treating expensive full-dataset scans as normal refresh behavior.

Reason:

- The product should stay usable once users have real buckets and non-trivial object counts.

### 5. Add Product Acceptance Coverage

- Docker build/start smoke test.
- Restart-persistence test for credentials and policies.
- Cluster-config smoke test.
- One browser-level smoke test for login, bucket browse, and config view.

Reason:

- The repo is productization code. Acceptance coverage matters more here than unit-test density.

## Dependency-Owned Work Completed

The dependency boundary work landed in `@push.rocks/smartstorage` `v6.4.0` and is now consumed here through the Deno import map.

- Runtime storage stats and bucket summaries come from `smartstorage` instead of product-side S3 scans.
- Runtime credential listing/replacement uses supported `smartstorage` APIs instead of mutating engine internals.
- Cluster and drive health are exposed through `smartstorage` and surfaced through the management API/UI.

## Suggested Execution Order

1. Fix Dockerfile and add a container smoke test.
2. Make credential changes durable and test restart behavior.
3. Clean up bucket/policy deletion behavior and UI error reporting.
4. Keep product code on supported `smartstorage` runtime APIs for stats, credentials, and cluster health.
5. Add browser-level smoke coverage for login, bucket browse, and config views.

## Success Criteria

- Docker image builds from a clean checkout.
- Runtime-managed credentials survive restart or are explicitly documented as ephemeral until fixed.
- Status and bucket views no longer require full object scans for routine refreshes.
- Bucket deletion and policy cleanup complete without noisy missing-bucket follow-up errors.
- Cluster mode exposes live health, not just configured values.

## Enterprise Readiness Plan

### Acceptance Criteria Implemented In-Repo

- Runtime health: expose unauthenticated `/livez`, `/readyz`, `/healthz`, and `/metrics` on the management server.
- Container health: Docker `HEALTHCHECK` must use `/readyz` and the image must run as a non-root user.
- Admin auditability: login, logout, bucket mutation, object mutation, credential mutation, and named-policy mutation must emit append-only audit events under `.objectstorage/audit.log`.
- Audit access: admins can query recent audit entries through a typed management API.
- Session security: JWT role is validated from verified claims, logout revokes the active token, and repeated failed logins are rate-limited.
- Secret persistence: admin metadata files are written with restrictive file mode where the OS supports it.
- Default-secret guardrail: persistent `/data` deployments refuse `admin/admin` defaults unless explicitly allowed for disposable development.
- Frontend session storage: admin identity is no longer stored in persistent browser state.

### Dependency Work Implemented In `@push.rocks/smartstorage`

- Cluster identity and topology snapshots persist under `.smartstorage/cluster/`.
- Cluster startup refuses unsafe seed-node fallback instead of silently forming a split-brain cluster.
- Heartbeats probe all known peers, including suspect/offline peers, using configured timeout.
- Operational S3-side endpoints expose `/-/live`, `/-/ready`, `/-/health`, and `/-/metrics`.
- Runtime credential listing returns metadata only; secrets are write-only.
- Multi-node tests cover topology convergence, remote drive routing, and restart/rejoin identity durability.

### Production Requirements Outside This Repo

- Run the management UI behind TLS or add native typedserver TLS configuration for the deployment.
- Configure real admin and S3 credentials through secrets management, not image defaults.
- Decide cluster transport policy: pinned CA or mTLS with operational certificate rotation. Do not operate QUIC insecure-dev transport on untrusted networks.
- Define backup/restore procedures for `/data`, `.objectstorage`, `.smartstorage/cluster`, bucket manifests, and policies.
- Add load, soak, and failure-injection runs to release qualification for large datasets and network partitions.