modelgrid/readme.ui.md

# 🖥️ ModelGrid — UI Concept

**A browser-based operations console for ModelGrid, served by the same daemon that
already exposes the OpenAI-compatible API.**

This document sketches the user interface that will sit on top of the ModelGrid
daemon: what it shows, how it is organized, how an operator moves through it,
and how it stays in sync with a running node or a small cluster. It is a
concept, not a final spec — the goal is to lock the shape of the product
before any frontend code is written.

The structural idioms (tabbed top-level views, route-origin awareness,
embedded ops dashboard on a dedicated port, API-first with a thin UI on top)
are adapted from `@serve.zone/dcrouter`'s Ops dashboard. ModelGrid's UI should
feel familiar to anyone who has operated dcrouter, while staying grounded in
ModelGrid's own domain: GPUs, vLLM deployments, a public model catalog, and a
cluster of gateway-capable nodes.

## 🎯 Purpose & Audience

- **Primary user:** the operator of one or a few ModelGrid nodes. Often the
  same person who provisioned the GPU host and ran `modelgrid service enable`.
- **Secondary user:** a platform engineer wiring ModelGrid into an internal
  AI platform who needs to manage API keys, audit deployments, and watch
  request traffic.
- **Not an end-user chat UI.** Consumers of the OpenAI-compatible API keep
  using their own SDKs and tools. The browser UI is for operating the fleet,
  not for prompting models.

The UI should collapse gracefully from a full cluster view down to a
single-node, standalone deployment, because both shapes are first-class in
ModelGrid's `cluster.role` model (`standalone` / `control-plane` / `worker`).

## 🧭 Top-Level Information Architecture

URLs follow `/{view}` for flat views and `/{view}/{subview}` for tabbed
views, matching dcrouter's routing idiom.

```
/overview
  /stats
  /configuration

/cluster
  /nodes
  /placements
  /desired

/gpus
  /devices
  /drivers

/deployments
  /active
  /history

/models
  /catalog
  /deployed

/access
  /apikeys
  /clients

/logs                  (flat)
/metrics               (flat)
/settings              (flat)
```

Rationale for the split:

- **Overview** is the landing page — one screen that answers "is the fleet
  healthy right now?"
- **Cluster / GPUs / Deployments / Models** are the four nouns an operator
  actually reasons about when running ModelGrid. Keeping them at the top
  level matches the CLI verbs (`modelgrid cluster`, `modelgrid gpu`,
  `modelgrid container`, `modelgrid model`) so muscle memory transfers.
- **Access** consolidates the authn/authz surface (API keys today,
  user/OIDC later) into one place, the way dcrouter groups `apitokens` and
  `users` under `access`.
- **Logs** and **Metrics** are flat because they are cross-cutting streams,
  not noun-scoped tabs.

The navigation chrome itself is a persistent left rail on desktop, collapsing
into a top hamburger on narrow viewports. The selected view is indicated
there; subviews surface as a tab strip at the top of the content area.

```
┌────────────┬──────────────────────────────────────────────────────────────┐
│  ModelGrid │  Overview ▸ Stats  Configuration                             │
│            ├──────────────────────────────────────────────────────────────┤
│  Overview ●│                                                              │
│  Cluster   │   ┌─ Fleet Health ─────────────────────────────────────┐     │
│  GPUs      │   │  2 nodes  •  3 GPUs  •  4 deployments  •  api OK   │     │
│  Deploys   │   └───────────────────────────────────────────────────┘     │
│  Models    │   ┌─ Live Traffic ──────────────┐ ┌─ GPU Utilization ─┐     │
│  Access    │   │  42 req/s   p95 820 ms      │ │  ▁▂▄▅▇█▇▅▄▂▁      │     │
│            │   │  ▁▂▃▅▇▇▅▃▂▁▁▂▄▆             │ │  avg 64%          │     │
│  Logs      │   └─────────────────────────────┘ └───────────────────┘     │
│  Metrics   │   ┌─ Deployments ────────────────────────────────────┐      │
│  Settings  │   │  llama-3.1-8b      running    2/2  nvidia-0,1    │      │
│            │   │  qwen2.5-7b        running    1/1  nvidia-2      │      │
│ node: ctrl │   │  bge-m3            pending    0/1  (no capacity) │      │
│ v1.1.0     │   └──────────────────────────────────────────────────┘      │
└────────────┴──────────────────────────────────────────────────────────────┘
```

The footer of the rail surfaces the local node's identity (`nodeName`,
`role`), the daemon version, and a small link to the API base URL —
equivalent to how dcrouter surfaces its runtime identity in the sidebar.

## 📄 Per-View Sketches

### Overview ▸ Stats (landing page)

A dashboard of the things that an on-call operator wants to see in under
two seconds:

- **Fleet health band**: green/yellow/red status tiles for nodes, GPUs,
  deployments, API.
- **Live traffic**: requests/sec, p50/p95/p99 latency, error rate. Sparkline
  for the last 15 minutes, streaming from `/metrics` and a server-pushed
  channel.
- **GPU utilization strip**: one micro-sparkline per GPU, colored by VRAM
  pressure.
- **Deployment summary**: the `modelgrid ps` output, but clickable. Each
  row deep-links into Deployments ▸ Active.
- **Catalog drift**: a small callout when `list.modelgrid.com` has newer
  model entries than the node's cached catalog.

### Overview ▸ Configuration

A read-only rendering of the resolved `/etc/modelgrid/config.json` with
section headers (`api`, `docker`, `gpus`, `models`, `cluster`). Operators
can copy the JSON; editing config is intentionally kept to the Settings view
(or the CLI) to avoid a "two sources of truth" problem.

### Cluster ▸ Nodes

Mirrors `modelgrid cluster nodes`. Each row: node name, role badge
(`standalone` / `control-plane` / `worker`), advertised URL, last heartbeat,
GPU inventory summary, status (`active` / `cordoned` / `draining`).

Row actions: `cordon`, `drain`, `activate` — the same verbs as the CLI.
Hitting an action fires the corresponding control-plane call and shows an
in-row toast on success.

```
┌ Nodes ───────────────────────────────────────────────────────────────────┐
│  Name          Role            Advertised URL              Heartbeat     │
│  ──────────────────────────────────────────────────────────────────────  │
│  control-a     control-plane   http://ctrl.internal:8080   2s ago    ●   │
│  worker-a      worker          http://wa.internal:8080     3s ago    ●   │
│  worker-b      worker          http://wb.internal:8080     41s ago   ◐   │
│                                                            [cordon] [drain]
└──────────────────────────────────────────────────────────────────────────┘
```

### Cluster ▸ Placements

A live map of where every deployed model is currently running, read from
the control-plane's placement state. Grouped by model, with a column per
node. Cells show replica count and health. This is where the operator
answers "where did `llama-3.1-8b` actually end up?".

### Cluster ▸ Desired

The companion to Placements: the desired-state table. Each row is a model
with a target replica count. Rows can be added (`cluster ensure`), edited
(`cluster scale`), or removed (`cluster clear`). The reconciler's pending
work is surfaced as a diff badge: e.g. `+1 replica`, `moving from worker-b
→ worker-a`.

### GPUs ▸ Devices

Mirrors `modelgrid gpu list` / `gpu status`, rendered as a card per GPU:
vendor, model, VRAM free/total, driver version, temperature, current
utilization, and which deployment is using it. Cards stream their
utilization via the realtime channel; no full page reloads.

### GPUs ▸ Drivers

Status per vendor (NVIDIA / AMD / Intel): driver installed? version? any
known issue? Includes a button to run `modelgrid gpu install`
interactively — but since the install flow is privileged and interactive,
the UI only kicks off the CLI walk-through in a terminal session rather
than trying to reimplement it in the browser. A small "copy the command"
affordance makes this explicit.

### Deployments ▸ Active

The core operational table. One row per active vLLM deployment:

- container ID, display name, model, GPU bindings, port, uptime, request
  rate, error rate
- status pill (`running`, `pending`, `restarting`, `failed`)
- row actions: `logs`, `stop`, `restart`, `remove`

Clicking a row opens a detail drawer with sub-tabs:

- **Summary** — the effective container config and the scheduling
  decision that landed it on this node
- **Logs** — a live tail (SSE)
- **Metrics** — request latency histogram, token throughput, VRAM
  occupancy
- **Events** — a timeline of lifecycle events (scheduled, pulled image,
  started, health check, restart, stopped)

### Deployments ▸ History

Deployments that have been stopped or removed, with the reason and the
last-known logs. Useful for post-mortem on a failed deploy.

### Models ▸ Catalog

The current catalog resolved from `list.modelgrid.com`, with a "refresh"
action that calls `modelgrid model refresh`. Each entry shows canonical
ID, aliases, capabilities (chat / completions / embeddings), minimum
VRAM, default GPU count, and a `Deploy` button. Deploying opens a small
form that mirrors `modelgrid run`: target node (or auto), desired replica
count, optional env overrides (e.g. `HF_TOKEN`).

A visible "source" badge marks whether the entry came from the public
catalog or a custom `registryUrl`, so operators can tell at a glance which
models the cluster will actually trust for auto-deploy.

### Models ▸ Deployed

Shows the union of what is running across the cluster, with replica
counts, keyed by canonical model ID. This is the view a developer asks
the operator for when they want to know "what models can I hit on this
endpoint?". It is effectively a pretty rendering of `/v1/models`.

### Access ▸ API Keys

Mirrors `modelgrid config apikey list`. Columns: label, prefix (first
8 chars), created, last used, status. Actions: `generate`, `revoke`.
Generating a key shows the secret once in a modal with a copy button,
then never shows it again — the same contract as dcrouter's API tokens.

### Access ▸ Clients

Placeholder for per-consumer rate limits, quotas, and request labels.
This view is explicitly future work; it renders as "not yet configured"
until the daemon exposes client records. Listing it now reserves the IA
slot so it doesn't have to be retrofitted later.

### Logs

A unified tail across daemon, scheduler, and deployments, with filters
by source (`daemon`, `scheduler`, `deployment:<id>`), level, and
free-text. Streamed via SSE. A "pause" toggle freezes the view for
reading; a "download" action exports the current buffer as NDJSON.

### Metrics

The `/metrics` endpoint rendered as a small set of charts (request rate,
latency, error rate, VRAM occupancy, model throughput). This is
deliberately lightweight — serious monitoring is expected to come from
Prometheus scraping `/metrics` into Grafana, and the UI says so with a
link to the recommended dashboard snippet.

### Settings

Editable configuration, grouped to match the config file:

- **API** — port, bind host, CORS, rate limit
- **Docker** — runtime, network name, socket path
- **GPUs** — auto-detect toggle, per-GPU assignments
- **Models** — registry URL, auto-deploy, default engine, auto-load list
- **Cluster** — role, advertise URL, control-plane URL, shared secret,
  heartbeat interval, seeds

Edits write through the daemon's config API (to be defined) and reload
without a restart wherever possible. Settings that require a restart are
marked with a `restart required` badge, and the UI surfaces a single
"restart daemon" action at the top of the view when any are pending.

## 🛤️ Key User Journeys

### Deploy a model from the catalog

1. Operator opens **Models ▸ Catalog**, filters for chat-capable models
   with VRAM ≤ 24 GB.
2. Clicks `Deploy` on `meta-llama/Llama-3.1-8B-Instruct`.
3. Dialog appears with target node (`auto` / specific worker), replica
   count (default from catalog), optional env (`HF_TOKEN`).
4. On submit, the UI calls the control plane (`cluster ensure` + `scale`
   under the hood). The dialog closes and the new row appears in
   **Deployments ▸ Active** in `pending` state.
5. SSE updates walk the row through `pulling image → starting → running`.
6. A toast links to the deployment detail drawer for logs.

### Add a worker node to an existing control plane

1. Operator opens **Cluster ▸ Nodes** on the control plane.
2. Clicks `Add node`, which opens a helper that pre-fills the worker's
   expected `cluster` config block — role, control-plane URL, shared
   secret — and exposes a one-liner install command.
3. The operator runs the install command on the worker host. The UI does
   **not** SSH into anything; it just hands out the exact snippet.
4. Once the worker's daemon starts and registers, the new node appears
   in the Nodes table with its first heartbeat. The helper closes
   automatically.

### Rotate an API key

1. **Access ▸ API Keys** → `Generate`.
2. Name the key, pick a scope (today: single scope; later: per-model).
3. The secret is shown once in a modal; copy-to-clipboard and a clear
   "you will not see this again" note.
4. Old key row gets a `revoke` action. Revoke is a confirm-then-apply
   flow because it will break live traffic.

### Investigate a failing deployment

1. **Overview ▸ Stats** shows a red tile: `1 deployment failed`.
2. Click drills into **Deployments ▸ Active**, filtered to `failed`.
3. Open the row drawer → **Events** tab to see the lifecycle timeline.
4. Jump to **Logs** tab for the live tail. If the deployment is down,
   fall back to the last 500 lines from its event buffer.
5. From the drawer, `restart` retries the deployment; if it fails again,
   the `Summary` tab shows the scheduling decision so the operator can
   see whether VRAM, GPU pinning, or image pull is the root cause.

## 📡 Realtime, Auth, and API Contract

- **Realtime updates.** Metrics, logs, GPU utilization, heartbeats, and
  deployment state changes stream over Server-Sent Events. A single
  `/v1/_ui/events?topics=...` endpoint is preferred over per-feature
  sockets so the browser holds exactly one connection. WebSocket is
  reserved for bidirectional features (e.g. an interactive install
  walkthrough) that we do not need in v1.
- **Auth model.** The UI runs behind the same daemon process as the
  OpenAI-compatible API, on a dedicated `uiPort` (default `8081`) to
  keep the data-plane clean. Login uses a session cookie; the first-boot
  bootstrap seeds an `admin` user with a one-time password printed to
  `journalctl -u modelgrid`, the same way dcrouter prints its initial
  `admin`/`admin`. SSO/OIDC is a later add-on.
- **API contract.** Every UI action maps to an HTTP endpoint on the
  daemon (`/v1/_ui/...`). The UI must not talk to any private internals
  directly; this keeps `@modelgrid.com/modelgrid-apiclient` (a future
  sibling to `@serve.zone/dcrouter-apiclient`) able to do everything the
  UI can do, from scripts.
- **Origin badges.** Similar to dcrouter's `config` / `email` / `dns` /
  `api` route-origin model, ModelGrid should tag each deployment with
  its origin: `config` (seeded via `containers` in config.json),
  `catalog` (auto-deployed from `models.autoLoad`), `api` (created via
  UI/API). Origin determines what the UI allows: `config`-origin
  deployments are toggle-only, `api`-origin deployments are full CRUD.

## 🧱 Implementation Notes (non-binding)

- **Web component stack.** Match the dcrouter OpsServer approach:
  component-per-view under `ts_web/elements/<area>/`, a tiny
  SmartRouter-style client router (`ts_web/router.ts`), and a single
  `appstate.ts` as the store.
- **Bundled into the binary via `ts_bundled/bundle.ts`.** ModelGrid is a
  Deno project that ships as a `deno compile` single binary, so the UI
  follows the `@stack.gallery/registry` pattern: a build step bundles
  the `ts_web/` sources (HTML, JS, CSS, fonts, icons) into a single
  generated `ts_bundled/bundle.ts` module that exports a
  `{ path → bytes | string }` map. The daemon dynamically imports that
  module at startup and hands the map to **typedserver**, which serves
  it on the UI port. Result: no external asset directory, no runtime
  filesystem dependency, one binary still ships the entire console.
- **Dev vs prod asset source.** In `deno task dev`, typedserver is
  pointed at `ts_web/` on disk so UI edits are hot-reloadable without
  re-running the bundler. In `deno task compile` / prod, the bundler
  regenerates `ts_bundled/bundle.ts` first and the compiled binary
  serves exclusively from the embedded map. A single flag
  (`UI_ASSET_SOURCE=disk|bundle`, default `bundle`) picks the strategy
  at runtime.
- **Bundler placement.** Mirrors `@stack.gallery/registry`: keep the
  bundler in `scripts/bundle-ui.ts`, invoke it from a `deno task
  bundle:ui` that the `compile:all` task depends on, and `.gitignore`
  the generated `ts_bundled/bundle.ts` so it is only produced during
  release builds (or regenerated on demand for local prod testing).
- **Packaging.** Follow dcrouter's module split: `@modelgrid.com/modelgrid`
  ships the daemon and the embedded UI bundle; a future
  `@modelgrid.com/modelgrid-web` can carve out the web sources as their
  own publishable boundary if the bundle grows large or the UI needs to
  be consumed independently.
- **Dark theme default** (black background, high-contrast foreground) to
  match dcrouter and the expected server-ops environment. Light theme
  is a later toggle.
- **No server-side rendering.** The UI is a static SPA; typedserver
  returns the asset map's `index.html` for the app shell and the rest
  of the state comes from the API. This keeps the runtime surface
  small and makes the UI-less `curl` story identical to the UI story.

## ❓ Open Questions

- **Edit config from the UI or keep it CLI/file-first?** Current lean:
  UI is authoritative only for API keys, deployments, and cluster
  actions. Config editing is exposed but optional, with CLI still the
  canonical path for reproducible installs.
- **Do we expose a model prompt playground?** Nice to have for smoke
  tests, but it blurs the operator/consumer line. Defer to v2.
- **Cluster-wide vs per-node view.** On a worker node, should the UI
  show only local state, or proxy the control plane's cluster view? The
  current lean: workers show local-only, and link to the control plane
  for cluster views. This avoids split-brain confusion.
- **Access control granularity.** API keys today are coarse (all or
  nothing). A future model might scope keys per deployment or per
  model. Reserve the column in the Access ▸ API Keys table now.

## 🛑 Out of Scope (for this concept)

- End-user chat or prompt UIs for the OpenAI-compatible API.
- Billing, quotas, or usage-based pricing dashboards.
- Multi-tenant isolation beyond per-API-key separation.
- Anything specific to non-vLLM runtimes — the UI assumes the v1.1.0
  reorientation around vLLM as the only first-class runtime.