feat(base-images): add managed base image bundles with cache retention, hosted manifests, and opt-in integration boot testing

2026-05-01 13:30:51 +00:00
parent 0ace928886
commit 9d0a57c5de
19 changed files with 2015 additions and 148 deletions
@@ -14,6 +14,14 @@ pnpm install @push.rocks/smartvm

 > ⚡ **Prerequisites**: Firecracker requires a Linux host with KVM support (`/dev/kvm`). Networking features (TAP devices, bridges, NAT) require root privileges.

+Runtime host requirements:
+
+- Linux with `/dev/kvm` available to the running process
+- A Firecracker binary downloaded by `ensureBinary()` or supplied through `firecrackerBinaryPath`
+- Root privileges for automatic bridge, TAP, IP forwarding, and iptables NAT setup
+- Host tools available for networking: `ip`, `sysctl`, and `iptables`
+- IPv4 CIDR subnets with prefix length `1-30`; the bridge uses the first usable address as gateway and guests start at the second usable address
+
 ## Quick Start

 ```typescript
@@ -118,6 +126,8 @@ const smartvm = new SmartVM({
  firecrackerVersion: 'v1.7.0',  // default: latest from GitHub
  arch: 'x86_64',                // default: x86_64 (also: aarch64)
  firecrackerBinaryPath: '/usr/bin/firecracker', // optional: skip download
+  baseImageCacheDir: '/tmp/.smartvm/base-images', // default: /tmp/.smartvm/base-images
+  maxStoredBaseImages: 2,       // default: keep at most 2 cached base image bundles
  bridgeName: 'svbr0',           // default: svbr0
  subnet: '172.30.0.0/24',       // default: 172.30.0.0/24
 });
@@ -126,6 +136,7 @@ const smartvm = new SmartVM({
 | Method | Description |
 |---|---|
 | `ensureBinary()` | Downloads Firecracker from GitHub if not cached. Returns path to binary. |
+| `ensureBaseImage(options)` | Downloads/caches a Firecracker CI base image bundle. Defaults to the `latest` preset. |
 | `createVM(config)` | Creates a `MicroVM` instance (not yet started). Returns the VM. |
 | `getVM(id)` | Look up an active VM by ID. |
 | `listVMs()` | Returns an array of active VM IDs. |
@@ -249,6 +260,57 @@ const clonePath = await imageManager.cloneRootfs(rootfsPath, 'ubuntu-clone.ext4'
  sockets/<vmId>.sock
 ```

+### `BaseImageManager` — Base Images
+
+Downloads known base image bundles into a `/tmp` cache for integration tests and quick local smoke tests. The default preset is `latest`; `lts` maps to a pinned Firecracker CI train (`v1.7`) for a stable fallback. Hosted project-owned manifests are also supported for pinned Alpine/BusyBox-style bundles.
+
+```typescript
+const baseImage = await smartvm.ensureBaseImage(); // same as { preset: 'latest' }
+
+const ltsBaseImage = await smartvm.ensureBaseImage({ preset: 'lts' });
+
+const hostedBaseImage = await smartvm.ensureBaseImage({
+  manifestUrl: 'https://assets.example.com/push.rocks/smartvm/base-images/smartvm-minimal-v1/x86_64/manifest.json',
+});
+
+const vm = await smartvm.createVM({
+  bootSource: {
+    kernelImagePath: baseImage.kernelImagePath,
+    bootArgs: baseImage.bootArgs,
+  },
+  machineConfig: { vcpuCount: 1, memSizeMib: 256 },
+  drives: [
+    {
+      driveId: 'rootfs',
+      pathOnHost: baseImage.rootfsPath,
+      isRootDevice: true,
+      isReadOnly: baseImage.rootfsIsReadOnly,
+    },
+  ],
+});
+```
+
+**Cache behavior:**
+
+- Default cache directory: `/tmp/.smartvm/base-images`
+- Default retention: at most `2` base image bundles
+- Configure retention with `maxStoredBaseImages`
+- Configure location with `baseImageCacheDir`
+- When a new download causes the retention limit to be exceeded, older bundles are removed and a console warning is emitted
+- Downloaded bundles include a local `manifest.json` with source URLs/keys, file paths, sizes, and computed SHA256 hashes
+
+Example configuration:
+
+```typescript
+const smartvm = new SmartVM({
+  baseImageCacheDir: '/tmp/.smartvm/base-images',
+  maxStoredBaseImages: 4,
+  baseImageManifestUrl: 'https://assets.example.com/push.rocks/smartvm/base-images/smartvm-minimal-v1/x86_64/manifest.json',
+});
+```
+
+Hosted manifest format examples live in `assets/base-images/`. Hosted URL artifacts require SHA256 hashes; `smartvm` verifies them during download before returning the bundle paths.
+
 ### `NetworkManager` — Host Networking

 Automatically manages TAP devices, a Linux bridge, and iptables NAT masquerade rules so VMs get internet access out of the box.
@@ -273,9 +335,10 @@ const bootArgs = networkManager.getGuestNetworkBootArgs(tap);
 ```

 **Networking architecture:**
- Creates a Linux bridge (default: `svbr0`) with gateway at `.1`
+- Creates a Linux bridge (default: `svbr0`) with gateway at the first usable subnet address
 - Each VM gets a TAP device attached to the bridge
- Sequential IP allocation from `.2` onwards
+- Sequential IP allocation from the second usable subnet address onwards
+- Subnet input is normalized to the network address and allocation fails with `IP_EXHAUSTED` when no guest addresses remain
 - iptables NAT masquerade for outbound internet
 - Deterministic MAC generation (`02:xx:xx:xx:xx:xx` locally-administered)
 - TAP names fit Linux's 15-char IFNAMSIZ limit
@@ -352,6 +415,14 @@ try {
 | `BINARY_NOT_FOUND` | Firecracker binary not at expected path |
 | `DOWNLOAD_FAILED` | Failed to download binary/kernel/rootfs |
 | `VERSION_FETCH_FAILED` | Couldn't query GitHub for latest version |
+| `BASE_IMAGE_RESOLVE_FAILED` | Failed to resolve Firecracker CI base image artifacts |
+| `BASE_IMAGE_MANIFEST_FAILED` | Failed to load or use a hosted base image manifest |
+| `BASE_IMAGE_PREPARE_FAILED` | Failed to download or prepare a base image bundle |
+| `INVALID_BASE_IMAGE_MANIFEST` | Hosted base image manifest is invalid |
+| `INVALID_BASE_IMAGE_CACHE_LIMIT` | Base image cache retention limit is invalid |
+| `INVALID_SUBNET` | Subnet is not a supported IPv4 CIDR range |
+| `INVALID_INTERFACE_NAME` | Bridge or TAP interface name is invalid |
+| `IP_EXHAUSTED` | No guest IP addresses remain in the configured subnet |
 | `BRIDGE_SETUP_FAILED` | Failed to create network bridge |
 | `TAP_CREATE_FAILED` | Failed to create TAP device |
 | `ROOTFS_CREATE_FAILED` | Failed to create blank rootfs |
@@ -434,6 +505,36 @@ await smartvm.stopAll();
 await smartvm.cleanup();
 ```

+## Testing
+
+The default test suite is unit-level and safe to run without KVM or root privileges:
+
+```bash
+pnpm test
+pnpm run build
+```
+
+These tests cover config validation, Firecracker payload generation, lifecycle guard errors, VM tracking, and subnet/IP allocation. They do not boot a real microVM.
+
+Real Firecracker boot testing should be run on a Linux/KVM host with the runtime requirements above. At minimum, verify `ensureBinary()`, `createVM()`, `start()`, `getInfo()`, `stop()`, and `cleanup()` against a known-good kernel and rootfs image before relying on a new host setup.
+
+An opt-in integration test scaffold is included and skipped by default:
+
+```bash
+SMARTVM_RUN_INTEGRATION=true pnpm test
+```
+
+Useful integration-test environment variables:
+
+- `SMARTVM_BASE_IMAGE_PRESET`: `latest` or `lts` (default: `latest`)
+- `SMARTVM_BASE_IMAGE_MANIFEST_URL`: use a hosted/project-owned base image manifest instead of a preset
+- `SMARTVM_BASE_IMAGE_MANIFEST_PATH`: use a local base image manifest instead of a preset
+- `SMARTVM_BASE_IMAGE_CACHE_DIR`: override `/tmp/.smartvm/base-images`
+- `SMARTVM_MAX_STORED_BASE_IMAGES`: override the default retention limit of `2`
+- `SMARTVM_FIRECRACKER_VERSION`: override the Firecracker binary version; otherwise the base image's recommended version is used
+- `SMARTVM_ARCH`: `x86_64` or `aarch64`; defaults from the host architecture
+- `SMARTVM_INTEGRATION_DATA_DIR`: override the Firecracker binary/socket data directory
+
 ## TypeScript Interfaces

 All configuration interfaces are fully exported for type-safe usage:
@@ -451,6 +552,11 @@ import type {
  IMmdsConfig,
  ILoggerConfig,
  IMetricsConfig,
+  IBaseImageManagerOptions,
+  IEnsureBaseImageOptions,
+  IBaseImageBundle,
+  IBaseImageHostedManifest,
+  IBaseImageArtifactManifest,
  ISnapshotCreateParams,
  ISnapshotLoadParams,
  IRateLimiter,
@@ -463,6 +569,8 @@ import type {
  TCacheType,
  TSnapshotType,
  TLogLevel,
+  TBaseImagePreset,
+  TBaseImageRootfsType,
 } from '@push.rocks/smartvm';
 ```