v2.3.0

changelog.md

@@ -1,5 +1,71 @@
 # Changelog
 
+## 2025-11-27 - 2.3.0 - feat(upstream)
+Add upstream proxy/cache subsystem and integrate per-protocol upstreams
+
+- Introduce a complete upstream subsystem (BaseUpstream, UpstreamCache, CircuitBreaker) with caching, negative-cache, stale-while-revalidate, retries, exponential backoff and per-upstream circuit breakers.
+- Add upstream interfaces and defaults (ts/upstream/interfaces.upstream.ts) and export upstream utilities from ts/upstream/index.ts and root ts/index.ts.
+- Implement protocol-specific upstream clients for npm, pypi, maven, composer, cargo and rubygems (classes.*upstream.ts) to fetch metadata and artifacts from configured upstream registries.
+- Integrate upstream usage into registries: registries now accept an upstream config, attempt to fetch missing metadata/artifacts from upstreams, cache results locally, and expose destroy() to stop upstream resources.
+- Add SmartRequest and minimatch to dependencies and expose smartrequest/minimatch via ts/plugins.ts for HTTP requests and glob-based scope matching.
+- Update package.json to add @push.rocks/smartrequest and minimatch dependencies.
+- Various registry implementations updated to utilize upstreams (npm, pypi, maven, composer, cargo, rubygems, oci) including URL rewrites and caching behavior.
+
+## 2025-11-27 - 2.2.3 - fix(tests)
+Use unique test run IDs and add S3 cleanup in test helpers to avoid cross-run conflicts
+
+- Add generateTestRunId() helper in test/helpers/registry.ts to produce unique IDs for each test run
+- Update PyPI and Composer native CLI tests to use generated testPackageName / unauth-pkg-<id> to avoid package name collisions between runs
+- Import smartbucket and add S3 bucket cleanup logic in test helpers to remove leftover objects between test runs
+- Improve test robustness by skipping upload-dependent checks when tools (twine/composer) are not available and logging outputs for debugging
+
+## 2025-11-25 - 2.2.2 - fix(npm)
+Replace console logging with structured Smartlog in NPM registry and silence RubyGems helper error logging
+
+- Replaced console.log calls with this.logger.log (Smartlog) in ts/npm/classes.npmregistry.ts for debug/info/success events
+- Converted console.error in NpmRegistry.handleSearch to structured logger.log('error', ...) including the error message
+- Removed console.error from ts/rubygems/helpers.rubygems.ts; gem metadata extraction failures are now handled silently by returning null
+
+## 2025-11-25 - 2.2.1 - fix(core)
+Normalize binary data handling across registries and add buffer helpers
+
+- Add core/helpers.buffer.ts with isBinaryData and toBuffer utilities to consistently handle Buffer, Uint8Array, string and object inputs.
+- Composer: accept Uint8Array uploads, convert to Buffer before ZIP extraction, SHA-1 calculation and storage.
+- PyPI: accept multipart file content as Buffer or Uint8Array and normalize to Buffer before processing and storage.
+- Maven: normalize artifact body input with toBuffer before validation and storage.
+- OCI: improve upload id generation by using substring for correct random length.
+
+## 2025-11-25 - 2.2.0 - feat(core/registrystorage)
+Persist OCI manifest content-type in sidecar and normalize manifest body handling
+
+- Add getOciManifestContentType(repository, digest) to read stored manifest Content-Type
+- Store manifest Content-Type in a .type sidecar file when putOciManifest is called
+- Update putOciManifest to persist both manifest data and its content type
+- OciRegistry now retrieves stored content type (with fallback to detectManifestContentType) when serving manifests
+- Add toBuffer helper in OciRegistry to consistently convert various request body forms to Buffer for digest calculation and uploads
+
+## 2025-11-25 - 2.1.2 - fix(oci)
+Prefer raw request body for content-addressable OCI operations and expose rawBody on request context
+
+- Add rawBody?: Buffer to IRequestContext to allow callers to provide the exact raw request bytes for digest calculation (falls back to body if absent).
+- OCI registry handlers now prefer context.rawBody over context.body for content-addressable operations (manifests, blobs, and blob uploads) to preserve exact bytes and ensure digest calculation matches client expectations.
+- Upload flow updates: upload init, PATCH (upload chunk) and PUT (complete upload) now pass rawBody when available.
+
+## 2025-11-25 - 2.1.1 - fix(oci)
+Preserve raw manifest bytes for digest calculation and handle string/JSON manifest bodies in OCI registry
+
+- Preserve the exact bytes of the manifest payload when computing the sha256 digest to comply with the OCI spec and avoid mismatches caused by re-serialization.
+- Accept string request bodies (converted using UTF-8) and treat already-parsed JSON objects by re-serializing as a fallback.
+- Keep existing content-type fallback logic while ensuring accurate digest calculation prior to storing manifests.
+
+## 2025-11-25 - 2.1.0 - feat(oci)
+Support configurable OCI token realm/service and centralize unauthorized responses
+
+- SmartRegistry now forwards optional ociTokens (realm and service) from auth configuration to OciRegistry when OCI is enabled
+- OciRegistry constructor accepts an optional ociTokens parameter and stores it for use in auth headers
+- Replaced repeated construction of WWW-Authenticate headers with createUnauthorizedResponse and createUnauthorizedHeadResponse helpers that use configured realm/service
+- Behavior is backwards-compatible: when ociTokens are not configured the registry falls back to the previous defaults (realm: <basePath>/v2/token, service: "registry")
+
 ## 2025-11-25 - 2.0.0 - BREAKING CHANGE(pypi,rubygems)
 Revise PyPI and RubyGems handling: normalize error payloads, fix .gem parsing/packing, adjust PyPI JSON API and tests, and export smartarchive plugin
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@push.rocks/smartregistry",
-  "version": "2.0.0",
+  "version": "2.3.0",
   "private": false,
   "description": "A composable TypeScript library implementing OCI, NPM, Maven, Cargo, Composer, PyPI, and RubyGems registries for building unified container and package registries",
   "main": "dist_ts/index.js",
@@ -50,8 +50,10 @@
     "@push.rocks/smartbucket": "^4.3.0",
     "@push.rocks/smartlog": "^3.1.10",
     "@push.rocks/smartpath": "^6.0.0",
+    "@push.rocks/smartrequest": "^5.0.1",
     "@tsclass/tsclass": "^9.3.0",
-    "adm-zip": "^0.5.10"
+    "adm-zip": "^0.5.10",
+    "minimatch": "^10.1.1"
   },
   "packageManager": "pnpm@10.18.1+sha512.77a884a165cbba2d8d1c19e3b4880eee6d2fcabd0d879121e282196b80042351d5eb3ca0935fa599da1dc51265cc68816ad2bddd2a2de5ea9fdf92adbec7cd34"
 }

pnpm-lock.yaml

@@ -20,12 +20,18 @@ importers:
       '@push.rocks/smartpath':
         specifier: ^6.0.0
         version: 6.0.0
+      '@push.rocks/smartrequest':
+        specifier: ^5.0.1
+        version: 5.0.1
       '@tsclass/tsclass':
         specifier: ^9.3.0
         version: 9.3.0
       adm-zip:
         specifier: ^0.5.10
         version: 0.5.16
+      minimatch:
+        specifier: ^10.1.1
+        version: 10.1.1
     devDependencies:
       '@git.zone/tsbuild':
         specifier: ^3.1.0

test/helpers/registry.ts

@@ -1,11 +1,63 @@
 import * as qenv from '@push.rocks/qenv';
 import * as crypto from 'crypto';
 import * as smartarchive from '@push.rocks/smartarchive';
+import * as smartbucket from '@push.rocks/smartbucket';
 import { SmartRegistry } from '../../ts/classes.smartregistry.js';
 import type { IRegistryConfig } from '../../ts/core/interfaces.core.js';
 
 const testQenv = new qenv.Qenv('./', './.nogit');
 
+/**
+ * Clean up S3 bucket contents for a fresh test run
+ * @param prefix Optional prefix to delete (e.g., 'cargo/', 'npm/', 'composer/')
+ */
+/**
+ * Generate a unique test run ID for avoiding conflicts between test runs
+ * Uses timestamp + random suffix for uniqueness
+ */
+export function generateTestRunId(): string {
+  const timestamp = Date.now().toString(36);
+  const random = Math.random().toString(36).substring(2, 6);
+  return `${timestamp}${random}`;
+}
+
+export async function cleanupS3Bucket(prefix?: string): Promise<void> {
+  const s3AccessKey = await testQenv.getEnvVarOnDemand('S3_ACCESSKEY');
+  const s3SecretKey = await testQenv.getEnvVarOnDemand('S3_SECRETKEY');
+  const s3Endpoint = await testQenv.getEnvVarOnDemand('S3_ENDPOINT');
+  const s3Port = await testQenv.getEnvVarOnDemand('S3_PORT');
+
+  const s3 = new smartbucket.SmartBucket({
+    accessKey: s3AccessKey || 'minioadmin',
+    accessSecret: s3SecretKey || 'minioadmin',
+    endpoint: s3Endpoint || 'localhost',
+    port: parseInt(s3Port || '9000', 10),
+    useSsl: false,
+  });
+
+  try {
+    const bucket = await s3.getBucket('test-registry');
+    if (bucket) {
+      if (prefix) {
+        // Delete only objects with the given prefix
+        const files = await bucket.fastList({ prefix });
+        for (const file of files) {
+          await bucket.fastRemove({ path: file.name });
+        }
+      } else {
+        // Delete all objects in the bucket
+        const files = await bucket.fastList({});
+        for (const file of files) {
+          await bucket.fastRemove({ path: file.name });
+        }
+      }
+    }
+  } catch (error) {
+    // Bucket might not exist yet, that's fine
+    console.log('Cleanup: No bucket to clean or error:', error);
+  }
+}
+
 /**
  * Create a test SmartRegistry instance with all protocols enabled
  */

test/test.composer.nativecli.node.ts

@@ -0,0 +1,468 @@
+/**
+ * Native Composer CLI Testing
+ * Tests the Composer registry implementation using the actual composer CLI
+ */
+
+import { expect, tap } from '@git.zone/tstest/tapbundle';
+import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
+import { SmartRegistry } from '../ts/index.js';
+import { createTestRegistry, createTestTokens, createComposerZip, generateTestRunId } from './helpers/registry.js';
+import type { IRequestContext, IResponse } from '../ts/core/interfaces.core.js';
+import * as http from 'http';
+import * as url from 'url';
+import * as fs from 'fs';
+import * as path from 'path';
+
+// Test context
+let registry: SmartRegistry;
+let server: http.Server;
+let registryUrl: string;
+let registryPort: number;
+let composerToken: string;
+let testDir: string;
+let composerHome: string;
+let hasComposer = false;
+
+// Unique test run ID to avoid conflicts between test runs
+const testRunId = generateTestRunId();
+const testPackageName = `testvendor/test-pkg-${testRunId}`;
+
+/**
+ * Create HTTP server wrapper around SmartRegistry
+ */
+async function createHttpServer(
+  registryInstance: SmartRegistry,
+  port: number
+): Promise<{ server: http.Server; url: string }> {
+  return new Promise((resolve, reject) => {
+    const httpServer = http.createServer(async (req, res) => {
+      try {
+        // Parse request
+        const parsedUrl = url.parse(req.url || '', true);
+        const pathname = parsedUrl.pathname || '/';
+        const query = parsedUrl.query;
+
+        // Read body
+        const chunks: Buffer[] = [];
+        for await (const chunk of req) {
+          chunks.push(chunk);
+        }
+        const bodyBuffer = Buffer.concat(chunks);
+
+        // Parse body based on content type
+        let body: any;
+        if (bodyBuffer.length > 0) {
+          const contentType = req.headers['content-type'] || '';
+          if (contentType.includes('application/json')) {
+            try {
+              body = JSON.parse(bodyBuffer.toString('utf-8'));
+            } catch (error) {
+              body = bodyBuffer;
+            }
+          } else {
+            body = bodyBuffer;
+          }
+        }
+
+        // Convert to IRequestContext
+        const context: IRequestContext = {
+          method: req.method || 'GET',
+          path: pathname,
+          headers: req.headers as Record<string, string>,
+          query: query as Record<string, string>,
+          body: body,
+        };
+
+        // Handle request
+        const response: IResponse = await registryInstance.handleRequest(context);
+
+        // Convert IResponse to HTTP response
+        res.statusCode = response.status;
+
+        // Set headers
+        for (const [key, value] of Object.entries(response.headers || {})) {
+          res.setHeader(key, value);
+        }
+
+        // Send body
+        if (response.body) {
+          if (Buffer.isBuffer(response.body)) {
+            res.end(response.body);
+          } else if (typeof response.body === 'string') {
+            res.end(response.body);
+          } else {
+            res.setHeader('Content-Type', 'application/json');
+            res.end(JSON.stringify(response.body));
+          }
+        } else {
+          res.end();
+        }
+      } catch (error) {
+        console.error('Server error:', error);
+        res.statusCode = 500;
+        res.setHeader('Content-Type', 'application/json');
+        res.end(JSON.stringify({ error: 'INTERNAL_ERROR', message: String(error) }));
+      }
+    });
+
+    httpServer.listen(port, () => {
+      const serverUrl = `http://localhost:${port}`;
+      resolve({ server: httpServer, url: serverUrl });
+    });
+
+    httpServer.on('error', reject);
+  });
+}
+
+/**
+ * Setup Composer auth.json for authentication
+ */
+function setupComposerAuth(
+  token: string,
+  composerHomeArg: string,
+  serverUrl: string,
+  port: number
+): string {
+  fs.mkdirSync(composerHomeArg, { recursive: true });
+
+  const authJson = {
+    'http-basic': {
+      [`localhost:${port}`]: {
+        username: 'testuser',
+        password: token,
+      },
+    },
+  };
+
+  const authPath = path.join(composerHomeArg, 'auth.json');
+  fs.writeFileSync(authPath, JSON.stringify(authJson, null, 2), 'utf-8');
+
+  return authPath;
+}
+
+/**
+ * Create a Composer project that uses our registry
+ */
+function createComposerProject(
+  projectDir: string,
+  serverUrl: string
+): void {
+  fs.mkdirSync(projectDir, { recursive: true });
+
+  const composerJson = {
+    name: 'test/consumer-project',
+    description: 'Test consumer project for Composer CLI tests',
+    type: 'project',
+    require: {},
+    repositories: [
+      {
+        type: 'composer',
+        url: `${serverUrl}/composer`,
+      },
+    ],
+    config: {
+      'secure-http': false,
+    },
+  };
+
+  fs.writeFileSync(
+    path.join(projectDir, 'composer.json'),
+    JSON.stringify(composerJson, null, 2),
+    'utf-8'
+  );
+}
+
+/**
+ * Run Composer command with custom home directory
+ */
+async function runComposerCommand(
+  command: string,
+  cwd: string
+): Promise<{ stdout: string; stderr: string; exitCode: number }> {
+  const fullCommand = `cd "${cwd}" && COMPOSER_HOME="${composerHome}" composer ${command}`;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    return {
+      stdout: result.stdout || '',
+      stderr: result.stderr || '',
+      exitCode: result.exitCode || 0,
+    };
+  } catch (error: any) {
+    return {
+      stdout: error.stdout || '',
+      stderr: error.stderr || String(error),
+      exitCode: error.exitCode || 1,
+    };
+  }
+}
+
+/**
+ * Upload a Composer package via HTTP API
+ */
+async function uploadComposerPackage(
+  vendorPackage: string,
+  version: string,
+  token: string,
+  serverUrl: string
+): Promise<void> {
+  const zipData = await createComposerZip(vendorPackage, version);
+
+  const response = await fetch(`${serverUrl}/composer/packages/${vendorPackage}`, {
+    method: 'PUT',
+    headers: {
+      'Content-Type': 'application/zip',
+      Authorization: `Bearer ${token}`,
+    },
+    body: zipData,
+  });
+
+  if (!response.ok) {
+    const body = await response.text();
+    throw new Error(`Failed to upload package: ${response.status} ${body}`);
+  }
+}
+
+/**
+ * Cleanup test directory
+ */
+function cleanupTestDir(dir: string): void {
+  if (fs.existsSync(dir)) {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+// ========================================================================
+// TESTS
+// ========================================================================
+
+tap.test('Composer CLI: should verify composer is installed', async () => {
+  try {
+    const result = await tapNodeTools.runCommand('composer --version');
+    console.log('Composer version output:', result.stdout.substring(0, 200));
+    hasComposer = result.exitCode === 0;
+    expect(result.exitCode).toEqual(0);
+  } catch (error) {
+    console.log('Composer CLI not available, skipping native CLI tests');
+    hasComposer = false;
+  }
+});
+
+tap.test('Composer CLI: should setup registry and HTTP server', async () => {
+  // Create registry
+  registry = await createTestRegistry();
+  const tokens = await createTestTokens(registry);
+  composerToken = tokens.composerToken;
+
+  expect(registry).toBeInstanceOf(SmartRegistry);
+  expect(composerToken).toBeTypeOf('string');
+
+  // Use port 38000 (avoids conflicts with other tests)
+  registryPort = 38000;
+  const serverSetup = await createHttpServer(registry, registryPort);
+  server = serverSetup.server;
+  registryUrl = serverSetup.url;
+
+  expect(server).toBeDefined();
+  expect(registryUrl).toEqual(`http://localhost:${registryPort}`);
+
+  // Setup test directory
+  testDir = path.join(process.cwd(), '.nogit', 'test-composer-cli');
+  cleanupTestDir(testDir);
+  fs.mkdirSync(testDir, { recursive: true });
+
+  // Setup COMPOSER_HOME directory
+  composerHome = path.join(testDir, '.composer');
+  fs.mkdirSync(composerHome, { recursive: true });
+
+  // Setup Composer auth
+  const authPath = setupComposerAuth(composerToken, composerHome, registryUrl, registryPort);
+  expect(fs.existsSync(authPath)).toEqual(true);
+});
+
+tap.test('Composer CLI: should verify server is responding', async () => {
+  // Check server is up by doing a direct HTTP request
+  const response = await fetch(`${registryUrl}/composer/packages.json`);
+  expect(response.status).toBeGreaterThanOrEqual(200);
+  expect(response.status).toBeLessThan(500);
+});
+
+tap.test('Composer CLI: should upload a package via API', async () => {
+  const version = '1.0.0';
+
+  await uploadComposerPackage(testPackageName, version, composerToken, registryUrl);
+
+  // Verify package exists via p2 metadata endpoint (more reliable than packages.json for new packages)
+  const metadataResponse = await fetch(`${registryUrl}/composer/p2/${testPackageName}.json`);
+  expect(metadataResponse.status).toEqual(200);
+
+  const metadata = await metadataResponse.json();
+  expect(metadata.packages).toBeDefined();
+  expect(metadata.packages[testPackageName]).toBeDefined();
+  expect(metadata.packages[testPackageName].length).toBeGreaterThan(0);
+});
+
+tap.test('Composer CLI: should require package from registry', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+  createComposerProject(projectDir, registryUrl);
+
+  // Try to require the package we uploaded
+  const result = await runComposerCommand(
+    `require ${testPackageName}:1.0.0 --no-interaction`,
+    projectDir
+  );
+  console.log('composer require output:', result.stdout);
+  console.log('composer require stderr:', result.stderr);
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Composer CLI: should verify package in vendor directory', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+  // Parse vendor/package from testPackageName (e.g., "testvendor/test-pkg-abc123")
+  const [vendor, pkg] = testPackageName.split('/');
+  const packageDir = path.join(projectDir, 'vendor', vendor, pkg);
+
+  expect(fs.existsSync(packageDir)).toEqual(true);
+
+  // Check composer.json exists in package
+  const packageComposerPath = path.join(packageDir, 'composer.json');
+  expect(fs.existsSync(packageComposerPath)).toEqual(true);
+});
+
+tap.test('Composer CLI: should upload second version', async () => {
+  const version = '2.0.0';
+
+  await uploadComposerPackage(testPackageName, version, composerToken, registryUrl);
+
+  // Verify both versions exist via p2 metadata endpoint (Composer v2 format)
+  const response = await fetch(`${registryUrl}/composer/p2/${testPackageName}.json`);
+  expect(response.status).toEqual(200);
+
+  const metadata = await response.json();
+  expect(metadata.packages).toBeDefined();
+  expect(metadata.packages[testPackageName]).toBeDefined();
+  // Check that both versions are present
+  const versions = metadata.packages[testPackageName];
+  expect(versions.length).toBeGreaterThanOrEqual(2);
+  const versionNumbers = versions.map((v: any) => v.version);
+  expect(versionNumbers).toContain('1.0.0');
+  expect(versionNumbers).toContain('2.0.0');
+});
+
+tap.test('Composer CLI: should update to new version', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+
+  // Update to version 2.0.0
+  const result = await runComposerCommand(
+    `require ${testPackageName}:2.0.0 --no-interaction`,
+    projectDir
+  );
+  console.log('composer update output:', result.stdout);
+
+  expect(result.exitCode).toEqual(0);
+
+  // Verify composer.lock has the new version
+  const lockPath = path.join(projectDir, 'composer.lock');
+  expect(fs.existsSync(lockPath)).toEqual(true);
+
+  const lockContent = JSON.parse(fs.readFileSync(lockPath, 'utf-8'));
+  const pkg = lockContent.packages.find((p: any) => p.name === testPackageName);
+  expect(pkg?.version).toEqual('2.0.0');
+});
+
+tap.test('Composer CLI: should search for packages', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+
+  // Search for packages (may not work on all Composer versions)
+  const result = await runComposerCommand(
+    'search testvendor --no-interaction 2>&1 || true',
+    projectDir
+  );
+  console.log('composer search output:', result.stdout);
+
+  // Search may or may not work depending on registry implementation
+  // Just verify it doesn't crash
+  expect(result.exitCode).toBeLessThanOrEqual(1);
+});
+
+tap.test('Composer CLI: should show package info', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+
+  const result = await runComposerCommand(
+    `show ${testPackageName} --no-interaction`,
+    projectDir
+  );
+  console.log('composer show output:', result.stdout);
+
+  expect(result.exitCode).toEqual(0);
+  expect(result.stdout).toContain(testPackageName);
+});
+
+tap.test('Composer CLI: should remove package', async () => {
+  if (!hasComposer) {
+    console.log('Skipping - composer not available');
+    return;
+  }
+
+  const projectDir = path.join(testDir, 'consumer-project');
+
+  const result = await runComposerCommand(
+    `remove ${testPackageName} --no-interaction`,
+    projectDir
+  );
+  console.log('composer remove output:', result.stdout);
+
+  expect(result.exitCode).toEqual(0);
+
+  // Verify package is removed from vendor
+  const [vendor, pkg] = testPackageName.split('/');
+  const packageDir = path.join(projectDir, 'vendor', vendor, pkg);
+  expect(fs.existsSync(packageDir)).toEqual(false);
+});
+
+tap.postTask('cleanup composer cli tests', async () => {
+  // Stop server
+  if (server) {
+    await new Promise<void>((resolve) => {
+      server.close(() => resolve());
+    });
+  }
+
+  // Cleanup test directory
+  if (testDir) {
+    cleanupTestDir(testDir);
+  }
+
+  // Destroy registry
+  if (registry) {
+    registry.destroy();
+  }
+});
+
+export default tap.start();

test/test.maven.nativecli.node.ts

@@ -0,0 +1,490 @@
+/**
+ * Native Maven CLI Testing
+ * Tests the Maven registry implementation using the actual mvn CLI
+ */
+
+import { expect, tap } from '@git.zone/tstest/tapbundle';
+import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
+import { SmartRegistry } from '../ts/index.js';
+import { createTestRegistry, createTestTokens, createTestPom, createTestJar } from './helpers/registry.js';
+import type { IRequestContext, IResponse } from '../ts/core/interfaces.core.js';
+import * as http from 'http';
+import * as url from 'url';
+import * as fs from 'fs';
+import * as path from 'path';
+
+// Test context
+let registry: SmartRegistry;
+let server: http.Server;
+let registryUrl: string;
+let registryPort: number;
+let mavenToken: string;
+let testDir: string;
+let m2Dir: string;
+
+/**
+ * Create HTTP server wrapper around SmartRegistry
+ */
+async function createHttpServer(
+  registryInstance: SmartRegistry,
+  port: number
+): Promise<{ server: http.Server; url: string }> {
+  return new Promise((resolve, reject) => {
+    const httpServer = http.createServer(async (req, res) => {
+      try {
+        // Parse request
+        const parsedUrl = url.parse(req.url || '', true);
+        const pathname = parsedUrl.pathname || '/';
+        const query = parsedUrl.query;
+
+        // Read body
+        const chunks: Buffer[] = [];
+        for await (const chunk of req) {
+          chunks.push(chunk);
+        }
+        const bodyBuffer = Buffer.concat(chunks);
+
+        // Parse body based on content type
+        let body: any;
+        if (bodyBuffer.length > 0) {
+          const contentType = req.headers['content-type'] || '';
+          if (contentType.includes('application/json')) {
+            try {
+              body = JSON.parse(bodyBuffer.toString('utf-8'));
+            } catch (error) {
+              body = bodyBuffer;
+            }
+          } else {
+            body = bodyBuffer;
+          }
+        }
+
+        // Convert to IRequestContext
+        const context: IRequestContext = {
+          method: req.method || 'GET',
+          path: pathname,
+          headers: req.headers as Record<string, string>,
+          query: query as Record<string, string>,
+          body: body,
+        };
+
+        // Handle request
+        const response: IResponse = await registryInstance.handleRequest(context);
+
+        // Convert IResponse to HTTP response
+        res.statusCode = response.status;
+
+        // Set headers
+        for (const [key, value] of Object.entries(response.headers || {})) {
+          res.setHeader(key, value);
+        }
+
+        // Send body
+        if (response.body) {
+          if (Buffer.isBuffer(response.body)) {
+            res.end(response.body);
+          } else if (typeof response.body === 'string') {
+            res.end(response.body);
+          } else {
+            res.setHeader('Content-Type', 'application/json');
+            res.end(JSON.stringify(response.body));
+          }
+        } else {
+          res.end();
+        }
+      } catch (error) {
+        console.error('Server error:', error);
+        res.statusCode = 500;
+        res.setHeader('Content-Type', 'application/json');
+        res.end(JSON.stringify({ error: 'INTERNAL_ERROR', message: String(error) }));
+      }
+    });
+
+    httpServer.listen(port, () => {
+      const serverUrl = `http://localhost:${port}`;
+      resolve({ server: httpServer, url: serverUrl });
+    });
+
+    httpServer.on('error', reject);
+  });
+}
+
+/**
+ * Setup Maven settings.xml for authentication
+ */
+function setupMavenSettings(
+  token: string,
+  m2DirArg: string,
+  serverUrl: string
+): string {
+  fs.mkdirSync(m2DirArg, { recursive: true });
+
+  const settingsXml = `<?xml version="1.0" encoding="UTF-8"?>
+<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
+          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
+          http://maven.apache.org/xsd/settings-1.0.0.xsd">
+  <servers>
+    <server>
+      <id>test-registry</id>
+      <username>testuser</username>
+      <password>${token}</password>
+    </server>
+  </servers>
+  <profiles>
+    <profile>
+      <id>test-registry</id>
+      <repositories>
+        <repository>
+          <id>test-registry</id>
+          <url>${serverUrl}/maven</url>
+          <releases>
+            <enabled>true</enabled>
+          </releases>
+          <snapshots>
+            <enabled>true</enabled>
+          </snapshots>
+        </repository>
+      </repositories>
+    </profile>
+  </profiles>
+  <activeProfiles>
+    <activeProfile>test-registry</activeProfile>
+  </activeProfiles>
+</settings>
+`;
+
+  const settingsPath = path.join(m2DirArg, 'settings.xml');
+  fs.writeFileSync(settingsPath, settingsXml, 'utf-8');
+
+  return settingsPath;
+}
+
+/**
+ * Create a minimal Maven project for testing
+ */
+function createMavenProject(
+  projectDir: string,
+  groupId: string,
+  artifactId: string,
+  version: string,
+  registryUrl: string
+): void {
+  fs.mkdirSync(projectDir, { recursive: true });
+
+  const pomXml = `<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
+         http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <groupId>${groupId}</groupId>
+  <artifactId>${artifactId}</artifactId>
+  <version>${version}</version>
+  <packaging>jar</packaging>
+  <name>${artifactId}</name>
+  <description>Test Maven project for SmartRegistry CLI tests</description>
+
+  <distributionManagement>
+    <repository>
+      <id>test-registry</id>
+      <url>${registryUrl}/maven</url>
+    </repository>
+    <snapshotRepository>
+      <id>test-registry</id>
+      <url>${registryUrl}/maven</url>
+    </snapshotRepository>
+  </distributionManagement>
+
+  <build>
+    <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-compiler-plugin</artifactId>
+        <version>3.8.1</version>
+        <configuration>
+          <source>1.8</source>
+          <target>1.8</target>
+        </configuration>
+      </plugin>
+    </plugins>
+  </build>
+</project>
+`;
+
+  fs.writeFileSync(path.join(projectDir, 'pom.xml'), pomXml, 'utf-8');
+
+  // Create minimal Java source
+  const srcDir = path.join(projectDir, 'src', 'main', 'java', 'com', 'test');
+  fs.mkdirSync(srcDir, { recursive: true });
+
+  const javaSource = `package com.test;
+
+public class Main {
+    public static void main(String[] args) {
+        System.out.println("Hello from SmartRegistry test!");
+    }
+}
+`;
+  fs.writeFileSync(path.join(srcDir, 'Main.java'), javaSource, 'utf-8');
+}
+
+/**
+ * Run Maven command with custom settings
+ */
+async function runMavenCommand(
+  command: string,
+  cwd: string
+): Promise<{ stdout: string; stderr: string; exitCode: number }> {
+  const settingsPath = path.join(m2Dir, 'settings.xml');
+  const fullCommand = `cd "${cwd}" && mvn -s "${settingsPath}" ${command}`;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    return {
+      stdout: result.stdout || '',
+      stderr: result.stderr || '',
+      exitCode: result.exitCode || 0,
+    };
+  } catch (error: any) {
+    return {
+      stdout: error.stdout || '',
+      stderr: error.stderr || String(error),
+      exitCode: error.exitCode || 1,
+    };
+  }
+}
+
+/**
+ * Cleanup test directory
+ */
+function cleanupTestDir(dir: string): void {
+  if (fs.existsSync(dir)) {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+// ========================================================================
+// TESTS
+// ========================================================================
+
+tap.test('Maven CLI: should verify mvn is installed', async () => {
+  try {
+    const result = await tapNodeTools.runCommand('mvn -version');
+    console.log('Maven version output:', result.stdout.substring(0, 200));
+    expect(result.exitCode).toEqual(0);
+  } catch (error) {
+    console.log('Maven CLI not available, skipping native CLI tests');
+    // Skip remaining tests if Maven is not installed
+    tap.skip.test('Maven CLI: remaining tests skipped - mvn not available');
+    return;
+  }
+});
+
+tap.test('Maven CLI: should setup registry and HTTP server', async () => {
+  // Create registry
+  registry = await createTestRegistry();
+  const tokens = await createTestTokens(registry);
+  mavenToken = tokens.mavenToken;
+
+  expect(registry).toBeInstanceOf(SmartRegistry);
+  expect(mavenToken).toBeTypeOf('string');
+
+  // Use port 37000 (avoids conflicts with other tests)
+  registryPort = 37000;
+  const serverSetup = await createHttpServer(registry, registryPort);
+  server = serverSetup.server;
+  registryUrl = serverSetup.url;
+
+  expect(server).toBeDefined();
+  expect(registryUrl).toEqual(`http://localhost:${registryPort}`);
+
+  // Setup test directory
+  testDir = path.join(process.cwd(), '.nogit', 'test-maven-cli');
+  cleanupTestDir(testDir);
+  fs.mkdirSync(testDir, { recursive: true });
+
+  // Setup .m2 directory
+  m2Dir = path.join(testDir, '.m2');
+  fs.mkdirSync(m2Dir, { recursive: true });
+
+  // Setup Maven settings
+  const settingsPath = setupMavenSettings(mavenToken, m2Dir, registryUrl);
+  expect(fs.existsSync(settingsPath)).toEqual(true);
+});
+
+tap.test('Maven CLI: should verify server is responding', async () => {
+  // Check server is up by doing a direct HTTP request
+  const response = await fetch(`${registryUrl}/maven/`);
+  expect(response.status).toBeGreaterThanOrEqual(200);
+  expect(response.status).toBeLessThan(500);
+});
+
+tap.test('Maven CLI: should deploy a JAR artifact', async () => {
+  const groupId = 'com.test';
+  const artifactId = 'test-artifact';
+  const version = '1.0.0';
+
+  const projectDir = path.join(testDir, 'test-project');
+  createMavenProject(projectDir, groupId, artifactId, version, registryUrl);
+
+  // Build and deploy
+  const result = await runMavenCommand('clean package deploy -DskipTests', projectDir);
+  console.log('mvn deploy output:', result.stdout.substring(0, 500));
+  console.log('mvn deploy stderr:', result.stderr.substring(0, 500));
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Maven CLI: should verify artifact in registry via API', async () => {
+  const groupId = 'com.test';
+  const artifactId = 'test-artifact';
+  const version = '1.0.0';
+
+  // Maven path: /maven/{groupId path}/{artifactId}/{version}/{artifactId}-{version}.jar
+  const jarPath = `/maven/com/test/${artifactId}/${version}/${artifactId}-${version}.jar`;
+  const response = await fetch(`${registryUrl}${jarPath}`, {
+    headers: { Authorization: `Bearer ${mavenToken}` },
+  });
+
+  expect(response.status).toEqual(200);
+
+  const jarData = await response.arrayBuffer();
+  expect(jarData.byteLength).toBeGreaterThan(0);
+});
+
+tap.test('Maven CLI: should verify POM in registry', async () => {
+  const groupId = 'com.test';
+  const artifactId = 'test-artifact';
+  const version = '1.0.0';
+
+  const pomPath = `/maven/com/test/${artifactId}/${version}/${artifactId}-${version}.pom`;
+  const response = await fetch(`${registryUrl}${pomPath}`, {
+    headers: { Authorization: `Bearer ${mavenToken}` },
+  });
+
+  expect(response.status).toEqual(200);
+
+  const pomContent = await response.text();
+  expect(pomContent).toContain(groupId);
+  expect(pomContent).toContain(artifactId);
+  expect(pomContent).toContain(version);
+});
+
+tap.test('Maven CLI: should verify checksums exist', async () => {
+  const artifactId = 'test-artifact';
+  const version = '1.0.0';
+
+  // Check JAR checksums
+  const basePath = `/maven/com/test/${artifactId}/${version}/${artifactId}-${version}.jar`;
+
+  // MD5
+  const md5Response = await fetch(`${registryUrl}${basePath}.md5`, {
+    headers: { Authorization: `Bearer ${mavenToken}` },
+  });
+  expect(md5Response.status).toEqual(200);
+
+  // SHA1
+  const sha1Response = await fetch(`${registryUrl}${basePath}.sha1`, {
+    headers: { Authorization: `Bearer ${mavenToken}` },
+  });
+  expect(sha1Response.status).toEqual(200);
+});
+
+tap.test('Maven CLI: should deploy second version', async () => {
+  const groupId = 'com.test';
+  const artifactId = 'test-artifact';
+  const version = '2.0.0';
+
+  const projectDir = path.join(testDir, 'test-project-v2');
+  createMavenProject(projectDir, groupId, artifactId, version, registryUrl);
+
+  const result = await runMavenCommand('clean package deploy -DskipTests', projectDir);
+  console.log('mvn deploy v2 output:', result.stdout.substring(0, 500));
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Maven CLI: should verify metadata.xml exists', async () => {
+  const artifactId = 'test-artifact';
+
+  // Maven metadata is stored at /maven/{groupId path}/{artifactId}/maven-metadata.xml
+  const metadataPath = `/maven/com/test/${artifactId}/maven-metadata.xml`;
+  const response = await fetch(`${registryUrl}${metadataPath}`, {
+    headers: { Authorization: `Bearer ${mavenToken}` },
+  });
+
+  expect(response.status).toEqual(200);
+
+  const metadataXml = await response.text();
+  expect(metadataXml).toContain(artifactId);
+  expect(metadataXml).toContain('1.0.0');
+  expect(metadataXml).toContain('2.0.0');
+});
+
+tap.test('Maven CLI: should resolve dependency from registry', async () => {
+  const groupId = 'com.consumer';
+  const artifactId = 'consumer-app';
+  const version = '1.0.0';
+
+  const projectDir = path.join(testDir, 'consumer-project');
+  fs.mkdirSync(projectDir, { recursive: true });
+
+  // Create a consumer project that depends on our test artifact
+  const pomXml = `<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
+         http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <groupId>${groupId}</groupId>
+  <artifactId>${artifactId}</artifactId>
+  <version>${version}</version>
+  <packaging>jar</packaging>
+
+  <repositories>
+    <repository>
+      <id>test-registry</id>
+      <url>${registryUrl}/maven</url>
+    </repository>
+  </repositories>
+
+  <dependencies>
+    <dependency>
+      <groupId>com.test</groupId>
+      <artifactId>test-artifact</artifactId>
+      <version>1.0.0</version>
+    </dependency>
+  </dependencies>
+</project>
+`;
+
+  fs.writeFileSync(path.join(projectDir, 'pom.xml'), pomXml, 'utf-8');
+
+  // Try to resolve dependencies
+  const result = await runMavenCommand('dependency:resolve', projectDir);
+  console.log('mvn dependency:resolve output:', result.stdout.substring(0, 500));
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.postTask('cleanup maven cli tests', async () => {
+  // Stop server
+  if (server) {
+    await new Promise<void>((resolve) => {
+      server.close(() => resolve());
+    });
+  }
+
+  // Cleanup test directory
+  if (testDir) {
+    cleanupTestDir(testDir);
+  }
+
+  // Destroy registry
+  if (registry) {
+    registry.destroy();
+  }
+});
+
+export default tap.start();

test/test.oci.nativecli.node.ts

@@ -0,0 +1,406 @@
+/**
+ * Native Docker CLI Testing
+ * Tests the OCI registry implementation using the actual Docker CLI
+ */
+
+import { expect, tap } from '@git.zone/tstest/tapbundle';
+import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
+import { SmartRegistry } from '../ts/index.js';
+import type { IRequestContext, IResponse, IRegistryConfig } from '../ts/core/interfaces.core.js';
+import * as qenv from '@push.rocks/qenv';
+import * as http from 'http';
+import * as url from 'url';
+import * as fs from 'fs';
+import * as path from 'path';
+
+const testQenv = new qenv.Qenv('./', './.nogit');
+
+/**
+ * Create a test registry with local token endpoint realm
+ */
+async function createDockerTestRegistry(port: number): Promise<SmartRegistry> {
+  const s3AccessKey = await testQenv.getEnvVarOnDemand('S3_ACCESSKEY');
+  const s3SecretKey = await testQenv.getEnvVarOnDemand('S3_SECRETKEY');
+  const s3Endpoint = await testQenv.getEnvVarOnDemand('S3_ENDPOINT');
+  const s3Port = await testQenv.getEnvVarOnDemand('S3_PORT');
+
+  const config: IRegistryConfig = {
+    storage: {
+      accessKey: s3AccessKey || 'minioadmin',
+      accessSecret: s3SecretKey || 'minioadmin',
+      endpoint: s3Endpoint || 'localhost',
+      port: parseInt(s3Port || '9000', 10),
+      useSsl: false,
+      region: 'us-east-1',
+      bucketName: 'test-registry',
+    },
+    auth: {
+      jwtSecret: 'test-secret-key',
+      tokenStore: 'memory',
+      npmTokens: {
+        enabled: true,
+      },
+      ociTokens: {
+        enabled: true,
+        realm: `http://localhost:${port}/v2/token`,
+        service: 'test-registry',
+      },
+    },
+    oci: {
+      enabled: true,
+      basePath: '/oci',
+    },
+  };
+
+  const reg = new SmartRegistry(config);
+  await reg.init();
+  return reg;
+}
+
+/**
+ * Create test tokens for the registry
+ */
+async function createDockerTestTokens(reg: SmartRegistry) {
+  const authManager = reg.getAuthManager();
+
+  const userId = await authManager.authenticate({
+    username: 'testuser',
+    password: 'testpass',
+  });
+
+  if (!userId) {
+    throw new Error('Failed to authenticate test user');
+  }
+
+  // Create OCI token with full access
+  const ociToken = await authManager.createOciToken(
+    userId,
+    ['oci:repository:*:*'],
+    3600
+  );
+
+  return { ociToken, userId };
+}
+
+// Test context
+let registry: SmartRegistry;
+let server: http.Server;
+let registryUrl: string;
+let registryPort: number;
+let ociToken: string;
+let testDir: string;
+let testImageName: string;
+
+/**
+ * Create HTTP server wrapper around SmartRegistry
+ * CRITICAL: Always passes rawBody for content-addressable operations (OCI manifests/blobs)
+ *
+ * Docker expects registry at /v2/ but SmartRegistry serves at /oci/v2/
+ * This wrapper rewrites paths for Docker compatibility
+ *
+ * Also implements a simple /v2/token endpoint for Docker Bearer auth flow
+ */
+async function createHttpServer(
+  registryInstance: SmartRegistry,
+  port: number,
+  tokenForAuth: string
+): Promise<{ server: http.Server; url: string }> {
+  return new Promise((resolve, reject) => {
+    const httpServer = http.createServer(async (req, res) => {
+      try {
+        // Parse request
+        const parsedUrl = url.parse(req.url || '', true);
+        let pathname = parsedUrl.pathname || '/';
+        const query = parsedUrl.query;
+
+        // Handle token endpoint for Docker Bearer auth
+        if (pathname === '/v2/token' || pathname === '/token') {
+          console.log(`[Token Request] ${req.method} ${req.url}`);
+          res.statusCode = 200;
+          res.setHeader('Content-Type', 'application/json');
+          res.end(JSON.stringify({
+            token: tokenForAuth,
+            access_token: tokenForAuth,
+            expires_in: 3600,
+            issued_at: new Date().toISOString(),
+          }));
+          return;
+        }
+
+        // Log all requests for debugging
+        console.log(`[Registry] ${req.method} ${pathname}`);
+
+        // Docker expects /v2/ but SmartRegistry serves at /oci/v2/
+        if (pathname.startsWith('/v2')) {
+          pathname = '/oci' + pathname;
+        }
+
+        // Read raw body - ALWAYS preserve exact bytes for OCI
+        const chunks: Buffer[] = [];
+        for await (const chunk of req) {
+          chunks.push(chunk);
+        }
+        const bodyBuffer = Buffer.concat(chunks);
+
+        // Parse body based on content type (for non-OCI protocols that need it)
+        let parsedBody: any;
+        if (bodyBuffer.length > 0) {
+          const contentType = req.headers['content-type'] || '';
+          if (contentType.includes('application/json')) {
+            try {
+              parsedBody = JSON.parse(bodyBuffer.toString('utf-8'));
+            } catch (error) {
+              parsedBody = bodyBuffer;
+            }
+          } else {
+            parsedBody = bodyBuffer;
+          }
+        }
+
+        // Convert to IRequestContext
+        const context: IRequestContext = {
+          method: req.method || 'GET',
+          path: pathname,
+          headers: req.headers as Record<string, string>,
+          query: query as Record<string, string>,
+          body: parsedBody,
+          rawBody: bodyBuffer,
+        };
+
+        // Handle request
+        const response: IResponse = await registryInstance.handleRequest(context);
+        console.log(`[Registry] Response: ${response.status} for ${pathname}`);
+
+        // Convert IResponse to HTTP response
+        res.statusCode = response.status;
+
+        // Set headers
+        for (const [key, value] of Object.entries(response.headers || {})) {
+          res.setHeader(key, value);
+        }
+
+        // Send body
+        if (response.body) {
+          if (Buffer.isBuffer(response.body)) {
+            res.end(response.body);
+          } else if (typeof response.body === 'string') {
+            res.end(response.body);
+          } else {
+            res.setHeader('Content-Type', 'application/json');
+            res.end(JSON.stringify(response.body));
+          }
+        } else {
+          res.end();
+        }
+      } catch (error) {
+        console.error('Server error:', error);
+        res.statusCode = 500;
+        res.setHeader('Content-Type', 'application/json');
+        res.end(JSON.stringify({ error: 'INTERNAL_ERROR', message: String(error) }));
+      }
+    });
+
+    httpServer.listen(port, '0.0.0.0', () => {
+      const serverUrl = `http://localhost:${port}`;
+      resolve({ server: httpServer, url: serverUrl });
+    });
+
+    httpServer.on('error', reject);
+  });
+}
+
+/**
+ * Create a test Dockerfile
+ */
+function createTestDockerfile(targetDir: string, content?: string): string {
+  const dockerfilePath = path.join(targetDir, 'Dockerfile');
+  const dockerfileContent = content || `FROM alpine:latest
+RUN echo "Hello from SmartRegistry test" > /hello.txt
+CMD ["cat", "/hello.txt"]
+`;
+  fs.writeFileSync(dockerfilePath, dockerfileContent, 'utf-8');
+  return dockerfilePath;
+}
+
+/**
+ * Run Docker command using the main Docker daemon (not rootless)
+ * Rootless Docker runs in its own network namespace and can't access host localhost
+ *
+ * IMPORTANT: DOCKER_HOST env var overrides --context flag, so we must unset it
+ * and explicitly set the socket path to use the main Docker daemon.
+ */
+async function runDockerCommand(
+  command: string,
+  cwd?: string
+): Promise<{ stdout: string; stderr: string; exitCode: number }> {
+  // First unset DOCKER_HOST then set it to main Docker daemon socket
+  // Using both unset and export ensures we override any inherited env var
+  const dockerCommand = `unset DOCKER_HOST && export DOCKER_HOST=unix:///var/run/docker.sock && ${command}`;
+  const fullCommand = cwd ? `cd "${cwd}" && ${dockerCommand}` : dockerCommand;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    return {
+      stdout: result.stdout || '',
+      stderr: result.stderr || '',
+      exitCode: result.exitCode || 0,
+    };
+  } catch (error: any) {
+    return {
+      stdout: error.stdout || '',
+      stderr: error.stderr || String(error),
+      exitCode: error.exitCode || 1,
+    };
+  }
+}
+
+/**
+ * Cleanup test directory
+ */
+function cleanupTestDir(dir: string): void {
+  if (fs.existsSync(dir)) {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+/**
+ * Cleanup Docker resources
+ */
+async function cleanupDocker(imageName: string): Promise<void> {
+  await runDockerCommand(`docker rmi ${imageName} 2>/dev/null || true`);
+  await runDockerCommand(`docker rmi ${imageName}:v1 2>/dev/null || true`);
+  await runDockerCommand(`docker rmi ${imageName}:v2 2>/dev/null || true`);
+}
+
+// ========================================================================
+// TESTS
+// ========================================================================
+
+tap.test('Docker CLI: should verify Docker is installed', async () => {
+  const result = await runDockerCommand('docker version');
+  console.log('Docker version output:', result.stdout.substring(0, 200));
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Docker CLI: should setup registry and HTTP server', async () => {
+  // Use localhost - Docker allows HTTP for localhost without any special config
+  registryPort = 15000 + Math.floor(Math.random() * 1000);
+  console.log(`Using port: ${registryPort}`);
+
+  registry = await createDockerTestRegistry(registryPort);
+  const tokens = await createDockerTestTokens(registry);
+  ociToken = tokens.ociToken;
+
+  expect(registry).toBeInstanceOf(SmartRegistry);
+  expect(ociToken).toBeTypeOf('string');
+
+  const serverSetup = await createHttpServer(registry, registryPort, ociToken);
+  server = serverSetup.server;
+  registryUrl = serverSetup.url;
+
+  expect(server).toBeDefined();
+  console.log(`Registry server started at ${registryUrl}`);
+
+  // Setup test directory
+  testDir = path.join(process.cwd(), '.nogit', 'test-docker-cli');
+  cleanupTestDir(testDir);
+  fs.mkdirSync(testDir, { recursive: true });
+
+  testImageName = `localhost:${registryPort}/test-image`;
+});
+
+tap.test('Docker CLI: should verify server is responding', async () => {
+  // Give the server a moment to fully initialize
+  await new Promise(resolve => setTimeout(resolve, 500));
+
+  const response = await fetch(`${registryUrl}/oci/v2/`);
+  expect(response.status).toEqual(200);
+  console.log('OCI v2 response:', await response.json());
+});
+
+tap.test('Docker CLI: should login to registry', async () => {
+  const result = await runDockerCommand(
+    `echo "${ociToken}" | docker login localhost:${registryPort} -u testuser --password-stdin`
+  );
+  console.log('docker login output:', result.stdout);
+  console.log('docker login stderr:', result.stderr);
+
+  const combinedOutput = result.stdout + result.stderr;
+  expect(combinedOutput).toContain('Login Succeeded');
+});
+
+tap.test('Docker CLI: should build test image', async () => {
+  createTestDockerfile(testDir);
+
+  const result = await runDockerCommand(
+    `docker build -t ${testImageName}:v1 .`,
+    testDir
+  );
+  console.log('docker build output:', result.stdout.substring(0, 500));
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Docker CLI: should push image to registry', async () => {
+  // This is the critical test - if the digest mismatch bug is fixed,
+  // this should succeed. The manifest bytes must be preserved exactly.
+  const result = await runDockerCommand(`docker push ${testImageName}:v1`);
+  console.log('docker push output:', result.stdout);
+  console.log('docker push stderr:', result.stderr);
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Docker CLI: should verify manifest in registry via API', async () => {
+  const response = await fetch(`${registryUrl}/oci/v2/test-image/tags/list`, {
+    headers: { Authorization: `Bearer ${ociToken}` },
+  });
+
+  expect(response.status).toEqual(200);
+
+  const tagList = await response.json();
+  console.log('Tags list:', tagList);
+
+  expect(tagList.name).toEqual('test-image');
+  expect(tagList.tags).toContain('v1');
+});
+
+tap.test('Docker CLI: should pull pushed image', async () => {
+  // First remove the local image
+  await runDockerCommand(`docker rmi ${testImageName}:v1 || true`);
+
+  const result = await runDockerCommand(`docker pull ${testImageName}:v1`);
+  console.log('docker pull output:', result.stdout);
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('Docker CLI: should run pulled image', async () => {
+  const result = await runDockerCommand(`docker run --rm ${testImageName}:v1`);
+  console.log('docker run output:', result.stdout);
+
+  expect(result.exitCode).toEqual(0);
+  expect(result.stdout).toContain('Hello from SmartRegistry test');
+});
+
+tap.postTask('cleanup docker cli tests', async () => {
+  if (testImageName) {
+    await cleanupDocker(testImageName);
+  }
+
+  if (server) {
+    await new Promise<void>((resolve) => {
+      server.close(() => resolve());
+    });
+  }
+
+  if (testDir) {
+    cleanupTestDir(testDir);
+  }
+
+  if (registry) {
+    registry.destroy();
+  }
+});
+
+export default tap.start();

test/test.pypi.nativecli.node.ts

@@ -0,0 +1,521 @@
+/**
+ * Native PyPI CLI Testing
+ * Tests the PyPI registry implementation using pip and twine CLI tools
+ */
+
+import { expect, tap } from '@git.zone/tstest/tapbundle';
+import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
+import { SmartRegistry } from '../ts/index.js';
+import { createTestRegistry, createTestTokens, createPythonWheel, createPythonSdist, generateTestRunId } from './helpers/registry.js';
+import type { IRequestContext, IResponse } from '../ts/core/interfaces.core.js';
+import * as http from 'http';
+import * as url from 'url';
+import * as fs from 'fs';
+import * as path from 'path';
+
+// Test context
+let registry: SmartRegistry;
+let server: http.Server;
+let registryUrl: string;
+let registryPort: number;
+let pypiToken: string;
+let testDir: string;
+let pipHome: string;
+let hasPip = false;
+let hasTwine = false;
+
+// Unique test run ID to avoid conflicts between test runs
+const testRunId = generateTestRunId();
+const testPackageName = `test-pypi-pkg-${testRunId}`;
+
+/**
+ * Create HTTP server wrapper around SmartRegistry
+ */
+async function createHttpServer(
+  registryInstance: SmartRegistry,
+  port: number
+): Promise<{ server: http.Server; url: string }> {
+  return new Promise((resolve, reject) => {
+    const httpServer = http.createServer(async (req, res) => {
+      try {
+        // Parse request
+        const parsedUrl = url.parse(req.url || '', true);
+        const pathname = parsedUrl.pathname || '/';
+        const query = parsedUrl.query;
+
+        // Read body
+        const chunks: Buffer[] = [];
+        for await (const chunk of req) {
+          chunks.push(chunk);
+        }
+        const bodyBuffer = Buffer.concat(chunks);
+
+        // Parse body based on content type
+        let body: any;
+        if (bodyBuffer.length > 0) {
+          const contentType = req.headers['content-type'] || '';
+          if (contentType.includes('application/json')) {
+            try {
+              body = JSON.parse(bodyBuffer.toString('utf-8'));
+            } catch (error) {
+              body = bodyBuffer;
+            }
+          } else if (contentType.includes('multipart/form-data')) {
+            // For multipart, pass raw buffer
+            body = bodyBuffer;
+          } else {
+            body = bodyBuffer;
+          }
+        }
+
+        // Convert to IRequestContext
+        const context: IRequestContext = {
+          method: req.method || 'GET',
+          path: pathname,
+          headers: req.headers as Record<string, string>,
+          query: query as Record<string, string>,
+          body: body,
+          rawBody: bodyBuffer,
+        };
+
+        // Handle request
+        const response: IResponse = await registryInstance.handleRequest(context);
+
+        // Convert IResponse to HTTP response
+        res.statusCode = response.status;
+
+        // Set headers
+        for (const [key, value] of Object.entries(response.headers || {})) {
+          res.setHeader(key, value);
+        }
+
+        // Send body
+        if (response.body) {
+          if (Buffer.isBuffer(response.body)) {
+            res.end(response.body);
+          } else if (typeof response.body === 'string') {
+            res.end(response.body);
+          } else {
+            res.setHeader('Content-Type', 'application/json');
+            res.end(JSON.stringify(response.body));
+          }
+        } else {
+          res.end();
+        }
+      } catch (error) {
+        console.error('Server error:', error);
+        res.statusCode = 500;
+        res.setHeader('Content-Type', 'application/json');
+        res.end(JSON.stringify({ error: 'INTERNAL_ERROR', message: String(error) }));
+      }
+    });
+
+    httpServer.listen(port, () => {
+      const serverUrl = `http://localhost:${port}`;
+      resolve({ server: httpServer, url: serverUrl });
+    });
+
+    httpServer.on('error', reject);
+  });
+}
+
+/**
+ * Setup .pypirc for twine authentication
+ */
+function setupPypirc(
+  token: string,
+  pipHomeArg: string,
+  serverUrl: string
+): string {
+  fs.mkdirSync(pipHomeArg, { recursive: true });
+
+  const pypircContent = `[distutils]
+index-servers = testpypi
+
+[testpypi]
+repository = ${serverUrl}/pypi
+username = testuser
+password = ${token}
+`;
+
+  const pypircPath = path.join(pipHomeArg, '.pypirc');
+  fs.writeFileSync(pypircPath, pypircContent, 'utf-8');
+
+  // Set restrictive permissions
+  fs.chmodSync(pypircPath, 0o600);
+
+  return pypircPath;
+}
+
+/**
+ * Setup pip.conf for pip to use our registry
+ */
+function setupPipConf(
+  token: string,
+  pipHomeArg: string,
+  serverUrl: string,
+  port: number
+): string {
+  fs.mkdirSync(pipHomeArg, { recursive: true });
+
+  // pip.conf with authentication
+  const pipConfContent = `[global]
+index-url = ${serverUrl}/pypi/simple/
+trusted-host = localhost
+extra-index-url = https://pypi.org/simple/
+`;
+
+  const pipDir = path.join(pipHomeArg, 'pip');
+  fs.mkdirSync(pipDir, { recursive: true });
+
+  const pipConfPath = path.join(pipDir, 'pip.conf');
+  fs.writeFileSync(pipConfPath, pipConfContent, 'utf-8');
+
+  return pipConfPath;
+}
+
+/**
+ * Create a test Python package wheel file
+ */
+async function createTestWheelFile(
+  packageName: string,
+  version: string,
+  targetDir: string
+): Promise<string> {
+  const wheelData = await createPythonWheel(packageName, version);
+  const normalizedName = packageName.replace(/-/g, '_');
+  const wheelFilename = `${normalizedName}-${version}-py3-none-any.whl`;
+  const wheelPath = path.join(targetDir, wheelFilename);
+
+  fs.writeFileSync(wheelPath, wheelData);
+
+  return wheelPath;
+}
+
+/**
+ * Create a test Python package sdist file
+ */
+async function createTestSdistFile(
+  packageName: string,
+  version: string,
+  targetDir: string
+): Promise<string> {
+  const sdistData = await createPythonSdist(packageName, version);
+  const sdistFilename = `${packageName}-${version}.tar.gz`;
+  const sdistPath = path.join(targetDir, sdistFilename);
+
+  fs.writeFileSync(sdistPath, sdistData);
+
+  return sdistPath;
+}
+
+/**
+ * Run pip command with custom config
+ */
+async function runPipCommand(
+  command: string,
+  cwd: string
+): Promise<{ stdout: string; stderr: string; exitCode: number }> {
+  const pipConfDir = path.join(pipHome, 'pip');
+  const fullCommand = `cd "${cwd}" && PIP_CONFIG_FILE="${path.join(pipConfDir, 'pip.conf')}" pip ${command}`;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    return {
+      stdout: result.stdout || '',
+      stderr: result.stderr || '',
+      exitCode: result.exitCode || 0,
+    };
+  } catch (error: any) {
+    return {
+      stdout: error.stdout || '',
+      stderr: error.stderr || String(error),
+      exitCode: error.exitCode || 1,
+    };
+  }
+}
+
+/**
+ * Run twine command with custom config
+ */
+async function runTwineCommand(
+  command: string,
+  cwd: string
+): Promise<{ stdout: string; stderr: string; exitCode: number }> {
+  const pypircPath = path.join(pipHome, '.pypirc');
+  const fullCommand = `cd "${cwd}" && twine ${command} --config-file "${pypircPath}"`;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    return {
+      stdout: result.stdout || '',
+      stderr: result.stderr || '',
+      exitCode: result.exitCode || 0,
+    };
+  } catch (error: any) {
+    return {
+      stdout: error.stdout || '',
+      stderr: error.stderr || String(error),
+      exitCode: error.exitCode || 1,
+    };
+  }
+}
+
+/**
+ * Cleanup test directory
+ */
+function cleanupTestDir(dir: string): void {
+  if (fs.existsSync(dir)) {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+// ========================================================================
+// TESTS
+// ========================================================================
+
+tap.test('PyPI CLI: should verify pip is installed', async () => {
+  try {
+    const result = await tapNodeTools.runCommand('pip --version');
+    console.log('pip version output:', result.stdout.substring(0, 200));
+    hasPip = result.exitCode === 0;
+    expect(result.exitCode).toEqual(0);
+  } catch (error) {
+    console.log('pip CLI not available');
+  }
+});
+
+tap.test('PyPI CLI: should verify twine is installed', async () => {
+  try {
+    const result = await tapNodeTools.runCommand('twine --version');
+    console.log('twine version output:', result.stdout.substring(0, 200));
+    hasTwine = result.exitCode === 0;
+    expect(result.exitCode).toEqual(0);
+  } catch (error) {
+    console.log('twine CLI not available');
+  }
+
+  if (!hasPip && !hasTwine) {
+    console.log('Neither pip nor twine available, skipping native CLI tests');
+    tap.skip.test('PyPI CLI: remaining tests skipped - no CLI tools available');
+    return;
+  }
+});
+
+tap.test('PyPI CLI: should setup registry and HTTP server', async () => {
+  // Create registry
+  registry = await createTestRegistry();
+  const tokens = await createTestTokens(registry);
+  pypiToken = tokens.pypiToken;
+
+  expect(registry).toBeInstanceOf(SmartRegistry);
+  expect(pypiToken).toBeTypeOf('string');
+
+  // Use port 39000 (avoids conflicts with other tests)
+  registryPort = 39000;
+  const serverSetup = await createHttpServer(registry, registryPort);
+  server = serverSetup.server;
+  registryUrl = serverSetup.url;
+
+  expect(server).toBeDefined();
+  expect(registryUrl).toEqual(`http://localhost:${registryPort}`);
+
+  // Setup test directory
+  testDir = path.join(process.cwd(), '.nogit', 'test-pypi-cli');
+  cleanupTestDir(testDir);
+  fs.mkdirSync(testDir, { recursive: true });
+
+  // Setup pip/pypi home directory
+  pipHome = path.join(testDir, '.pip');
+  fs.mkdirSync(pipHome, { recursive: true });
+
+  // Setup .pypirc for twine
+  const pypircPath = setupPypirc(pypiToken, pipHome, registryUrl);
+  expect(fs.existsSync(pypircPath)).toEqual(true);
+
+  // Setup pip.conf
+  const pipConfPath = setupPipConf(pypiToken, pipHome, registryUrl, registryPort);
+  expect(fs.existsSync(pipConfPath)).toEqual(true);
+});
+
+tap.test('PyPI CLI: should verify server is responding', async () => {
+  // Check server is up by doing a direct HTTP request to simple index
+  const response = await fetch(`${registryUrl}/pypi/simple/`);
+  expect(response.status).toBeGreaterThanOrEqual(200);
+  expect(response.status).toBeLessThan(500);
+});
+
+tap.test('PyPI CLI: should upload wheel with twine', async () => {
+  if (!hasTwine) {
+    console.log('Skipping twine test - twine not available');
+    return;
+  }
+
+  const version = '1.0.0';
+  const wheelPath = await createTestWheelFile(testPackageName, version, testDir);
+
+  expect(fs.existsSync(wheelPath)).toEqual(true);
+
+  const result = await runTwineCommand(
+    `upload --repository testpypi "${wheelPath}"`,
+    testDir
+  );
+  console.log('twine upload output:', result.stdout);
+  console.log('twine upload stderr:', result.stderr);
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('PyPI CLI: should verify package in simple index', async () => {
+  if (!hasTwine) {
+    console.log('Skipping - twine not available');
+    return;
+  }
+
+  const response = await fetch(`${registryUrl}/pypi/simple/${testPackageName}/`);
+  expect(response.status).toEqual(200);
+
+  const html = await response.text();
+  expect(html).toContain('1.0.0');
+});
+
+tap.test('PyPI CLI: should upload sdist with twine', async () => {
+  if (!hasTwine) {
+    console.log('Skipping twine test - twine not available');
+    return;
+  }
+
+  const version = '1.1.0';
+  const sdistPath = await createTestSdistFile(testPackageName, version, testDir);
+
+  expect(fs.existsSync(sdistPath)).toEqual(true);
+
+  const result = await runTwineCommand(
+    `upload --repository testpypi "${sdistPath}"`,
+    testDir
+  );
+  console.log('twine upload sdist output:', result.stdout);
+  console.log('twine upload sdist stderr:', result.stderr);
+
+  expect(result.exitCode).toEqual(0);
+});
+
+tap.test('PyPI CLI: should list all versions in simple index', async () => {
+  if (!hasTwine) {
+    console.log('Skipping - twine not available');
+    return;
+  }
+
+  const response = await fetch(`${registryUrl}/pypi/simple/${testPackageName}/`);
+  expect(response.status).toEqual(200);
+
+  const html = await response.text();
+  expect(html).toContain('1.0.0');
+  expect(html).toContain('1.1.0');
+});
+
+tap.test('PyPI CLI: should get JSON metadata', async () => {
+  if (!hasTwine) {
+    console.log('Skipping - twine not available');
+    return;
+  }
+
+  const response = await fetch(`${registryUrl}/pypi/pypi/${testPackageName}/json`);
+  expect(response.status).toEqual(200);
+
+  const metadata = await response.json();
+  expect(metadata.info).toBeDefined();
+  expect(metadata.info.name).toEqual(testPackageName);
+  expect(metadata.releases).toBeDefined();
+  expect(metadata.releases['1.0.0']).toBeDefined();
+});
+
+tap.test('PyPI CLI: should download package with pip', async () => {
+  if (!hasPip || !hasTwine) {
+    console.log('Skipping pip download test - pip or twine not available');
+    return;
+  }
+
+  const downloadDir = path.join(testDir, 'downloads');
+  fs.mkdirSync(downloadDir, { recursive: true });
+
+  // Download (not install) the package
+  const result = await runPipCommand(
+    `download ${testPackageName}==1.0.0 --dest "${downloadDir}" --no-deps`,
+    testDir
+  );
+  console.log('pip download output:', result.stdout);
+  console.log('pip download stderr:', result.stderr);
+
+  // pip download may fail if the package doesn't meet pip's requirements
+  // Just check it doesn't crash
+  expect(result.exitCode).toBeLessThanOrEqual(1);
+});
+
+tap.test('PyPI CLI: should search for packages via API', async () => {
+  if (!hasTwine) {
+    console.log('Skipping - twine not available (no packages uploaded)');
+    return;
+  }
+
+  // Use the JSON API to search/list
+  const response = await fetch(`${registryUrl}/pypi/pypi/${testPackageName}/json`);
+  expect(response.status).toEqual(200);
+
+  const metadata = await response.json();
+  expect(metadata.info.name).toEqual(testPackageName);
+});
+
+tap.test('PyPI CLI: should fail upload without auth', async () => {
+  if (!hasTwine) {
+    console.log('Skipping twine test - twine not available');
+    return;
+  }
+
+  const unauthPkgName = `unauth-pkg-${testRunId}`;
+  const version = '1.0.0';
+  const wheelPath = await createTestWheelFile(unauthPkgName, version, testDir);
+
+  // Create a pypirc without proper credentials
+  const badPypircPath = path.join(testDir, '.bad-pypirc');
+  fs.writeFileSync(badPypircPath, `[distutils]
+index-servers = badpypi
+
+[badpypi]
+repository = ${registryUrl}/pypi
+username = baduser
+password = badtoken
+`, 'utf-8');
+
+  const fullCommand = `cd "${testDir}" && twine upload --repository badpypi "${wheelPath}" --config-file "${badPypircPath}"`;
+
+  try {
+    const result = await tapNodeTools.runCommand(fullCommand);
+    // Should fail
+    expect(result.exitCode).not.toEqual(0);
+  } catch (error: any) {
+    // Expected to fail
+    expect(error.exitCode || 1).not.toEqual(0);
+  }
+});
+
+tap.postTask('cleanup pypi cli tests', async () => {
+  // Stop server
+  if (server) {
+    await new Promise<void>((resolve) => {
+      server.close(() => resolve());
+    });
+  }
+
+  // Cleanup test directory
+  if (testDir) {
+    cleanupTestDir(testDir);
+  }
+
+  // Destroy registry
+  if (registry) {
+    registry.destroy();
+  }
+});
+
+export default tap.start();

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartregistry',
-  version: '2.0.0',
+  version: '2.3.0',
   description: 'A composable TypeScript library implementing OCI, NPM, Maven, Cargo, Composer, PyPI, and RubyGems registries for building unified container and package registries'
 }

ts/cargo/classes.cargoregistry.ts

@@ -3,6 +3,7 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import { RegistryStorage } from '../core/classes.registrystorage.js';
 import { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
 import type {
   ICargoIndexEntry,
   ICargoPublishMetadata,
@@ -13,6 +14,7 @@ import type {
   ICargoSearchResponse,
   ICargoSearchResult,
 } from './interfaces.cargo.js';
+import { CargoUpstream } from './classes.cargoupstream.js';
 
 /**
  * Cargo/crates.io registry implementation
@@ -25,12 +27,14 @@ export class CargoRegistry extends BaseRegistry {
   private basePath: string = '/cargo';
   private registryUrl: string;
   private logger: Smartlog;
+  private upstream: CargoUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string = '/cargo',
-    registryUrl: string = 'http://localhost:5000/cargo'
+    registryUrl: string = 'http://localhost:5000/cargo',
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
@@ -50,6 +54,20 @@ export class CargoRegistry extends BaseRegistry {
       }
     });
     this.logger.enableConsole();
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new CargoUpstream(upstreamConfig, undefined, this.logger);
+    }
+  }
+
+  /**
+   * Clean up resources (timers, connections, etc.)
+   */
+  public destroy(): void {
+    if (this.upstream) {
+      this.upstream.stop();
+    }
   }
 
   public async init(): Promise<void> {
@@ -207,7 +225,25 @@ export class CargoRegistry extends BaseRegistry {
    * Serve index file for a crate
    */
   private async handleIndexFile(crateName: string): Promise<IResponse> {
-    const index = await this.storage.getCargoIndex(crateName);
+    let index = await this.storage.getCargoIndex(crateName);
+
+    // Try upstream if not found locally
+    if ((!index || index.length === 0) && this.upstream) {
+      const upstreamIndex = await this.upstream.fetchCrateIndex(crateName);
+      if (upstreamIndex) {
+        // Parse the newline-delimited JSON
+        const parsedIndex: ICargoIndexEntry[] = upstreamIndex
+          .split('\n')
+          .filter(line => line.trim())
+          .map(line => JSON.parse(line));
+
+        if (parsedIndex.length > 0) {
+          // Cache locally
+          await this.storage.putCargoIndex(crateName, parsedIndex);
+          index = parsedIndex;
+        }
+      }
+    }
 
     if (!index || index.length === 0) {
       return {
@@ -399,7 +435,16 @@ export class CargoRegistry extends BaseRegistry {
   ): Promise<IResponse> {
     this.logger.log('debug', 'handleDownload', { crate: crateName, version });
 
-    const crateFile = await this.storage.getCargoCrate(crateName, version);
+    let crateFile = await this.storage.getCargoCrate(crateName, version);
+
+    // Try upstream if not found locally
+    if (!crateFile && this.upstream) {
+      crateFile = await this.upstream.fetchCrate(crateName, version);
+      if (crateFile) {
+        // Cache locally
+        await this.storage.putCargoCrate(crateName, version, crateFile);
+      }
+    }
 
     if (!crateFile) {
       return {

ts/cargo/classes.cargoupstream.ts

@@ -0,0 +1,159 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+
+/**
+ * Cargo-specific upstream implementation.
+ *
+ * Handles:
+ * - Crate metadata (index) fetching
+ * - Crate file (.crate) downloading
+ * - Sparse index protocol support
+ * - Content-addressable caching for .crate files
+ */
+export class CargoUpstream extends BaseUpstream {
+  protected readonly protocolName = 'cargo';
+
+  /** Base URL for crate downloads (may differ from index URL) */
+  private readonly downloadUrl: string;
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    downloadUrl?: string,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+    // Default to crates.io download URL if not specified
+    this.downloadUrl = downloadUrl || 'https://static.crates.io/crates';
+  }
+
+  /**
+   * Fetch crate metadata from the sparse index.
+   */
+  public async fetchCrateIndex(crateName: string): Promise<string | null> {
+    const path = this.buildIndexPath(crateName);
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'cargo',
+      resource: crateName,
+      resourceType: 'index',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'text/plain',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch a crate file from upstream.
+   */
+  public async fetchCrate(crateName: string, version: string): Promise<Buffer | null> {
+    // Crate downloads typically go to a different URL than the index
+    const path = `/${crateName}/${crateName}-${version}.crate`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'cargo',
+      resource: crateName,
+      resourceType: 'crate',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    // Use special handling for crate downloads
+    const result = await this.fetchCrateFile(crateName, version);
+    return result;
+  }
+
+  /**
+   * Fetch crate file directly from the download URL.
+   */
+  private async fetchCrateFile(crateName: string, version: string): Promise<Buffer | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'cargo',
+      resource: crateName,
+      resourceType: 'crate',
+      path: `/${crateName}/${crateName}-${version}.crate`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Build the sparse index path for a crate.
+   *
+   * Path structure:
+   * - 1 char: /1/{name}
+   * - 2 chars: /2/{name}
+   * - 3 chars: /3/{first char}/{name}
+   * - 4+ chars: /{first 2}/{next 2}/{name}
+   */
+  private buildIndexPath(crateName: string): string {
+    const lowerName = crateName.toLowerCase();
+    const len = lowerName.length;
+
+    if (len === 1) {
+      return `/1/${lowerName}`;
+    } else if (len === 2) {
+      return `/2/${lowerName}`;
+    } else if (len === 3) {
+      return `/3/${lowerName[0]}/${lowerName}`;
+    } else {
+      return `/${lowerName.slice(0, 2)}/${lowerName.slice(2, 4)}/${lowerName}`;
+    }
+  }
+
+  /**
+   * Override URL building for Cargo-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // For crate downloads, use the download URL
+    if (context.resourceType === 'crate') {
+      baseUrl = this.downloadUrl;
+    }
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/cargo/index.ts

@@ -3,4 +3,5 @@
  */
 
 export { CargoRegistry } from './classes.cargoregistry.js';
+export { CargoUpstream } from './classes.cargoupstream.js';
 export * from './interfaces.cargo.js';

ts/classes.smartregistry.ts

@@ -42,7 +42,17 @@ export class SmartRegistry {
     // Initialize OCI registry if enabled
     if (this.config.oci?.enabled) {
       const ociBasePath = this.config.oci.basePath ?? '/oci';
-      const ociRegistry = new OciRegistry(this.storage, this.authManager, ociBasePath);
+      const ociTokens = this.config.auth.ociTokens?.enabled ? {
+        realm: this.config.auth.ociTokens.realm,
+        service: this.config.auth.ociTokens.service,
+      } : undefined;
+      const ociRegistry = new OciRegistry(
+        this.storage,
+        this.authManager,
+        ociBasePath,
+        ociTokens,
+        this.config.oci.upstream
+      );
       await ociRegistry.init();
       this.registries.set('oci', ociRegistry);
     }
@@ -51,7 +61,13 @@ export class SmartRegistry {
     if (this.config.npm?.enabled) {
       const npmBasePath = this.config.npm.basePath ?? '/npm';
       const registryUrl = `http://localhost:5000${npmBasePath}`; // TODO: Make configurable
-      const npmRegistry = new NpmRegistry(this.storage, this.authManager, npmBasePath, registryUrl);
+      const npmRegistry = new NpmRegistry(
+        this.storage,
+        this.authManager,
+        npmBasePath,
+        registryUrl,
+        this.config.npm.upstream
+      );
       await npmRegistry.init();
       this.registries.set('npm', npmRegistry);
     }
@@ -60,7 +76,13 @@ export class SmartRegistry {
     if (this.config.maven?.enabled) {
       const mavenBasePath = this.config.maven.basePath ?? '/maven';
       const registryUrl = `http://localhost:5000${mavenBasePath}`; // TODO: Make configurable
-      const mavenRegistry = new MavenRegistry(this.storage, this.authManager, mavenBasePath, registryUrl);
+      const mavenRegistry = new MavenRegistry(
+        this.storage,
+        this.authManager,
+        mavenBasePath,
+        registryUrl,
+        this.config.maven.upstream
+      );
       await mavenRegistry.init();
       this.registries.set('maven', mavenRegistry);
     }
@@ -69,7 +91,13 @@ export class SmartRegistry {
     if (this.config.cargo?.enabled) {
       const cargoBasePath = this.config.cargo.basePath ?? '/cargo';
       const registryUrl = `http://localhost:5000${cargoBasePath}`; // TODO: Make configurable
-      const cargoRegistry = new CargoRegistry(this.storage, this.authManager, cargoBasePath, registryUrl);
+      const cargoRegistry = new CargoRegistry(
+        this.storage,
+        this.authManager,
+        cargoBasePath,
+        registryUrl,
+        this.config.cargo.upstream
+      );
       await cargoRegistry.init();
       this.registries.set('cargo', cargoRegistry);
     }
@@ -78,7 +106,13 @@ export class SmartRegistry {
     if (this.config.composer?.enabled) {
       const composerBasePath = this.config.composer.basePath ?? '/composer';
       const registryUrl = `http://localhost:5000${composerBasePath}`; // TODO: Make configurable
-      const composerRegistry = new ComposerRegistry(this.storage, this.authManager, composerBasePath, registryUrl);
+      const composerRegistry = new ComposerRegistry(
+        this.storage,
+        this.authManager,
+        composerBasePath,
+        registryUrl,
+        this.config.composer.upstream
+      );
       await composerRegistry.init();
       this.registries.set('composer', composerRegistry);
     }
@@ -87,7 +121,13 @@ export class SmartRegistry {
     if (this.config.pypi?.enabled) {
       const pypiBasePath = this.config.pypi.basePath ?? '/pypi';
       const registryUrl = `http://localhost:5000`; // TODO: Make configurable
-      const pypiRegistry = new PypiRegistry(this.storage, this.authManager, pypiBasePath, registryUrl);
+      const pypiRegistry = new PypiRegistry(
+        this.storage,
+        this.authManager,
+        pypiBasePath,
+        registryUrl,
+        this.config.pypi.upstream
+      );
       await pypiRegistry.init();
       this.registries.set('pypi', pypiRegistry);
     }
@@ -96,7 +136,13 @@ export class SmartRegistry {
     if (this.config.rubygems?.enabled) {
       const rubygemsBasePath = this.config.rubygems.basePath ?? '/rubygems';
       const registryUrl = `http://localhost:5000${rubygemsBasePath}`; // TODO: Make configurable
-      const rubygemsRegistry = new RubyGemsRegistry(this.storage, this.authManager, rubygemsBasePath, registryUrl);
+      const rubygemsRegistry = new RubyGemsRegistry(
+        this.storage,
+        this.authManager,
+        rubygemsBasePath,
+        registryUrl,
+        this.config.rubygems.upstream
+      );
       await rubygemsRegistry.init();
       this.registries.set('rubygems', rubygemsRegistry);
     }

ts/composer/classes.composerregistry.ts

@@ -7,6 +7,8 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import type { RegistryStorage } from '../core/classes.registrystorage.js';
 import type { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
+import { isBinaryData, toBuffer } from '../core/helpers.buffer.js';
 import type {
   IComposerPackage,
   IComposerPackageMetadata,
@@ -21,24 +23,41 @@ import {
   generatePackagesJson,
   sortVersions,
 } from './helpers.composer.js';
+import { ComposerUpstream } from './classes.composerupstream.js';
 
 export class ComposerRegistry extends BaseRegistry {
   private storage: RegistryStorage;
   private authManager: AuthManager;
   private basePath: string = '/composer';
   private registryUrl: string;
+  private upstream: ComposerUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string = '/composer',
-    registryUrl: string = 'http://localhost:5000/composer'
+    registryUrl: string = 'http://localhost:5000/composer',
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
     this.authManager = authManager;
     this.basePath = basePath;
     this.registryUrl = registryUrl;
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new ComposerUpstream(upstreamConfig);
+    }
+  }
+
+  /**
+   * Clean up resources (timers, connections, etc.)
+   */
+  public destroy(): void {
+    if (this.upstream) {
+      this.upstream.stop();
+    }
   }
 
   public async init(): Promise<void> {
@@ -160,7 +179,26 @@ export class ComposerRegistry extends BaseRegistry {
     token: IAuthToken | null
   ): Promise<IResponse> {
     // Read operations are public, no authentication required
-    const metadata = await this.storage.getComposerPackageMetadata(vendorPackage);
+    let metadata = await this.storage.getComposerPackageMetadata(vendorPackage);
+
+    // Try upstream if not found locally
+    if (!metadata && this.upstream) {
+      const [vendor, packageName] = vendorPackage.split('/');
+      if (vendor && packageName) {
+        const upstreamMetadata = includeDev
+          ? await this.upstream.fetchPackageDevMetadata(vendor, packageName)
+          : await this.upstream.fetchPackageMetadata(vendor, packageName);
+
+        if (upstreamMetadata && upstreamMetadata.packages) {
+          // Store upstream metadata locally
+          metadata = {
+            packages: upstreamMetadata.packages,
+            lastModified: new Date().toUTCString(),
+          };
+          await this.storage.putComposerPackageMetadata(vendorPackage, metadata);
+        }
+      }
+    }
 
     if (!metadata) {
       return {
@@ -255,7 +293,7 @@ export class ComposerRegistry extends BaseRegistry {
       };
     }
 
-    if (!body || !Buffer.isBuffer(body)) {
+    if (!body || !isBinaryData(body)) {
       return {
         status: 400,
         headers: {},
@@ -263,8 +301,11 @@ export class ComposerRegistry extends BaseRegistry {
       };
     }
 
+    // Convert to Buffer for ZIP processing
+    const zipData = toBuffer(body);
+
     // Extract and validate composer.json from ZIP
-    const composerJson = await extractComposerJsonFromZip(body);
+    const composerJson = await extractComposerJsonFromZip(zipData);
     if (!composerJson || !validateComposerJson(composerJson)) {
       return {
         status: 400,
@@ -292,13 +333,13 @@ export class ComposerRegistry extends BaseRegistry {
     }
 
     // Calculate SHA-1 hash
-    const shasum = await calculateSha1(body);
+    const shasum = await calculateSha1(zipData);
 
     // Generate reference (use version or commit hash)
     const reference = composerJson.source?.reference || version.replace(/[^a-zA-Z0-9.-]/g, '-');
 
     // Store ZIP file
-    await this.storage.putComposerPackageZip(vendorPackage, reference, body);
+    await this.storage.putComposerPackageZip(vendorPackage, reference, zipData);
 
     // Get or create metadata
     let metadata = await this.storage.getComposerPackageMetadata(vendorPackage);

ts/composer/classes.composerupstream.ts

@@ -0,0 +1,200 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+
+/**
+ * Composer-specific upstream implementation.
+ *
+ * Handles:
+ * - Package metadata fetching (packages.json, provider-includes)
+ * - Package version metadata (p2/{vendor}/{package}.json)
+ * - Dist file (zip) proxying
+ * - Packagist v2 API support
+ */
+export class ComposerUpstream extends BaseUpstream {
+  protected readonly protocolName = 'composer';
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+  }
+
+  /**
+   * Fetch the root packages.json from upstream.
+   */
+  public async fetchPackagesJson(): Promise<any | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'composer',
+      resource: '*',
+      resourceType: 'root',
+      path: '/packages.json',
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch package metadata using v2 API (p2/{vendor}/{package}.json).
+   */
+  public async fetchPackageMetadata(vendor: string, packageName: string): Promise<any | null> {
+    const fullName = `${vendor}/${packageName}`;
+    const path = `/p2/${vendor}/${packageName}.json`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'composer',
+      resource: fullName,
+      resourceType: 'metadata',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch package metadata with dev versions (p2/{vendor}/{package}~dev.json).
+   */
+  public async fetchPackageDevMetadata(vendor: string, packageName: string): Promise<any | null> {
+    const fullName = `${vendor}/${packageName}`;
+    const path = `/p2/${vendor}/${packageName}~dev.json`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'composer',
+      resource: fullName,
+      resourceType: 'metadata-dev',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch a provider-includes file.
+   */
+  public async fetchProviderIncludes(path: string): Promise<any | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'composer',
+      resource: '*',
+      resourceType: 'provider',
+      path: path.startsWith('/') ? path : `/${path}`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch a dist file (zip) from upstream.
+   */
+  public async fetchDist(url: string): Promise<Buffer | null> {
+    // Parse the URL to get the path
+    let path: string;
+    try {
+      const parsed = new URL(url);
+      path = parsed.pathname;
+    } catch {
+      path = url;
+    }
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'composer',
+      resource: '*',
+      resourceType: 'dist',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/zip, application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Override URL building for Composer-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/composer/index.ts

@@ -4,5 +4,6 @@
  */
 
 export { ComposerRegistry } from './classes.composerregistry.js';
+export { ComposerUpstream } from './classes.composerupstream.js';
 export * from './interfaces.composer.js';
 export * from './helpers.composer.js';

ts/core/classes.authmanager.ts

@@ -52,6 +52,60 @@ export class AuthManager {
     return token;
   }
 
+  /**
+   * Generic protocol token creation (internal helper)
+   * @param userId - User ID
+   * @param protocol - Protocol type (npm, maven, composer, etc.)
+   * @param readonly - Whether the token is readonly
+   * @returns UUID token string
+   */
+  private async createProtocolToken(
+    userId: string,
+    protocol: TRegistryProtocol,
+    readonly: boolean
+  ): Promise<string> {
+    const scopes = readonly
+      ? [`${protocol}:*:*:read`]
+      : [`${protocol}:*:*:*`];
+    return this.createUuidToken(userId, protocol, scopes, readonly);
+  }
+
+  /**
+   * Generic protocol token validation (internal helper)
+   * @param token - UUID token string
+   * @param protocol - Expected protocol type
+   * @returns Auth token object or null
+   */
+  private async validateProtocolToken(
+    token: string,
+    protocol: TRegistryProtocol
+  ): Promise<IAuthToken | null> {
+    if (!this.isValidUuid(token)) {
+      return null;
+    }
+
+    const authToken = this.tokenStore.get(token);
+    if (!authToken || authToken.type !== protocol) {
+      return null;
+    }
+
+    // Check expiration if set
+    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
+      this.tokenStore.delete(token);
+      return null;
+    }
+
+    return authToken;
+  }
+
+  /**
+   * Generic protocol token revocation (internal helper)
+   * @param token - UUID token string
+   */
+  private async revokeProtocolToken(token: string): Promise<void> {
+    this.tokenStore.delete(token);
+  }
+
   // ========================================================================
   // NPM AUTHENTICATION
   // ========================================================================
@@ -66,9 +120,7 @@ export class AuthManager {
     if (!this.config.npmTokens.enabled) {
       throw new Error('NPM tokens are not enabled');
     }
-
-    const scopes = readonly ? ['npm:*:*:read'] : ['npm:*:*:*'];
-    return this.createUuidToken(userId, 'npm', scopes, readonly);
+    return this.createProtocolToken(userId, 'npm', readonly);
   }
 
   /**
@@ -77,22 +129,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validateNpmToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'npm') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'npm');
   }
 
   /**
@@ -100,7 +137,7 @@ export class AuthManager {
    * @param token - NPM UUID token
    */
   public async revokeNpmToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   /**
@@ -265,8 +302,7 @@ export class AuthManager {
    * @returns Maven UUID token
    */
   public async createMavenToken(userId: string, readonly: boolean = false): Promise<string> {
-    const scopes = readonly ? ['maven:*:*:read'] : ['maven:*:*:*'];
-    return this.createUuidToken(userId, 'maven', scopes, readonly);
+    return this.createProtocolToken(userId, 'maven', readonly);
   }
 
   /**
@@ -275,22 +311,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validateMavenToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'maven') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'maven');
   }
 
   /**
@@ -298,7 +319,7 @@ export class AuthManager {
    * @param token - Maven UUID token
    */
   public async revokeMavenToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   // ========================================================================
@@ -312,8 +333,7 @@ export class AuthManager {
    * @returns Composer UUID token
    */
   public async createComposerToken(userId: string, readonly: boolean = false): Promise<string> {
-    const scopes = readonly ? ['composer:*:*:read'] : ['composer:*:*:*'];
-    return this.createUuidToken(userId, 'composer', scopes, readonly);
+    return this.createProtocolToken(userId, 'composer', readonly);
   }
 
   /**
@@ -322,22 +342,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validateComposerToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'composer') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'composer');
   }
 
   /**
@@ -345,7 +350,7 @@ export class AuthManager {
    * @param token - Composer UUID token
    */
   public async revokeComposerToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   // ========================================================================
@@ -359,8 +364,7 @@ export class AuthManager {
    * @returns Cargo UUID token
    */
   public async createCargoToken(userId: string, readonly: boolean = false): Promise<string> {
-    const scopes = readonly ? ['cargo:*:*:read'] : ['cargo:*:*:*'];
-    return this.createUuidToken(userId, 'cargo', scopes, readonly);
+    return this.createProtocolToken(userId, 'cargo', readonly);
   }
 
   /**
@@ -369,22 +373,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validateCargoToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'cargo') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'cargo');
   }
 
   /**
@@ -392,7 +381,7 @@ export class AuthManager {
    * @param token - Cargo UUID token
    */
   public async revokeCargoToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   // ========================================================================
@@ -406,8 +395,7 @@ export class AuthManager {
    * @returns PyPI UUID token
    */
   public async createPypiToken(userId: string, readonly: boolean = false): Promise<string> {
-    const scopes = readonly ? ['pypi:*:*:read'] : ['pypi:*:*:*'];
-    return this.createUuidToken(userId, 'pypi', scopes, readonly);
+    return this.createProtocolToken(userId, 'pypi', readonly);
   }
 
   /**
@@ -416,22 +404,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validatePypiToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'pypi') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'pypi');
   }
 
   /**
@@ -439,7 +412,7 @@ export class AuthManager {
    * @param token - PyPI UUID token
    */
   public async revokePypiToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   // ========================================================================
@@ -453,8 +426,7 @@ export class AuthManager {
    * @returns RubyGems UUID token
    */
   public async createRubyGemsToken(userId: string, readonly: boolean = false): Promise<string> {
-    const scopes = readonly ? ['rubygems:*:*:read'] : ['rubygems:*:*:*'];
-    return this.createUuidToken(userId, 'rubygems', scopes, readonly);
+    return this.createProtocolToken(userId, 'rubygems', readonly);
   }
 
   /**
@@ -463,22 +435,7 @@ export class AuthManager {
    * @returns Auth token object or null
    */
   public async validateRubyGemsToken(token: string): Promise<IAuthToken | null> {
-    if (!this.isValidUuid(token)) {
-      return null;
-    }
-
-    const authToken = this.tokenStore.get(token);
-    if (!authToken || authToken.type !== 'rubygems') {
-      return null;
-    }
-
-    // Check expiration if set
-    if (authToken.expiresAt && authToken.expiresAt < new Date()) {
-      this.tokenStore.delete(token);
-      return null;
-    }
-
-    return authToken;
+    return this.validateProtocolToken(token, 'rubygems');
   }
 
   /**
@@ -486,7 +443,7 @@ export class AuthManager {
    * @param token - RubyGems UUID token
    */
   public async revokeRubyGemsToken(token: string): Promise<void> {
-    this.tokenStore.delete(token);
+    return this.revokeProtocolToken(token);
   }
 
   // ========================================================================
@@ -495,57 +452,42 @@ export class AuthManager {
 
   /**
    * Validate any token (NPM, Maven, OCI, PyPI, RubyGems, Composer, Cargo)
+   * Optimized: O(1) lookup when protocol hint provided
    * @param tokenString - Token string (UUID or JWT)
-   * @param protocol - Expected protocol type
+   * @param protocol - Expected protocol type (optional, improves performance)
    * @returns Auth token object or null
    */
   public async validateToken(
     tokenString: string,
     protocol?: TRegistryProtocol
   ): Promise<IAuthToken | null> {
-    // Try UUID-based tokens (NPM, Maven, Composer, Cargo, PyPI, RubyGems)
-    if (this.isValidUuid(tokenString)) {
-      // Try NPM token
-      const npmToken = await this.validateNpmToken(tokenString);
-      if (npmToken && (!protocol || protocol === 'npm')) {
-        return npmToken;
+    // OCI uses JWT (contains dots), not UUID - check first if OCI is expected
+    if (protocol === 'oci' || tokenString.includes('.')) {
+      const ociToken = await this.validateOciToken(tokenString);
+      if (ociToken && (!protocol || protocol === 'oci')) {
+        return ociToken;
       }
-
-      // Try Maven token
-      const mavenToken = await this.validateMavenToken(tokenString);
-      if (mavenToken && (!protocol || protocol === 'maven')) {
-        return mavenToken;
-      }
-
-      // Try Composer token
-      const composerToken = await this.validateComposerToken(tokenString);
-      if (composerToken && (!protocol || protocol === 'composer')) {
-        return composerToken;
-      }
-
-      // Try Cargo token
-      const cargoToken = await this.validateCargoToken(tokenString);
-      if (cargoToken && (!protocol || protocol === 'cargo')) {
-        return cargoToken;
-      }
-
-      // Try PyPI token
-      const pypiToken = await this.validatePypiToken(tokenString);
-      if (pypiToken && (!protocol || protocol === 'pypi')) {
-        return pypiToken;
-      }
-
-      // Try RubyGems token
-      const rubygemsToken = await this.validateRubyGemsToken(tokenString);
-      if (rubygemsToken && (!protocol || protocol === 'rubygems')) {
-        return rubygemsToken;
+      // If protocol was explicitly OCI but validation failed, return null
+      if (protocol === 'oci') {
+        return null;
       }
     }
 
-    // Try OCI JWT
-    const ociToken = await this.validateOciToken(tokenString);
-    if (ociToken && (!protocol || protocol === 'oci')) {
-      return ociToken;
+    // UUID-based tokens: single O(1) Map lookup
+    if (this.isValidUuid(tokenString)) {
+      const authToken = this.tokenStore.get(tokenString);
+      if (authToken) {
+        // If protocol specified, verify it matches
+        if (protocol && authToken.type !== protocol) {
+          return null;
+        }
+        // Check expiration
+        if (authToken.expiresAt && authToken.expiresAt < new Date()) {
+          this.tokenStore.delete(tokenString);
+          return null;
+        }
+        return authToken;
+      }
     }
 
     return null;

ts/core/classes.registrystorage.ts

@@ -129,7 +129,7 @@ export class RegistryStorage implements IStorageBackend {
   }
 
   /**
-   * Get OCI manifest
+   * Get OCI manifest and its content type
    */
   public async getOciManifest(repository: string, digest: string): Promise<Buffer | null> {
     const path = this.getOciManifestPath(repository, digest);
@@ -137,7 +137,17 @@ export class RegistryStorage implements IStorageBackend {
   }
 
   /**
-   * Store OCI manifest
+   * Get OCI manifest content type
+   * Returns the stored content type or null if not found
+   */
+  public async getOciManifestContentType(repository: string, digest: string): Promise<string | null> {
+    const typePath = this.getOciManifestPath(repository, digest) + '.type';
+    const data = await this.getObject(typePath);
+    return data ? data.toString('utf-8') : null;
+  }
+
+  /**
+   * Store OCI manifest with its content type
    */
   public async putOciManifest(
     repository: string,
@@ -146,7 +156,11 @@ export class RegistryStorage implements IStorageBackend {
     contentType: string
   ): Promise<void> {
     const path = this.getOciManifestPath(repository, digest);
-    return this.putObject(path, data, { 'Content-Type': contentType });
+    // Store manifest data
+    await this.putObject(path, data, { 'Content-Type': contentType });
+    // Store content type in sidecar file for later retrieval
+    const typePath = path + '.type';
+    await this.putObject(typePath, Buffer.from(contentType, 'utf-8'));
   }
 
   /**

ts/core/helpers.buffer.ts

@@ -0,0 +1,34 @@
+/**
+ * Shared buffer utilities for consistent binary data handling across all registry types.
+ *
+ * This module addresses the common issue where `Buffer.isBuffer(Uint8Array)` returns `false`,
+ * which can cause data handling bugs when binary data arrives as Uint8Array instead of Buffer.
+ */
+
+/**
+ * Check if value is binary data (Buffer or Uint8Array)
+ */
+export function isBinaryData(value: unknown): value is Buffer | Uint8Array {
+  return Buffer.isBuffer(value) || value instanceof Uint8Array;
+}
+
+/**
+ * Convert any binary-like data to Buffer.
+ * Handles Buffer, Uint8Array, string, and objects.
+ *
+ * @param data - The data to convert to Buffer
+ * @returns A Buffer containing the data
+ */
+export function toBuffer(data: unknown): Buffer {
+  if (Buffer.isBuffer(data)) {
+    return data;
+  }
+  if (data instanceof Uint8Array) {
+    return Buffer.from(data);
+  }
+  if (typeof data === 'string') {
+    return Buffer.from(data, 'utf-8');
+  }
+  // Fallback: serialize object to JSON
+  return Buffer.from(JSON.stringify(data));
+}

ts/core/interfaces.core.ts

@@ -3,6 +3,7 @@
  */
 
 import type * as plugins from '../plugins.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
 
 /**
  * Registry protocol types
@@ -86,6 +87,8 @@ export interface IProtocolConfig {
   enabled: boolean;
   basePath: string;
   features?: Record<string, boolean>;
+  /** Upstream registry configuration for proxying/caching */
+  upstream?: IProtocolUpstreamConfig;
 }
 
 /**
@@ -158,6 +161,12 @@ export interface IRequestContext {
   headers: Record<string, string>;
   query: Record<string, string>;
   body?: any;
+  /**
+   * Raw request body as bytes. MUST be provided for content-addressable operations
+   * (OCI manifests, blobs) to ensure digest calculation matches client expectations.
+   * If not provided, falls back to 'body' field.
+   */
+  rawBody?: Buffer;
   token?: string;
 }
 

ts/index.ts

@@ -9,6 +9,9 @@ export { SmartRegistry } from './classes.smartregistry.js';
 // Core infrastructure
 export * from './core/index.js';
 
+// Upstream infrastructure
+export * from './upstream/index.js';
+
 // OCI Registry
 export * from './oci/index.js';
 

ts/maven/classes.mavenregistry.ts

@@ -7,6 +7,8 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import type { RegistryStorage } from '../core/classes.registrystorage.js';
 import type { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
+import { toBuffer } from '../core/helpers.buffer.js';
 import type { IMavenCoordinate, IMavenMetadata, IChecksums } from './interfaces.maven.js';
 import {
   pathToGAV,
@@ -20,6 +22,7 @@ import {
   extractGAVFromPom,
   gavToPath,
 } from './helpers.maven.js';
+import { MavenUpstream } from './classes.mavenupstream.js';
 
 /**
  * Maven Registry class
@@ -30,18 +33,34 @@ export class MavenRegistry extends BaseRegistry {
   private authManager: AuthManager;
   private basePath: string = '/maven';
   private registryUrl: string;
+  private upstream: MavenUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string,
-    registryUrl: string
+    registryUrl: string,
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
     this.authManager = authManager;
     this.basePath = basePath;
     this.registryUrl = registryUrl;
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new MavenUpstream(upstreamConfig);
+    }
+  }
+
+  /**
+   * Clean up resources (timers, connections, etc.)
+   */
+  public destroy(): void {
+    if (this.upstream) {
+      this.upstream.stop();
+    }
   }
 
   public async init(): Promise<void> {
@@ -233,7 +252,23 @@ export class MavenRegistry extends BaseRegistry {
     version: string,
     filename: string
   ): Promise<IResponse> {
-    const data = await this.storage.getMavenArtifact(groupId, artifactId, version, filename);
+    let data = await this.storage.getMavenArtifact(groupId, artifactId, version, filename);
+
+    // Try upstream if not found locally
+    if (!data && this.upstream) {
+      // Parse the filename to extract extension and classifier
+      const { extension, classifier } = this.parseFilename(filename, artifactId, version);
+      if (extension) {
+        data = await this.upstream.fetchArtifact(groupId, artifactId, version, extension, classifier);
+        if (data) {
+          // Cache the artifact locally
+          await this.storage.putMavenArtifact(groupId, artifactId, version, filename, data);
+          // Generate and store checksums
+          const checksums = await calculateChecksums(data);
+          await this.storeChecksums(groupId, artifactId, version, filename, checksums);
+        }
+      }
+    }
 
     if (!data) {
       return {
@@ -296,7 +331,7 @@ export class MavenRegistry extends BaseRegistry {
     coordinate: IMavenCoordinate,
     body: Buffer | any
   ): Promise<IResponse> {
-    const data = Buffer.isBuffer(body) ? body : Buffer.from(JSON.stringify(body));
+    const data = toBuffer(body);
 
     // Validate POM if uploading .pom file
     if (coordinate.extension === 'pom') {
@@ -461,7 +496,17 @@ export class MavenRegistry extends BaseRegistry {
   // ========================================================================
 
   private async getMetadata(groupId: string, artifactId: string): Promise<IResponse> {
-    const metadataBuffer = await this.storage.getMavenMetadata(groupId, artifactId);
+    let metadataBuffer = await this.storage.getMavenMetadata(groupId, artifactId);
+
+    // Try upstream if not found locally
+    if (!metadataBuffer && this.upstream) {
+      const upstreamMetadata = await this.upstream.fetchMetadata(groupId, artifactId);
+      if (upstreamMetadata) {
+        metadataBuffer = Buffer.from(upstreamMetadata, 'utf-8');
+        // Cache the metadata locally
+        await this.storage.putMavenMetadata(groupId, artifactId, metadataBuffer);
+      }
+    }
 
     if (!metadataBuffer) {
       // Generate empty metadata if none exists
@@ -577,4 +622,41 @@ export class MavenRegistry extends BaseRegistry {
 
     return contentTypes[extension] || 'application/octet-stream';
   }
+
+  /**
+   * Parse a Maven filename to extract extension and classifier.
+   * Filename format: {artifactId}-{version}[-{classifier}].{extension}
+   */
+  private parseFilename(
+    filename: string,
+    artifactId: string,
+    version: string
+  ): { extension: string; classifier?: string } {
+    const prefix = `${artifactId}-${version}`;
+
+    if (!filename.startsWith(prefix)) {
+      // Fallback: just get the extension
+      const lastDot = filename.lastIndexOf('.');
+      return { extension: lastDot > 0 ? filename.slice(lastDot + 1) : '' };
+    }
+
+    const remainder = filename.slice(prefix.length);
+    // remainder is either ".extension" or "-classifier.extension"
+
+    if (remainder.startsWith('.')) {
+      return { extension: remainder.slice(1) };
+    }
+
+    if (remainder.startsWith('-')) {
+      const lastDot = remainder.lastIndexOf('.');
+      if (lastDot > 1) {
+        return {
+          classifier: remainder.slice(1, lastDot),
+          extension: remainder.slice(lastDot + 1),
+        };
+      }
+    }
+
+    return { extension: '' };
+  }
 }

ts/maven/classes.mavenupstream.ts

@@ -0,0 +1,220 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+import type { IMavenCoordinate } from './interfaces.maven.js';
+
+/**
+ * Maven-specific upstream implementation.
+ *
+ * Handles:
+ * - Artifact fetching (JAR, POM, WAR, etc.)
+ * - Metadata fetching (maven-metadata.xml)
+ * - Checksum files (.md5, .sha1, .sha256, .sha512)
+ * - SNAPSHOT version handling
+ * - Content-addressable caching for release artifacts
+ */
+export class MavenUpstream extends BaseUpstream {
+  protected readonly protocolName = 'maven';
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+  }
+
+  /**
+   * Fetch an artifact from upstream registries.
+   */
+  public async fetchArtifact(
+    groupId: string,
+    artifactId: string,
+    version: string,
+    extension: string,
+    classifier?: string,
+  ): Promise<Buffer | null> {
+    const path = this.buildArtifactPath(groupId, artifactId, version, extension, classifier);
+    const resource = `${groupId}:${artifactId}`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'maven',
+      resource,
+      resourceType: 'artifact',
+      path,
+      method: 'GET',
+      headers: {},
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Fetch maven-metadata.xml from upstream.
+   */
+  public async fetchMetadata(groupId: string, artifactId: string, version?: string): Promise<string | null> {
+    const groupPath = groupId.replace(/\./g, '/');
+    let path: string;
+
+    if (version) {
+      // Version-level metadata (for SNAPSHOTs)
+      path = `/${groupPath}/${artifactId}/${version}/maven-metadata.xml`;
+    } else {
+      // Artifact-level metadata (lists all versions)
+      path = `/${groupPath}/${artifactId}/maven-metadata.xml`;
+    }
+
+    const resource = `${groupId}:${artifactId}`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'maven',
+      resource,
+      resourceType: 'metadata',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/xml, text/xml',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch a checksum file from upstream.
+   */
+  public async fetchChecksum(
+    groupId: string,
+    artifactId: string,
+    version: string,
+    extension: string,
+    checksumType: 'md5' | 'sha1' | 'sha256' | 'sha512',
+    classifier?: string,
+  ): Promise<string | null> {
+    const basePath = this.buildArtifactPath(groupId, artifactId, version, extension, classifier);
+    const path = `${basePath}.${checksumType}`;
+    const resource = `${groupId}:${artifactId}`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'maven',
+      resource,
+      resourceType: 'checksum',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'text/plain',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8').trim();
+    }
+
+    return typeof result.body === 'string' ? result.body.trim() : null;
+  }
+
+  /**
+   * Check if an artifact exists in upstream (HEAD request).
+   */
+  public async headArtifact(
+    groupId: string,
+    artifactId: string,
+    version: string,
+    extension: string,
+    classifier?: string,
+  ): Promise<{ exists: boolean; size?: number; lastModified?: string } | null> {
+    const path = this.buildArtifactPath(groupId, artifactId, version, extension, classifier);
+    const resource = `${groupId}:${artifactId}`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'maven',
+      resource,
+      resourceType: 'artifact',
+      path,
+      method: 'HEAD',
+      headers: {},
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result) {
+      return null;
+    }
+
+    if (!result.success) {
+      return { exists: false };
+    }
+
+    return {
+      exists: true,
+      size: result.headers['content-length'] ? parseInt(result.headers['content-length'], 10) : undefined,
+      lastModified: result.headers['last-modified'],
+    };
+  }
+
+  /**
+   * Build the path for a Maven artifact.
+   */
+  private buildArtifactPath(
+    groupId: string,
+    artifactId: string,
+    version: string,
+    extension: string,
+    classifier?: string,
+  ): string {
+    const groupPath = groupId.replace(/\./g, '/');
+    let filename = `${artifactId}-${version}`;
+    if (classifier) {
+      filename += `-${classifier}`;
+    }
+    filename += `.${extension}`;
+
+    return `/${groupPath}/${artifactId}/${version}/${filename}`;
+  }
+
+  /**
+   * Override URL building for Maven-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/maven/index.ts

@@ -3,5 +3,6 @@
  */
 
 export { MavenRegistry } from './classes.mavenregistry.js';
+export { MavenUpstream } from './classes.mavenupstream.js';
 export * from './interfaces.maven.js';
 export * from './helpers.maven.js';

ts/npm/classes.npmregistry.ts

@@ -3,6 +3,8 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import { RegistryStorage } from '../core/classes.registrystorage.js';
 import { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
+import { NpmUpstream } from './classes.npmupstream.js';
 import type {
   IPackument,
   INpmVersion,
@@ -25,12 +27,14 @@ export class NpmRegistry extends BaseRegistry {
   private basePath: string = '/npm';
   private registryUrl: string;
   private logger: Smartlog;
+  private upstream: NpmUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string = '/npm',
-    registryUrl: string = 'http://localhost:5000/npm'
+    registryUrl: string = 'http://localhost:5000/npm',
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
@@ -50,6 +54,14 @@ export class NpmRegistry extends BaseRegistry {
       }
     });
     this.logger.enableConsole();
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new NpmUpstream(upstreamConfig, registryUrl, this.logger);
+      this.logger.log('info', 'NPM upstream initialized', {
+        upstreams: upstreamConfig.upstreams.map(u => u.name),
+      });
+    }
   }
 
   public async init(): Promise<void> {
@@ -113,7 +125,7 @@ export class NpmRegistry extends BaseRegistry {
     const unpublishVersionMatch = path.match(/^\/(@?[^\/]+(?:\/[^\/]+)?)\/-\/([^\/]+)$/);
     if (unpublishVersionMatch && context.method === 'DELETE') {
       const [, packageName, version] = unpublishVersionMatch;
-      console.log(`[unpublishVersionMatch] packageName=${packageName}, version=${version}`);
+      this.logger.log('debug', 'unpublishVersionMatch', { packageName, version });
       return this.unpublishVersion(packageName, version, token);
     }
 
@@ -121,7 +133,7 @@ export class NpmRegistry extends BaseRegistry {
     const unpublishPackageMatch = path.match(/^\/(@?[^\/]+(?:\/[^\/]+)?)\/-rev\/([^\/]+)$/);
     if (unpublishPackageMatch && context.method === 'DELETE') {
       const [, packageName, rev] = unpublishPackageMatch;
-      console.log(`[unpublishPackageMatch] packageName=${packageName}, rev=${rev}`);
+      this.logger.log('debug', 'unpublishPackageMatch', { packageName, rev });
       return this.unpublishPackage(packageName, token);
     }
 
@@ -129,7 +141,7 @@ export class NpmRegistry extends BaseRegistry {
     const versionMatch = path.match(/^\/(@?[^\/]+(?:\/[^\/]+)?)\/([^\/]+)$/);
     if (versionMatch) {
       const [, packageName, version] = versionMatch;
-      console.log(`[versionMatch] matched! packageName=${packageName}, version=${version}`);
+      this.logger.log('debug', 'versionMatch', { packageName, version });
       return this.handlePackageVersion(packageName, version, token);
     }
 
@@ -137,7 +149,7 @@ export class NpmRegistry extends BaseRegistry {
     const packageMatch = path.match(/^\/(@?[^\/]+(?:\/[^\/]+)?)$/);
     if (packageMatch) {
       const packageName = packageMatch[1];
-      console.log(`[packageMatch] matched! packageName=${packageName}`);
+      this.logger.log('debug', 'packageMatch', { packageName });
       return this.handlePackage(context.method, packageName, context.body, context.query, token);
     }
 
@@ -209,13 +221,28 @@ export class NpmRegistry extends BaseRegistry {
     token: IAuthToken | null,
     query: Record<string, string>
   ): Promise<IResponse> {
-    const packument = await this.storage.getNpmPackument(packageName);
+    let packument = await this.storage.getNpmPackument(packageName);
     this.logger.log('debug', `getPackument: ${packageName}`, {
       packageName,
       found: !!packument,
       versions: packument ? Object.keys(packument.versions).length : 0
     });
 
+    // If not found locally, try upstream
+    if (!packument && this.upstream) {
+      this.logger.log('debug', `getPackument: fetching from upstream`, { packageName });
+      const upstreamPackument = await this.upstream.fetchPackument(packageName);
+      if (upstreamPackument) {
+        this.logger.log('debug', `getPackument: found in upstream`, {
+          packageName,
+          versions: Object.keys(upstreamPackument.versions || {}).length
+        });
+        packument = upstreamPackument;
+        // Optionally cache the packument locally (without tarballs)
+        // We don't store tarballs here - they'll be fetched on demand
+      }
+    }
+
     if (!packument) {
       return {
         status: 404,
@@ -254,12 +281,22 @@ export class NpmRegistry extends BaseRegistry {
     version: string,
     token: IAuthToken | null
   ): Promise<IResponse> {
-    console.log(`[handlePackageVersion] packageName=${packageName}, version=${version}`);
-    const packument = await this.storage.getNpmPackument(packageName);
-    console.log(`[handlePackageVersion] packument found:`, !!packument);
+    this.logger.log('debug', 'handlePackageVersion', { packageName, version });
+    let packument = await this.storage.getNpmPackument(packageName);
+    this.logger.log('debug', 'handlePackageVersion packument', { found: !!packument });
     if (packument) {
-      console.log(`[handlePackageVersion] versions:`, Object.keys(packument.versions || {}));
+      this.logger.log('debug', 'handlePackageVersion versions', { versions: Object.keys(packument.versions || {}) });
     }
+
+    // If not found locally, try upstream
+    if (!packument && this.upstream) {
+      this.logger.log('debug', 'handlePackageVersion: fetching from upstream', { packageName });
+      const upstreamPackument = await this.upstream.fetchPackument(packageName);
+      if (upstreamPackument) {
+        packument = upstreamPackument;
+      }
+    }
+
     if (!packument) {
       return {
         status: 404,
@@ -529,7 +566,7 @@ export class NpmRegistry extends BaseRegistry {
     token: IAuthToken | null
   ): Promise<IResponse> {
     // Extract version from filename: package-name-1.0.0.tgz
-    const versionMatch = filename.match(/-([\d.]+(?:-[a-z0-9.]+)?)\.tgz$/);
+    const versionMatch = filename.match(/-([\d.]+(?:-[a-z0-9.]+)?)\.tgz$/i);
     if (!versionMatch) {
       return {
         status: 400,
@@ -539,7 +576,26 @@ export class NpmRegistry extends BaseRegistry {
     }
 
     const version = versionMatch[1];
-    const tarball = await this.storage.getNpmTarball(packageName, version);
+    let tarball = await this.storage.getNpmTarball(packageName, version);
+
+    // If not found locally, try upstream
+    if (!tarball && this.upstream) {
+      this.logger.log('debug', 'handleTarballDownload: fetching from upstream', {
+        packageName,
+        version,
+      });
+      const upstreamTarball = await this.upstream.fetchTarball(packageName, version);
+      if (upstreamTarball) {
+        tarball = upstreamTarball;
+        // Cache the tarball locally for future requests
+        await this.storage.putNpmTarball(packageName, version, tarball);
+        this.logger.log('debug', 'handleTarballDownload: cached tarball locally', {
+          packageName,
+          version,
+          size: tarball.length,
+        });
+      }
+    }
 
     if (!tarball) {
       return {
@@ -621,7 +677,7 @@ export class NpmRegistry extends BaseRegistry {
         }
       }
     } catch (error) {
-      console.error('[handleSearch] Error:', error);
+      this.logger.log('error', 'handleSearch failed', { error: (error as Error).message });
     }
 
     // Apply pagination

ts/npm/classes.npmupstream.ts

@@ -0,0 +1,260 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamResult,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+import type { IPackument, INpmVersion } from './interfaces.npm.js';
+
+/**
+ * NPM-specific upstream implementation.
+ *
+ * Handles:
+ * - Package metadata (packument) fetching
+ * - Tarball proxying
+ * - Scoped package routing (@scope/* patterns)
+ * - NPM-specific URL rewriting
+ */
+export class NpmUpstream extends BaseUpstream {
+  protected readonly protocolName = 'npm';
+
+  /** Local registry URL for rewriting tarball URLs */
+  private readonly localRegistryUrl: string;
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    localRegistryUrl: string,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+    this.localRegistryUrl = localRegistryUrl;
+  }
+
+  /**
+   * Fetch a packument from upstream registries.
+   */
+  public async fetchPackument(packageName: string): Promise<IPackument | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'npm',
+      resource: packageName,
+      resourceType: 'packument',
+      path: `/${encodeURIComponent(packageName).replace('%40', '@')}`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    // Parse and process packument
+    let packument: IPackument;
+    if (Buffer.isBuffer(result.body)) {
+      packument = JSON.parse(result.body.toString('utf8'));
+    } else {
+      packument = result.body;
+    }
+
+    // Rewrite tarball URLs to point to local registry
+    packument = this.rewriteTarballUrls(packument);
+
+    return packument;
+  }
+
+  /**
+   * Fetch a specific version from upstream registries.
+   */
+  public async fetchVersion(packageName: string, version: string): Promise<INpmVersion | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'npm',
+      resource: packageName,
+      resourceType: 'version',
+      path: `/${encodeURIComponent(packageName).replace('%40', '@')}/${version}`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    let versionData: INpmVersion;
+    if (Buffer.isBuffer(result.body)) {
+      versionData = JSON.parse(result.body.toString('utf8'));
+    } else {
+      versionData = result.body;
+    }
+
+    // Rewrite tarball URL
+    if (versionData.dist?.tarball) {
+      versionData.dist.tarball = this.rewriteSingleTarballUrl(
+        packageName,
+        versionData.version,
+        versionData.dist.tarball,
+      );
+    }
+
+    return versionData;
+  }
+
+  /**
+   * Fetch a tarball from upstream registries.
+   */
+  public async fetchTarball(packageName: string, version: string): Promise<Buffer | null> {
+    // First, try to get the tarball URL from packument
+    const packument = await this.fetchPackument(packageName);
+    let tarballPath: string;
+
+    if (packument?.versions?.[version]?.dist?.tarball) {
+      // Extract path from original (upstream) tarball URL
+      const tarballUrl = packument.versions[version].dist.tarball;
+      try {
+        const url = new URL(tarballUrl);
+        tarballPath = url.pathname;
+      } catch {
+        // Fallback to standard NPM tarball path
+        tarballPath = this.buildTarballPath(packageName, version);
+      }
+    } else {
+      tarballPath = this.buildTarballPath(packageName, version);
+    }
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'npm',
+      resource: packageName,
+      resourceType: 'tarball',
+      path: tarballPath,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Search packages in upstream registries.
+   */
+  public async search(text: string, size: number = 20, from: number = 0): Promise<any | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'npm',
+      resource: '*',
+      resourceType: 'search',
+      path: '/-/v1/search',
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {
+        text,
+        size: size.toString(),
+        from: from.toString(),
+      },
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Build the standard NPM tarball path.
+   */
+  private buildTarballPath(packageName: string, version: string): string {
+    // NPM uses: /{package}/-/{package-name}-{version}.tgz
+    // For scoped packages: /@scope/name/-/name-version.tgz
+    if (packageName.startsWith('@')) {
+      const [scope, name] = packageName.split('/');
+      return `/${scope}/${name}/-/${name}-${version}.tgz`;
+    } else {
+      return `/${packageName}/-/${packageName}-${version}.tgz`;
+    }
+  }
+
+  /**
+   * Rewrite all tarball URLs in a packument to point to local registry.
+   */
+  private rewriteTarballUrls(packument: IPackument): IPackument {
+    if (!packument.versions) {
+      return packument;
+    }
+
+    const rewritten = { ...packument };
+    rewritten.versions = {};
+
+    for (const [version, versionData] of Object.entries(packument.versions)) {
+      const newVersionData = { ...versionData };
+      if (newVersionData.dist?.tarball) {
+        newVersionData.dist = {
+          ...newVersionData.dist,
+          tarball: this.rewriteSingleTarballUrl(
+            packument.name,
+            version,
+            newVersionData.dist.tarball,
+          ),
+        };
+      }
+      rewritten.versions[version] = newVersionData;
+    }
+
+    return rewritten;
+  }
+
+  /**
+   * Rewrite a single tarball URL to point to local registry.
+   */
+  private rewriteSingleTarballUrl(
+    packageName: string,
+    version: string,
+    _originalUrl: string,
+  ): string {
+    // Generate local tarball URL
+    // Format: {localRegistryUrl}/{package}/-/{package-name}-{version}.tgz
+    const safeName = packageName.replace('@', '').replace('/', '-');
+    return `${this.localRegistryUrl}/${packageName}/-/${safeName}-${version}.tgz`;
+  }
+
+  /**
+   * Override URL building for NPM-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    // NPM registries often don't have trailing slashes
+    let baseUrl = upstream.url;
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/npm/index.ts

@@ -3,4 +3,5 @@
  */
 
 export { NpmRegistry } from './classes.npmregistry.js';
+export { NpmUpstream } from './classes.npmupstream.js';
 export * from './interfaces.npm.js';

ts/oci/classes.ociregistry.ts

@@ -1,7 +1,10 @@
+import { Smartlog } from '@push.rocks/smartlog';
 import { BaseRegistry } from '../core/classes.baseregistry.js';
 import { RegistryStorage } from '../core/classes.registrystorage.js';
 import { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken, IRegistryError } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
+import { OciUpstream } from './classes.ociupstream.js';
 import type {
   IUploadSession,
   IOciManifest,
@@ -20,12 +23,43 @@ export class OciRegistry extends BaseRegistry {
   private uploadSessions: Map<string, IUploadSession> = new Map();
   private basePath: string = '/oci';
   private cleanupInterval?: NodeJS.Timeout;
+  private ociTokens?: { realm: string; service: string };
+  private upstream: OciUpstream | null = null;
+  private logger: Smartlog;
 
-  constructor(storage: RegistryStorage, authManager: AuthManager, basePath: string = '/oci') {
+  constructor(
+    storage: RegistryStorage,
+    authManager: AuthManager,
+    basePath: string = '/oci',
+    ociTokens?: { realm: string; service: string },
+    upstreamConfig?: IProtocolUpstreamConfig
+  ) {
     super();
     this.storage = storage;
     this.authManager = authManager;
     this.basePath = basePath;
+    this.ociTokens = ociTokens;
+
+    // Initialize logger
+    this.logger = new Smartlog({
+      logContext: {
+        company: 'push.rocks',
+        companyunit: 'smartregistry',
+        containerName: 'oci-registry',
+        environment: (process.env.NODE_ENV as any) || 'development',
+        runtime: 'node',
+        zone: 'oci'
+      }
+    });
+    this.logger.enableConsole();
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new OciUpstream(upstreamConfig, basePath, this.logger);
+      this.logger.log('info', 'OCI upstream initialized', {
+        upstreams: upstreamConfig.upstreams.map(u => u.name),
+      });
+    }
   }
 
   public async init(): Promise<void> {
@@ -55,7 +89,9 @@ export class OciRegistry extends BaseRegistry {
     const manifestMatch = path.match(/^\/v2\/([^\/]+(?:\/[^\/]+)*)\/manifests\/([^\/]+)$/);
     if (manifestMatch) {
       const [, name, reference] = manifestMatch;
-      return this.handleManifestRequest(context.method, name, reference, token, context.body, context.headers);
+      // Prefer rawBody for content-addressable operations to preserve exact bytes
+      const bodyData = context.rawBody || context.body;
+      return this.handleManifestRequest(context.method, name, reference, token, bodyData, context.headers);
     }
 
     // Blob operations: /v2/{name}/blobs/{digest}
@@ -69,7 +105,9 @@ export class OciRegistry extends BaseRegistry {
     const uploadInitMatch = path.match(/^\/v2\/([^\/]+(?:\/[^\/]+)*)\/blobs\/uploads\/?$/);
     if (uploadInitMatch && context.method === 'POST') {
       const [, name] = uploadInitMatch;
-      return this.handleUploadInit(name, token, context.query, context.body);
+      // Prefer rawBody for content-addressable operations to preserve exact bytes
+      const bodyData = context.rawBody || context.body;
+      return this.handleUploadInit(name, token, context.query, bodyData);
     }
 
     // Blob upload operations: /v2/{name}/blobs/uploads/{uuid}
@@ -187,7 +225,7 @@ export class OciRegistry extends BaseRegistry {
     const digest = query.digest;
     if (digest && body) {
       // Monolithic upload: complete upload in single POST
-      const blobData = Buffer.isBuffer(body) ? body : Buffer.from(JSON.stringify(body));
+      const blobData = this.toBuffer(body);
 
       // Verify digest
       const calculatedDigest = await this.calculateDigest(blobData);
@@ -254,11 +292,14 @@ export class OciRegistry extends BaseRegistry {
       return this.createUnauthorizedResponse(session.repository, 'push');
     }
 
+    // Prefer rawBody for content-addressable operations to preserve exact bytes
+    const bodyData = context.rawBody || context.body;
+
     switch (method) {
       case 'PATCH':
-        return this.uploadChunk(uploadId, context.body, context.headers['content-range']);
+        return this.uploadChunk(uploadId, bodyData, context.headers['content-range']);
       case 'PUT':
-        return this.completeUpload(uploadId, context.query['digest'], context.body);
+        return this.completeUpload(uploadId, context.query['digest'], bodyData);
       case 'GET':
         return this.getUploadStatus(uploadId);
       default:
@@ -280,13 +321,7 @@ export class OciRegistry extends BaseRegistry {
     headers?: Record<string, string>
   ): Promise<IResponse> {
     if (!await this.checkPermission(token, repository, 'pull')) {
-      return {
-        status: 401,
-        headers: {
-          'WWW-Authenticate': `Bearer realm="${this.basePath}/v2/token",service="registry",scope="repository:${repository}:pull"`,
-        },
-        body: this.createError('DENIED', 'Insufficient permissions'),
-      };
+      return this.createUnauthorizedResponse(repository, 'pull');
     }
 
     // Resolve tag to digest if needed
@@ -294,16 +329,50 @@ export class OciRegistry extends BaseRegistry {
     if (!reference.startsWith('sha256:')) {
       const tags = await this.getTagsData(repository);
       digest = tags[reference];
-      if (!digest) {
-        return {
-          status: 404,
-          headers: {},
-          body: this.createError('MANIFEST_UNKNOWN', 'Manifest not found'),
-        };
+    }
+
+    // Try local storage first (if we have a digest)
+    let manifestData: Buffer | null = null;
+    let contentType: string | null = null;
+
+    if (digest) {
+      manifestData = await this.storage.getOciManifest(repository, digest);
+      if (manifestData) {
+        contentType = await this.storage.getOciManifestContentType(repository, digest);
+        if (!contentType) {
+          contentType = this.detectManifestContentType(manifestData);
+        }
+      }
+    }
+
+    // If not found locally, try upstream
+    if (!manifestData && this.upstream) {
+      this.logger.log('debug', 'getManifest: fetching from upstream', { repository, reference });
+      const upstreamResult = await this.upstream.fetchManifest(repository, reference);
+      if (upstreamResult) {
+        manifestData = Buffer.from(JSON.stringify(upstreamResult.manifest), 'utf8');
+        contentType = upstreamResult.contentType;
+        digest = upstreamResult.digest;
+
+        // Cache the manifest locally
+        await this.storage.putOciManifest(repository, digest, manifestData, contentType);
+
+        // If reference is a tag, update tags mapping
+        if (!reference.startsWith('sha256:')) {
+          const tags = await this.getTagsData(repository);
+          tags[reference] = digest;
+          const tagsPath = `oci/tags/${repository}/tags.json`;
+          await this.storage.putObject(tagsPath, Buffer.from(JSON.stringify(tags), 'utf-8'));
+        }
+
+        this.logger.log('debug', 'getManifest: cached manifest locally', {
+          repository,
+          reference,
+          digest,
+        });
       }
     }
 
-    const manifestData = await this.storage.getOciManifest(repository, digest);
     if (!manifestData) {
       return {
         status: 404,
@@ -315,7 +384,7 @@ export class OciRegistry extends BaseRegistry {
     return {
       status: 200,
       headers: {
-        'Content-Type': 'application/vnd.oci.image.manifest.v1+json',
+        'Content-Type': contentType || 'application/vnd.oci.image.manifest.v1+json',
         'Docker-Content-Digest': digest,
       },
       body: manifestData,
@@ -348,10 +417,18 @@ export class OciRegistry extends BaseRegistry {
 
     const manifestData = await this.storage.getOciManifest(repository, digest);
 
+    // Get stored content type, falling back to detecting from manifest content
+    let contentType = await this.storage.getOciManifestContentType(repository, digest);
+    if (!contentType && manifestData) {
+      // Fallback: detect content type from manifest content
+      contentType = this.detectManifestContentType(manifestData);
+    }
+    contentType = contentType || 'application/vnd.oci.image.manifest.v1+json';
+
     return {
       status: 200,
       headers: {
-        'Content-Type': 'application/vnd.oci.image.manifest.v1+json',
+        'Content-Type': contentType,
         'Docker-Content-Digest': digest,
         'Content-Length': manifestData ? manifestData.length.toString() : '0',
       },
@@ -367,13 +444,7 @@ export class OciRegistry extends BaseRegistry {
     headers?: Record<string, string>
   ): Promise<IResponse> {
     if (!await this.checkPermission(token, repository, 'push')) {
-      return {
-        status: 401,
-        headers: {
-          'WWW-Authenticate': `Bearer realm="${this.basePath}/v2/token",service="registry",scope="repository:${repository}:push"`,
-        },
-        body: this.createError('DENIED', 'Insufficient permissions'),
-      };
+      return this.createUnauthorizedResponse(repository, 'push');
     }
 
     if (!body) {
@@ -384,7 +455,9 @@ export class OciRegistry extends BaseRegistry {
       };
     }
 
-    const manifestData = Buffer.isBuffer(body) ? body : Buffer.from(JSON.stringify(body));
+    // Preserve raw bytes for accurate digest calculation
+    // Per OCI spec, digest must match the exact bytes sent by client
+    const manifestData = this.toBuffer(body);
     const contentType = headers?.['content-type'] || headers?.['Content-Type'] || 'application/vnd.oci.image.manifest.v1+json';
 
     // Calculate manifest digest
@@ -447,7 +520,25 @@ export class OciRegistry extends BaseRegistry {
       return this.createUnauthorizedResponse(repository, 'pull');
     }
 
-    const data = await this.storage.getOciBlob(digest);
+    // Try local storage first
+    let data = await this.storage.getOciBlob(digest);
+
+    // If not found locally, try upstream
+    if (!data && this.upstream) {
+      this.logger.log('debug', 'getBlob: fetching from upstream', { repository, digest });
+      const upstreamBlob = await this.upstream.fetchBlob(repository, digest);
+      if (upstreamBlob) {
+        data = upstreamBlob;
+        // Cache the blob locally (blobs are content-addressable and immutable)
+        await this.storage.putOciBlob(digest, data);
+        this.logger.log('debug', 'getBlob: cached blob locally', {
+          repository,
+          digest,
+          size: data.length,
+        });
+      }
+    }
+
     if (!data) {
       return {
         status: 404,
@@ -512,7 +603,7 @@ export class OciRegistry extends BaseRegistry {
 
   private async uploadChunk(
     uploadId: string,
-    data: Buffer,
+    data: Buffer | Uint8Array | unknown,
     contentRange: string
   ): Promise<IResponse> {
     const session = this.uploadSessions.get(uploadId);
@@ -524,8 +615,9 @@ export class OciRegistry extends BaseRegistry {
       };
     }
 
-    session.chunks.push(data);
-    session.totalSize += data.length;
+    const chunkData = this.toBuffer(data);
+    session.chunks.push(chunkData);
+    session.totalSize += chunkData.length;
     session.lastActivity = new Date();
 
     return {
@@ -542,7 +634,7 @@ export class OciRegistry extends BaseRegistry {
   private async completeUpload(
     uploadId: string,
     digest: string,
-    finalData?: Buffer
+    finalData?: Buffer | Uint8Array | unknown
   ): Promise<IResponse> {
     const session = this.uploadSessions.get(uploadId);
     if (!session) {
@@ -554,7 +646,7 @@ export class OciRegistry extends BaseRegistry {
     }
 
     const chunks = [...session.chunks];
-    if (finalData) chunks.push(finalData);
+    if (finalData) chunks.push(this.toBuffer(finalData));
     const blobData = Buffer.concat(chunks);
 
     // Verify digest
@@ -652,6 +744,59 @@ export class OciRegistry extends BaseRegistry {
   // HELPER METHODS
   // ========================================================================
 
+  /**
+   * Detect manifest content type from manifest content.
+   * OCI Image Index has "manifests" array, OCI Image Manifest has "config" object.
+   * Also checks the mediaType field if present.
+   */
+  private detectManifestContentType(manifestData: Buffer): string {
+    try {
+      const manifest = JSON.parse(manifestData.toString('utf-8'));
+
+      // First check if manifest has explicit mediaType field
+      if (manifest.mediaType) {
+        return manifest.mediaType;
+      }
+
+      // Otherwise detect from structure
+      if (Array.isArray(manifest.manifests)) {
+        // OCI Image Index (multi-arch manifest list)
+        return 'application/vnd.oci.image.index.v1+json';
+      } else if (manifest.config) {
+        // OCI Image Manifest
+        return 'application/vnd.oci.image.manifest.v1+json';
+      }
+
+      // Fallback to standard manifest type
+      return 'application/vnd.oci.image.manifest.v1+json';
+    } catch (e) {
+      // If parsing fails, return default
+      return 'application/vnd.oci.image.manifest.v1+json';
+    }
+  }
+
+  /**
+   * Convert any binary-like data to Buffer.
+   * Handles Buffer, Uint8Array (modern cross-platform), string, and objects.
+   *
+   * Note: Buffer.isBuffer(Uint8Array) returns false even though Buffer extends Uint8Array.
+   * This is because Uint8Array is the modern, cross-platform standard while Buffer is Node.js-specific.
+   * Many HTTP frameworks pass request bodies as Uint8Array for better compatibility.
+   */
+  private toBuffer(data: unknown): Buffer {
+    if (Buffer.isBuffer(data)) {
+      return data;
+    }
+    if (data instanceof Uint8Array) {
+      return Buffer.from(data);
+    }
+    if (typeof data === 'string') {
+      return Buffer.from(data, 'utf-8');
+    }
+    // Fallback: serialize object to JSON (may cause digest mismatch for manifests)
+    return Buffer.from(JSON.stringify(data));
+  }
+
   private async getTagsData(repository: string): Promise<Record<string, string>> {
     const path = `oci/tags/${repository}/tags.json`;
     const data = await this.storage.getObject(path);
@@ -665,7 +810,7 @@ export class OciRegistry extends BaseRegistry {
   }
 
   private generateUploadId(): string {
-    return `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
+    return `${Date.now()}-${Math.random().toString(36).substring(2, 11)}`;
   }
 
   private async calculateDigest(data: Buffer): Promise<string> {
@@ -685,10 +830,12 @@ export class OciRegistry extends BaseRegistry {
    * Per OCI Distribution Spec, 401 responses MUST include WWW-Authenticate header.
    */
   private createUnauthorizedResponse(repository: string, action: string): IResponse {
+    const realm = this.ociTokens?.realm || `${this.basePath}/v2/token`;
+    const service = this.ociTokens?.service || 'registry';
     return {
       status: 401,
       headers: {
-        'WWW-Authenticate': `Bearer realm="${this.basePath}/v2/token",service="registry",scope="repository:${repository}:${action}"`,
+        'WWW-Authenticate': `Bearer realm="${realm}",service="${service}",scope="repository:${repository}:${action}"`,
       },
       body: this.createError('DENIED', 'Insufficient permissions'),
     };
@@ -698,10 +845,12 @@ export class OciRegistry extends BaseRegistry {
    * Create an unauthorized HEAD response (no body per HTTP spec).
    */
   private createUnauthorizedHeadResponse(repository: string, action: string): IResponse {
+    const realm = this.ociTokens?.realm || `${this.basePath}/v2/token`;
+    const service = this.ociTokens?.service || 'registry';
     return {
       status: 401,
       headers: {
-        'WWW-Authenticate': `Bearer realm="${this.basePath}/v2/token",service="registry",scope="repository:${repository}:${action}"`,
+        'WWW-Authenticate': `Bearer realm="${realm}",service="${service}",scope="repository:${repository}:${action}"`,
       },
       body: null,
     };

ts/oci/classes.ociupstream.ts

@@ -0,0 +1,263 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamResult,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+import type { IOciManifest, IOciImageIndex, ITagList } from './interfaces.oci.js';
+
+/**
+ * OCI-specific upstream implementation.
+ *
+ * Handles:
+ * - Manifest fetching (image manifests and index manifests)
+ * - Blob proxying (layers, configs)
+ * - Tag list fetching
+ * - Content-addressable caching (blobs are immutable)
+ * - Docker Hub authentication flow
+ */
+export class OciUpstream extends BaseUpstream {
+  protected readonly protocolName = 'oci';
+
+  /** Local registry base path for URL building */
+  private readonly localBasePath: string;
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    localBasePath: string = '/oci',
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+    this.localBasePath = localBasePath;
+  }
+
+  /**
+   * Fetch a manifest from upstream registries.
+   */
+  public async fetchManifest(
+    repository: string,
+    reference: string,
+  ): Promise<{ manifest: IOciManifest | IOciImageIndex; contentType: string; digest: string } | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'oci',
+      resource: repository,
+      resourceType: 'manifest',
+      path: `/v2/${repository}/manifests/${reference}`,
+      method: 'GET',
+      headers: {
+        'accept': [
+          'application/vnd.oci.image.manifest.v1+json',
+          'application/vnd.oci.image.index.v1+json',
+          'application/vnd.docker.distribution.manifest.v2+json',
+          'application/vnd.docker.distribution.manifest.list.v2+json',
+          'application/vnd.docker.distribution.manifest.v1+json',
+        ].join(', '),
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    let manifest: IOciManifest | IOciImageIndex;
+    if (Buffer.isBuffer(result.body)) {
+      manifest = JSON.parse(result.body.toString('utf8'));
+    } else {
+      manifest = result.body;
+    }
+
+    const contentType = result.headers['content-type'] || 'application/vnd.oci.image.manifest.v1+json';
+    const digest = result.headers['docker-content-digest'] || '';
+
+    return { manifest, contentType, digest };
+  }
+
+  /**
+   * Check if a manifest exists in upstream (HEAD request).
+   */
+  public async headManifest(
+    repository: string,
+    reference: string,
+  ): Promise<{ exists: boolean; contentType?: string; digest?: string; size?: number } | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'oci',
+      resource: repository,
+      resourceType: 'manifest',
+      path: `/v2/${repository}/manifests/${reference}`,
+      method: 'HEAD',
+      headers: {
+        'accept': [
+          'application/vnd.oci.image.manifest.v1+json',
+          'application/vnd.oci.image.index.v1+json',
+          'application/vnd.docker.distribution.manifest.v2+json',
+          'application/vnd.docker.distribution.manifest.list.v2+json',
+        ].join(', '),
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result) {
+      return null;
+    }
+
+    if (!result.success) {
+      return { exists: false };
+    }
+
+    return {
+      exists: true,
+      contentType: result.headers['content-type'],
+      digest: result.headers['docker-content-digest'],
+      size: result.headers['content-length'] ? parseInt(result.headers['content-length'], 10) : undefined,
+    };
+  }
+
+  /**
+   * Fetch a blob from upstream registries.
+   */
+  public async fetchBlob(repository: string, digest: string): Promise<Buffer | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'oci',
+      resource: repository,
+      resourceType: 'blob',
+      path: `/v2/${repository}/blobs/${digest}`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Check if a blob exists in upstream (HEAD request).
+   */
+  public async headBlob(
+    repository: string,
+    digest: string,
+  ): Promise<{ exists: boolean; size?: number } | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'oci',
+      resource: repository,
+      resourceType: 'blob',
+      path: `/v2/${repository}/blobs/${digest}`,
+      method: 'HEAD',
+      headers: {},
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result) {
+      return null;
+    }
+
+    if (!result.success) {
+      return { exists: false };
+    }
+
+    return {
+      exists: true,
+      size: result.headers['content-length'] ? parseInt(result.headers['content-length'], 10) : undefined,
+    };
+  }
+
+  /**
+   * Fetch the tag list for a repository.
+   */
+  public async fetchTags(repository: string, n?: number, last?: string): Promise<ITagList | null> {
+    const query: Record<string, string> = {};
+    if (n) query.n = n.toString();
+    if (last) query.last = last;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'oci',
+      resource: repository,
+      resourceType: 'tags',
+      path: `/v2/${repository}/tags/list`,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query,
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    let tagList: ITagList;
+    if (Buffer.isBuffer(result.body)) {
+      tagList = JSON.parse(result.body.toString('utf8'));
+    } else {
+      tagList = result.body;
+    }
+
+    return tagList;
+  }
+
+  /**
+   * Override URL building for OCI-specific handling.
+   * OCI registries use /v2/ prefix and may require special handling for Docker Hub.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    // Handle Docker Hub special case
+    // Docker Hub uses registry-1.docker.io but library images need special handling
+    if (baseUrl.includes('docker.io') || baseUrl.includes('registry-1.docker.io')) {
+      // For library images (e.g., "nginx" -> "library/nginx")
+      const pathParts = context.path.match(/^\/v2\/([^\/]+)\/(.+)$/);
+      if (pathParts) {
+        const [, repository, rest] = pathParts;
+        // If repository doesn't contain a slash, it's a library image
+        if (!repository.includes('/')) {
+          return `${baseUrl}/v2/library/${repository}/${rest}`;
+        }
+      }
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+
+  /**
+   * Override header building for OCI-specific authentication.
+   * OCI registries may require token-based auth obtained from a separate endpoint.
+   */
+  protected buildHeaders(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): Record<string, string> {
+    const headers = super.buildHeaders(upstream, context);
+
+    // OCI registries typically use Docker-Distribution-API-Version header
+    headers['docker-distribution-api-version'] = 'registry/2.0';
+
+    return headers;
+  }
+}

ts/oci/index.ts

@@ -3,4 +3,5 @@
  */
 
 export { OciRegistry } from './classes.ociregistry.js';
+export { OciUpstream } from './classes.ociupstream.js';
 export * from './interfaces.oci.js';

ts/plugins.ts

@@ -8,10 +8,16 @@ import * as smartarchive from '@push.rocks/smartarchive';
 import * as smartbucket from '@push.rocks/smartbucket';
 import * as smartlog from '@push.rocks/smartlog';
 import * as smartpath from '@push.rocks/smartpath';
+import * as smartrequest from '@push.rocks/smartrequest';
 
-export { smartarchive, smartbucket, smartlog, smartpath };
+export { smartarchive, smartbucket, smartlog, smartpath, smartrequest };
 
 // @tsclass scope
 import * as tsclass from '@tsclass/tsclass';
 
 export { tsclass };
+
+// third party
+import { minimatch } from 'minimatch';
+
+export { minimatch };

ts/pypi/classes.pypiregistry.ts

@@ -3,6 +3,8 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import { RegistryStorage } from '../core/classes.registrystorage.js';
 import { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
+import { isBinaryData, toBuffer } from '../core/helpers.buffer.js';
 import type {
   IPypiPackageMetadata,
   IPypiFile,
@@ -10,6 +12,7 @@ import type {
   IPypiUploadResponse,
 } from './interfaces.pypi.js';
 import * as helpers from './helpers.pypi.js';
+import { PypiUpstream } from './classes.pypiupstream.js';
 
 /**
  * PyPI registry implementation
@@ -21,12 +24,14 @@ export class PypiRegistry extends BaseRegistry {
   private basePath: string = '/pypi';
   private registryUrl: string;
   private logger: Smartlog;
+  private upstream: PypiUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string = '/pypi',
-    registryUrl: string = 'http://localhost:5000'
+    registryUrl: string = 'http://localhost:5000',
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
@@ -46,6 +51,20 @@ export class PypiRegistry extends BaseRegistry {
       }
     });
     this.logger.enableConsole();
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new PypiUpstream(upstreamConfig, registryUrl, this.logger);
+    }
+  }
+
+  /**
+   * Clean up resources (timers, connections, etc.)
+   */
+  public destroy(): void {
+    if (this.upstream) {
+      this.upstream.stop();
+    }
   }
 
   public async init(): Promise<void> {
@@ -213,7 +232,45 @@ export class PypiRegistry extends BaseRegistry {
     const normalized = helpers.normalizePypiPackageName(packageName);
 
     // Get package metadata
-    const metadata = await this.storage.getPypiPackageMetadata(normalized);
+    let metadata = await this.storage.getPypiPackageMetadata(normalized);
+
+    // Try upstream if not found locally
+    if (!metadata && this.upstream) {
+      const upstreamHtml = await this.upstream.fetchSimplePackage(normalized);
+      if (upstreamHtml) {
+        // Parse the HTML to extract file information and cache it
+        // For now, just return the upstream HTML directly (caching can be improved later)
+        const acceptHeader = context.headers['accept'] || context.headers['Accept'] || '';
+        const preferJson = acceptHeader.includes('application/vnd.pypi.simple') &&
+                           acceptHeader.includes('json');
+
+        if (preferJson) {
+          // Try to get JSON format from upstream
+          const upstreamJson = await this.upstream.fetchPackageJson(normalized);
+          if (upstreamJson) {
+            return {
+              status: 200,
+              headers: {
+                'Content-Type': 'application/vnd.pypi.simple.v1+json',
+                'Cache-Control': 'public, max-age=300'
+              },
+              body: upstreamJson,
+            };
+          }
+        }
+
+        // Return HTML format
+        return {
+          status: 200,
+          headers: {
+            'Content-Type': 'text/html; charset=utf-8',
+            'Cache-Control': 'public, max-age=300'
+          },
+          body: upstreamHtml,
+        };
+      }
+    }
+
     if (!metadata) {
       return this.errorResponse(404, 'Package not found');
     }
@@ -328,8 +385,9 @@ export class PypiRegistry extends BaseRegistry {
       const version = formData.version;
       // Support both: formData.content.filename (multipart parsed) and formData.filename (flat)
       const filename = formData.content?.filename || formData.filename;
-      // Support both: formData.content.data (multipart parsed) and formData.content (Buffer directly)
-      const fileData = (formData.content?.data || (Buffer.isBuffer(formData.content) ? formData.content : null)) as Buffer;
+      // Support both: formData.content.data (multipart parsed) and formData.content (Buffer/Uint8Array directly)
+      const rawContent = formData.content?.data || (isBinaryData(formData.content) ? formData.content : null);
+      const fileData = rawContent ? toBuffer(rawContent) : null;
       const filetype = formData.filetype; // 'bdist_wheel' or 'sdist'
       const pyversion = formData.pyversion;
 
@@ -447,7 +505,16 @@ export class PypiRegistry extends BaseRegistry {
    */
   private async handleDownload(packageName: string, filename: string): Promise<IResponse> {
     const normalized = helpers.normalizePypiPackageName(packageName);
-    const fileData = await this.storage.getPypiPackageFile(normalized, filename);
+    let fileData = await this.storage.getPypiPackageFile(normalized, filename);
+
+    // Try upstream if not found locally
+    if (!fileData && this.upstream) {
+      fileData = await this.upstream.fetchPackageFile(normalized, filename);
+      if (fileData) {
+        // Cache locally
+        await this.storage.putPypiPackageFile(normalized, filename, fileData);
+      }
+    }
 
     if (!fileData) {
       return {

ts/pypi/classes.pypiupstream.ts

@@ -0,0 +1,211 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+
+/**
+ * PyPI-specific upstream implementation.
+ *
+ * Handles:
+ * - Simple API (HTML) - PEP 503
+ * - JSON API - PEP 691
+ * - Package file downloads (wheels, sdists)
+ * - Package name normalization
+ */
+export class PypiUpstream extends BaseUpstream {
+  protected readonly protocolName = 'pypi';
+
+  /** Local registry URL for rewriting download URLs */
+  private readonly localRegistryUrl: string;
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    localRegistryUrl: string,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+    this.localRegistryUrl = localRegistryUrl;
+  }
+
+  /**
+   * Fetch Simple API index (list of all packages) in HTML format.
+   */
+  public async fetchSimpleIndex(): Promise<string | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'pypi',
+      resource: '*',
+      resourceType: 'index',
+      path: '/simple/',
+      method: 'GET',
+      headers: {
+        'accept': 'text/html',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch Simple API package page (list of files) in HTML format.
+   */
+  public async fetchSimplePackage(packageName: string): Promise<string | null> {
+    const normalizedName = this.normalizePackageName(packageName);
+    const path = `/simple/${normalizedName}/`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'pypi',
+      resource: packageName,
+      resourceType: 'simple',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'text/html',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch package metadata using JSON API (PEP 691).
+   */
+  public async fetchPackageJson(packageName: string): Promise<any | null> {
+    const normalizedName = this.normalizePackageName(packageName);
+    const path = `/simple/${normalizedName}/`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'pypi',
+      resource: packageName,
+      resourceType: 'metadata',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/vnd.pypi.simple.v1+json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch full package info from PyPI JSON API (/pypi/{package}/json).
+   */
+  public async fetchPypiJson(packageName: string): Promise<any | null> {
+    const normalizedName = this.normalizePackageName(packageName);
+    const path = `/pypi/${normalizedName}/json`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'pypi',
+      resource: packageName,
+      resourceType: 'pypi-json',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return result.body;
+  }
+
+  /**
+   * Fetch a package file (wheel or sdist) from upstream.
+   */
+  public async fetchPackageFile(packageName: string, filename: string): Promise<Buffer | null> {
+    const normalizedName = this.normalizePackageName(packageName);
+    const path = `/packages/${normalizedName}/${filename}`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'pypi',
+      resource: packageName,
+      resourceType: 'package',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Normalize a PyPI package name according to PEP 503.
+   * - Lowercase all characters
+   * - Replace runs of ., -, _ with single -
+   */
+  private normalizePackageName(name: string): string {
+    return name.toLowerCase().replace(/[-_.]+/g, '-');
+  }
+
+  /**
+   * Override URL building for PyPI-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/pypi/index.ts

@@ -5,4 +5,5 @@
 
 export * from './interfaces.pypi.js';
 export * from './classes.pypiregistry.js';
+export { PypiUpstream } from './classes.pypiupstream.js';
 export * as pypiHelpers from './helpers.pypi.js';

ts/rubygems/classes.rubygemsregistry.ts

@@ -3,6 +3,7 @@ import { BaseRegistry } from '../core/classes.baseregistry.js';
 import { RegistryStorage } from '../core/classes.registrystorage.js';
 import { AuthManager } from '../core/classes.authmanager.js';
 import type { IRequestContext, IResponse, IAuthToken } from '../core/interfaces.core.js';
+import type { IProtocolUpstreamConfig } from '../upstream/interfaces.upstream.js';
 import type {
   IRubyGemsMetadata,
   IRubyGemsVersionMetadata,
@@ -12,6 +13,7 @@ import type {
   ICompactIndexInfoEntry,
 } from './interfaces.rubygems.js';
 import * as helpers from './helpers.rubygems.js';
+import { RubygemsUpstream } from './classes.rubygemsupstream.js';
 
 /**
  * RubyGems registry implementation
@@ -23,12 +25,14 @@ export class RubyGemsRegistry extends BaseRegistry {
   private basePath: string = '/rubygems';
   private registryUrl: string;
   private logger: Smartlog;
+  private upstream: RubygemsUpstream | null = null;
 
   constructor(
     storage: RegistryStorage,
     authManager: AuthManager,
     basePath: string = '/rubygems',
-    registryUrl: string = 'http://localhost:5000/rubygems'
+    registryUrl: string = 'http://localhost:5000/rubygems',
+    upstreamConfig?: IProtocolUpstreamConfig
   ) {
     super();
     this.storage = storage;
@@ -48,6 +52,20 @@ export class RubyGemsRegistry extends BaseRegistry {
       }
     });
     this.logger.enableConsole();
+
+    // Initialize upstream if configured
+    if (upstreamConfig?.enabled) {
+      this.upstream = new RubygemsUpstream(upstreamConfig, this.logger);
+    }
+  }
+
+  /**
+   * Clean up resources (timers, connections, etc.)
+   */
+  public destroy(): void {
+    if (this.upstream) {
+      this.upstream.stop();
+    }
   }
 
   public async init(): Promise<void> {
@@ -215,7 +233,17 @@ export class RubyGemsRegistry extends BaseRegistry {
    * Handle /info/{gem} endpoint (Compact Index)
    */
   private async handleInfoFile(gemName: string): Promise<IResponse> {
-    const content = await this.storage.getRubyGemsInfo(gemName);
+    let content = await this.storage.getRubyGemsInfo(gemName);
+
+    // Try upstream if not found locally
+    if (!content && this.upstream) {
+      const upstreamInfo = await this.upstream.fetchInfo(gemName);
+      if (upstreamInfo) {
+        // Cache locally
+        await this.storage.putRubyGemsInfo(gemName, upstreamInfo);
+        content = upstreamInfo;
+      }
+    }
 
     if (!content) {
       return {
@@ -245,12 +273,21 @@ export class RubyGemsRegistry extends BaseRegistry {
       return this.errorResponse(400, 'Invalid gem filename');
     }
 
-    const gemData = await this.storage.getRubyGemsGem(
+    let gemData = await this.storage.getRubyGemsGem(
       parsed.name,
       parsed.version,
       parsed.platform
     );
 
+    // Try upstream if not found locally
+    if (!gemData && this.upstream) {
+      gemData = await this.upstream.fetchGem(parsed.name, parsed.version);
+      if (gemData) {
+        // Cache locally
+        await this.storage.putRubyGemsGem(parsed.name, parsed.version, gemData, parsed.platform);
+      }
+    }
+
     if (!gemData) {
       return this.errorResponse(404, 'Gem not found');
     }

ts/rubygems/classes.rubygemsupstream.ts

@@ -0,0 +1,230 @@
+import * as plugins from '../plugins.js';
+import { BaseUpstream } from '../upstream/classes.baseupstream.js';
+import type {
+  IProtocolUpstreamConfig,
+  IUpstreamFetchContext,
+  IUpstreamRegistryConfig,
+} from '../upstream/interfaces.upstream.js';
+
+/**
+ * RubyGems-specific upstream implementation.
+ *
+ * Handles:
+ * - Compact Index format (/versions, /info/{gem}, /names)
+ * - Gem file (.gem) downloading
+ * - Gem spec fetching
+ * - HTTP Range requests for incremental updates
+ */
+export class RubygemsUpstream extends BaseUpstream {
+  protected readonly protocolName = 'rubygems';
+
+  constructor(
+    config: IProtocolUpstreamConfig,
+    logger?: plugins.smartlog.Smartlog,
+  ) {
+    super(config, logger);
+  }
+
+  /**
+   * Fetch the /versions file (master list of all gems).
+   */
+  public async fetchVersions(etag?: string): Promise<{ data: string; etag?: string } | null> {
+    const headers: Record<string, string> = {
+      'accept': 'text/plain',
+    };
+
+    if (etag) {
+      headers['if-none-match'] = etag;
+    }
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: '*',
+      resourceType: 'versions',
+      path: '/versions',
+      method: 'GET',
+      headers,
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    let data: string;
+    if (Buffer.isBuffer(result.body)) {
+      data = result.body.toString('utf8');
+    } else if (typeof result.body === 'string') {
+      data = result.body;
+    } else {
+      return null;
+    }
+
+    return {
+      data,
+      etag: result.headers['etag'],
+    };
+  }
+
+  /**
+   * Fetch gem info file (/info/{gemname}).
+   */
+  public async fetchInfo(gemName: string): Promise<string | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: gemName,
+      resourceType: 'info',
+      path: `/info/${gemName}`,
+      method: 'GET',
+      headers: {
+        'accept': 'text/plain',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch the /names file (list of all gem names).
+   */
+  public async fetchNames(): Promise<string | null> {
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: '*',
+      resourceType: 'names',
+      path: '/names',
+      method: 'GET',
+      headers: {
+        'accept': 'text/plain',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return result.body.toString('utf8');
+    }
+
+    return typeof result.body === 'string' ? result.body : null;
+  }
+
+  /**
+   * Fetch a gem file.
+   */
+  public async fetchGem(gemName: string, version: string): Promise<Buffer | null> {
+    const path = `/gems/${gemName}-${version}.gem`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: gemName,
+      resourceType: 'gem',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Fetch gem spec (quick spec).
+   */
+  public async fetchQuickSpec(gemName: string, version: string): Promise<Buffer | null> {
+    const path = `/quick/Marshal.4.8/${gemName}-${version}.gemspec.rz`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: gemName,
+      resourceType: 'spec',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/octet-stream',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    return Buffer.isBuffer(result.body) ? result.body : Buffer.from(result.body);
+  }
+
+  /**
+   * Fetch gem versions JSON from API.
+   */
+  public async fetchVersionsJson(gemName: string): Promise<any[] | null> {
+    const path = `/api/v1/versions/${gemName}.json`;
+
+    const context: IUpstreamFetchContext = {
+      protocol: 'rubygems',
+      resource: gemName,
+      resourceType: 'versions-json',
+      path,
+      method: 'GET',
+      headers: {
+        'accept': 'application/json',
+      },
+      query: {},
+    };
+
+    const result = await this.fetch(context);
+
+    if (!result || !result.success) {
+      return null;
+    }
+
+    if (Buffer.isBuffer(result.body)) {
+      return JSON.parse(result.body.toString('utf8'));
+    }
+
+    return Array.isArray(result.body) ? result.body : null;
+  }
+
+  /**
+   * Override URL building for RubyGems-specific handling.
+   */
+  protected buildUpstreamUrl(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): string {
+    let baseUrl = upstream.url;
+
+    // Remove trailing slash
+    if (baseUrl.endsWith('/')) {
+      baseUrl = baseUrl.slice(0, -1);
+    }
+
+    return `${baseUrl}${context.path}`;
+  }
+}

ts/rubygems/helpers.rubygems.ts

@@ -455,9 +455,8 @@ export async function extractGemMetadata(gemData: Buffer): Promise<{
     }
 
     return null;
-  } catch (error) {
-    // Log error for debugging but return null gracefully
-    console.error('Failed to extract gem metadata:', error);
+  } catch (_error) {
+    // Error handled gracefully - return null and let caller handle missing metadata
     return null;
   }
 }

ts/rubygems/index.ts

@@ -5,4 +5,5 @@
 
 export * from './interfaces.rubygems.js';
 export * from './classes.rubygemsregistry.js';
+export { RubygemsUpstream } from './classes.rubygemsupstream.js';
 export * as rubygemsHelpers from './helpers.rubygems.js';

ts/upstream/classes.baseupstream.ts

@@ -0,0 +1,521 @@
+import * as plugins from '../plugins.js';
+import type {
+  IUpstreamRegistryConfig,
+  IUpstreamAuthConfig,
+  IUpstreamCacheConfig,
+  IUpstreamResilienceConfig,
+  IUpstreamResult,
+  IUpstreamFetchContext,
+  IProtocolUpstreamConfig,
+  IUpstreamScopeRule,
+  TCircuitState,
+} from './interfaces.upstream.js';
+import {
+  DEFAULT_CACHE_CONFIG,
+  DEFAULT_RESILIENCE_CONFIG,
+} from './interfaces.upstream.js';
+import { CircuitBreaker, CircuitOpenError, withCircuitBreaker } from './classes.circuitbreaker.js';
+import { UpstreamCache } from './classes.upstreamcache.js';
+
+/**
+ * Base class for protocol-specific upstream implementations.
+ *
+ * Provides:
+ * - Multi-upstream routing with priority
+ * - Scope-based filtering (glob patterns)
+ * - Authentication handling
+ * - Circuit breaker per upstream
+ * - Caching with TTL
+ * - Retry with exponential backoff
+ * - 429 rate limit handling
+ */
+export abstract class BaseUpstream {
+  /** Protocol name for logging */
+  protected abstract readonly protocolName: string;
+
+  /** Upstream configuration */
+  protected readonly config: IProtocolUpstreamConfig;
+
+  /** Resolved cache configuration */
+  protected readonly cacheConfig: IUpstreamCacheConfig;
+
+  /** Resolved resilience configuration */
+  protected readonly resilienceConfig: IUpstreamResilienceConfig;
+
+  /** Circuit breakers per upstream */
+  protected readonly circuitBreakers: Map<string, CircuitBreaker> = new Map();
+
+  /** Upstream cache */
+  protected readonly cache: UpstreamCache;
+
+  /** Logger instance */
+  protected readonly logger: plugins.smartlog.Smartlog;
+
+  constructor(config: IProtocolUpstreamConfig, logger?: plugins.smartlog.Smartlog) {
+    this.config = config;
+    this.cacheConfig = { ...DEFAULT_CACHE_CONFIG, ...config.cache };
+    this.resilienceConfig = { ...DEFAULT_RESILIENCE_CONFIG, ...config.resilience };
+    this.cache = new UpstreamCache(this.cacheConfig);
+    this.logger = logger || new plugins.smartlog.Smartlog({
+      logContext: {
+        company: 'smartregistry',
+        companyunit: 'upstream',
+        environment: 'production',
+        runtime: 'node',
+      }
+    });
+
+    // Initialize circuit breakers for each upstream
+    for (const upstream of config.upstreams) {
+      const upstreamResilience = { ...this.resilienceConfig, ...upstream.resilience };
+      this.circuitBreakers.set(upstream.id, new CircuitBreaker(upstream.id, upstreamResilience));
+    }
+  }
+
+  /**
+   * Check if upstream is enabled.
+   */
+  public isEnabled(): boolean {
+    return this.config.enabled;
+  }
+
+  /**
+   * Get all configured upstreams.
+   */
+  public getUpstreams(): IUpstreamRegistryConfig[] {
+    return this.config.upstreams;
+  }
+
+  /**
+   * Get circuit breaker state for an upstream.
+   */
+  public getCircuitState(upstreamId: string): TCircuitState | null {
+    const breaker = this.circuitBreakers.get(upstreamId);
+    return breaker ? breaker.getState() : null;
+  }
+
+  /**
+   * Get cache statistics.
+   */
+  public getCacheStats() {
+    return this.cache.getStats();
+  }
+
+  /**
+   * Fetch a resource from upstreams.
+   * Tries upstreams in priority order, respecting circuit breakers and scope rules.
+   */
+  public async fetch(context: IUpstreamFetchContext): Promise<IUpstreamResult | null> {
+    if (!this.config.enabled) {
+      return null;
+    }
+
+    // Check cache first
+    const cached = this.cache.get(context);
+    if (cached && !cached.stale) {
+      return {
+        success: true,
+        status: 200,
+        headers: cached.headers,
+        body: cached.data,
+        upstreamId: cached.upstreamId,
+        fromCache: true,
+        latencyMs: 0,
+      };
+    }
+
+    // Check for negative cache (recent 404)
+    if (this.cache.hasNegative(context)) {
+      return {
+        success: false,
+        status: 404,
+        headers: {},
+        upstreamId: 'cache',
+        fromCache: true,
+        latencyMs: 0,
+      };
+    }
+
+    // Get applicable upstreams sorted by priority
+    const applicableUpstreams = this.getApplicableUpstreams(context.resource);
+
+    if (applicableUpstreams.length === 0) {
+      return null;
+    }
+
+    // If we have stale cache, return it immediately and revalidate in background
+    if (cached?.stale && this.cacheConfig.staleWhileRevalidate) {
+      // Fire and forget revalidation
+      this.revalidateInBackground(context, applicableUpstreams);
+      return {
+        success: true,
+        status: 200,
+        headers: cached.headers,
+        body: cached.data,
+        upstreamId: cached.upstreamId,
+        fromCache: true,
+        latencyMs: 0,
+      };
+    }
+
+    // Try each upstream in order
+    let lastError: Error | null = null;
+
+    for (const upstream of applicableUpstreams) {
+      const breaker = this.circuitBreakers.get(upstream.id);
+      if (!breaker) continue;
+
+      try {
+        const result = await withCircuitBreaker(
+          breaker,
+          () => this.fetchFromUpstream(upstream, context),
+        );
+
+        // Cache successful responses
+        if (result.success && result.body) {
+          this.cache.set(
+            context,
+            Buffer.isBuffer(result.body) ? result.body : Buffer.from(JSON.stringify(result.body)),
+            result.headers['content-type'] || 'application/octet-stream',
+            result.headers,
+            upstream.id,
+          );
+        }
+
+        // Cache 404 responses
+        if (result.status === 404) {
+          this.cache.setNegative(context, upstream.id);
+        }
+
+        return result;
+      } catch (error) {
+        if (error instanceof CircuitOpenError) {
+          this.logger.log('debug', `Circuit open for upstream ${upstream.id}, trying next`);
+        } else {
+          this.logger.log('warn', `Upstream ${upstream.id} failed: ${(error as Error).message}`);
+        }
+        lastError = error as Error;
+        // Continue to next upstream
+      }
+    }
+
+    // All upstreams failed
+    if (lastError) {
+      this.logger.log('error', `All upstreams failed for ${context.resource}: ${lastError.message}`);
+    }
+
+    return null;
+  }
+
+  /**
+   * Invalidate cache for a resource pattern.
+   */
+  public invalidateCache(pattern: RegExp): number {
+    return this.cache.invalidatePattern(pattern);
+  }
+
+  /**
+   * Clear all cache entries.
+   */
+  public clearCache(): void {
+    this.cache.clear();
+  }
+
+  /**
+   * Stop the upstream (cleanup resources).
+   */
+  public stop(): void {
+    this.cache.stop();
+  }
+
+  /**
+   * Get upstreams that apply to a resource, sorted by priority.
+   */
+  protected getApplicableUpstreams(resource: string): IUpstreamRegistryConfig[] {
+    return this.config.upstreams
+      .filter(upstream => {
+        if (!upstream.enabled) return false;
+
+        // Check circuit breaker
+        const breaker = this.circuitBreakers.get(upstream.id);
+        if (breaker && !breaker.canRequest()) return false;
+
+        // Check scope rules
+        return this.matchesScopeRules(resource, upstream.scopeRules);
+      })
+      .sort((a, b) => a.priority - b.priority);
+  }
+
+  /**
+   * Check if a resource matches scope rules.
+   * Empty rules = match all.
+   */
+  protected matchesScopeRules(resource: string, rules?: IUpstreamScopeRule[]): boolean {
+    if (!rules || rules.length === 0) {
+      return true;
+    }
+
+    // Process rules in order
+    // Start with default exclude (nothing matches)
+    let matched = false;
+
+    for (const rule of rules) {
+      const isMatch = plugins.minimatch(resource, rule.pattern);
+      if (isMatch) {
+        matched = rule.action === 'include';
+      }
+    }
+
+    return matched;
+  }
+
+  /**
+   * Fetch from a specific upstream with retry logic.
+   */
+  protected async fetchFromUpstream(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): Promise<IUpstreamResult> {
+    const upstreamResilience = { ...this.resilienceConfig, ...upstream.resilience };
+    const startTime = Date.now();
+
+    let lastError: Error | null = null;
+
+    for (let attempt = 0; attempt <= upstreamResilience.maxRetries; attempt++) {
+      try {
+        const result = await this.executeRequest(upstream, context, upstreamResilience.timeoutMs);
+        return {
+          ...result,
+          upstreamId: upstream.id,
+          fromCache: false,
+          latencyMs: Date.now() - startTime,
+        };
+      } catch (error) {
+        lastError = error as Error;
+
+        // Don't retry on 4xx errors (except 429)
+        if (this.isNonRetryableError(error)) {
+          break;
+        }
+
+        // Calculate delay with exponential backoff and jitter
+        if (attempt < upstreamResilience.maxRetries) {
+          const delay = this.calculateBackoffDelay(
+            attempt,
+            upstreamResilience.retryDelayMs,
+            upstreamResilience.retryMaxDelayMs,
+          );
+          await this.sleep(delay);
+        }
+      }
+    }
+
+    throw lastError || new Error('Request failed');
+  }
+
+  /**
+   * Execute a single HTTP request to an upstream.
+   */
+  protected async executeRequest(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+    timeoutMs: number,
+  ): Promise<Omit<IUpstreamResult, 'upstreamId' | 'fromCache' | 'latencyMs'>> {
+    // Build the full URL
+    const url = this.buildUpstreamUrl(upstream, context);
+
+    // Build headers with auth
+    const headers = this.buildHeaders(upstream, context);
+
+    // Make the request using SmartRequest
+    const request = plugins.smartrequest.SmartRequest.create()
+      .url(url)
+      .method(context.method as any)
+      .headers(headers)
+      .timeout(timeoutMs)
+      .handle429Backoff({ maxRetries: 3, fallbackDelay: 1000, maxWaitTime: 30000 });
+
+    // Add query params if present
+    if (Object.keys(context.query).length > 0) {
+      request.query(context.query);
+    }
+
+    let response: plugins.smartrequest.ICoreResponse;
+
+    switch (context.method.toUpperCase()) {
+      case 'GET':
+        response = await request.get();
+        break;
+      case 'HEAD':
+        // SmartRequest doesn't have head(), use options
+        response = await request.method('HEAD').get();
+        break;
+      default:
+        response = await request.get();
+    }
+
+    // Parse response
+    const responseHeaders: Record<string, string> = {};
+    for (const [key, value] of Object.entries(response.headers)) {
+      responseHeaders[key.toLowerCase()] = Array.isArray(value) ? value[0] : value;
+    }
+
+    let body: Buffer | any;
+    const contentType = responseHeaders['content-type'] || '';
+
+    if (response.ok) {
+      if (contentType.includes('application/json')) {
+        body = await response.json();
+      } else {
+        const arrayBuffer = await response.arrayBuffer();
+        body = Buffer.from(arrayBuffer);
+      }
+    }
+
+    return {
+      success: response.ok,
+      status: response.status,
+      headers: responseHeaders,
+      body,
+    };
+  }
+
+  /**
+   * Build the full URL for an upstream request.
+   * Subclasses can override for protocol-specific URL building.
+   */
+  protected buildUpstreamUrl(upstream: IUpstreamRegistryConfig, context: IUpstreamFetchContext): string {
+    // Remove leading slash if URL already has trailing slash
+    let path = context.path;
+    if (upstream.url.endsWith('/') && path.startsWith('/')) {
+      path = path.slice(1);
+    }
+    return `${upstream.url}${path}`;
+  }
+
+  /**
+   * Build headers including authentication.
+   */
+  protected buildHeaders(
+    upstream: IUpstreamRegistryConfig,
+    context: IUpstreamFetchContext,
+  ): Record<string, string> {
+    const headers: Record<string, string> = { ...context.headers };
+
+    // Remove host header (will be set by HTTP client)
+    delete headers['host'];
+
+    // Add authentication
+    this.addAuthHeaders(headers, upstream.auth);
+
+    return headers;
+  }
+
+  /**
+   * Add authentication headers based on auth config.
+   */
+  protected addAuthHeaders(headers: Record<string, string>, auth: IUpstreamAuthConfig): void {
+    switch (auth.type) {
+      case 'basic':
+        if (auth.username && auth.password) {
+          const credentials = Buffer.from(`${auth.username}:${auth.password}`).toString('base64');
+          headers['authorization'] = `Basic ${credentials}`;
+        }
+        break;
+      case 'bearer':
+        if (auth.token) {
+          headers['authorization'] = `Bearer ${auth.token}`;
+        }
+        break;
+      case 'api-key':
+        if (auth.token) {
+          const headerName = auth.headerName || 'authorization';
+          headers[headerName.toLowerCase()] = auth.token;
+        }
+        break;
+      case 'none':
+      default:
+        // No authentication
+        break;
+    }
+  }
+
+  /**
+   * Check if an error should not be retried.
+   */
+  protected isNonRetryableError(error: unknown): boolean {
+    // Check for HTTP status errors
+    if (error && typeof error === 'object' && 'status' in error) {
+      const status = (error as { status: number }).status;
+      // Don't retry 4xx errors except 429 (rate limited)
+      if (status >= 400 && status < 500 && status !== 429) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  /**
+   * Calculate backoff delay with exponential backoff and jitter.
+   */
+  protected calculateBackoffDelay(
+    attempt: number,
+    baseDelayMs: number,
+    maxDelayMs: number,
+  ): number {
+    // Exponential backoff: delay = base * 2^attempt
+    const exponentialDelay = baseDelayMs * Math.pow(2, attempt);
+
+    // Cap at max delay
+    const cappedDelay = Math.min(exponentialDelay, maxDelayMs);
+
+    // Add jitter (±25%)
+    const jitter = cappedDelay * 0.25 * (Math.random() * 2 - 1);
+
+    return Math.floor(cappedDelay + jitter);
+  }
+
+  /**
+   * Sleep for a specified duration.
+   */
+  protected sleep(ms: number): Promise<void> {
+    return new Promise(resolve => setTimeout(resolve, ms));
+  }
+
+  /**
+   * Revalidate cache in background.
+   */
+  protected async revalidateInBackground(
+    context: IUpstreamFetchContext,
+    upstreams: IUpstreamRegistryConfig[],
+  ): Promise<void> {
+    try {
+      for (const upstream of upstreams) {
+        const breaker = this.circuitBreakers.get(upstream.id);
+        if (!breaker || !breaker.canRequest()) continue;
+
+        try {
+          const result = await withCircuitBreaker(
+            breaker,
+            () => this.fetchFromUpstream(upstream, context),
+          );
+
+          if (result.success && result.body) {
+            this.cache.set(
+              context,
+              Buffer.isBuffer(result.body) ? result.body : Buffer.from(JSON.stringify(result.body)),
+              result.headers['content-type'] || 'application/octet-stream',
+              result.headers,
+              upstream.id,
+            );
+            return; // Successfully revalidated
+          }
+        } catch {
+          // Continue to next upstream
+        }
+      }
+    } catch (error) {
+      this.logger.log('debug', `Background revalidation failed: ${(error as Error).message}`);
+    }
+  }
+}

ts/upstream/classes.circuitbreaker.ts

@@ -0,0 +1,238 @@
+import type { TCircuitState, IUpstreamResilienceConfig } from './interfaces.upstream.js';
+import { DEFAULT_RESILIENCE_CONFIG } from './interfaces.upstream.js';
+
+/**
+ * Circuit breaker implementation for upstream resilience.
+ *
+ * States:
+ * - CLOSED: Normal operation, requests pass through
+ * - OPEN: Circuit is tripped, requests fail fast
+ * - HALF_OPEN: Testing if upstream has recovered
+ *
+ * Transitions:
+ * - CLOSED → OPEN: When failure count exceeds threshold
+ * - OPEN → HALF_OPEN: After reset timeout expires
+ * - HALF_OPEN → CLOSED: On successful request
+ * - HALF_OPEN → OPEN: On failed request
+ */
+export class CircuitBreaker {
+  /** Unique identifier for logging and metrics */
+  public readonly id: string;
+
+  /** Current circuit state */
+  private state: TCircuitState = 'CLOSED';
+
+  /** Count of consecutive failures */
+  private failureCount: number = 0;
+
+  /** Timestamp when circuit was opened */
+  private openedAt: number = 0;
+
+  /** Number of successful requests in half-open state */
+  private halfOpenSuccesses: number = 0;
+
+  /** Configuration */
+  private readonly config: IUpstreamResilienceConfig;
+
+  /** Number of successes required to close circuit from half-open */
+  private readonly halfOpenThreshold: number = 2;
+
+  constructor(id: string, config?: Partial<IUpstreamResilienceConfig>) {
+    this.id = id;
+    this.config = { ...DEFAULT_RESILIENCE_CONFIG, ...config };
+  }
+
+  /**
+   * Get current circuit state.
+   */
+  public getState(): TCircuitState {
+    // Check if we should transition from OPEN to HALF_OPEN
+    if (this.state === 'OPEN') {
+      const elapsed = Date.now() - this.openedAt;
+      if (elapsed >= this.config.circuitBreakerResetMs) {
+        this.transitionTo('HALF_OPEN');
+      }
+    }
+    return this.state;
+  }
+
+  /**
+   * Check if circuit allows requests.
+   * Returns true if requests should be allowed.
+   */
+  public canRequest(): boolean {
+    const currentState = this.getState();
+    return currentState !== 'OPEN';
+  }
+
+  /**
+   * Record a successful request.
+   * May transition circuit from HALF_OPEN to CLOSED.
+   */
+  public recordSuccess(): void {
+    if (this.state === 'HALF_OPEN') {
+      this.halfOpenSuccesses++;
+      if (this.halfOpenSuccesses >= this.halfOpenThreshold) {
+        this.transitionTo('CLOSED');
+      }
+    } else if (this.state === 'CLOSED') {
+      // Reset failure count on success
+      this.failureCount = 0;
+    }
+  }
+
+  /**
+   * Record a failed request.
+   * May transition circuit from CLOSED/HALF_OPEN to OPEN.
+   */
+  public recordFailure(): void {
+    if (this.state === 'HALF_OPEN') {
+      // Any failure in half-open immediately opens circuit
+      this.transitionTo('OPEN');
+    } else if (this.state === 'CLOSED') {
+      this.failureCount++;
+      if (this.failureCount >= this.config.circuitBreakerThreshold) {
+        this.transitionTo('OPEN');
+      }
+    }
+  }
+
+  /**
+   * Force circuit to open state.
+   * Useful for manual intervention or external health checks.
+   */
+  public forceOpen(): void {
+    this.transitionTo('OPEN');
+  }
+
+  /**
+   * Force circuit to closed state.
+   * Useful for manual intervention after fixing upstream issues.
+   */
+  public forceClose(): void {
+    this.transitionTo('CLOSED');
+  }
+
+  /**
+   * Reset circuit to initial state.
+   */
+  public reset(): void {
+    this.state = 'CLOSED';
+    this.failureCount = 0;
+    this.openedAt = 0;
+    this.halfOpenSuccesses = 0;
+  }
+
+  /**
+   * Get circuit metrics for monitoring.
+   */
+  public getMetrics(): ICircuitBreakerMetrics {
+    return {
+      id: this.id,
+      state: this.getState(),
+      failureCount: this.failureCount,
+      openedAt: this.openedAt > 0 ? new Date(this.openedAt) : null,
+      timeUntilHalfOpen: this.state === 'OPEN'
+        ? Math.max(0, this.config.circuitBreakerResetMs - (Date.now() - this.openedAt))
+        : 0,
+      halfOpenSuccesses: this.halfOpenSuccesses,
+      threshold: this.config.circuitBreakerThreshold,
+      resetMs: this.config.circuitBreakerResetMs,
+    };
+  }
+
+  /**
+   * Transition to a new state with proper cleanup.
+   */
+  private transitionTo(newState: TCircuitState): void {
+    const previousState = this.state;
+    this.state = newState;
+
+    switch (newState) {
+      case 'OPEN':
+        this.openedAt = Date.now();
+        this.halfOpenSuccesses = 0;
+        break;
+      case 'HALF_OPEN':
+        this.halfOpenSuccesses = 0;
+        break;
+      case 'CLOSED':
+        this.failureCount = 0;
+        this.openedAt = 0;
+        this.halfOpenSuccesses = 0;
+        break;
+    }
+
+    // Log state transition (useful for debugging and monitoring)
+    // In production, this would emit events or metrics
+    if (previousState !== newState) {
+      // State changed - could emit event here
+    }
+  }
+}
+
+/**
+ * Metrics for circuit breaker monitoring.
+ */
+export interface ICircuitBreakerMetrics {
+  /** Circuit breaker identifier */
+  id: string;
+  /** Current state */
+  state: TCircuitState;
+  /** Number of consecutive failures */
+  failureCount: number;
+  /** When circuit was opened (null if never opened) */
+  openedAt: Date | null;
+  /** Milliseconds until circuit transitions to half-open (0 if not open) */
+  timeUntilHalfOpen: number;
+  /** Number of successes in half-open state */
+  halfOpenSuccesses: number;
+  /** Failure threshold for opening circuit */
+  threshold: number;
+  /** Reset timeout in milliseconds */
+  resetMs: number;
+}
+
+/**
+ * Execute a function with circuit breaker protection.
+ *
+ * @param breaker The circuit breaker to use
+ * @param fn The async function to execute
+ * @param fallback Optional fallback function when circuit is open
+ * @returns The result of fn or fallback
+ * @throws CircuitOpenError if circuit is open and no fallback provided
+ */
+export async function withCircuitBreaker<T>(
+  breaker: CircuitBreaker,
+  fn: () => Promise<T>,
+  fallback?: () => Promise<T>,
+): Promise<T> {
+  if (!breaker.canRequest()) {
+    if (fallback) {
+      return fallback();
+    }
+    throw new CircuitOpenError(breaker.id);
+  }
+
+  try {
+    const result = await fn();
+    breaker.recordSuccess();
+    return result;
+  } catch (error) {
+    breaker.recordFailure();
+    throw error;
+  }
+}
+
+/**
+ * Error thrown when circuit is open and no fallback is provided.
+ */
+export class CircuitOpenError extends Error {
+  public readonly circuitId: string;
+
+  constructor(circuitId: string) {
+    super(`Circuit breaker '${circuitId}' is open`);
+    this.name = 'CircuitOpenError';
+    this.circuitId = circuitId;
+  }
+}

ts/upstream/classes.upstreamcache.ts

@@ -0,0 +1,423 @@
+import type {
+  ICacheEntry,
+  IUpstreamCacheConfig,
+  IUpstreamFetchContext,
+} from './interfaces.upstream.js';
+import { DEFAULT_CACHE_CONFIG } from './interfaces.upstream.js';
+
+/**
+ * In-memory cache for upstream responses.
+ *
+ * Features:
+ * - TTL-based expiration
+ * - Stale-while-revalidate support
+ * - Negative caching (404s)
+ * - Content-type aware caching
+ * - ETag support for conditional requests
+ *
+ * Note: This is an in-memory implementation. For production with persistence,
+ * extend this class to use RegistryStorage for S3-backed caching.
+ */
+export class UpstreamCache {
+  /** Cache storage */
+  private readonly cache: Map<string, ICacheEntry> = new Map();
+
+  /** Configuration */
+  private readonly config: IUpstreamCacheConfig;
+
+  /** Maximum cache entries (prevents memory bloat) */
+  private readonly maxEntries: number;
+
+  /** Cleanup interval handle */
+  private cleanupInterval: ReturnType<typeof setInterval> | null = null;
+
+  constructor(config?: Partial<IUpstreamCacheConfig>, maxEntries: number = 10000) {
+    this.config = { ...DEFAULT_CACHE_CONFIG, ...config };
+    this.maxEntries = maxEntries;
+
+    // Start periodic cleanup if caching is enabled
+    if (this.config.enabled) {
+      this.startCleanup();
+    }
+  }
+
+  /**
+   * Check if caching is enabled.
+   */
+  public isEnabled(): boolean {
+    return this.config.enabled;
+  }
+
+  /**
+   * Get cached entry for a request context.
+   * Returns null if not found or expired (unless stale-while-revalidate).
+   */
+  public get(context: IUpstreamFetchContext): ICacheEntry | null {
+    if (!this.config.enabled) {
+      return null;
+    }
+
+    const key = this.buildCacheKey(context);
+    const entry = this.cache.get(key);
+
+    if (!entry) {
+      return null;
+    }
+
+    const now = new Date();
+
+    // Check if entry is expired
+    if (entry.expiresAt && entry.expiresAt < now) {
+      // Check if we can serve stale content
+      if (this.config.staleWhileRevalidate && !entry.stale) {
+        const staleAge = (now.getTime() - entry.expiresAt.getTime()) / 1000;
+        if (staleAge <= this.config.staleMaxAgeSeconds) {
+          // Mark as stale and return
+          entry.stale = true;
+          return entry;
+        }
+      }
+      // Entry is too old, remove it
+      this.cache.delete(key);
+      return null;
+    }
+
+    return entry;
+  }
+
+  /**
+   * Store a response in the cache.
+   */
+  public set(
+    context: IUpstreamFetchContext,
+    data: Buffer,
+    contentType: string,
+    headers: Record<string, string>,
+    upstreamId: string,
+    options?: ICacheSetOptions,
+  ): void {
+    if (!this.config.enabled) {
+      return;
+    }
+
+    // Enforce max entries limit
+    if (this.cache.size >= this.maxEntries) {
+      this.evictOldest();
+    }
+
+    const key = this.buildCacheKey(context);
+    const now = new Date();
+
+    // Determine TTL based on content type
+    const ttlSeconds = options?.ttlSeconds ?? this.determineTtl(context, contentType, headers);
+
+    const entry: ICacheEntry = {
+      data,
+      contentType,
+      headers,
+      cachedAt: now,
+      expiresAt: ttlSeconds > 0 ? new Date(now.getTime() + ttlSeconds * 1000) : undefined,
+      etag: headers['etag'] || options?.etag,
+      upstreamId,
+      stale: false,
+    };
+
+    this.cache.set(key, entry);
+  }
+
+  /**
+   * Store a negative cache entry (404 response).
+   */
+  public setNegative(context: IUpstreamFetchContext, upstreamId: string): void {
+    if (!this.config.enabled || this.config.negativeCacheTtlSeconds <= 0) {
+      return;
+    }
+
+    const key = this.buildCacheKey(context);
+    const now = new Date();
+
+    const entry: ICacheEntry = {
+      data: Buffer.from(''),
+      contentType: 'application/octet-stream',
+      headers: {},
+      cachedAt: now,
+      expiresAt: new Date(now.getTime() + this.config.negativeCacheTtlSeconds * 1000),
+      upstreamId,
+      stale: false,
+    };
+
+    this.cache.set(key, entry);
+  }
+
+  /**
+   * Check if there's a negative cache entry for this context.
+   */
+  public hasNegative(context: IUpstreamFetchContext): boolean {
+    const entry = this.get(context);
+    return entry !== null && entry.data.length === 0;
+  }
+
+  /**
+   * Invalidate a specific cache entry.
+   */
+  public invalidate(context: IUpstreamFetchContext): boolean {
+    const key = this.buildCacheKey(context);
+    return this.cache.delete(key);
+  }
+
+  /**
+   * Invalidate all entries matching a pattern.
+   * Useful for invalidating all versions of a package.
+   */
+  public invalidatePattern(pattern: RegExp): number {
+    let count = 0;
+    for (const key of this.cache.keys()) {
+      if (pattern.test(key)) {
+        this.cache.delete(key);
+        count++;
+      }
+    }
+    return count;
+  }
+
+  /**
+   * Invalidate all entries from a specific upstream.
+   */
+  public invalidateUpstream(upstreamId: string): number {
+    let count = 0;
+    for (const [key, entry] of this.cache.entries()) {
+      if (entry.upstreamId === upstreamId) {
+        this.cache.delete(key);
+        count++;
+      }
+    }
+    return count;
+  }
+
+  /**
+   * Clear all cache entries.
+   */
+  public clear(): void {
+    this.cache.clear();
+  }
+
+  /**
+   * Get cache statistics.
+   */
+  public getStats(): ICacheStats {
+    let freshCount = 0;
+    let staleCount = 0;
+    let negativeCount = 0;
+    let totalSize = 0;
+    const now = new Date();
+
+    for (const entry of this.cache.values()) {
+      totalSize += entry.data.length;
+
+      if (entry.data.length === 0) {
+        negativeCount++;
+      } else if (entry.stale || (entry.expiresAt && entry.expiresAt < now)) {
+        staleCount++;
+      } else {
+        freshCount++;
+      }
+    }
+
+    return {
+      totalEntries: this.cache.size,
+      freshEntries: freshCount,
+      staleEntries: staleCount,
+      negativeEntries: negativeCount,
+      totalSizeBytes: totalSize,
+      maxEntries: this.maxEntries,
+      enabled: this.config.enabled,
+    };
+  }
+
+  /**
+   * Stop the cache and cleanup.
+   */
+  public stop(): void {
+    if (this.cleanupInterval) {
+      clearInterval(this.cleanupInterval);
+      this.cleanupInterval = null;
+    }
+  }
+
+  /**
+   * Build a unique cache key for a request context.
+   */
+  private buildCacheKey(context: IUpstreamFetchContext): string {
+    // Include method, protocol, path, and sorted query params
+    const queryString = Object.keys(context.query)
+      .sort()
+      .map(k => `${k}=${context.query[k]}`)
+      .join('&');
+
+    return `${context.protocol}:${context.method}:${context.path}${queryString ? '?' + queryString : ''}`;
+  }
+
+  /**
+   * Determine TTL based on content characteristics.
+   */
+  private determineTtl(
+    context: IUpstreamFetchContext,
+    contentType: string,
+    headers: Record<string, string>,
+  ): number {
+    // Check for Cache-Control header
+    const cacheControl = headers['cache-control'];
+    if (cacheControl) {
+      const maxAgeMatch = cacheControl.match(/max-age=(\d+)/);
+      if (maxAgeMatch) {
+        return parseInt(maxAgeMatch[1], 10);
+      }
+      if (cacheControl.includes('no-store') || cacheControl.includes('no-cache')) {
+        return 0;
+      }
+    }
+
+    // Check if content is immutable (content-addressable)
+    if (this.isImmutableContent(context, contentType)) {
+      return this.config.immutableTtlSeconds;
+    }
+
+    // Default TTL for mutable content
+    return this.config.defaultTtlSeconds;
+  }
+
+  /**
+   * Check if content is immutable (content-addressable).
+   */
+  private isImmutableContent(context: IUpstreamFetchContext, contentType: string): boolean {
+    // OCI blobs with digest are immutable
+    if (context.protocol === 'oci' && context.resourceType === 'blob') {
+      return true;
+    }
+
+    // NPM tarballs are immutable (versioned)
+    if (context.protocol === 'npm' && context.resourceType === 'tarball') {
+      return true;
+    }
+
+    // Maven artifacts with version are immutable
+    if (context.protocol === 'maven' && context.resourceType === 'artifact') {
+      return true;
+    }
+
+    // Cargo crate files are immutable
+    if (context.protocol === 'cargo' && context.resourceType === 'crate') {
+      return true;
+    }
+
+    // Composer dist files are immutable
+    if (context.protocol === 'composer' && context.resourceType === 'dist') {
+      return true;
+    }
+
+    // PyPI package files are immutable
+    if (context.protocol === 'pypi' && context.resourceType === 'package') {
+      return true;
+    }
+
+    // RubyGems .gem files are immutable
+    if (context.protocol === 'rubygems' && context.resourceType === 'gem') {
+      return true;
+    }
+
+    return false;
+  }
+
+  /**
+   * Evict oldest entries to make room for new ones.
+   */
+  private evictOldest(): void {
+    // Evict 10% of max entries
+    const evictCount = Math.ceil(this.maxEntries * 0.1);
+    let evicted = 0;
+
+    // First, try to evict stale entries
+    const now = new Date();
+    for (const [key, entry] of this.cache.entries()) {
+      if (evicted >= evictCount) break;
+      if (entry.stale || (entry.expiresAt && entry.expiresAt < now)) {
+        this.cache.delete(key);
+        evicted++;
+      }
+    }
+
+    // If not enough evicted, evict oldest by cachedAt
+    if (evicted < evictCount) {
+      const entries = Array.from(this.cache.entries())
+        .sort((a, b) => a[1].cachedAt.getTime() - b[1].cachedAt.getTime());
+
+      for (const [key] of entries) {
+        if (evicted >= evictCount) break;
+        this.cache.delete(key);
+        evicted++;
+      }
+    }
+  }
+
+  /**
+   * Start periodic cleanup of expired entries.
+   */
+  private startCleanup(): void {
+    // Run cleanup every minute
+    this.cleanupInterval = setInterval(() => {
+      this.cleanup();
+    }, 60000);
+
+    // Don't keep the process alive just for cleanup
+    if (this.cleanupInterval.unref) {
+      this.cleanupInterval.unref();
+    }
+  }
+
+  /**
+   * Remove all expired entries.
+   */
+  private cleanup(): void {
+    const now = new Date();
+    const staleDeadline = new Date(now.getTime() - this.config.staleMaxAgeSeconds * 1000);
+
+    for (const [key, entry] of this.cache.entries()) {
+      if (entry.expiresAt) {
+        // Remove if past stale deadline
+        if (entry.expiresAt < staleDeadline) {
+          this.cache.delete(key);
+        }
+      }
+    }
+  }
+}
+
+/**
+ * Options for cache set operation.
+ */
+export interface ICacheSetOptions {
+  /** Override TTL in seconds */
+  ttlSeconds?: number;
+  /** ETag for conditional requests */
+  etag?: string;
+}
+
+/**
+ * Cache statistics.
+ */
+export interface ICacheStats {
+  /** Total number of cached entries */
+  totalEntries: number;
+  /** Number of fresh (non-expired) entries */
+  freshEntries: number;
+  /** Number of stale entries (expired but still usable) */
+  staleEntries: number;
+  /** Number of negative cache entries */
+  negativeEntries: number;
+  /** Total size of cached data in bytes */
+  totalSizeBytes: number;
+  /** Maximum allowed entries */
+  maxEntries: number;
+  /** Whether caching is enabled */
+  enabled: boolean;
+}

ts/upstream/index.ts

@@ -0,0 +1,11 @@
+// Interfaces and types
+export * from './interfaces.upstream.js';
+
+// Classes
+export { CircuitBreaker, CircuitOpenError, withCircuitBreaker } from './classes.circuitbreaker.js';
+export type { ICircuitBreakerMetrics } from './classes.circuitbreaker.js';
+
+export { UpstreamCache } from './classes.upstreamcache.js';
+export type { ICacheSetOptions, ICacheStats } from './classes.upstreamcache.js';
+
+export { BaseUpstream } from './classes.baseupstream.js';

ts/upstream/interfaces.upstream.ts

@@ -0,0 +1,195 @@
+import type { TRegistryProtocol } from '../core/interfaces.core.js';
+
+/**
+ * Scope rule for routing requests to specific upstreams.
+ * Uses glob patterns for flexible matching.
+ */
+export interface IUpstreamScopeRule {
+  /** Glob pattern (e.g., "@company/*", "com.example.*", "library/*") */
+  pattern: string;
+  /** Whether matching resources should be included or excluded */
+  action: 'include' | 'exclude';
+}
+
+/**
+ * Authentication configuration for an upstream registry.
+ * Supports multiple auth strategies.
+ */
+export interface IUpstreamAuthConfig {
+  /** Authentication type */
+  type: 'none' | 'basic' | 'bearer' | 'api-key';
+  /** Username for basic auth */
+  username?: string;
+  /** Password for basic auth */
+  password?: string;
+  /** Token for bearer or api-key auth */
+  token?: string;
+  /** Custom header name for api-key auth (default: 'Authorization') */
+  headerName?: string;
+}
+
+/**
+ * Cache configuration for upstream content.
+ */
+export interface IUpstreamCacheConfig {
+  /** Whether caching is enabled */
+  enabled: boolean;
+  /** Default TTL in seconds for mutable content (default: 300 = 5 min) */
+  defaultTtlSeconds: number;
+  /** TTL in seconds for immutable/content-addressable content (default: 2592000 = 30 days) */
+  immutableTtlSeconds: number;
+  /** Whether to serve stale content while revalidating in background */
+  staleWhileRevalidate: boolean;
+  /** Maximum age in seconds for stale content (default: 3600 = 1 hour) */
+  staleMaxAgeSeconds: number;
+  /** TTL in seconds for negative cache entries (404s) (default: 60 = 1 min) */
+  negativeCacheTtlSeconds: number;
+}
+
+/**
+ * Resilience configuration for upstream requests.
+ */
+export interface IUpstreamResilienceConfig {
+  /** Request timeout in milliseconds (default: 30000) */
+  timeoutMs: number;
+  /** Maximum number of retry attempts (default: 3) */
+  maxRetries: number;
+  /** Initial retry delay in milliseconds (default: 1000) */
+  retryDelayMs: number;
+  /** Maximum retry delay in milliseconds (default: 30000) */
+  retryMaxDelayMs: number;
+  /** Number of failures before circuit breaker opens (default: 5) */
+  circuitBreakerThreshold: number;
+  /** Time in milliseconds before circuit breaker attempts reset (default: 30000) */
+  circuitBreakerResetMs: number;
+}
+
+/**
+ * Configuration for a single upstream registry.
+ */
+export interface IUpstreamRegistryConfig {
+  /** Unique identifier for this upstream */
+  id: string;
+  /** Human-readable name */
+  name: string;
+  /** Base URL of the upstream registry (e.g., "https://registry.npmjs.org") */
+  url: string;
+  /** Priority for routing (lower = higher priority, 1 = first) */
+  priority: number;
+  /** Whether this upstream is enabled */
+  enabled: boolean;
+  /** Scope rules for routing (empty = match all) */
+  scopeRules?: IUpstreamScopeRule[];
+  /** Authentication configuration */
+  auth: IUpstreamAuthConfig;
+  /** Cache configuration overrides */
+  cache?: Partial<IUpstreamCacheConfig>;
+  /** Resilience configuration overrides */
+  resilience?: Partial<IUpstreamResilienceConfig>;
+}
+
+/**
+ * Protocol-level upstream configuration.
+ * Configures upstream behavior for a specific protocol (npm, oci, etc.)
+ */
+export interface IProtocolUpstreamConfig {
+  /** Whether upstream is enabled for this protocol */
+  enabled: boolean;
+  /** List of upstream registries, ordered by priority */
+  upstreams: IUpstreamRegistryConfig[];
+  /** Protocol-level cache configuration defaults */
+  cache?: Partial<IUpstreamCacheConfig>;
+  /** Protocol-level resilience configuration defaults */
+  resilience?: Partial<IUpstreamResilienceConfig>;
+}
+
+/**
+ * Result of an upstream fetch operation.
+ */
+export interface IUpstreamResult {
+  /** Whether the fetch was successful (2xx status) */
+  success: boolean;
+  /** HTTP status code */
+  status: number;
+  /** Response headers */
+  headers: Record<string, string>;
+  /** Response body (Buffer for binary, object for JSON) */
+  body?: Buffer | any;
+  /** ID of the upstream that served the request */
+  upstreamId: string;
+  /** Whether the response was served from cache */
+  fromCache: boolean;
+  /** Request latency in milliseconds */
+  latencyMs: number;
+}
+
+/**
+ * Circuit breaker state.
+ */
+export type TCircuitState = 'CLOSED' | 'OPEN' | 'HALF_OPEN';
+
+/**
+ * Context for an upstream fetch request.
+ */
+export interface IUpstreamFetchContext {
+  /** Protocol type */
+  protocol: TRegistryProtocol;
+  /** Resource identifier (package name, artifact name, etc.) */
+  resource: string;
+  /** Type of resource being fetched (packument, tarball, manifest, blob, etc.) */
+  resourceType: string;
+  /** Original request path */
+  path: string;
+  /** HTTP method */
+  method: string;
+  /** Request headers */
+  headers: Record<string, string>;
+  /** Query parameters */
+  query: Record<string, string>;
+}
+
+/**
+ * Cache entry stored in the upstream cache.
+ */
+export interface ICacheEntry {
+  /** Cached data */
+  data: Buffer;
+  /** Content type of the cached data */
+  contentType: string;
+  /** Original response headers */
+  headers: Record<string, string>;
+  /** When the entry was cached */
+  cachedAt: Date;
+  /** When the entry expires */
+  expiresAt?: Date;
+  /** ETag for conditional requests */
+  etag?: string;
+  /** ID of the upstream that provided the data */
+  upstreamId: string;
+  /** Whether the entry is stale but still usable */
+  stale?: boolean;
+}
+
+/**
+ * Default cache configuration values.
+ */
+export const DEFAULT_CACHE_CONFIG: IUpstreamCacheConfig = {
+  enabled: true,
+  defaultTtlSeconds: 300,           // 5 minutes
+  immutableTtlSeconds: 2592000,     // 30 days
+  staleWhileRevalidate: true,
+  staleMaxAgeSeconds: 3600,         // 1 hour
+  negativeCacheTtlSeconds: 60,      // 1 minute
+};
+
+/**
+ * Default resilience configuration values.
+ */
+export const DEFAULT_RESILIENCE_CONFIG: IUpstreamResilienceConfig = {
+  timeoutMs: 30000,
+  maxRetries: 3,
+  retryDelayMs: 1000,
+  retryMaxDelayMs: 30000,
+  circuitBreakerThreshold: 5,
+  circuitBreakerResetMs: 30000,
+};