487 lines
12 KiB
Markdown
487 lines
12 KiB
Markdown
# @push.rocks/smartregistry
|
|
|
|
> 🚀 A composable TypeScript library implementing both **OCI Distribution Specification v1.1** and **NPM Registry API** for building unified container and package registries.
|
|
|
|
## ✨ Features
|
|
|
|
### 🔄 Dual Protocol Support
|
|
- **OCI Distribution Spec v1.1**: Full container registry with manifest/blob operations
|
|
- **NPM Registry API**: Complete package registry with publish/install/search
|
|
|
|
### 🏗️ Unified Architecture
|
|
- **Composable Design**: Core infrastructure with protocol plugins
|
|
- **Shared Storage**: Cloud-agnostic S3-compatible backend ([@push.rocks/smartbucket](https://www.npmjs.com/package/@push.rocks/smartbucket))
|
|
- **Unified Authentication**: Scope-based permissions across both protocols
|
|
- **Path-based Routing**: `/oci/*` for containers, `/npm/*` for packages
|
|
|
|
### 🔐 Authentication & Authorization
|
|
- NPM UUID tokens for package operations
|
|
- OCI JWT tokens for container operations
|
|
- Unified scope system: `npm:package:foo:write`, `oci:repository:bar:push`
|
|
- Pluggable via async callbacks
|
|
|
|
### 📦 Comprehensive Feature Set
|
|
|
|
**OCI Features:**
|
|
- ✅ Pull operations (manifests, blobs)
|
|
- ✅ Push operations (chunked uploads)
|
|
- ✅ Content discovery (tags, referrers API)
|
|
- ✅ Content management (deletion)
|
|
|
|
**NPM Features:**
|
|
- ✅ Package publish/unpublish
|
|
- ✅ Package download (tarballs)
|
|
- ✅ Metadata & search
|
|
- ✅ Dist-tag management
|
|
- ✅ Token management
|
|
|
|
## 📥 Installation
|
|
|
|
```bash
|
|
# Using npm
|
|
npm install @push.rocks/smartregistry
|
|
|
|
# Using pnpm (recommended)
|
|
pnpm add @push.rocks/smartregistry
|
|
```
|
|
|
|
## 🚀 Quick Start
|
|
|
|
```typescript
|
|
import { SmartRegistry, IRegistryConfig } from '@push.rocks/smartregistry';
|
|
|
|
const config: IRegistryConfig = {
|
|
storage: {
|
|
accessKey: 'your-s3-key',
|
|
accessSecret: 'your-s3-secret',
|
|
endpoint: 's3.amazonaws.com',
|
|
port: 443,
|
|
useSsl: true,
|
|
region: 'us-east-1',
|
|
bucketName: 'my-registry',
|
|
},
|
|
auth: {
|
|
jwtSecret: 'your-secret-key',
|
|
tokenStore: 'memory',
|
|
npmTokens: { enabled: true },
|
|
ociTokens: {
|
|
enabled: true,
|
|
realm: 'https://auth.example.com/token',
|
|
service: 'my-registry',
|
|
},
|
|
},
|
|
oci: {
|
|
enabled: true,
|
|
basePath: '/oci',
|
|
},
|
|
npm: {
|
|
enabled: true,
|
|
basePath: '/npm',
|
|
},
|
|
};
|
|
|
|
const registry = new SmartRegistry(config);
|
|
await registry.init();
|
|
|
|
// Handle requests
|
|
const response = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
```
|
|
|
|
## 🏛️ Architecture
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
ts/
|
|
├── core/ # Shared infrastructure
|
|
│ ├── classes.baseregistry.ts
|
|
│ ├── classes.registrystorage.ts
|
|
│ ├── classes.authmanager.ts
|
|
│ └── interfaces.core.ts
|
|
├── oci/ # OCI implementation
|
|
│ ├── classes.ociregistry.ts
|
|
│ └── interfaces.oci.ts
|
|
├── npm/ # NPM implementation
|
|
│ ├── classes.npmregistry.ts
|
|
│ └── interfaces.npm.ts
|
|
└── classes.smartregistry.ts # Main orchestrator
|
|
```
|
|
|
|
### Request Flow
|
|
|
|
```
|
|
HTTP Request
|
|
↓
|
|
SmartRegistry (orchestrator)
|
|
↓
|
|
Path-based routing
|
|
├─→ /oci/* → OciRegistry
|
|
└─→ /npm/* → NpmRegistry
|
|
↓
|
|
Shared Storage & Auth
|
|
↓
|
|
S3-compatible backend
|
|
```
|
|
|
|
## 💡 Usage Examples
|
|
|
|
### 🐳 OCI Registry (Container Images)
|
|
|
|
```typescript
|
|
// Pull an image
|
|
const response = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/oci/v2/library/nginx/manifests/latest',
|
|
headers: {
|
|
'Authorization': 'Bearer <token>',
|
|
},
|
|
query: {},
|
|
});
|
|
|
|
// Push a blob
|
|
const uploadInit = await registry.handleRequest({
|
|
method: 'POST',
|
|
path: '/oci/v2/myapp/blobs/uploads/',
|
|
headers: { 'Authorization': 'Bearer <token>' },
|
|
query: {},
|
|
});
|
|
|
|
const uploadId = uploadInit.headers['Docker-Upload-UUID'];
|
|
|
|
await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: `/oci/v2/myapp/blobs/uploads/${uploadId}`,
|
|
headers: { 'Authorization': 'Bearer <token>' },
|
|
query: { digest: 'sha256:abc123...' },
|
|
body: blobData,
|
|
});
|
|
```
|
|
|
|
### 📦 NPM Registry (Packages)
|
|
|
|
```typescript
|
|
// Install a package (get metadata)
|
|
const metadata = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Download tarball
|
|
const tarball = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express/-/express-4.18.0.tgz',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Publish a package
|
|
const publishResponse = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/npm/my-package',
|
|
headers: { 'Authorization': 'Bearer <npm-token>' },
|
|
query: {},
|
|
body: {
|
|
name: 'my-package',
|
|
versions: {
|
|
'1.0.0': { /* version metadata */ },
|
|
},
|
|
'dist-tags': { latest: '1.0.0' },
|
|
_attachments: {
|
|
'my-package-1.0.0.tgz': {
|
|
content_type: 'application/octet-stream',
|
|
data: '<base64-tarball>',
|
|
length: 12345,
|
|
},
|
|
},
|
|
},
|
|
});
|
|
|
|
// Search packages
|
|
const searchResults = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/-/v1/search',
|
|
headers: {},
|
|
query: { text: 'express', size: '20' },
|
|
});
|
|
```
|
|
|
|
### 🔐 Authentication
|
|
|
|
```typescript
|
|
// Get auth manager instance
|
|
const authManager = registry.getAuthManager();
|
|
|
|
// Authenticate user
|
|
const userId = await authManager.authenticate({
|
|
username: 'user',
|
|
password: 'pass',
|
|
});
|
|
|
|
// Create NPM token
|
|
const npmToken = await authManager.createNpmToken(userId, false);
|
|
|
|
// Create OCI token with scopes
|
|
const ociToken = await authManager.createOciToken(
|
|
userId,
|
|
['oci:repository:myapp:push', 'oci:repository:myapp:pull'],
|
|
3600
|
|
);
|
|
|
|
// Validate any token
|
|
const token = await authManager.validateToken(npmToken, 'npm');
|
|
|
|
// Check permissions
|
|
const canWrite = await authManager.authorize(
|
|
token,
|
|
'npm:package:my-package',
|
|
'write'
|
|
);
|
|
```
|
|
|
|
## ⚙️ Configuration
|
|
|
|
### Storage Configuration
|
|
|
|
```typescript
|
|
storage: {
|
|
accessKey: string; // S3 access key
|
|
accessSecret: string; // S3 secret key
|
|
endpoint: string; // S3 endpoint
|
|
port?: number; // Default: 443
|
|
useSsl?: boolean; // Default: true
|
|
region?: string; // Default: 'us-east-1'
|
|
bucketName: string; // Bucket name
|
|
}
|
|
```
|
|
|
|
### Authentication Configuration
|
|
|
|
```typescript
|
|
auth: {
|
|
jwtSecret: string; // Secret for signing JWTs
|
|
tokenStore: 'memory' | 'redis' | 'database';
|
|
npmTokens: {
|
|
enabled: boolean;
|
|
defaultReadonly?: boolean;
|
|
};
|
|
ociTokens: {
|
|
enabled: boolean;
|
|
realm: string; // Auth server URL
|
|
service: string; // Service name
|
|
};
|
|
}
|
|
```
|
|
|
|
### Protocol Configuration
|
|
|
|
```typescript
|
|
oci?: {
|
|
enabled: boolean;
|
|
basePath: string; // Default: '/oci'
|
|
features?: {
|
|
referrers?: boolean;
|
|
deletion?: boolean;
|
|
};
|
|
}
|
|
|
|
npm?: {
|
|
enabled: boolean;
|
|
basePath: string; // Default: '/npm'
|
|
features?: {
|
|
publish?: boolean;
|
|
unpublish?: boolean;
|
|
search?: boolean;
|
|
};
|
|
}
|
|
```
|
|
|
|
## 📚 API Reference
|
|
|
|
### Core Classes
|
|
|
|
#### SmartRegistry
|
|
|
|
Main orchestrator class that routes requests to appropriate protocol handlers.
|
|
|
|
**Methods:**
|
|
- `init()` - Initialize the registry
|
|
- `handleRequest(context)` - Handle HTTP request
|
|
- `getStorage()` - Get storage instance
|
|
- `getAuthManager()` - Get auth manager
|
|
- `getRegistry(protocol)` - Get protocol handler
|
|
|
|
#### RegistryStorage
|
|
|
|
Unified storage abstraction for both OCI and NPM content.
|
|
|
|
**OCI Methods:**
|
|
- `getOciBlob(digest)` - Get blob
|
|
- `putOciBlob(digest, data)` - Store blob
|
|
- `getOciManifest(repo, digest)` - Get manifest
|
|
- `putOciManifest(repo, digest, data, type)` - Store manifest
|
|
|
|
**NPM Methods:**
|
|
- `getNpmPackument(name)` - Get package metadata
|
|
- `putNpmPackument(name, data)` - Store package metadata
|
|
- `getNpmTarball(name, version)` - Get tarball
|
|
- `putNpmTarball(name, version, data)` - Store tarball
|
|
|
|
#### AuthManager
|
|
|
|
Unified authentication manager supporting both NPM and OCI authentication schemes.
|
|
|
|
**Methods:**
|
|
- `authenticate(credentials)` - Validate user credentials
|
|
- `createNpmToken(userId, readonly)` - Create NPM token
|
|
- `createOciToken(userId, scopes, expiresIn)` - Create OCI JWT
|
|
- `validateToken(token, protocol)` - Validate any token
|
|
- `authorize(token, resource, action)` - Check permissions
|
|
|
|
### Protocol Handlers
|
|
|
|
#### OciRegistry
|
|
|
|
OCI Distribution Specification v1.1 compliant registry.
|
|
|
|
**Endpoints:**
|
|
- `GET /v2/` - Version check
|
|
- `GET /v2/{name}/manifests/{ref}` - Get manifest
|
|
- `PUT /v2/{name}/manifests/{ref}` - Push manifest
|
|
- `GET /v2/{name}/blobs/{digest}` - Get blob
|
|
- `POST /v2/{name}/blobs/uploads/` - Initiate upload
|
|
- `PUT /v2/{name}/blobs/uploads/{uuid}` - Complete upload
|
|
- `GET /v2/{name}/tags/list` - List tags
|
|
- `GET /v2/{name}/referrers/{digest}` - Get referrers
|
|
|
|
#### NpmRegistry
|
|
|
|
NPM registry API compliant implementation.
|
|
|
|
**Endpoints:**
|
|
- `GET /{package}` - Get package metadata
|
|
- `PUT /{package}` - Publish package
|
|
- `GET /{package}/-/{tarball}` - Download tarball
|
|
- `GET /-/v1/search` - Search packages
|
|
- `PUT /-/user/org.couchdb.user:{user}` - Login
|
|
- `GET /-/npm/v1/tokens` - List tokens
|
|
- `POST /-/npm/v1/tokens` - Create token
|
|
- `PUT /-/package/{pkg}/dist-tags/{tag}` - Update tag
|
|
|
|
## 🗄️ Storage Structure
|
|
|
|
```
|
|
bucket/
|
|
├── oci/
|
|
│ ├── blobs/
|
|
│ │ └── sha256/{hash}
|
|
│ ├── manifests/
|
|
│ │ └── {repository}/{digest}
|
|
│ └── tags/
|
|
│ └── {repository}/tags.json
|
|
└── npm/
|
|
├── packages/
|
|
│ ├── {name}/
|
|
│ │ ├── index.json # Packument
|
|
│ │ └── {name}-{ver}.tgz # Tarball
|
|
│ └── @{scope}/{name}/
|
|
│ ├── index.json
|
|
│ └── {name}-{ver}.tgz
|
|
└── users/
|
|
└── {username}.json
|
|
```
|
|
|
|
## 🎯 Scope Format
|
|
|
|
Unified scope format across protocols:
|
|
|
|
```
|
|
{protocol}:{type}:{name}:{action}
|
|
|
|
Examples:
|
|
npm:package:express:read # Read express package
|
|
npm:package:*:write # Write any package
|
|
npm:*:*:* # Full NPM access
|
|
oci:repository:nginx:pull # Pull nginx image
|
|
oci:repository:*:push # Push any image
|
|
oci:*:*:* # Full OCI access
|
|
```
|
|
|
|
## 🔌 Integration Examples
|
|
|
|
### Express Server
|
|
|
|
```typescript
|
|
import express from 'express';
|
|
import { SmartRegistry } from '@push.rocks/smartregistry';
|
|
|
|
const app = express();
|
|
const registry = new SmartRegistry(config);
|
|
await registry.init();
|
|
|
|
app.all('*', async (req, res) => {
|
|
const response = await registry.handleRequest({
|
|
method: req.method,
|
|
path: req.path,
|
|
headers: req.headers as Record<string, string>,
|
|
query: req.query as Record<string, string>,
|
|
body: req.body,
|
|
});
|
|
|
|
res.status(response.status);
|
|
Object.entries(response.headers).forEach(([key, value]) => {
|
|
res.setHeader(key, value);
|
|
});
|
|
|
|
if (response.body) {
|
|
if (Buffer.isBuffer(response.body)) {
|
|
res.send(response.body);
|
|
} else {
|
|
res.json(response.body);
|
|
}
|
|
} else {
|
|
res.end();
|
|
}
|
|
});
|
|
|
|
app.listen(5000);
|
|
```
|
|
|
|
## 🛠️ Development
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pnpm install
|
|
|
|
# Build
|
|
pnpm run build
|
|
|
|
# Test
|
|
pnpm test
|
|
```
|
|
|
|
## License and Legal Information
|
|
|
|
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
|
|
|
|
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
|
|
|
|
### Trademarks
|
|
|
|
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
|
|
|
|
### Company Information
|
|
|
|
Task Venture Capital GmbH
|
|
Registered at District court Bremen HRB 35230 HB, Germany
|
|
|
|
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
|
|
|
|
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|