1045 lines
29 KiB
Markdown
1045 lines
29 KiB
Markdown
# @push.rocks/smartregistry
|
|
|
|
> 🚀 A composable TypeScript library implementing **OCI Distribution Specification v1.1**, **NPM Registry API**, **Maven Repository**, **Cargo/crates.io Registry**, **Composer/Packagist**, **PyPI (Python Package Index)**, and **RubyGems Registry** for building unified container and package registries.
|
|
|
|
## Issue Reporting and Security
|
|
|
|
For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who want to sign a contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.
|
|
|
|
## ✨ Features
|
|
|
|
### 🔄 Multi-Protocol Support
|
|
- **OCI Distribution Spec v1.1**: Full container registry with manifest/blob operations
|
|
- **NPM Registry API**: Complete package registry with publish/install/search
|
|
- **Maven Repository**: Java/JVM artifact management with POM support
|
|
- **Cargo/crates.io Registry**: Rust crate registry with sparse HTTP protocol
|
|
- **Composer/Packagist**: PHP package registry with Composer v2 protocol
|
|
- **PyPI (Python Package Index)**: Python package registry with PEP 503/691 support
|
|
- **RubyGems Registry**: Ruby gem registry with compact index protocol
|
|
|
|
### 🏗️ Unified Architecture
|
|
- **Composable Design**: Core infrastructure with protocol plugins
|
|
- **Shared Storage**: Cloud-agnostic S3-compatible backend using [@push.rocks/smartbucket](https://www.npmjs.com/package/@push.rocks/smartbucket) with standardized `IS3Descriptor` from [@tsclass/tsclass](https://www.npmjs.com/package/@tsclass/tsclass)
|
|
- **Unified Authentication**: Scope-based permissions across all protocols
|
|
- **Path-based Routing**: `/oci/*` for containers, `/npm/*` for packages, `/maven/*` for Java artifacts, `/cargo/*` for Rust crates, `/composer/*` for PHP packages, `/pypi/*` for Python packages, `/rubygems/*` for Ruby gems
|
|
|
|
### 🔐 Authentication & Authorization
|
|
- NPM UUID tokens for package operations
|
|
- OCI JWT tokens for container operations
|
|
- Unified scope system: `npm:package:foo:write`, `oci:repository:bar:push`
|
|
- Pluggable via async callbacks
|
|
|
|
### 📦 Comprehensive Feature Set
|
|
|
|
**OCI Features:**
|
|
- ✅ Pull operations (manifests, blobs)
|
|
- ✅ Push operations (chunked uploads)
|
|
- ✅ Content discovery (tags, referrers API)
|
|
- ✅ Content management (deletion)
|
|
|
|
**NPM Features:**
|
|
- ✅ Package publish/unpublish
|
|
- ✅ Package download (tarballs)
|
|
- ✅ Metadata & search
|
|
- ✅ Dist-tag management
|
|
- ✅ Token management
|
|
|
|
**Maven Features:**
|
|
- ✅ Artifact upload/download
|
|
- ✅ POM and metadata management
|
|
- ✅ Snapshot and release versions
|
|
- ✅ Checksum verification (MD5, SHA1)
|
|
|
|
**Cargo Features:**
|
|
- ✅ Crate publish (.crate files)
|
|
- ✅ Sparse HTTP protocol (modern index)
|
|
- ✅ Version yank/unyank
|
|
- ✅ Dependency resolution
|
|
- ✅ Search functionality
|
|
|
|
**Composer Features:**
|
|
- ✅ Package publish/download (ZIP format)
|
|
- ✅ Composer v2 repository API
|
|
- ✅ Package metadata (packages.json)
|
|
- ✅ Version management
|
|
- ✅ Dependency resolution
|
|
- ✅ PSR-4/PSR-0 autoloading support
|
|
|
|
**PyPI Features:**
|
|
- ✅ PEP 503 Simple Repository API (HTML)
|
|
- ✅ PEP 691 JSON-based Simple API
|
|
- ✅ Package upload (wheel and sdist)
|
|
- ✅ Package name normalization
|
|
- ✅ Hash verification (SHA256, MD5, Blake2b)
|
|
- ✅ Content negotiation (JSON/HTML)
|
|
- ✅ Metadata API (JSON endpoints)
|
|
|
|
**RubyGems Features:**
|
|
- ✅ Compact Index protocol (modern Bundler)
|
|
- ✅ Gem publish/download (.gem files)
|
|
- ✅ Version yank/unyank
|
|
- ✅ Platform-specific gems
|
|
- ✅ Dependency resolution
|
|
- ✅ Legacy API compatibility
|
|
|
|
## 📥 Installation
|
|
|
|
```bash
|
|
# Using npm
|
|
npm install @push.rocks/smartregistry
|
|
|
|
# Using pnpm (recommended)
|
|
pnpm add @push.rocks/smartregistry
|
|
```
|
|
|
|
## 🚀 Quick Start
|
|
|
|
```typescript
|
|
import { SmartRegistry, IRegistryConfig } from '@push.rocks/smartregistry';
|
|
|
|
const config: IRegistryConfig = {
|
|
storage: {
|
|
accessKey: 'your-s3-key',
|
|
accessSecret: 'your-s3-secret',
|
|
endpoint: 's3.amazonaws.com',
|
|
port: 443,
|
|
useSsl: true,
|
|
region: 'us-east-1',
|
|
bucketName: 'my-registry',
|
|
},
|
|
auth: {
|
|
jwtSecret: 'your-secret-key',
|
|
tokenStore: 'memory',
|
|
npmTokens: { enabled: true },
|
|
ociTokens: {
|
|
enabled: true,
|
|
realm: 'https://auth.example.com/token',
|
|
service: 'my-registry',
|
|
},
|
|
},
|
|
oci: {
|
|
enabled: true,
|
|
basePath: '/oci',
|
|
},
|
|
npm: {
|
|
enabled: true,
|
|
basePath: '/npm',
|
|
},
|
|
maven: {
|
|
enabled: true,
|
|
basePath: '/maven',
|
|
},
|
|
cargo: {
|
|
enabled: true,
|
|
basePath: '/cargo',
|
|
},
|
|
composer: {
|
|
enabled: true,
|
|
basePath: '/composer',
|
|
},
|
|
pypi: {
|
|
enabled: true,
|
|
basePath: '/pypi',
|
|
},
|
|
rubygems: {
|
|
enabled: true,
|
|
basePath: '/rubygems',
|
|
},
|
|
};
|
|
|
|
const registry = new SmartRegistry(config);
|
|
await registry.init();
|
|
|
|
// Handle requests
|
|
const response = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
```
|
|
|
|
## 🏛️ Architecture
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
ts/
|
|
├── core/ # Shared infrastructure
|
|
│ ├── classes.baseregistry.ts
|
|
│ ├── classes.registrystorage.ts
|
|
│ ├── classes.authmanager.ts
|
|
│ └── interfaces.core.ts
|
|
├── oci/ # OCI implementation
|
|
│ ├── classes.ociregistry.ts
|
|
│ └── interfaces.oci.ts
|
|
├── npm/ # NPM implementation
|
|
│ ├── classes.npmregistry.ts
|
|
│ └── interfaces.npm.ts
|
|
├── maven/ # Maven implementation
|
|
├── cargo/ # Cargo implementation
|
|
├── composer/ # Composer implementation
|
|
├── pypi/ # PyPI implementation
|
|
├── rubygems/ # RubyGems implementation
|
|
└── classes.smartregistry.ts # Main orchestrator
|
|
```
|
|
|
|
### Request Flow
|
|
|
|
```
|
|
HTTP Request
|
|
↓
|
|
SmartRegistry (orchestrator)
|
|
↓
|
|
Path-based routing
|
|
├─→ /oci/* → OciRegistry
|
|
├─→ /npm/* → NpmRegistry
|
|
├─→ /maven/* → MavenRegistry
|
|
├─→ /cargo/* → CargoRegistry
|
|
├─→ /composer/* → ComposerRegistry
|
|
├─→ /pypi/* → PypiRegistry
|
|
└─→ /rubygems/* → RubyGemsRegistry
|
|
↓
|
|
Shared Storage & Auth
|
|
↓
|
|
S3-compatible backend
|
|
```
|
|
|
|
## 💡 Usage Examples
|
|
|
|
### 🐳 OCI Registry (Container Images)
|
|
|
|
```typescript
|
|
// Pull an image
|
|
const response = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/oci/v2/library/nginx/manifests/latest',
|
|
headers: {
|
|
'Authorization': 'Bearer <token>',
|
|
},
|
|
query: {},
|
|
});
|
|
|
|
// Push a blob
|
|
const uploadInit = await registry.handleRequest({
|
|
method: 'POST',
|
|
path: '/oci/v2/myapp/blobs/uploads/',
|
|
headers: { 'Authorization': 'Bearer <token>' },
|
|
query: {},
|
|
});
|
|
|
|
const uploadId = uploadInit.headers['Docker-Upload-UUID'];
|
|
|
|
await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: `/oci/v2/myapp/blobs/uploads/${uploadId}`,
|
|
headers: { 'Authorization': 'Bearer <token>' },
|
|
query: { digest: 'sha256:abc123...' },
|
|
body: blobData,
|
|
});
|
|
```
|
|
|
|
### 📦 NPM Registry (Packages)
|
|
|
|
```typescript
|
|
// Install a package (get metadata)
|
|
const metadata = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Download tarball
|
|
const tarball = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/express/-/express-4.18.0.tgz',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Publish a package
|
|
const publishResponse = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/npm/my-package',
|
|
headers: { 'Authorization': 'Bearer <npm-token>' },
|
|
query: {},
|
|
body: {
|
|
name: 'my-package',
|
|
versions: {
|
|
'1.0.0': { /* version metadata */ },
|
|
},
|
|
'dist-tags': { latest: '1.0.0' },
|
|
_attachments: {
|
|
'my-package-1.0.0.tgz': {
|
|
content_type: 'application/octet-stream',
|
|
data: '<base64-tarball>',
|
|
length: 12345,
|
|
},
|
|
},
|
|
},
|
|
});
|
|
|
|
// Search packages
|
|
const searchResults = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/npm/-/v1/search',
|
|
headers: {},
|
|
query: { text: 'express', size: '20' },
|
|
});
|
|
```
|
|
|
|
### 🦀 Cargo Registry (Rust Crates)
|
|
|
|
```typescript
|
|
// Get config.json (required for Cargo)
|
|
const config = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/cargo/config.json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Get index file for a crate
|
|
const index = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/cargo/se/rd/serde', // Path based on crate name length
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Download a crate file
|
|
const crateFile = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/cargo/api/v1/crates/serde/1.0.0/download',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Publish a crate (binary format: [4 bytes JSON len][JSON][4 bytes crate len][.crate])
|
|
const publishResponse = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/cargo/api/v1/crates/new',
|
|
headers: { 'Authorization': '<cargo-token>' }, // No "Bearer" prefix
|
|
query: {},
|
|
body: binaryPublishData, // Length-prefixed binary format
|
|
});
|
|
|
|
// Yank a version (deprecate without deleting)
|
|
const yankResponse = await registry.handleRequest({
|
|
method: 'DELETE',
|
|
path: '/cargo/api/v1/crates/my-crate/0.1.0/yank',
|
|
headers: { 'Authorization': '<cargo-token>' },
|
|
query: {},
|
|
});
|
|
|
|
// Unyank a version
|
|
const unyankResponse = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/cargo/api/v1/crates/my-crate/0.1.0/unyank',
|
|
headers: { 'Authorization': '<cargo-token>' },
|
|
query: {},
|
|
});
|
|
|
|
// Search crates
|
|
const search = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/cargo/api/v1/crates',
|
|
headers: {},
|
|
query: { q: 'serde', per_page: '10' },
|
|
});
|
|
```
|
|
|
|
**Using with Cargo CLI:**
|
|
|
|
```toml
|
|
# .cargo/config.toml
|
|
[registries.myregistry]
|
|
index = "sparse+https://registry.example.com/cargo/"
|
|
|
|
[registries.myregistry.credential-provider]
|
|
# Or use credentials directly:
|
|
# [registries.myregistry]
|
|
# token = "your-api-token"
|
|
```
|
|
|
|
```bash
|
|
# Publish to custom registry
|
|
cargo publish --registry=myregistry
|
|
|
|
# Install from custom registry
|
|
cargo install --registry=myregistry my-crate
|
|
|
|
# Search custom registry
|
|
cargo search --registry=myregistry tokio
|
|
```
|
|
|
|
### 🎼 Composer Registry (PHP Packages)
|
|
|
|
```typescript
|
|
// Get repository root (packages.json)
|
|
const packagesJson = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/composer/packages.json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Get package metadata
|
|
const metadata = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/composer/p2/vendor/package.json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Upload a package (ZIP with composer.json)
|
|
const zipBuffer = await readFile('package.zip');
|
|
const uploadResponse = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/composer/packages/vendor/package',
|
|
headers: { 'Authorization': `Bearer <composer-token>` },
|
|
query: {},
|
|
body: zipBuffer,
|
|
});
|
|
|
|
// Download package ZIP
|
|
const download = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/composer/dists/vendor/package/ref123.zip',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// List all packages
|
|
const list = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/composer/packages/list.json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Delete a specific version
|
|
const deleteVersion = await registry.handleRequest({
|
|
method: 'DELETE',
|
|
path: '/composer/packages/vendor/package/1.0.0',
|
|
headers: { 'Authorization': `Bearer <composer-token>` },
|
|
query: {},
|
|
});
|
|
```
|
|
|
|
**Using with Composer CLI:**
|
|
|
|
```json
|
|
// composer.json
|
|
{
|
|
"repositories": [
|
|
{
|
|
"type": "composer",
|
|
"url": "https://registry.example.com/composer"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
```bash
|
|
# Install from custom registry
|
|
composer require vendor/package
|
|
|
|
# Update packages
|
|
composer update
|
|
```
|
|
|
|
### 🐍 PyPI Registry (Python Packages)
|
|
|
|
```typescript
|
|
// Get package index (PEP 503 HTML format)
|
|
const htmlIndex = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/simple/requests/',
|
|
headers: { 'Accept': 'text/html' },
|
|
query: {},
|
|
});
|
|
|
|
// Get package index (PEP 691 JSON format)
|
|
const jsonIndex = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/simple/requests/',
|
|
headers: { 'Accept': 'application/vnd.pypi.simple.v1+json' },
|
|
query: {},
|
|
});
|
|
|
|
// Upload a Python package (wheel or sdist)
|
|
const formData = new FormData();
|
|
formData.append(':action', 'file_upload');
|
|
formData.append('protocol_version', '1');
|
|
formData.append('name', 'my-package');
|
|
formData.append('version', '1.0.0');
|
|
formData.append('filetype', 'bdist_wheel');
|
|
formData.append('pyversion', 'py3');
|
|
formData.append('metadata_version', '2.1');
|
|
formData.append('sha256_digest', 'abc123...');
|
|
formData.append('content', packageFile, { filename: 'my_package-1.0.0-py3-none-any.whl' });
|
|
|
|
const upload = await registry.handleRequest({
|
|
method: 'POST',
|
|
path: '/pypi/legacy/',
|
|
headers: {
|
|
'Authorization': `Bearer <pypi-token>`,
|
|
'Content-Type': 'multipart/form-data',
|
|
},
|
|
query: {},
|
|
body: formData,
|
|
});
|
|
|
|
// Get package metadata (PyPI JSON API)
|
|
const metadata = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/pypi/my-package/json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Download a specific version
|
|
const download = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/packages/my-package/my_package-1.0.0-py3-none-any.whl',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
```
|
|
|
|
**Using with pip:**
|
|
|
|
```bash
|
|
# Install from custom registry
|
|
pip install --index-url https://registry.example.com/simple/ my-package
|
|
|
|
# Upload to custom registry
|
|
python -m twine upload --repository-url https://registry.example.com/pypi/legacy/ dist/*
|
|
|
|
# Configure in pip.conf or pip.ini
|
|
[global]
|
|
index-url = https://registry.example.com/simple/
|
|
```
|
|
|
|
### 💎 RubyGems Registry (Ruby Gems)
|
|
|
|
```typescript
|
|
// Get versions file (compact index)
|
|
const versions = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/rubygems/versions',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Get gem-specific info
|
|
const gemInfo = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/rubygems/info/rails',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Get list of all gem names
|
|
const names = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/rubygems/names',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Upload a gem file
|
|
const gemBuffer = await readFile('my-gem-1.0.0.gem');
|
|
const uploadGem = await registry.handleRequest({
|
|
method: 'POST',
|
|
path: '/rubygems/api/v1/gems',
|
|
headers: { 'Authorization': '<rubygems-api-key>' },
|
|
query: {},
|
|
body: gemBuffer,
|
|
});
|
|
|
|
// Yank a version (make unavailable for install)
|
|
const yank = await registry.handleRequest({
|
|
method: 'DELETE',
|
|
path: '/rubygems/api/v1/gems/yank',
|
|
headers: { 'Authorization': '<rubygems-api-key>' },
|
|
query: { gem_name: 'my-gem', version: '1.0.0' },
|
|
});
|
|
|
|
// Unyank a version
|
|
const unyank = await registry.handleRequest({
|
|
method: 'PUT',
|
|
path: '/rubygems/api/v1/gems/unyank',
|
|
headers: { 'Authorization': '<rubygems-api-key>' },
|
|
query: { gem_name: 'my-gem', version: '1.0.0' },
|
|
});
|
|
|
|
// Get gem version metadata
|
|
const versionMeta = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/rubygems/api/v1/versions/rails.json',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
|
|
// Download gem file
|
|
const gemDownload = await registry.handleRequest({
|
|
method: 'GET',
|
|
path: '/rubygems/gems/rails-7.0.0.gem',
|
|
headers: {},
|
|
query: {},
|
|
});
|
|
```
|
|
|
|
**Using with Bundler:**
|
|
|
|
```ruby
|
|
# Gemfile
|
|
source 'https://registry.example.com/rubygems' do
|
|
gem 'my-gem'
|
|
gem 'rails'
|
|
end
|
|
```
|
|
|
|
```bash
|
|
# Install gems
|
|
bundle install
|
|
|
|
# Push gem to custom registry
|
|
gem push my-gem-1.0.0.gem --host https://registry.example.com/rubygems
|
|
|
|
# Configure gem source
|
|
gem sources --add https://registry.example.com/rubygems/
|
|
gem sources --remove https://rubygems.org/
|
|
```
|
|
|
|
### 🔐 Authentication
|
|
|
|
```typescript
|
|
// Get auth manager instance
|
|
const authManager = registry.getAuthManager();
|
|
|
|
// Authenticate user
|
|
const userId = await authManager.authenticate({
|
|
username: 'user',
|
|
password: 'pass',
|
|
});
|
|
|
|
// Create NPM token
|
|
const npmToken = await authManager.createNpmToken(userId, false);
|
|
|
|
// Create OCI token with scopes
|
|
const ociToken = await authManager.createOciToken(
|
|
userId,
|
|
['oci:repository:myapp:push', 'oci:repository:myapp:pull'],
|
|
3600
|
|
);
|
|
|
|
// Validate any token
|
|
const token = await authManager.validateToken(npmToken, 'npm');
|
|
|
|
// Check permissions
|
|
const canWrite = await authManager.authorize(
|
|
token,
|
|
'npm:package:my-package',
|
|
'write'
|
|
);
|
|
```
|
|
|
|
## ⚙️ Configuration
|
|
|
|
### Storage Configuration
|
|
|
|
The storage configuration extends `IS3Descriptor` from `@tsclass/tsclass` for standardized S3 configuration:
|
|
|
|
```typescript
|
|
import type { IS3Descriptor } from '@tsclass/tsclass';
|
|
|
|
storage: IS3Descriptor & {
|
|
bucketName: string; // Bucket name for registry storage
|
|
}
|
|
|
|
// Example:
|
|
storage: {
|
|
accessKey: string; // S3 access key
|
|
accessSecret: string; // S3 secret key
|
|
endpoint: string; // S3 endpoint (e.g., 's3.amazonaws.com')
|
|
port?: number; // Default: 443
|
|
useSsl?: boolean; // Default: true
|
|
region?: string; // AWS region (e.g., 'us-east-1')
|
|
bucketName: string; // Bucket name for this registry
|
|
}
|
|
```
|
|
|
|
### Authentication Configuration
|
|
|
|
```typescript
|
|
auth: {
|
|
jwtSecret: string; // Secret for signing JWTs
|
|
tokenStore: 'memory' | 'redis' | 'database';
|
|
npmTokens: {
|
|
enabled: boolean;
|
|
defaultReadonly?: boolean;
|
|
};
|
|
ociTokens: {
|
|
enabled: boolean;
|
|
realm: string; // Auth server URL
|
|
service: string; // Service name
|
|
};
|
|
}
|
|
```
|
|
|
|
### Protocol Configuration
|
|
|
|
```typescript
|
|
oci?: {
|
|
enabled: boolean;
|
|
basePath: string; // Default: '/oci'
|
|
features?: {
|
|
referrers?: boolean;
|
|
deletion?: boolean;
|
|
};
|
|
}
|
|
|
|
npm?: {
|
|
enabled: boolean;
|
|
basePath: string; // Default: '/npm'
|
|
features?: {
|
|
publish?: boolean;
|
|
unpublish?: boolean;
|
|
search?: boolean;
|
|
};
|
|
}
|
|
```
|
|
|
|
## 📚 API Reference
|
|
|
|
### Core Classes
|
|
|
|
#### SmartRegistry
|
|
|
|
Main orchestrator class that routes requests to appropriate protocol handlers.
|
|
|
|
**Methods:**
|
|
- `init()` - Initialize the registry
|
|
- `handleRequest(context)` - Handle HTTP request
|
|
- `getStorage()` - Get storage instance
|
|
- `getAuthManager()` - Get auth manager
|
|
- `getRegistry(protocol)` - Get protocol handler
|
|
|
|
#### RegistryStorage
|
|
|
|
Unified storage abstraction for both OCI and NPM content.
|
|
|
|
**OCI Methods:**
|
|
- `getOciBlob(digest)` - Get blob
|
|
- `putOciBlob(digest, data)` - Store blob
|
|
- `getOciManifest(repo, digest)` - Get manifest
|
|
- `putOciManifest(repo, digest, data, type)` - Store manifest
|
|
|
|
**NPM Methods:**
|
|
- `getNpmPackument(name)` - Get package metadata
|
|
- `putNpmPackument(name, data)` - Store package metadata
|
|
- `getNpmTarball(name, version)` - Get tarball
|
|
- `putNpmTarball(name, version, data)` - Store tarball
|
|
|
|
**PyPI Methods:**
|
|
- `getPypiPackageMetadata(name)` - Get package metadata
|
|
- `putPypiPackageMetadata(name, data)` - Store package metadata
|
|
- `getPypiPackageFile(name, filename)` - Get package file
|
|
- `putPypiPackageFile(name, filename, data)` - Store package file
|
|
|
|
**RubyGems Methods:**
|
|
- `getRubyGemsVersions()` - Get versions index
|
|
- `putRubyGemsVersions(data)` - Store versions index
|
|
- `getRubyGemsInfo(gemName)` - Get gem info
|
|
- `putRubyGemsInfo(gemName, data)` - Store gem info
|
|
- `getRubyGem(gemName, version)` - Get .gem file
|
|
- `putRubyGem(gemName, version, data)` - Store .gem file
|
|
|
|
#### AuthManager
|
|
|
|
Unified authentication manager supporting both NPM and OCI authentication schemes.
|
|
|
|
**Methods:**
|
|
- `authenticate(credentials)` - Validate user credentials
|
|
- `createNpmToken(userId, readonly)` - Create NPM token
|
|
- `createOciToken(userId, scopes, expiresIn)` - Create OCI JWT
|
|
- `validateToken(token, protocol)` - Validate any token
|
|
- `authorize(token, resource, action)` - Check permissions
|
|
|
|
### Protocol Handlers
|
|
|
|
#### OciRegistry
|
|
|
|
OCI Distribution Specification v1.1 compliant registry.
|
|
|
|
**Endpoints:**
|
|
- `GET /v2/` - Version check
|
|
- `GET /v2/{name}/manifests/{ref}` - Get manifest
|
|
- `PUT /v2/{name}/manifests/{ref}` - Push manifest
|
|
- `GET /v2/{name}/blobs/{digest}` - Get blob
|
|
- `POST /v2/{name}/blobs/uploads/` - Initiate upload
|
|
- `PUT /v2/{name}/blobs/uploads/{uuid}` - Complete upload
|
|
- `GET /v2/{name}/tags/list` - List tags
|
|
- `GET /v2/{name}/referrers/{digest}` - Get referrers
|
|
|
|
#### NpmRegistry
|
|
|
|
NPM registry API compliant implementation.
|
|
|
|
**Endpoints:**
|
|
- `GET /{package}` - Get package metadata
|
|
- `PUT /{package}` - Publish package
|
|
- `GET /{package}/-/{tarball}` - Download tarball
|
|
- `GET /-/v1/search` - Search packages
|
|
- `PUT /-/user/org.couchdb.user:{user}` - Login
|
|
- `GET /-/npm/v1/tokens` - List tokens
|
|
- `POST /-/npm/v1/tokens` - Create token
|
|
- `PUT /-/package/{pkg}/dist-tags/{tag}` - Update tag
|
|
|
|
#### CargoRegistry
|
|
|
|
Cargo/crates.io registry with sparse HTTP protocol support.
|
|
|
|
**Endpoints:**
|
|
- `GET /config.json` - Registry configuration (sparse protocol)
|
|
- `GET /index/{path}` - Index files (hierarchical structure)
|
|
- `/1/{name}` - 1-character crate names
|
|
- `/2/{name}` - 2-character crate names
|
|
- `/3/{c}/{name}` - 3-character crate names
|
|
- `/{p1}/{p2}/{name}` - 4+ character crate names
|
|
- `PUT /api/v1/crates/new` - Publish crate (binary format)
|
|
- `GET /api/v1/crates/{crate}/{version}/download` - Download .crate file
|
|
- `DELETE /api/v1/crates/{crate}/{version}/yank` - Yank (deprecate) version
|
|
- `PUT /api/v1/crates/{crate}/{version}/unyank` - Unyank version
|
|
- `GET /api/v1/crates?q={query}` - Search crates
|
|
|
|
**Index Format:**
|
|
- Newline-delimited JSON (one line per version)
|
|
- SHA256 checksums for .crate files
|
|
- Yanked flag (keep files, mark unavailable)
|
|
|
|
#### ComposerRegistry
|
|
|
|
Composer v2 repository API compliant implementation.
|
|
|
|
**Endpoints:**
|
|
- `GET /packages.json` - Repository metadata and configuration
|
|
- `GET /p2/{vendor}/{package}.json` - Package version metadata
|
|
- `GET /p2/{vendor}/{package}~dev.json` - Dev versions metadata
|
|
- `GET /packages/list.json` - List all packages
|
|
- `GET /dists/{vendor}/{package}/{ref}.zip` - Download package ZIP
|
|
- `PUT /packages/{vendor}/{package}` - Upload package (requires auth)
|
|
- `DELETE /packages/{vendor}/{package}` - Delete entire package
|
|
- `DELETE /packages/{vendor}/{package}/{version}` - Delete specific version
|
|
|
|
#### PypiRegistry
|
|
|
|
PyPI (Python Package Index) registry implementing PEP 503 and PEP 691.
|
|
|
|
**Endpoints:**
|
|
- `GET /simple/` - List all packages (HTML or JSON)
|
|
- `GET /simple/{package}/` - List package files (HTML or JSON)
|
|
- `POST /legacy/` - Upload package (multipart/form-data)
|
|
- `GET /pypi/{package}/json` - Package metadata API
|
|
- `GET /pypi/{package}/{version}/json` - Version-specific metadata
|
|
- `GET /packages/{package}/{filename}` - Download package file
|
|
|
|
**Features:**
|
|
- PEP 503 Simple Repository API (HTML)
|
|
- PEP 691 JSON-based Simple API
|
|
- Content negotiation via Accept header
|
|
- Package name normalization
|
|
- Hash verification (SHA256, MD5, Blake2b)
|
|
|
|
#### RubyGemsRegistry
|
|
|
|
RubyGems registry with compact index protocol for modern Bundler.
|
|
|
|
**Endpoints:**
|
|
- `GET /versions` - Master versions file (all gems)
|
|
- `GET /info/{gem}` - Gem-specific info file
|
|
- `GET /names` - List of all gem names
|
|
- `POST /api/v1/gems` - Upload gem file
|
|
- `DELETE /api/v1/gems/yank` - Yank (deprecate) version
|
|
- `PUT /api/v1/gems/unyank` - Unyank version
|
|
- `GET /api/v1/versions/{gem}.json` - Version metadata
|
|
- `GET /gems/{gem}-{version}.gem` - Download gem file
|
|
|
|
**Features:**
|
|
- Compact Index format (append-only text files)
|
|
- Platform-specific gems support
|
|
- Yank/unyank functionality
|
|
- Checksum calculations (MD5 for index, SHA256 for gems)
|
|
- Legacy Marshal API compatibility
|
|
|
|
## 🗄️ Storage Structure
|
|
|
|
```
|
|
bucket/
|
|
├── oci/
|
|
│ ├── blobs/
|
|
│ │ └── sha256/{hash}
|
|
│ ├── manifests/
|
|
│ │ └── {repository}/{digest}
|
|
│ └── tags/
|
|
│ └── {repository}/tags.json
|
|
├── npm/
|
|
│ ├── packages/
|
|
│ │ ├── {name}/
|
|
│ │ │ ├── index.json # Packument
|
|
│ │ │ └── {name}-{ver}.tgz # Tarball
|
|
│ │ └── @{scope}/{name}/
|
|
│ │ ├── index.json
|
|
│ │ └── {name}-{ver}.tgz
|
|
│ └── users/
|
|
│ └── {username}.json
|
|
├── maven/
|
|
│ ├── artifacts/
|
|
│ │ └── {group-path}/{artifact}/{version}/
|
|
│ │ ├── {artifact}-{version}.jar
|
|
│ │ ├── {artifact}-{version}.pom
|
|
│ │ └── {artifact}-{version}.{ext}
|
|
│ └── metadata/
|
|
│ └── {group-path}/{artifact}/maven-metadata.xml
|
|
├── cargo/
|
|
│ ├── config.json # Registry configuration (sparse protocol)
|
|
│ ├── index/ # Hierarchical index structure
|
|
│ │ ├── 1/{name} # 1-char crate names (e.g., "a")
|
|
│ │ ├── 2/{name} # 2-char crate names (e.g., "io")
|
|
│ │ ├── 3/{c}/{name} # 3-char crate names (e.g., "3/a/axo")
|
|
│ │ └── {p1}/{p2}/{name} # 4+ char (e.g., "se/rd/serde")
|
|
│ └── crates/
|
|
│ └── {name}/{name}-{version}.crate # Gzipped tar archives
|
|
├── composer/
|
|
│ └── packages/
|
|
│ └── {vendor}/{package}/
|
|
│ ├── metadata.json # All versions metadata
|
|
│ └── {reference}.zip # Package ZIP files
|
|
├── pypi/
|
|
│ ├── simple/ # PEP 503 HTML files
|
|
│ │ ├── index.html # All packages list
|
|
│ │ └── {package}/index.html # Package versions list
|
|
│ ├── packages/
|
|
│ │ └── {package}/{filename} # .whl and .tar.gz files
|
|
│ └── metadata/
|
|
│ └── {package}/metadata.json # Package metadata
|
|
└── rubygems/
|
|
├── versions # Master versions file
|
|
├── info/{gemname} # Per-gem info files
|
|
├── names # All gem names
|
|
└── gems/{gemname}-{version}.gem # .gem files
|
|
```
|
|
|
|
## 🎯 Scope Format
|
|
|
|
Unified scope format across protocols:
|
|
|
|
```
|
|
{protocol}:{type}:{name}:{action}
|
|
|
|
Examples:
|
|
npm:package:express:read # Read express package
|
|
npm:package:*:write # Write any package
|
|
npm:*:*:* # Full NPM access
|
|
|
|
oci:repository:nginx:pull # Pull nginx image
|
|
oci:repository:*:push # Push any image
|
|
oci:*:*:* # Full OCI access
|
|
|
|
maven:artifact:com.example:read # Read Maven artifact
|
|
maven:artifact:*:write # Write any artifact
|
|
maven:*:*:* # Full Maven access
|
|
|
|
cargo:crate:serde:write # Write serde crate
|
|
cargo:crate:*:read # Read any crate
|
|
cargo:*:*:* # Full Cargo access
|
|
|
|
composer:package:vendor/package:read # Read Composer package
|
|
composer:package:*:write # Write any package
|
|
composer:*:*:* # Full Composer access
|
|
|
|
pypi:package:my-package:read # Read PyPI package
|
|
pypi:package:*:write # Write any package
|
|
pypi:*:*:* # Full PyPI access
|
|
|
|
rubygems:gem:rails:read # Read RubyGems gem
|
|
rubygems:gem:*:write # Write any gem
|
|
rubygems:*:*:* # Full RubyGems access
|
|
```
|
|
|
|
## 🔌 Integration Examples
|
|
|
|
### Express Server
|
|
|
|
```typescript
|
|
import express from 'express';
|
|
import { SmartRegistry } from '@push.rocks/smartregistry';
|
|
|
|
const app = express();
|
|
const registry = new SmartRegistry(config);
|
|
await registry.init();
|
|
|
|
app.all('*', async (req, res) => {
|
|
const response = await registry.handleRequest({
|
|
method: req.method,
|
|
path: req.path,
|
|
headers: req.headers as Record<string, string>,
|
|
query: req.query as Record<string, string>,
|
|
body: req.body,
|
|
});
|
|
|
|
res.status(response.status);
|
|
Object.entries(response.headers).forEach(([key, value]) => {
|
|
res.setHeader(key, value);
|
|
});
|
|
|
|
if (response.body) {
|
|
if (Buffer.isBuffer(response.body)) {
|
|
res.send(response.body);
|
|
} else {
|
|
res.json(response.body);
|
|
}
|
|
} else {
|
|
res.end();
|
|
}
|
|
});
|
|
|
|
app.listen(5000);
|
|
```
|
|
|
|
## 🛠️ Development
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pnpm install
|
|
|
|
# Build
|
|
pnpm run build
|
|
|
|
# Test
|
|
pnpm test
|
|
```
|
|
|
|
## License and Legal Information
|
|
|
|
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
|
|
|
|
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
|
|
|
|
### Trademarks
|
|
|
|
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
|
|
|
|
### Company Information
|
|
|
|
Task Venture Capital GmbH
|
|
Registered at District court Bremen HRB 35230 HB, Germany
|
|
|
|
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
|
|
|
|
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|