stack.gallery/registry
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat(release,build,tests): add automated multi-platform release pipeline and align runtime, model, and test updates

Browse Source
This commit is contained in:
Jürgen Kunz
2026-03-20 13:56:43 +00:00
parent 4d561b3874
commit b05c53f967
25 changed files with 3747 additions and 941 deletions
Download Patch File Download Diff File Expand all files Collapse all files

438
readme.md
View File

@@ -1,6 +1,6 @@
# @stack.gallery/registry 📦
**Enterprise-grade multi-protocol package registry** built with Deno and TypeScript. Host your own private NPM, Docker/OCI, Maven, Cargo, PyPI, Composer, and RubyGems registry with a unified, beautiful web interface.
A self-hosted, multi-protocol package registry built with Deno and TypeScript. Run your own private **NPM**, **Docker/OCI**, **Maven**, **Cargo**, **PyPI**, **Composer**, and **RubyGems** registry — all behind a single binary with a modern web UI.
## Issue Reporting and Security
@@ -8,225 +8,357 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
## ✨ Features
- 🔐 **Multi-Protocol Support** - NPM, OCI/Docker, Maven, Cargo, PyPI, Composer, RubyGems
- 🏢 **Organizations & Teams** - Fine-grained access control with role-based permissions
- 🎫 **API Tokens** - Scoped tokens for CI/CD and programmatic access
- 🔍 **Upstream Caching** - Proxy and cache packages from public registries
- 📊 **Audit Logging** - Complete audit trail for compliance and security
- 🎨 **Modern Web UI** - Angular 19 dashboard for package management
- ⚡ **Deno Runtime** - Fast, secure, TypeScript-first backend
- 🗄️ **MongoDB + S3** - Scalable storage with smartdata ORM
- 🔌 **7 Protocol Support** — NPM, OCI/Docker, Maven, Cargo, PyPI, Composer, RubyGems via [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry)
- 🏢 **Organizations & Teams** — Hierarchical access control: orgs → teams → repositories
- 🔐 **Flexible Authentication** — Local JWT auth, OAuth/OIDC, and LDAP with JIT user provisioning
- 🎫 **Scoped API Tokens** — Per-protocol, per-scope tokens (`srg_` prefix) for CI/CD pipelines
- 🛡️ **RBAC Permissions** — Reader → Developer → Maintainer → Admin per repository
- 🔍 **Upstream Caching** — Transparently proxy and cache packages from public registries
- 📊 **Audit Logging** — Full audit trail on every action for compliance
- 🎨 **Modern Web UI** — Angular 19 dashboard with Tailwind CSS, embedded in the binary
- ⚡ **Single Binary** — Cross-compiled with `deno compile` for Linux and macOS (x64 + ARM64)
- 🗄️ **MongoDB + S3** — Metadata in MongoDB, artifacts in any S3-compatible store
## 🚀 Quick Start
### Prerequisites
- **Deno** >= 1.40
- **MongoDB** >= 4.4
- **S3-compatible storage** (MinIO, AWS S3, etc.)
- **Node.js** >= 18 (for UI development)
- **S3-compatible storage** (MinIO, AWS S3, Cloudflare R2, etc.)
### Installation
### Install from Binary
```bash
# Clone the repository
# One-liner install (latest version)
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash
# Install specific version
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash -s -- --version v1.3.0
# Install + set up systemd service
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash -s -- --setup-service
```
The installer:
- Detects your platform (Linux/macOS, x64/ARM64)
- Downloads the pre-compiled binary from Gitea releases
- Installs to `/opt/stack-gallery-registry/` with a symlink in `/usr/local/bin/`
- Optionally creates and enables a systemd service
### Run from Source
```bash
# Clone
git clone https://code.foss.global/stack.gallery/registry.git
cd registry
# Install UI dependencies
cd ui && pnpm install && cd ..
# Development mode (hot reload, reads .nogit/env.json)
deno task dev
# Build the UI
pnpm run build
# Production mode
deno task start
```
### Configuration
The registry is available at `http://localhost:3000`.
Create a `.nogit/env.json` file for local development:
## ⚙️ Configuration
Configuration is loaded from **environment variables** (production) or from **`.nogit/env.json`** when using the `--ephemeral` flag (development).
| Variable | Default | Description |
|----------|---------|-------------|
| `MONGODB_URL` | `mongodb://localhost:27017` | MongoDB connection string |
| `MONGODB_DB` | `stackgallery` | Database name |
| `S3_ENDPOINT` | `http://localhost:9000` | S3-compatible endpoint |
| `S3_ACCESS_KEY` | `minioadmin` | S3 access key |
| `S3_SECRET_KEY` | `minioadmin` | S3 secret key |
| `S3_BUCKET` | `registry` | S3 bucket name |
| `S3_REGION` | — | S3 region |
| `HOST` | `0.0.0.0` | Server bind address |
| `PORT` | `3000` | Server port |
| `JWT_SECRET` | `change-me-in-production` | JWT signing secret |
| `AUTH_ENCRYPTION_KEY` | *(ephemeral)* | 64-char hex for AES-256-GCM encryption of OAuth/LDAP secrets |
| `STORAGE_PATH` | `packages` | Base path in S3 for artifacts |
| `ENABLE_UPSTREAM_CACHE` | `true` | Cache packages from upstream registries |
| `UPSTREAM_CACHE_EXPIRY` | `24` | Cache TTL in hours |
**Example `.nogit/env.json`:**
```json
{
"MONGODB_URL": "mongodb://localhost:27017",
"MONGODB_URL": "mongodb://admin:pass@localhost:27017/stackregistry?authSource=admin",
"MONGODB_NAME": "stackregistry",
"S3_HOST": "localhost",
"S3_PORT": "9000",
"S3_ACCESSKEY": "minioadmin",
"S3_SECRETKEY": "minioadmin",
"S3_BUCKET": "registry",
"S3_USESSL": false,
"JWT_SECRET": "your-secure-secret-key",
"ADMIN_EMAIL": "admin@example.com",
"ADMIN_PASSWORD": "your-admin-password"
"S3_USESSL": false
}
```
Or use environment variables:
## 🔌 Protocol Endpoints
| Variable | Description | Default |
|----------|-------------|---------|
| `MONGODB_URL` | MongoDB connection string | `mongodb://localhost:27017` |
| `MONGODB_DB` | Database name | `stackgallery` |
| `S3_ENDPOINT` | S3 endpoint URL | `http://localhost:9000` |
| `S3_ACCESS_KEY` | S3 access key | `minioadmin` |
| `S3_SECRET_KEY` | S3 secret key | `minioadmin` |
| `S3_BUCKET` | S3 bucket name | `registry` |
| `JWT_SECRET` | JWT signing secret | `change-me-in-production` |
| `ADMIN_EMAIL` | Default admin email | `admin@stack.gallery` |
| `ADMIN_PASSWORD` | Default admin password | `admin` |
| `PORT` | HTTP server port | `3000` |
Each protocol is handled natively via [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry). Point your package manager at the registry:
### Running
| Protocol | Paths | Client Config Example |
|----------|-------|-----------------------|
| **NPM** | `/-/*`, `/@scope/*` | `npm config set registry http://registry:3000` |
| **OCI/Docker** | `/v2/*` | `docker login registry:3000` |
| **Maven** | `/maven2/*` | Add repository URL in `pom.xml` |
| **Cargo** | `/api/v1/crates/*` | Configure in `.cargo/config.toml` |
| **PyPI** | `/simple/*`, `/pypi/*` | `pip install --index-url http://registry:3000/simple/` |
| **Composer** | `/packages.json`, `/p/*` | Add repository in `composer.json` |
| **RubyGems** | `/api/v1/gems/*`, `/gems/*` | `gem sources -a http://registry:3000` |
```bash
# Development mode (with hot reload)
pnpm run watch
Authentication works with **Bearer tokens** (API tokens prefixed `srg_`) and **Basic auth** (email:password or username:token).
# Production mode
deno run --allow-all mod.ts server
## 🔐 Authentication & Security
# Or with Deno tasks
deno task start
### Local Auth
- JWT-based with **15-minute access tokens** and **7-day refresh tokens** (HS256)
- Session tracking — each login creates a session, tokens embed session IDs
- Password hashing with PBKDF2 (10,000 rounds SHA-256 + random salt)
### External Auth (OAuth/OIDC & LDAP)
- **OAuth/OIDC** — Connect to any OIDC-compliant provider (Keycloak, Okta, Auth0, Azure AD, etc.)
- **LDAP** — Bind + search authentication against Active Directory or OpenLDAP
- **JIT Provisioning** — Users are auto-created on first external login
- **Auto-linking** — External identities are linked to existing users by email match
- **Encrypted secrets** — Provider client secrets and bind passwords are stored AES-256-GCM encrypted
### RBAC Permissions
Access is resolved through a hierarchy:
```
Platform Admin (full access)
└─ Organization Owner/Admin
└─ Team Maintainer (read + write + delete on team repos)
└─ Team Member (read + write on team repos)
└─ Direct Repo Permission (reader / developer / maintainer / admin)
└─ Public Repository (read for everyone)
```
The registry will be available at `http://localhost:3000`
### Scoped API Tokens
Tokens are prefixed with `srg_` and can be scoped to:
- Specific **protocols** (e.g., npm + oci only)
- Specific **actions** (read / write / delete)
- Specific **organizations**
- Custom **expiration** dates
## 📡 REST API
All management endpoints live under `/api/v1/`. Authenticated via `Authorization: Bearer <jwt_or_api_token>`.
### Auth
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/auth/login` | Login (email + password) |
| `POST` | `/api/v1/auth/refresh` | Refresh access token |
| `POST` | `/api/v1/auth/logout` | Logout (invalidate session) |
| `GET` | `/api/v1/auth/me` | Current user info |
| `GET` | `/api/v1/auth/providers` | List active external auth providers |
| `GET` | `/api/v1/auth/oauth/:id/authorize` | Initiate OAuth flow |
| `GET` | `/api/v1/auth/oauth/:id/callback` | OAuth callback |
| `POST` | `/api/v1/auth/ldap/:id/login` | LDAP login |
### Users
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/users` | List users |
| `POST` | `/api/v1/users` | Create user |
| `GET` | `/api/v1/users/:id` | Get user |
| `PUT` | `/api/v1/users/:id` | Update user |
| `DELETE` | `/api/v1/users/:id` | Delete user |
### Organizations
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations` | List organizations |
| `POST` | `/api/v1/organizations` | Create organization |
| `GET` | `/api/v1/organizations/:id` | Get organization |
| `PUT` | `/api/v1/organizations/:id` | Update organization |
| `DELETE` | `/api/v1/organizations/:id` | Delete organization |
| `GET` | `/api/v1/organizations/:id/members` | List members |
| `POST` | `/api/v1/organizations/:id/members` | Add member |
| `PUT` | `/api/v1/organizations/:id/members/:userId` | Update member role |
| `DELETE` | `/api/v1/organizations/:id/members/:userId` | Remove member |
### Repositories
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations/:orgId/repositories` | List org repos |
| `POST` | `/api/v1/organizations/:orgId/repositories` | Create repo |
| `GET` | `/api/v1/repositories/:id` | Get repo |
| `PUT` | `/api/v1/repositories/:id` | Update repo |
| `DELETE` | `/api/v1/repositories/:id` | Delete repo |
### Packages
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/packages` | Search packages |
| `GET` | `/api/v1/packages/:id` | Get package details |
| `GET` | `/api/v1/packages/:id/versions` | List versions |
| `DELETE` | `/api/v1/packages/:id` | Delete package |
| `DELETE` | `/api/v1/packages/:id/versions/:version` | Delete version |
### Tokens
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/tokens` | List your tokens |
| `POST` | `/api/v1/tokens` | Create token |
| `DELETE` | `/api/v1/tokens/:id` | Revoke token |
### Audit
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/audit` | Query audit logs |
### Admin (Platform Admins Only)
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/auth/providers` | List all auth providers |
| `POST` | `/api/v1/admin/auth/providers` | Create auth provider |
| `GET` | `/api/v1/admin/auth/providers/:id` | Get provider details |
| `PUT` | `/api/v1/admin/auth/providers/:id` | Update provider |
| `DELETE` | `/api/v1/admin/auth/providers/:id` | Disable provider |
| `POST` | `/api/v1/admin/auth/providers/:id/test` | Test provider connection |
| `GET` | `/api/v1/admin/auth/settings` | Get platform settings |
| `PUT` | `/api/v1/admin/auth/settings` | Update platform settings |
### Health Check
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/health` or `/healthz` | Returns JSON status of MongoDB, S3, and registry |
## 🏗️ Architecture
```
registry/
├── mod.ts # Entry point
├── mod.ts # Deno entry point
├── deno.json # Deno config, tasks, imports
├── npmextra.json # tsdeno compile targets & gitzone config
├── install.sh # Binary installer script
├── .gitea/workflows/ # CI release pipeline
├── scripts/
│ └── bundle-ui.ts # Embeds Angular build as base64 TypeScript
├── ts/
│ ├── registry.ts # Main StackGalleryRegistry class
│ ├── cli.ts # CLI command handler
│ ├── plugins.ts # Centralized dependencies
│ ├── registry.ts # StackGalleryRegistry — main orchestrator
│ ├── cli.ts # CLI commands (smartcli)
│ ├── plugins.ts # Centralized dependency imports
│ ├── api/
│ │ ├── router.ts # REST API router with JWT auth
│ │ └── handlers/ # API endpoint handlers
│ ├── models/ # MongoDB models (smartdata)
│ │ ├── user.ts
│ │ ├── organization.ts
│ │ ├── repository.ts
│ │ ├── package.ts
│ │ ├── session.ts
│ │ └── ...
│ ├── services/ # Business logic
│ │ ├── auth.service.ts
│ │ ├── permission.service.ts
│ │ ├── token.service.ts
│ │ └── audit.service.ts
│ ├── providers/ # Registry protocol integrations
│ │ ├── auth.provider.ts
│ │ └── storage.provider.ts
│ └── interfaces/ # TypeScript types
└── ui/ # Angular 19 web interface
│ │ ├── router.ts # REST API router with JWT/token auth
│ │ └── handlers/ # auth, user, org, repo, package, token, audit, oauth, admin
│ ├── models/ # MongoDB models via @push.rocks/smartdata
│ │ ├── user.ts, organization.ts, team.ts
│ │ ├── repository.ts, package.ts
│ │ ├── apitoken.ts, session.ts, auditlog.ts
│ │ ├── auth.provider.ts, external.identity.ts, platform.settings.ts
│ │ └── *.member.ts, *.permission.ts
│ ├── services/ # Business logic
│ │ ├── auth.service.ts # JWT login/refresh/logout
│ │ ├── external.auth.service.ts # OAuth/OIDC & LDAP flows
│ │ ├── crypto.service.ts # AES-256-GCM encryption
│ │ ├── token.service.ts # API token CRUD
│ │ ├── permission.service.ts # RBAC resolution
│ │ └── audit.service.ts # Audit logging
│ ├── providers/ # smartregistry integration
│ │ ├── auth.provider.ts # IAuthProvider implementation
│ │ └── storage.provider.ts # IStorageHooks for quota/audit
│ └── interfaces/ # TypeScript interfaces & types
└── ui/ # Angular 19 + Tailwind CSS frontend
└── src/app/
├── features/ # Login, dashboard, orgs, repos, packages, tokens, admin
├── core/ # Services, guards, interceptors
└── shared/ # Layout, UI components
```
## 📡 API Endpoints
### Authentication
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/auth/login` | Login with email/password |
| `POST` | `/api/v1/auth/refresh` | Refresh access token |
| `POST` | `/api/v1/auth/logout` | Logout (invalidate session) |
| `GET` | `/api/v1/auth/me` | Get current user |
### Organizations
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations` | List organizations |
| `POST` | `/api/v1/organizations` | Create organization |
| `GET` | `/api/v1/organizations/:id` | Get organization details |
| `PUT` | `/api/v1/organizations/:id` | Update organization |
| `DELETE` | `/api/v1/organizations/:id` | Delete organization |
### Repositories
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations/:orgId/repositories` | List repositories |
| `POST` | `/api/v1/organizations/:orgId/repositories` | Create repository |
| `GET` | `/api/v1/repositories/:id` | Get repository details |
### Packages
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/packages` | List packages |
| `GET` | `/api/v1/packages/:id` | Get package details |
| `GET` | `/api/v1/packages/:id/versions` | List package versions |
### API Tokens
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/tokens` | List user's tokens |
| `POST` | `/api/v1/tokens` | Create new token |
| `DELETE` | `/api/v1/tokens/:id` | Revoke token |
## 🔌 Protocol Endpoints
The registry handles protocol-specific endpoints automatically via `@push.rocks/smartregistry`:
- **NPM**: `/-/*`, `/@scope/*`
- **OCI/Docker**: `/v2/*`
- **Maven**: `/maven2/*`
- **PyPI**: `/simple/*`, `/pypi/*`
- **Cargo**: `/api/v1/crates/*`
- **Composer**: `/packages.json`, `/p/*`
- **RubyGems**: `/api/v1/gems/*`, `/gems/*`
## 🔧 Technology Stack
| Component | Technology |
|-----------|------------|
| Runtime | Deno |
| Language | TypeScript |
| Database | MongoDB via `@push.rocks/smartdata` |
| Storage | S3 via `@push.rocks/smartbucket` |
| Registry | `@push.rocks/smartregistry` |
| Frontend | Angular 19 |
| Auth | JWT with session management |
| **Runtime** | Deno 2.x |
| **Language** | TypeScript (strict mode) |
| **Database** | MongoDB via [`@push.rocks/smartdata`](https://code.foss.global/push.rocks/smartdata) |
| **Storage** | S3 via [`@push.rocks/smartbucket`](https://code.foss.global/push.rocks/smartbucket) |
| **Registry Core** | [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry) |
| **Frontend** | Angular 19 (Signals, Zoneless) + Tailwind CSS |
| **Auth** | JWT (HS256) + OAuth/OIDC + LDAP |
| **Build** | [`@git.zone/tsdeno`](https://code.foss.global/git.zone/tsdeno) cross-compilation |
| **CI/CD** | Gitea Actions → binary releases |
## 🛡️ Security Features
## 🛠️ Development
- **JWT Authentication** - Short-lived access tokens with refresh flow
- **Session Management** - Track and invalidate active sessions
- **Scoped API Tokens** - Fine-grained permissions per token
- **RBAC** - Organization-level role-based access control
- **Audit Logging** - Comprehensive action logging
- **Password Hashing** - PBKDF2-style hashing with salts
## 📜 CLI Commands
### Commands
```bash
# Start the server
deno run --allow-all mod.ts server [--ephemeral]
# Start dev server with hot reload (reads .nogit/env.json)
deno task dev
# Show help
deno run --allow-all mod.ts help
# Watch mode: backend + UI + bundler concurrently
pnpm run watch
# Build Angular UI
deno task build
# Bundle UI into embedded TypeScript
deno task bundle-ui
# Cross-compile binaries for all platforms
deno task compile
# Type check / format / lint
deno task check
deno task fmt
deno task lint
# Run tests
deno task test # All tests
deno task test:unit # Unit tests only
deno task test:integration # Integration tests (requires running server)
deno task test:e2e # E2E tests (requires running server + services)
```
Options:
- `--ephemeral` / `-e` - Load config from `.nogit/env.json` instead of environment variables
### Build & Release
Releases are automated via Gitea Actions (`.gitea/workflows/release.yml`):
1. Push a `v*` tag
2. CI builds the Angular UI and bundles it into TypeScript
3. `tsdeno compile` produces binaries for 4 platforms (linux-x64, linux-arm64, macos-x64, macos-arm64)
4. Binaries + SHA256 checksums are uploaded as Gitea release assets
Compile targets are configured in `npmextra.json` under `@git.zone/tsdeno`.
### Storage Layout
Artifacts are stored in S3 at:
```
{storagePath}/{protocol}/{orgName}/{packageName}/{version}/{filename}
```
For example: `packages/npm/myorg/mypackage/1.0.0/mypackage-1.0.0.tgz`
## License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
Registered at District Court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
For any legal inquiries or further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.