stack.gallery/registry
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity
Files
15ca1a67f498fddf29461d20d9a8d97e286d70cd
registry/readme.md

365 lines
16 KiB
Markdown
Raw Blame History

# @stack.gallery/registry 📦
A self-hosted, multi-protocol package registry built with Deno and TypeScript. Run your own private **NPM**, **Docker/OCI**, **Maven**, **Cargo**, **PyPI**, **Composer**, and **RubyGems** registry — all behind a single binary with a modern web UI.
## Issue Reporting and Security
For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.
## ✨ Features
- 🔌 **7 Protocol Support** — NPM, OCI/Docker, Maven, Cargo, PyPI, Composer, RubyGems via [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry)
- 🏢 **Organizations & Teams** — Hierarchical access control: orgs → teams → repositories
- 🔐 **Flexible Authentication** — Local JWT auth, OAuth/OIDC, and LDAP with JIT user provisioning
- 🎫 **Scoped API Tokens** — Per-protocol, per-scope tokens (`srg_` prefix) for CI/CD pipelines
- 🛡️ **RBAC Permissions** — Reader → Developer → Maintainer → Admin per repository
- 🔍 **Upstream Caching** — Transparently proxy and cache packages from public registries
- 📊 **Audit Logging** — Full audit trail on every action for compliance
- 🎨 **Modern Web UI** — Angular 19 dashboard with Tailwind CSS, embedded in the binary
- ⚡ **Single Binary** — Cross-compiled with `deno compile` for Linux and macOS (x64 + ARM64)
- 🗄️ **MongoDB + S3** — Metadata in MongoDB, artifacts in any S3-compatible store
## 🚀 Quick Start
### Prerequisites
- **MongoDB** >= 4.4
- **S3-compatible storage** (MinIO, AWS S3, Cloudflare R2, etc.)
### Install from Binary
```bash
# One-liner install (latest version)
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash
# Install specific version
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash -s -- --version v1.3.0
# Install + set up systemd service
curl -sSL https://code.foss.global/stack.gallery/registry/raw/branch/main/install.sh | sudo bash -s -- --setup-service
```
The installer:
- Detects your platform (Linux/macOS, x64/ARM64)
- Downloads the pre-compiled binary from Gitea releases
- Installs to `/opt/stack-gallery-registry/` with a symlink in `/usr/local/bin/`
- Optionally creates and enables a systemd service
### Run from Source
```bash
# Clone
git clone https://code.foss.global/stack.gallery/registry.git
cd registry
# Development mode (hot reload, reads .nogit/env.json)
deno task dev
# Production mode
deno task start
```
The registry is available at `http://localhost:3000`.
## ⚙️ Configuration
Configuration is loaded from **environment variables** (production) or from **`.nogit/env.json`** when using the `--ephemeral` flag (development).
| Variable | Default | Description |
|----------|---------|-------------|
| `MONGODB_URL` | `mongodb://localhost:27017` | MongoDB connection string |
| `MONGODB_DB` | `stackgallery` | Database name |
| `S3_ENDPOINT` | `http://localhost:9000` | S3-compatible endpoint |
| `S3_ACCESS_KEY` | `minioadmin` | S3 access key |
| `S3_SECRET_KEY` | `minioadmin` | S3 secret key |
| `S3_BUCKET` | `registry` | S3 bucket name |
| `S3_REGION` | — | S3 region |
| `HOST` | `0.0.0.0` | Server bind address |
| `PORT` | `3000` | Server port |
| `JWT_SECRET` | `change-me-in-production` | JWT signing secret |
| `AUTH_ENCRYPTION_KEY` | *(ephemeral)* | 64-char hex for AES-256-GCM encryption of OAuth/LDAP secrets |
| `STORAGE_PATH` | `packages` | Base path in S3 for artifacts |
| `ENABLE_UPSTREAM_CACHE` | `true` | Cache packages from upstream registries |
| `UPSTREAM_CACHE_EXPIRY` | `24` | Cache TTL in hours |
**Example `.nogit/env.json`:**
```json
{
"MONGODB_URL": "mongodb://admin:pass@localhost:27017/stackregistry?authSource=admin",
"MONGODB_NAME": "stackregistry",
"S3_HOST": "localhost",
"S3_PORT": "9000",
"S3_ACCESSKEY": "minioadmin",
"S3_SECRETKEY": "minioadmin",
"S3_BUCKET": "registry",
"S3_USESSL": false
}
```
## 🔌 Protocol Endpoints
Each protocol is handled natively via [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry). Point your package manager at the registry:
| Protocol | Paths | Client Config Example |
|----------|-------|-----------------------|
| **NPM** | `/-/*`, `/@scope/*` | `npm config set registry http://registry:3000` |
| **OCI/Docker** | `/v2/*` | `docker login registry:3000` |
| **Maven** | `/maven2/*` | Add repository URL in `pom.xml` |
| **Cargo** | `/api/v1/crates/*` | Configure in `.cargo/config.toml` |
| **PyPI** | `/simple/*`, `/pypi/*` | `pip install --index-url http://registry:3000/simple/` |
| **Composer** | `/packages.json`, `/p/*` | Add repository in `composer.json` |
| **RubyGems** | `/api/v1/gems/*`, `/gems/*` | `gem sources -a http://registry:3000` |
Authentication works with **Bearer tokens** (API tokens prefixed `srg_`) and **Basic auth** (email:password or username:token).
## 🔐 Authentication & Security
### Local Auth
- JWT-based with **15-minute access tokens** and **7-day refresh tokens** (HS256)
- Session tracking — each login creates a session, tokens embed session IDs
- Password hashing with PBKDF2 (10,000 rounds SHA-256 + random salt)
### External Auth (OAuth/OIDC & LDAP)
- **OAuth/OIDC** — Connect to any OIDC-compliant provider (Keycloak, Okta, Auth0, Azure AD, etc.)
- **LDAP** — Bind + search authentication against Active Directory or OpenLDAP
- **JIT Provisioning** — Users are auto-created on first external login
- **Auto-linking** — External identities are linked to existing users by email match
- **Encrypted secrets** — Provider client secrets and bind passwords are stored AES-256-GCM encrypted
### RBAC Permissions
Access is resolved through a hierarchy:
```
Platform Admin (full access)
└─ Organization Owner/Admin
└─ Team Maintainer (read + write + delete on team repos)
└─ Team Member (read + write on team repos)
└─ Direct Repo Permission (reader / developer / maintainer / admin)
└─ Public Repository (read for everyone)
```
### Scoped API Tokens
Tokens are prefixed with `srg_` and can be scoped to:
- Specific **protocols** (e.g., npm + oci only)
- Specific **actions** (read / write / delete)
- Specific **organizations**
- Custom **expiration** dates
## 📡 REST API
All management endpoints live under `/api/v1/`. Authenticated via `Authorization: Bearer <jwt_or_api_token>`.
### Auth
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/auth/login` | Login (email + password) |
| `POST` | `/api/v1/auth/refresh` | Refresh access token |
| `POST` | `/api/v1/auth/logout` | Logout (invalidate session) |
| `GET` | `/api/v1/auth/me` | Current user info |
| `GET` | `/api/v1/auth/providers` | List active external auth providers |
| `GET` | `/api/v1/auth/oauth/:id/authorize` | Initiate OAuth flow |
| `GET` | `/api/v1/auth/oauth/:id/callback` | OAuth callback |
| `POST` | `/api/v1/auth/ldap/:id/login` | LDAP login |
### Users
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/users` | List users |
| `POST` | `/api/v1/users` | Create user |
| `GET` | `/api/v1/users/:id` | Get user |
| `PUT` | `/api/v1/users/:id` | Update user |
| `DELETE` | `/api/v1/users/:id` | Delete user |
### Organizations
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations` | List organizations |
| `POST` | `/api/v1/organizations` | Create organization |
| `GET` | `/api/v1/organizations/:id` | Get organization |
| `PUT` | `/api/v1/organizations/:id` | Update organization |
| `DELETE` | `/api/v1/organizations/:id` | Delete organization |
| `GET` | `/api/v1/organizations/:id/members` | List members |
| `POST` | `/api/v1/organizations/:id/members` | Add member |
| `PUT` | `/api/v1/organizations/:id/members/:userId` | Update member role |
| `DELETE` | `/api/v1/organizations/:id/members/:userId` | Remove member |
### Repositories
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/organizations/:orgId/repositories` | List org repos |
| `POST` | `/api/v1/organizations/:orgId/repositories` | Create repo |
| `GET` | `/api/v1/repositories/:id` | Get repo |
| `PUT` | `/api/v1/repositories/:id` | Update repo |
| `DELETE` | `/api/v1/repositories/:id` | Delete repo |
### Packages
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/packages` | Search packages |
| `GET` | `/api/v1/packages/:id` | Get package details |
| `GET` | `/api/v1/packages/:id/versions` | List versions |
| `DELETE` | `/api/v1/packages/:id` | Delete package |
| `DELETE` | `/api/v1/packages/:id/versions/:version` | Delete version |
### Tokens
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/tokens` | List your tokens |
| `POST` | `/api/v1/tokens` | Create token |
| `DELETE` | `/api/v1/tokens/:id` | Revoke token |
### Audit
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/audit` | Query audit logs |
### Admin (Platform Admins Only)
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/auth/providers` | List all auth providers |
| `POST` | `/api/v1/admin/auth/providers` | Create auth provider |
| `GET` | `/api/v1/admin/auth/providers/:id` | Get provider details |
| `PUT` | `/api/v1/admin/auth/providers/:id` | Update provider |
| `DELETE` | `/api/v1/admin/auth/providers/:id` | Disable provider |
| `POST` | `/api/v1/admin/auth/providers/:id/test` | Test provider connection |
| `GET` | `/api/v1/admin/auth/settings` | Get platform settings |
| `PUT` | `/api/v1/admin/auth/settings` | Update platform settings |
### Health Check
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/health` or `/healthz` | Returns JSON status of MongoDB, S3, and registry |
## 🏗️ Architecture
```
registry/
├── mod.ts # Deno entry point
├── deno.json # Deno config, tasks, imports
├── npmextra.json # tsdeno compile targets & gitzone config
├── install.sh # Binary installer script
├── .gitea/workflows/ # CI release pipeline
├── scripts/
│ └── bundle-ui.ts # Embeds Angular build as base64 TypeScript
├── ts/
│ ├── registry.ts # StackGalleryRegistry — main orchestrator
│ ├── cli.ts # CLI commands (smartcli)
│ ├── plugins.ts # Centralized dependency imports
│ ├── api/
│ │ ├── router.ts # REST API router with JWT/token auth
│ │ └── handlers/ # auth, user, org, repo, package, token, audit, oauth, admin
│ ├── models/ # MongoDB models via @push.rocks/smartdata
│ │ ├── user.ts, organization.ts, team.ts
│ │ ├── repository.ts, package.ts
│ │ ├── apitoken.ts, session.ts, auditlog.ts
│ │ ├── auth.provider.ts, external.identity.ts, platform.settings.ts
│ │ └── *.member.ts, *.permission.ts
│ ├── services/ # Business logic
│ │ ├── auth.service.ts # JWT login/refresh/logout
│ │ ├── external.auth.service.ts # OAuth/OIDC & LDAP flows
│ │ ├── crypto.service.ts # AES-256-GCM encryption
│ │ ├── token.service.ts # API token CRUD
│ │ ├── permission.service.ts # RBAC resolution
│ │ └── audit.service.ts # Audit logging
│ ├── providers/ # smartregistry integration
│ │ ├── auth.provider.ts # IAuthProvider implementation
│ │ └── storage.provider.ts # IStorageHooks for quota/audit
│ └── interfaces/ # TypeScript interfaces & types
└── ui/ # Angular 19 + Tailwind CSS frontend
└── src/app/
├── features/ # Login, dashboard, orgs, repos, packages, tokens, admin
├── core/ # Services, guards, interceptors
└── shared/ # Layout, UI components
```
## 🔧 Technology Stack
| Component | Technology |
|-----------|------------|
| **Runtime** | Deno 2.x |
| **Language** | TypeScript (strict mode) |
| **Database** | MongoDB via [`@push.rocks/smartdata`](https://code.foss.global/push.rocks/smartdata) |
| **Storage** | S3 via [`@push.rocks/smartbucket`](https://code.foss.global/push.rocks/smartbucket) |
| **Registry Core** | [`@push.rocks/smartregistry`](https://code.foss.global/push.rocks/smartregistry) |
| **Frontend** | Angular 19 (Signals, Zoneless) + Tailwind CSS |
| **Auth** | JWT (HS256) + OAuth/OIDC + LDAP |
| **Build** | [`@git.zone/tsdeno`](https://code.foss.global/git.zone/tsdeno) cross-compilation |
| **CI/CD** | Gitea Actions → binary releases |
## 🛠️ Development
### Commands
```bash
# Start dev server with hot reload (reads .nogit/env.json)
deno task dev
# Watch mode: backend + UI + bundler concurrently
pnpm run watch
# Build Angular UI
deno task build
# Bundle UI into embedded TypeScript
deno task bundle-ui
# Cross-compile binaries for all platforms
deno task compile
# Type check / format / lint
deno task check
deno task fmt
deno task lint
# Run tests
deno task test # All tests
deno task test:unit # Unit tests only
deno task test:integration # Integration tests (requires running server)
deno task test:e2e # E2E tests (requires running server + services)
```
### Build & Release
Releases are automated via Gitea Actions (`.gitea/workflows/release.yml`):
1. Push a `v*` tag
2. CI builds the Angular UI and bundles it into TypeScript
3. `tsdeno compile` produces binaries for 4 platforms (linux-x64, linux-arm64, macos-x64, macos-arm64)
4. Binaries + SHA256 checksums are uploaded as Gitea release assets
Compile targets are configured in `npmextra.json` under `@git.zone/tsdeno`.
### Storage Layout
Artifacts are stored in S3 at:
```
{storagePath}/{protocol}/{orgName}/{packageName}/{version}/{filename}
```
For example: `packages/npm/myorg/mypackage/1.0.0/mypackage-1.0.0.tgz`
## License and Legal Information
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
### Company Information
Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany
For any legal inquiries or further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
Reference in New Issue View Git Blame Copy Permalink