feat(core): Add PyPI and RubyGems registries, integrate into SmartRegistry, extend storage and auth

This commit is contained in:
2025-11-21 17:13:06 +00:00
parent ac51a94c8b
commit 0d73230d5a
17 changed files with 3514 additions and 33 deletions

View File

@@ -1,6 +1,8 @@
# Project Readme Hints
# Project Implementation Notes
## Python (PyPI) Protocol Implementation Notes
This file contains technical implementation details for PyPI and RubyGems protocols.
## Python (PyPI) Protocol Implementation ✅
### PEP 503: Simple Repository API (HTML-based)
@@ -114,7 +116,7 @@ Format: `#<hashname>=<hashvalue>`
---
## Ruby (RubyGems) Protocol Implementation Notes
## Ruby (RubyGems) Protocol Implementation
### Compact Index Format
@@ -222,7 +224,16 @@ gemname3
---
## Implementation Strategy
## Implementation Details
### Completed Protocols
- ✅ OCI Distribution Spec v1.1
- ✅ NPM Registry API
- ✅ Maven Repository
- ✅ Cargo/crates.io Registry
- ✅ Composer/Packagist
- ✅ PyPI (Python Package Index) - PEP 503/691
- ✅ RubyGems - Compact Index
### Storage Paths
@@ -333,3 +344,96 @@ rubygems:gem:{name}:{read|write|yank}
6. **HTML escaping** - Prevent XSS in generated HTML
7. **Metadata sanitization** - Clean user-provided strings
8. **Rate limiting** - Consider upload frequency limits
---
## Implementation Status (Completed)
### PyPI Implementation ✅
- **Files Created:**
- `ts/pypi/interfaces.pypi.ts` - Type definitions (354 lines)
- `ts/pypi/helpers.pypi.ts` - Helper functions (280 lines)
- `ts/pypi/classes.pypiregistry.ts` - Main registry (650 lines)
- `ts/pypi/index.ts` - Module exports
- **Features Implemented:**
- ✅ PEP 503 Simple API (HTML)
- ✅ PEP 691 JSON API
- ✅ Content negotiation (Accept header)
- ✅ Package name normalization
- ✅ File upload with multipart/form-data
- ✅ Hash verification (SHA256, MD5, Blake2b)
- ✅ Package metadata management
- ✅ JSON API endpoints (/pypi/{package}/json)
- ✅ Token-based authentication
- ✅ Scope-based permissions (read/write/delete)
- **Security Enhancements:**
- ✅ Hash verification on upload (validates client-provided hashes)
- ✅ Package name validation (regex check)
- ✅ HTML escaping in generated pages
- ✅ Permission checks on all mutating operations
### RubyGems Implementation ✅
- **Files Created:**
- `ts/rubygems/interfaces.rubygems.ts` - Type definitions (215 lines)
- `ts/rubygems/helpers.rubygems.ts` - Helper functions (350 lines)
- `ts/rubygems/classes.rubygemsregistry.ts` - Main registry (580 lines)
- `ts/rubygems/index.ts` - Module exports
- **Features Implemented:**
- ✅ Compact Index format (modern Bundler)
- ✅ /versions endpoint (all gems list)
- ✅ /info/{gem} endpoint (gem-specific metadata)
- ✅ /names endpoint (gem names list)
- ✅ Gem upload API
- ✅ Yank/unyank functionality
- ✅ Platform-specific gems support
- ✅ JSON API endpoints
- ✅ Legacy endpoints (specs.4.8.gz, Marshal.4.8)
- ✅ Token-based authentication
- ✅ Scope-based permissions
### Integration ✅
- **Core Updates:**
- ✅ Updated `IRegistryConfig` interface
- ✅ Updated `TRegistryProtocol` type
- ✅ Added authentication methods to `AuthManager`
- ✅ Added 30+ storage methods to `RegistryStorage`
- ✅ Updated `SmartRegistry` initialization and routing
- ✅ Module exports from `ts/index.ts`
- **Test Coverage:**
-`test/test.pypi.ts` - 25+ tests covering all PyPI endpoints
-`test/test.rubygems.ts` - 30+ tests covering all RubyGems endpoints
-`test/test.integration.pypi-rubygems.ts` - Integration tests
- ✅ Updated test helpers with PyPI and RubyGems support
### Known Limitations
1. **PyPI:**
- Does not implement legacy XML-RPC API
- No support for PGP signatures (data-gpg-sig always false)
- Metadata extraction from wheel files not implemented
2. **RubyGems:**
- Gem spec extraction from .gem files returns placeholder (Ruby Marshal parsing not implemented)
- Legacy Marshal endpoints return basic data only
- No support for gem dependencies resolution
### Configuration Example
```typescript
{
pypi: {
enabled: true,
basePath: '/pypi', // Also handles /simple
},
rubygems: {
enabled: true,
basePath: '/rubygems',
},
auth: {
pypiTokens: { enabled: true },
rubygemsTokens: { enabled: true },
}
}
```