npmextra/readme.md

# @push.rocks/npmextra 🚀

**Supercharge your npm projects with powerful configuration management, tool orchestration, and persistent key-value storage.**

## Install 📦

```bash
# Using npm
npm install @push.rocks/npmextra --save

# Using pnpm (recommended)
pnpm add @push.rocks/npmextra
```

## Overview 🎯

`@push.rocks/npmextra` is your Swiss Army knife for npm project configuration. It eliminates configuration sprawl by centralizing tool settings, providing intelligent key-value storage, and offering powerful environment variable mapping with automatic type conversions.

### Why npmextra? 

- **🎛️ Centralized Configuration**: Manage all your tool configs in one `npmextra.json` file
- **💾 Persistent Storage**: Smart key-value store with multiple storage strategies
- **🔐 Environment Mapping**: Sophisticated env var handling with automatic type conversion
- **🏗️ TypeScript First**: Full type safety and IntelliSense support
- **⚡ Zero Config**: Works out of the box with sensible defaults
- **🔄 Reactive**: Built-in change detection and observables

## Core Concepts 🏗️

### 1. Npmextra Configuration Management

Stop scattering configuration across dozens of files. Centralize everything in `npmextra.json`:

```typescript
import { Npmextra } from '@push.rocks/npmextra';

// Initialize with current directory
const npmextra = new Npmextra();

// Or specify a custom path
const npmextra = new Npmextra('/path/to/project');

// Get merged configuration for any tool
const eslintConfig = npmextra.dataFor<EslintConfig>('eslint', {
  // Default values if not in npmextra.json
  extends: 'standard',
  rules: {}
});
```

**npmextra.json example:**
```json
{
  "eslint": {
    "extends": "@company/eslint-config",
    "rules": {
      "no-console": "warn"
    }
  },
  "prettier": {
    "semi": false,
    "singleQuote": true
  }
}
```

### 2. KeyValueStore - Persistent Data Storage

A flexible key-value store that persists data between script executions:

```typescript
import { KeyValueStore } from '@push.rocks/npmextra';

interface UserSettings {
  username: string;
  apiKey: string;
  preferences: {
    theme: 'light' | 'dark';
    notifications: boolean;
  };
}

// Different storage strategies
const kvStore = new KeyValueStore<UserSettings>({
  typeArg: 'userHomeDir',  // Store in user's home directory
  identityArg: 'myApp',
  mandatoryKeys: ['username', 'apiKey']
});

// Write operations
await kvStore.writeKey('username', 'john_doe');
await kvStore.writeKey('preferences', {
  theme: 'dark',
  notifications: true
});

// Read operations
const username = await kvStore.readKey('username');
const allData = await kvStore.readAll();

// Check for missing mandatory keys
const missingKeys = await kvStore.getMissingMandatoryKeys();
if (missingKeys.length > 0) {
  console.log('Missing required configuration:', missingKeys);
}

// Wait for keys to be present
await kvStore.waitForKeysPresent(['apiKey']);
```

**Storage Types:**
- `userHomeDir`: Store in user's home directory
- `custom`: Specify your own path
- `ephemeral`: In-memory only (perfect for testing)

### 3. AppData - Advanced Environment Management 🌟

The crown jewel of npmextra - sophisticated environment variable mapping with automatic type conversion:

```typescript
import { AppData } from '@push.rocks/npmextra';

interface AppConfig {
  apiUrl: string;
  apiKey: string;
  port: number;
  features: {
    analytics: boolean;
    payment: boolean;
  };
  cache: {
    ttl: number;
    redis: {
      host: string;
      password: string;
    };
  };
}

const appData = await AppData.createAndInit<AppConfig>({
  dirPath: '/app/config',  // Optional: defaults to smart path selection
  requiredKeys: ['apiKey', 'apiUrl'],
  envMapping: {
    apiUrl: 'API_URL',                    // Simple mapping
    apiKey: 'hard:development-key-123',   // Hardcoded value
    port: 'hard:3000',                     // Hardcoded number
    features: {
      analytics: 'boolean:ENABLE_ANALYTICS',  // Force boolean conversion
      payment: 'hard_boolean:true'            // Hardcoded boolean
    },
    cache: {
      ttl: 'json:CACHE_CONFIG',  // Parse JSON from env var
      redis: {
        host: 'REDIS_HOST',
        password: 'base64:REDIS_PASSWORD_B64'  // Decode base64
      }
    }
  },
  overwriteObject: {
    // Force these values regardless of env vars
    port: 8080
  }
});

const store = await appData.getKvStore();
const apiUrl = await store.readKey('apiUrl');
```

## AppData Special Cases & Conversions 🎯

### Environment Variable Prefixes

AppData supports sophisticated type conversion through prefixes:

| Prefix | Description | Example | Result |
|--------|-------------|---------|--------|
| `hard:` | Hardcoded value | `hard:myvalue` | `"myvalue"` |
| `hard_boolean:` | Hardcoded boolean | `hard_boolean:true` | `true` |
| `hard_json:` | Hardcoded JSON | `hard_json:{"key":"value"}` | `{key: "value"}` |
| `hard_base64:` | Hardcoded base64 | `hard_base64:SGVsbG8=` | `"Hello"` |
| `boolean:` | Env var as boolean | `boolean:FEATURE_FLAG` | `true/false` |
| `json:` | Parse env var as JSON | `json:CONFIG_JSON` | Parsed object |
| `base64:` | Decode env var from base64 | `base64:SECRET_B64` | Decoded string |

### Automatic Suffix Detection

Variables ending with certain suffixes get automatic conversion:

```typescript
{
  envMapping: {
    // Automatically parsed as JSON if MY_CONFIG_JSON="{"enabled":true}"
    config: 'MY_CONFIG_JSON',
    
    // Automatically decoded from base64 if SECRET_KEY_BASE64="SGVsbG8="
    secret: 'SECRET_KEY_BASE64'
  }
}
```

### Complex Examples

```typescript
const appData = await AppData.createAndInit({
  envMapping: {
    // Simple environment variable
    apiUrl: 'API_URL',
    
    // Hardcoded values with type conversion
    debugMode: 'hard_boolean:false',
    maxRetries: 'hard:5',
    defaultConfig: 'hard_json:{"timeout":30,"retries":3}',
    
    // Environment variables with conversion
    features: 'json:FEATURE_FLAGS',  // Expects: {"feature1":true,"feature2":false}
    isProduction: 'boolean:IS_PROD', // Expects: "true" or "false"
    apiSecret: 'base64:API_SECRET',  // Expects: base64 encoded string
    
    // Nested structures
    database: {
      host: 'DB_HOST',
      port: 'hard:5432',
      credentials: {
        user: 'DB_USER',
        password: 'base64:DB_PASSWORD_ENCODED',
        ssl: 'boolean:DB_USE_SSL'
      }
    }
  },
  
  // Override any env mappings
  overwriteObject: {
    debugMode: true,  // Force debug mode regardless of env
    database: {
      host: 'localhost'  // Force localhost for development
    }
  }
});
```

### Boolean Conversion Rules

AppData intelligently handles boolean conversions:

1. **String "true"/"false"**: Converted to boolean
2. **With `boolean:` prefix**: Any env var value is converted (`"true"` → `true`, anything else → `false`)
3. **With `hard_boolean:` prefix**: Hardcoded boolean value
4. **Regular env vars**: Strings remain strings unless prefixed

```typescript
// Environment: FEATURE_A="true", FEATURE_B="yes", FEATURE_C="1"
{
  envMapping: {
    featureA: 'FEATURE_A',           // Result: "true" (string)
    featureB: 'boolean:FEATURE_B',   // Result: false (only "true" → true)
    featureC: 'boolean:FEATURE_C',   // Result: false (only "true" → true)
    featureD: 'hard_boolean:true'    // Result: true (hardcoded)
  }
}
```

## Advanced Patterns 🎨

### Reactive Configuration

Subscribe to configuration changes:

```typescript
const kvStore = new KeyValueStore<Config>({
  typeArg: 'custom',
  identityArg: 'myApp'
});

// Subscribe to changes
kvStore.changeSubject.subscribe((newData) => {
  console.log('Configuration changed:', newData);
});

// Changes trigger notifications
await kvStore.writeKey('theme', 'dark');
```

### Testing with Ephemeral Storage

Perfect for unit tests - no file system pollution:

```typescript
const testStore = new KeyValueStore<TestData>({
  typeArg: 'ephemeral',
  identityArg: 'test'
});

// All operations work normally, but nothing persists to disk
await testStore.writeKey('testKey', 'testValue');
```

### Smart Path Resolution

AppData automatically selects the best storage location:

1. Checks for `/app/data` (containerized environments)
2. Falls back to `/data` (alternate container path)
3. Uses `.nogit/appdata` (local development)

```typescript
// Automatic path selection
const appData = await AppData.createAndInit({
  // No dirPath specified - smart detection
  requiredKeys: ['apiKey']
});

// Or force ephemeral for testing
const testData = await AppData.createAndInit({
  ephemeral: true,  // No disk persistence
  requiredKeys: ['testKey']
});
```

### Waiting for Configuration

Block until required configuration is available:

```typescript
const appData = await AppData.createAndInit<Config>({
  requiredKeys: ['apiKey', 'apiUrl']
});

// Wait for specific key
const apiKey = await appData.waitForAndGetKey('apiKey');

// Check missing keys
const missingKeys = await appData.logMissingKeys();
// Logs: "The following mandatory keys are missing in the appdata:
//        -> apiKey,
//        -> apiUrl"
```

## Real-World Example 🌍

Here's a complete example of a CLI tool using npmextra:

```typescript
import { Npmextra, AppData, KeyValueStore } from '@push.rocks/npmextra';

interface CliConfig {
  githubToken: string;
  openaiKey: string;
  model: 'gpt-3' | 'gpt-4';
  cache: {
    enabled: boolean;
    ttl: number;
  };
}

class MyCLI {
  private npmextra: Npmextra;
  private appData: AppData<CliConfig>;
  private cache: KeyValueStore<{[key: string]: any}>;

  async initialize() {
    // Load tool configuration
    this.npmextra = new Npmextra();
    const config = this.npmextra.dataFor<any>('mycli', {
      defaultModel: 'gpt-3'
    });

    // Setup app data with env mapping
    this.appData = await AppData.createAndInit<CliConfig>({
      requiredKeys: ['githubToken', 'openaiKey'],
      envMapping: {
        githubToken: 'GITHUB_TOKEN',
        openaiKey: 'base64:OPENAI_KEY_ENCODED',
        model: 'hard:gpt-4',
        cache: {
          enabled: 'boolean:ENABLE_CACHE',
          ttl: 'hard:3600'
        }
      }
    });

    // Initialize cache
    this.cache = new KeyValueStore({
      typeArg: 'userHomeDir',
      identityArg: 'mycli-cache'
    });

    // Check for missing configuration
    const missingKeys = await this.appData.logMissingKeys();
    if (missingKeys.length > 0) {
      console.error('Please configure the missing keys');
      process.exit(1);
    }
  }

  async run() {
    await this.initialize();
    
    const config = await this.appData.getKvStore();
    const settings = await config.readAll();
    
    console.log(`Using model: ${settings.model}`);
    console.log(`Cache enabled: ${settings.cache.enabled}`);
    
    // Use the configuration...
  }
}

// Run the CLI
const cli = new MyCLI();
cli.run();
```

## API Reference 📚

### Npmextra Class

```typescript
new Npmextra(cwdArg?: string)
```
- `cwdArg`: Optional working directory path

**Methods:**
- `dataFor<T>(toolName: string, defaultOptions: T): T` - Get merged configuration

### KeyValueStore Class

```typescript
new KeyValueStore<T>(options: {
  typeArg: 'custom' | 'userHomeDir' | 'ephemeral';
  identityArg: string;
  customPath?: string;
  mandatoryKeys?: Array<keyof T>;
})
```

**Methods:**
- `readKey(key: string): Promise<T>` - Read single value
- `writeKey(key: string, value: T): Promise<void>` - Write single value
- `readAll(): Promise<T>` - Read all values
- `writeAll(data: T): Promise<void>` - Write all values
- `deleteKey(key: string): Promise<void>` - Delete a key
- `getMissingMandatoryKeys(): Promise<string[]>` - Check missing required keys
- `waitForKeysPresent(keys: string[]): Promise<void>` - Wait for keys

### AppData Class

```typescript
await AppData.createAndInit<T>(options: {
  dirPath?: string;
  requiredKeys?: Array<keyof T>;
  ephemeral?: boolean;
  envMapping?: PartialDeep<T>;
  overwriteObject?: PartialDeep<T>;
})
```

**Methods:**
- `getKvStore(): Promise<KeyValueStore<T>>` - Get underlying store
- `logMissingKeys(): Promise<Array<keyof T>>` - Log and return missing keys
- `waitForAndGetKey<K>(key: K): Promise<T[K]>` - Wait for and retrieve key

## License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

### Company Information

Task Venture Capital GmbH  
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.