smartlog/readme.md

# @push.rocks/smartlog 🚀
*The ultimate TypeScript logging solution for modern applications*

[![npm version](https://img.shields.io/npm/v/@push.rocks/smartlog.svg)](https://www.npmjs.com/package/@push.rocks/smartlog)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

> **smartlog** is a powerful, distributed, and extensible logging system designed for the cloud-native era. Whether you're debugging locally, monitoring production systems, or building complex microservices, smartlog adapts to your needs with style. 🎯

## Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.

## 🌟 Why smartlog?

- **🎨 Beautiful Console Output**: Color-coded, formatted logs that are actually readable
- **🔌 Extensible Architecture**: Plug in any destination — databases, files, remote servers
- **🌍 Distributed by Design**: Built for microservices with correlation and context tracking
- **⚡ Zero-Config Start**: Works out of the box, scales when you need it
- **🎭 Interactive CLI Tools**: Spinners and progress bars that handle non-TTY environments gracefully
- **📊 Structured Logging**: JSON-based for easy parsing and analysis
- **🔍 Smart Filtering**: Log levels and context-based filtering

## 📦 Installation

```bash
# Using pnpm (recommended)
pnpm add @push.rocks/smartlog

# Using npm
npm install @push.rocks/smartlog
```

## 🚀 Quick Start

### Your First Logger

```typescript
import { Smartlog } from '@push.rocks/smartlog';

// Create a logger with context
const logger = new Smartlog({
  logContext: {
    company: 'MyStartup',
    companyunit: 'Backend Team',
    containerName: 'api-gateway',
    environment: 'production',
    runtime: 'node',
    zone: 'eu-central'
  }
});

// Enable console output
logger.enableConsole();

// Start logging!
await logger.log('info', '🎉 Application started successfully');
await logger.log('error', '💥 Database connection failed', {
  errorCode: 'DB_TIMEOUT',
  attemptCount: 3
});
```

### Using `createForCommitinfo`

If you're integrating with a build system that provides commit info, you can use the static factory method:

```typescript
import { Smartlog } from '@push.rocks/smartlog';

const logger = Smartlog.createForCommitinfo({
  name: 'my-app',
  version: '1.0.0',
  description: 'My application'
});

logger.enableConsole();
await logger.log('lifecycle', '🔄 App starting...');
```

## 📚 Core Concepts

### Log Levels

smartlog supports semantic log levels for different scenarios:

```typescript
// Lifecycle events
await logger.log('lifecycle', '🔄 Container starting up...');

// Success states
await logger.log('success', '✅ Payment processed');
await logger.log('ok', '👍 Health check passed');

// Information and debugging
await logger.log('info', '📋 User profile updated');
await logger.log('note', '📌 Cache invalidated');
await logger.log('debug', '🔍 Query execution plan', { sql: 'SELECT * FROM users' });

// Warnings and errors
await logger.log('warn', '⚠️  Memory usage above 80%');
await logger.log('error', '❌ Failed to send email');

// Verbose output
await logger.log('silly', '🔬 Entering function processPayment()');
```

Available levels (from most to least verbose): `silly`, `debug`, `info`, `note`, `ok`, `success`, `warn`, `error`, `lifecycle`

### Log Types

Each log entry has a type that describes what kind of data it represents:

| Type | Description |
|------|-------------|
| `log` | Standard log message |
| `increment` | Counter/metric increment |
| `gauge` | Gauge measurement |
| `error` | Error event |
| `success` | Success event |
| `value` | Value recording |
| `finance` | Financial transaction |
| `compliance` | Compliance event |

### Log Groups for Correlation

Perfect for tracking request flows through your system:

```typescript
// Track a user request through multiple operations
const requestGroup = logger.createLogGroup('req-7f3a2b');

requestGroup.log('info', 'Received POST /api/users');
requestGroup.log('debug', 'Validating request body');
requestGroup.log('info', 'Creating user in database');
requestGroup.log('success', 'User created successfully', { userId: 'usr_123' });

// All logs in the group share the same group ID and transaction ID
```

### Log Context

Every log carries contextual information about the environment it was created in:

```typescript
interface ILogContext {
  commitinfo?: ICommitInfo;  // Build/version info
  company?: string;           // Company name
  companyunit?: string;       // Team or department
  containerName?: string;     // Container/service name
  environment?: 'local' | 'test' | 'staging' | 'production';
  runtime?: 'node' | 'chrome' | 'rust' | 'deno' | 'cloudflare_workers';
  zone?: string;              // Deployment zone/region
}
```

## 🎯 Log Destinations

smartlog routes log packages to any number of destinations simultaneously. Each destination implements the `ILogDestination` interface with a single `handleLog` method.

### Built-in Destinations

#### 🖥️ Local Console (Enhanced)

Beautiful, color-coded output for local development:

```typescript
import { DestinationLocal } from '@push.rocks/smartlog/destination-local';

const localDestination = new DestinationLocal();
logger.addLogDestination(localDestination);

// Output is color-coded per log level:
// 🔵 info: blue prefix, white text
// 🔴 error: red prefix, red text
// 🟢 ok/success: green prefix, green text
// 🟠 warn: orange prefix, orange text
// 🟣 note/debug: pink prefix, pink text
// 🔵 silly: blue background, blue text
```

The `DestinationLocal` also supports **reduced logging** — a mode where repeated identical log messages are suppressed:

```typescript
const localDest = new DestinationLocal();

localDest.logReduced('Waiting for connection...'); // Logged
localDest.logReduced('Waiting for connection...'); // Suppressed
localDest.logReduced('Waiting for connection...'); // Suppressed
localDest.logReduced('Connected!');                // Logged (new message)
```

#### 📁 File Logging

Persist logs to files with timestamped entries:

```typescript
import { SmartlogDestinationFile } from '@push.rocks/smartlog/destination-file';

// Path MUST be absolute
const fileDestination = new SmartlogDestinationFile('/var/log/myapp/app.log');
logger.addLogDestination(fileDestination);

// Log entries are written as timestamped lines:
// 2024-01-15T10:30:00.000Z: Application started
// 2024-01-15T10:30:01.123Z: Processing request
```

#### 🌐 Browser DevTools

Optimized for browser environments with styled console output:

```typescript
import { SmartlogDestinationDevtools } from '@push.rocks/smartlog/destination-devtools';

const devtools = new SmartlogDestinationDevtools();
logger.addLogDestination(devtools);

// Uses CSS styling in browser console for beautiful, categorized output
// Different colors for error (red), info (pink), ok (green), success (green),
// warn (orange), and note (blue) levels
```

#### 📊 ClickHouse Analytics

Store logs in ClickHouse for powerful time-series analytics:

```typescript
import { SmartlogDestinationClickhouse } from '@push.rocks/smartlog/destination-clickhouse';

const clickhouse = await SmartlogDestinationClickhouse.createAndStart({
  url: 'https://analytics.example.com:8123',
  database: 'logs',
  username: 'logger',
  password: process.env.CLICKHOUSE_PASSWORD
});

logger.addLogDestination(clickhouse);
```

#### 🔗 Remote Receiver

Send logs to a centralized logging service with authenticated transport:

```typescript
import { SmartlogDestinationReceiver } from '@push.rocks/smartlog/destination-receiver';

const receiver = new SmartlogDestinationReceiver({
  passphrase: process.env.LOG_PASSPHRASE,
  receiverEndpoint: 'https://logs.mycompany.com/ingest'
});

logger.addLogDestination(receiver);
```

Logs are sent as authenticated JSON payloads with SHA-256 hashed passphrases.

### 🛠️ Custom Destinations

Build your own destination for any logging backend by implementing the `ILogDestination` interface:

```typescript
import type { ILogDestination, ILogPackage } from '@push.rocks/smartlog/interfaces';

class ElasticsearchDestination implements ILogDestination {
  async handleLog(logPackage: ILogPackage): Promise<void> {
    await this.client.index({
      index: `logs-${new Date().toISOString().split('T')[0]}`,
      body: {
        '@timestamp': logPackage.timestamp,
        level: logPackage.level,
        message: logPackage.message,
        context: logPackage.context,
        data: logPackage.data,
        correlation: logPackage.correlation
      }
    });
  }
}

logger.addLogDestination(new ElasticsearchDestination());
```

## 🎨 Interactive Console Features

### Spinners

Create beautiful loading animations that degrade gracefully in CI/CD:

```typescript
import { SmartlogSourceInteractive } from '@push.rocks/smartlog/source-interactive';

const spinner = new SmartlogSourceInteractive();

// Basic usage
spinner.text('🔄 Fetching data from API...');
// ... perform async operation
spinner.finishSuccess('✅ Data fetched successfully!');

// Chain success and move to next task
spinner.text('📡 Connecting to database');
// ... connect
spinner.successAndNext('🔍 Running migrations');
// ... migrate
spinner.finishSuccess('🎉 Database ready!');

// Customize appearance
spinner
  .setSpinnerStyle('dots')  // 'dots' | 'line' | 'star' | 'simple'
  .setColor('cyan')         // any terminal color
  .setSpeed(80);            // animation speed in ms

spinner.text('🚀 Deploying application...');

// Handle failures
try {
  await deployApp();
  spinner.finishSuccess('✅ Deployed!');
} catch (error) {
  spinner.finishFail('❌ Deployment failed');
}
```

### Progress Bars

Track long-running operations with style:

```typescript
import { SmartlogProgressBar } from '@push.rocks/smartlog/source-interactive';

// Create a progress bar
const progress = new SmartlogProgressBar({
  total: 100,
  width: 40,          // Bar width in characters
  complete: '█',      // Fill character
  incomplete: '░',    // Empty character
  showEta: true,      // Show estimated time remaining
  showPercent: true,  // Show percentage
  showCount: true     // Show current/total count
});

// Update progress
for (let i = 0; i <= 100; i++) {
  progress.update(i);
  await someAsyncWork();
}

progress.complete();

// Or use increment for simpler tracking
const files = await getFiles();
const fileProgress = new SmartlogProgressBar({ total: files.length });

for (const file of files) {
  await processFile(file);
  fileProgress.increment();
}

fileProgress.complete();
```

### Non-Interactive Fallback

Both spinners and progress bars automatically detect non-interactive environments (CI/CD, Docker logs, piped output) and fall back to simple text output:

```
[Loading] Connecting to database
[Success] Connected to database
[Loading] Running migrations
Progress: 25% (25/100)
Progress: 50% (50/100)
Progress: 100% (100/100)
Completed: 100% (100/100)
```

Detection checks for: TTY capability, CI environment variables (GitHub Actions, Jenkins, GitLab CI, Travis, CircleCI), and `TERM=dumb`.

### Backward Compatibility

The `SmartlogSourceOra` class extends `SmartlogSourceInteractive` and provides a compatibility layer for code that previously used the `ora` npm package:

```typescript
import { SmartlogSourceOra } from '@push.rocks/smartlog/source-interactive';

const ora = new SmartlogSourceOra();
ora.oraInstance.start();
ora.oraInstance.succeed('Done!');
```

## 🔧 Advanced Features

### Minimum Log Level

Set a minimum log level that destinations can use to filter messages:

```typescript
const logger = new Smartlog({
  logContext: { environment: 'production', runtime: 'node' },
  minimumLogLevel: 'warn'  // Destinations can check this to filter
});

// The minimumLogLevel is available as a public property
console.log(logger.minimumLogLevel); // 'warn'
```

### Increment Logging

Track metrics and counters alongside your logs:

```typescript
// Track API calls
logger.increment('info', 'api.requests', { endpoint: '/users', method: 'GET' });

// Track error types
logger.increment('error', 'payment.failed', { reason: 'insufficient_funds' });
```

Increment logs are routed to all destinations with `type: 'increment'` so analytics backends can aggregate them.

### Capture All Console Output

Redirect all `console.log` and `console.error` through smartlog:

```typescript
logger.enableConsole({
  captureAll: true
});

console.log('This goes through smartlog as info!');
console.error('This goes through smartlog as error!');

// Strings containing "Error:" are automatically classified as error level
// All captured output is prefixed with "LOG =>" to prevent recursion
```

### Log Receiver Server

Accept authenticated log packages from remote services:

```typescript
import { SmartlogReceiver } from '@push.rocks/smartlog/receiver';

const receiver = new SmartlogReceiver({
  smartlogInstance: logger,
  passphrase: 'shared-secret',
  validatorFunction: async (logPackage) => {
    // Custom validation logic
    return logPackage.context.company === 'MyCompany';
  }
});

// Handle incoming authenticated log packages (e.g., from an HTTP endpoint)
app.post('/logs', async (req, res) => {
  const result = await receiver.handleAuthenticatedLog(req.body);
  res.json(result); // { status: 'ok' } or { status: 'error' }
});

// Or handle batches
app.post('/logs/batch', async (req, res) => {
  await receiver.handleManyAuthenticatedLogs(req.body);
  res.json({ status: 'ok' });
});
```

## 🏗️ Real-World Examples

### Microservice with Distributed Tracing

```typescript
import { Smartlog } from '@push.rocks/smartlog';
import { SmartlogDestinationClickhouse } from '@push.rocks/smartlog/destination-clickhouse';
import { DestinationLocal } from '@push.rocks/smartlog/destination-local';

// Initialize logger with service context
const logger = new Smartlog({
  logContext: {
    company: 'TechCorp',
    companyunit: 'Platform',
    containerName: 'user-service',
    environment: 'production',
    runtime: 'node',
    zone: 'eu-central'
  }
});

// Add ClickHouse for analytics
const clickhouse = await SmartlogDestinationClickhouse.createAndStart({
  url: process.env.CLICKHOUSE_URL,
  database: 'microservices_logs'
});
logger.addLogDestination(clickhouse);

// Add local console for container stdout
logger.addLogDestination(new DestinationLocal());

// Express middleware for request tracking
app.use((req, res, next) => {
  const logGroup = logger.createLogGroup(req.headers['x-request-id'] || 'unknown');

  logGroup.log('info', 'Incoming request', {
    method: req.method,
    path: req.path,
    ip: req.ip
  });

  res.on('finish', () => {
    logGroup.log('info', 'Request completed', {
      statusCode: res.statusCode,
      duration: Date.now() - req.startTime
    });
  });

  next();
});
```

### CLI Tool with Progress Tracking

```typescript
import { Smartlog } from '@push.rocks/smartlog';
import { SmartlogSourceInteractive, SmartlogProgressBar } from '@push.rocks/smartlog/source-interactive';

const logger = new Smartlog({
  logContext: {
    containerName: 'migration-tool',
    environment: 'local',
    runtime: 'node'
  }
});

logger.enableConsole();

async function migrateDatabase() {
  const spinner = new SmartlogSourceInteractive();

  spinner.text('🔌 Connecting to database...');
  await connectDB();
  spinner.finishSuccess('✅ Connected to database');

  spinner.text('📋 Loading migrations...');
  const migrations = await getMigrations();
  spinner.finishSuccess(`✅ Found ${migrations.length} migrations`);

  const progress = new SmartlogProgressBar({
    total: migrations.length,
    width: 40,
    showEta: true
  });

  for (const [index, migration] of migrations.entries()) {
    await logger.log('info', `Running migration: ${migration.name}`);
    await runMigration(migration);
    progress.update(index + 1);
  }

  progress.complete();
  await logger.log('success', '🎉 All migrations completed successfully!');
}
```

### Production Logging with Multiple Destinations

```typescript
import { Smartlog } from '@push.rocks/smartlog';
import { DestinationLocal } from '@push.rocks/smartlog/destination-local';
import { SmartlogDestinationFile } from '@push.rocks/smartlog/destination-file';
import { SmartlogDestinationReceiver } from '@push.rocks/smartlog/destination-receiver';

const logger = new Smartlog({
  logContext: {
    company: 'Enterprise Corp',
    containerName: 'payment-processor',
    environment: 'production',
    runtime: 'node',
    zone: 'us-east-1'
  },
  minimumLogLevel: 'info'
});

// Color-coded console for container logs
logger.addLogDestination(new DestinationLocal());

// File for audit trail
logger.addLogDestination(new SmartlogDestinationFile('/var/log/app/audit.log'));

// Central logging service
logger.addLogDestination(new SmartlogDestinationReceiver({
  passphrase: process.env.LOG_PASSPHRASE,
  receiverEndpoint: 'https://logs.enterprise.com/ingest'
}));

// Custom inline destination for critical alerts
logger.addLogDestination({
  async handleLog(logPackage) {
    if (logPackage.level === 'error' && logPackage.data?.critical) {
      await sendSlackAlert(`🚨 Critical error: ${logPackage.message}`);
    }
  }
});
```

## 📖 API Reference

### Smartlog Class

```typescript
class Smartlog {
  // Static factory
  static createForCommitinfo(commitinfo: ICommitInfo): Smartlog;

  // Constructor
  constructor(options: {
    logContext: ILogContext;
    minimumLogLevel?: TLogLevel;  // default: 'silly'
  });

  // Logging
  log(level: TLogLevel, message: string, data?: any, correlation?: ILogCorrelation): Promise<void>;
  increment(level: TLogLevel, message: string, data?: any, correlation?: ILogCorrelation): void;

  // Configuration
  enableConsole(options?: { captureAll: boolean }): void;
  addLogDestination(destination: ILogDestination): void;

  // Correlation
  createLogGroup(transactionId?: string): LogGroup;

  // Forwarding (for receiver pattern)
  handleLog(logPackage: ILogPackage): Promise<void>;

  // Instance identity
  uniInstanceId: string;
  logContext: ILogContext;
  minimumLogLevel: TLogLevel;
}
```

### LogGroup Class

```typescript
class LogGroup {
  groupId: string;          // Auto-generated unique ID
  transactionId: string;    // The transaction ID passed at creation
  smartlogRef: Smartlog;    // Reference to the parent Smartlog

  log(level: TLogLevel, message: string, data?: any): void;
}
```

### ILogDestination Interface

```typescript
interface ILogDestination {
  handleLog(logPackage: ILogPackage): Promise<void>;
}
```

### ILogPackage Interface

```typescript
interface ILogPackage<T = unknown> {
  timestamp: number;             // Unix timestamp in milliseconds
  type: TLogType;                // 'log' | 'increment' | 'gauge' | ...
  context: ILogContext;          // Environment context
  level: TLogLevel;              // Log severity
  correlation: ILogCorrelation;  // Correlation metadata
  message: string;               // The log message
  data?: T;                      // Optional structured data
}
```

### Available Log Levels

| Level | Description | Use Case |
|-------|-------------|----------|
| `silly` | Ultra-verbose | Function entry/exit, variable dumps |
| `debug` | Debug info | Development-time diagnostics |
| `info` | Informational | Standard operational messages |
| `note` | Notable events | Important but non-critical events |
| `ok` | Success confirmation | Health checks, validations |
| `success` | Major success | Completed operations |
| `warn` | Warnings | Degraded performance, approaching limits |
| `error` | Errors | Failures requiring attention |
| `lifecycle` | Lifecycle events | Start, stop, restart, deploy |

## License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

### Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.