smarts3/readme.md

# @push.rocks/smarts3 🚀

**Mock S3 made simple** - A powerful Node.js TypeScript package for creating a local S3 endpoint that simulates AWS S3 operations using mapped local directories. Perfect for development and testing! 

## 🌟 Features

- 🏃 **Lightning-fast local S3 simulation** - No more waiting for cloud operations during development
- 🔄 **Full AWS S3 API compatibility** - Drop-in replacement for S3 in your tests
- 📂 **Local directory mapping** - Your buckets live right on your filesystem  
- 🧪 **Perfect for testing** - Reliable, repeatable tests without cloud dependencies
- 🎯 **TypeScript-first** - Built with TypeScript for excellent type safety and IDE support
- 🔧 **Zero configuration** - Works out of the box with sensible defaults
- 🧹 **Clean slate mode** - Start fresh on every test run

## 📦 Installation

Install using your favorite package manager:

```bash
# Using npm
npm install @push.rocks/smarts3 --save-dev

# Using pnpm (recommended)
pnpm add @push.rocks/smarts3 -D

# Using yarn
yarn add @push.rocks/smarts3 --dev
```

## 🚀 Quick Start

Get up and running in seconds:

```typescript
import { Smarts3 } from '@push.rocks/smarts3';

// Start your local S3 server
const s3Server = await Smarts3.createAndStart({
  port: 3000,
  cleanSlate: true // Start with empty buckets
});

// Create a bucket
const bucket = await s3Server.createBucket('my-awesome-bucket');

// Get S3 connection details for use with AWS SDK or other S3 clients
const s3Config = await s3Server.getS3Descriptor();

// When you're done
await s3Server.stop();
```

## 📖 Detailed Usage Guide

### 🏗️ Setting Up Your S3 Server

The `Smarts3` class provides a simple interface for managing your local S3 server:

```typescript
import { Smarts3 } from '@push.rocks/smarts3';

// Configuration options
const config = {
  port: 3000,        // Port to run the server on (default: 3000)
  cleanSlate: true   // Clear all data on start (default: false)
};

// Create and start in one go
const s3Server = await Smarts3.createAndStart(config);

// Or create and start separately
const s3Server = new Smarts3(config);
await s3Server.start();
```

### 🪣 Working with Buckets

Creating and managing buckets is straightforward:

```typescript
// Create a new bucket
const bucket = await s3Server.createBucket('my-bucket');

// The bucket is now ready to use!
console.log(`Created bucket: ${bucket.name}`);
```

### 📤 Uploading Files

Use the powerful `SmartBucket` integration for file operations:

```typescript
import { SmartBucket } from '@push.rocks/smartbucket';

// Get connection configuration
const s3Config = await s3Server.getS3Descriptor();

// Create a SmartBucket instance
const smartbucket = new SmartBucket(s3Config);

// Get your bucket
const bucket = await smartbucket.getBucket('my-bucket');

// Upload a file
const baseDir = await bucket.getBaseDirectory();
await baseDir.fastStore('path/to/file.txt', 'Hello, S3! 🎉');

// Upload with more control
await baseDir.fastPut({
  path: 'documents/important.pdf',
  contents: Buffer.from(yourPdfData)
});
```

### 📥 Downloading Files

Retrieve your files easily:

```typescript
// Get file contents as string
const content = await baseDir.fastGet('path/to/file.txt');
console.log(content); // "Hello, S3! 🎉"

// Get file as Buffer
const buffer = await baseDir.fastGetBuffer('documents/important.pdf');
```

### 📋 Listing Files

Browse your bucket contents:

```typescript
// List all files in the bucket
const files = await baseDir.listFiles();

files.forEach(file => {
  console.log(`📄 ${file.name} (${file.size} bytes)`);
});

// List files with a specific prefix
const docs = await baseDir.listFiles('documents/');
```

### 🗑️ Deleting Files

Clean up when needed:

```typescript
// Delete a single file
await baseDir.fastDelete('old-file.txt');

// Delete multiple files
const filesToDelete = ['temp1.txt', 'temp2.txt', 'temp3.txt'];
for (const file of filesToDelete) {
  await baseDir.fastDelete(file);
}
```

## 🧪 Testing Integration

### Using with Jest

```typescript
import { Smarts3 } from '@push.rocks/smarts3';

describe('S3 Operations', () => {
  let s3Server: Smarts3;

  beforeAll(async () => {
    s3Server = await Smarts3.createAndStart({
      port: 9999,
      cleanSlate: true
    });
  });

  afterAll(async () => {
    await s3Server.stop();
  });

  test('should upload and retrieve a file', async () => {
    const bucket = await s3Server.createBucket('test-bucket');
    // Your test logic here
  });
});
```

### Using with Mocha

```typescript
import { Smarts3 } from '@push.rocks/smarts3';
import { expect } from 'chai';

describe('S3 Operations', () => {
  let s3Server: Smarts3;

  before(async () => {
    s3Server = await Smarts3.createAndStart({
      port: 9999,
      cleanSlate: true
    });
  });

  after(async () => {
    await s3Server.stop();
  });

  it('should upload and retrieve a file', async () => {
    const bucket = await s3Server.createBucket('test-bucket');
    // Your test logic here
  });
});
```

## 🔌 AWS SDK Integration

Use `smarts3` with the official AWS SDK:

```typescript
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { Smarts3 } from '@push.rocks/smarts3';

// Start local S3
const s3Server = await Smarts3.createAndStart({ port: 3000 });
const config = await s3Server.getS3Descriptor();

// Configure AWS SDK
const s3Client = new S3Client({
  endpoint: `http://${config.endpoint}:${config.port}`,
  region: 'us-east-1',
  credentials: {
    accessKeyId: config.accessKey,
    secretAccessKey: config.accessSecret
  },
  forcePathStyle: true
});

// Use AWS SDK as normal
const command = new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'test-file.txt',
  Body: 'Hello from AWS SDK!'
});

await s3Client.send(command);
```

## 🎯 Real-World Examples

### CI/CD Pipeline Testing

```typescript
// ci-test.ts
import { Smarts3 } from '@push.rocks/smarts3';

export async function setupTestEnvironment() {
  // Start S3 server for CI tests
  const s3 = await Smarts3.createAndStart({
    port: process.env.S3_PORT || 3000,
    cleanSlate: true
  });

  // Create test buckets
  await s3.createBucket('uploads');
  await s3.createBucket('processed');
  await s3.createBucket('archive');

  return s3;
}
```

### Microservice Development

```typescript
// dev-server.ts
import { Smarts3 } from '@push.rocks/smarts3';
import express from 'express';

async function startDevelopmentServer() {
  // Start local S3
  const s3 = await Smarts3.createAndStart({ port: 3000 });
  await s3.createBucket('user-uploads');

  // Start your API server
  const app = express();
  
  app.post('/upload', async (req, res) => {
    // Your upload logic using local S3
  });

  app.listen(8080, () => {
    console.log('🚀 Dev server running with local S3!');
  });
}
```

### Data Migration Testing

```typescript
import { Smarts3 } from '@push.rocks/smarts3';

async function testDataMigration() {
  const s3 = await Smarts3.createAndStart({ cleanSlate: true });
  
  // Create source and destination buckets
  const sourceBucket = await s3.createBucket('legacy-data');
  const destBucket = await s3.createBucket('new-data');

  // Populate source with test data
  const config = await s3.getS3Descriptor();
  const smartbucket = new SmartBucket(config);
  const source = await smartbucket.getBucket('legacy-data');
  const sourceDir = await source.getBaseDirectory();
  
  // Add test files
  await sourceDir.fastStore('user-1.json', JSON.stringify({ id: 1, name: 'Alice' }));
  await sourceDir.fastStore('user-2.json', JSON.stringify({ id: 2, name: 'Bob' }));

  // Run your migration logic
  await runMigration(config);

  // Verify migration results
  const dest = await smartbucket.getBucket('new-data');
  const destDir = await dest.getBaseDirectory();
  const migratedFiles = await destDir.listFiles();
  
  console.log(`✅ Migrated ${migratedFiles.length} files successfully!`);
}
```

## 🛠️ Advanced Configuration

### Custom S3 Descriptor Options

When integrating with different S3 clients, you can customize the connection details:

```typescript
const customDescriptor = await s3Server.getS3Descriptor({
  endpoint: 'localhost',  // Custom endpoint
  port: 3001,            // Different port
  useSsl: false,         // SSL configuration
  // Add any additional options your S3 client needs
});
```

### Environment-Based Configuration

```typescript
const config = {
  port: parseInt(process.env.S3_PORT || '3000'),
  cleanSlate: process.env.NODE_ENV === 'test'
};

const s3Server = await Smarts3.createAndStart(config);
```

## 🤝 Use Cases

- **🧪 Unit & Integration Testing** - Test S3 operations without AWS credentials or internet
- **🏗️ Local Development** - Develop cloud features offline with full S3 compatibility
- **📚 Teaching & Demos** - Perfect for workshops and tutorials without AWS setup
- **🔄 CI/CD Pipelines** - Reliable S3 operations in containerized test environments
- **🎭 Mocking & Stubbing** - Replace real S3 calls in test suites
- **📊 Data Migration Testing** - Safely test data migrations locally before production

## 🔧 API Reference

### Smarts3 Class

#### Constructor Options

```typescript
interface ISmarts3ContructorOptions {
  port?: number;        // Server port (default: 3000)
  cleanSlate?: boolean; // Clear storage on start (default: false)
}
```

#### Methods

- `static createAndStart(options)` - Create and start server in one call
- `start()` - Start the S3 server
- `stop()` - Stop the S3 server
- `createBucket(name)` - Create a new bucket
- `getS3Descriptor(options?)` - Get S3 connection configuration

## 🐛 Debugging Tips

1. **Enable verbose logging** - The server logs all operations by default
2. **Check the buckets directory** - Find your data in `.nogit/bucketsDir/`
3. **Use the correct endpoint** - Remember to use `127.0.0.1` or `localhost`
4. **Force path style** - Always use path-style URLs with local S3

## 📈 Performance

`@push.rocks/smarts3` is optimized for development and testing:

- ⚡ **Instant operations** - No network latency
- 💾 **Low memory footprint** - Efficient file system usage
- 🔄 **Fast cleanup** - Clean slate mode for quick test resets
- 🚀 **Parallel operations** - Handle multiple requests simultaneously

## 🔗 Related Packages

- [`@push.rocks/smartbucket`](https://www.npmjs.com/package/@push.rocks/smartbucket) - Powerful S3 abstraction layer
- [`@push.rocks/smartfile`](https://www.npmjs.com/package/@push.rocks/smartfile) - Advanced file system operations
- [`@tsclass/tsclass`](https://www.npmjs.com/package/@tsclass/tsclass) - TypeScript class helpers
- [`s3rver`](https://www.npmjs.com/package/s3rver) - The underlying S3 server implementation

## License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

### Company Information

Task Venture Capital GmbH  
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.