smartai/readme.md

# @push.rocks/smartai

[![npm version](https://badge.fury.io/js/%40push.rocks%2Fsmartai.svg)](https://www.npmjs.com/package/@push.rocks/smartai)  
[![Build Status](https://github.com/push.rocks/smartai/workflows/CI/badge.svg)](https://github.com/push.rocks/smartai/actions)  
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

SmartAi is a comprehensive TypeScript library that provides a standardized interface for integrating and interacting with multiple AI models. It supports a range of operations from synchronous and streaming chat to audio generation, document processing, and vision tasks.

## Table of Contents

- [Features](#features)
- [Installation](#installation)
- [Supported AI Providers](#supported-ai-providers)
- [Quick Start](#quick-start)
- [Usage Examples](#usage-examples)
  - [Chat Interactions](#chat-interactions)
  - [Streaming Chat](#streaming-chat)
  - [Audio Generation](#audio-generation)
  - [Document Processing](#document-processing)
  - [Vision Processing](#vision-processing)
- [Error Handling](#error-handling)
- [Development](#development)
  - [Running Tests](#running-tests)
  - [Building the Project](#building-the-project)
- [Contributing](#contributing)
- [License](#license)
- [Legal Information](#legal-information)

## Features

- **Unified API:** Seamlessly integrate multiple AI providers with a consistent interface.
- **Chat & Streaming:** Support for both synchronous and real-time streaming chat interactions.
- **Audio & Vision:** Generate audio responses and perform detailed image analysis.
- **Document Processing:** Analyze PDFs and other documents using vision models.
- **Extensible:** Easily extend the library to support additional AI providers.

## Installation

To install SmartAi, run the following command:

```bash
npm install @push.rocks/smartai
```

This will add the package to your project’s dependencies.

## Supported AI Providers

SmartAi supports multiple AI providers. Configure each provider with its corresponding token or settings:

### OpenAI

- **Models:** GPT-4, GPT-3.5-turbo, GPT-4-vision-preview
- **Features:** Chat, Streaming, Audio Generation, Vision, Document Processing
- **Configuration Example:**
  
  ```typescript
  openaiToken: 'your-openai-token'
  ```

### X.AI

- **Models:** Grok-2-latest
- **Features:** Chat, Streaming, Document Processing
- **Configuration Example:**
  
  ```typescript
  xaiToken: 'your-xai-token'
  ```

### Anthropic

- **Models:** Claude-3-opus-20240229
- **Features:** Chat, Streaming, Vision, Document Processing
- **Configuration Example:**
  
  ```typescript
  anthropicToken: 'your-anthropic-token'
  ```

### Perplexity

- **Models:** Mixtral-8x7b-instruct
- **Features:** Chat, Streaming
- **Configuration Example:**
  
  ```typescript
  perplexityToken: 'your-perplexity-token'
  ```

### Groq

- **Models:** Llama-3.3-70b-versatile
- **Features:** Chat, Streaming
- **Configuration Example:**
  
  ```typescript
  groqToken: 'your-groq-token'
  ```

### Ollama

- **Models:** Configurable (default: llama2; use llava for vision/document tasks)
- **Features:** Chat, Streaming, Vision, Document Processing
- **Configuration Example:**
  
  ```typescript
  ollama: {
    baseUrl: 'http://localhost:11434', // Optional
    model: 'llama2',                  // Optional
    visionModel: 'llava'               // Optional for vision and document tasks
  }
  ```

## Quick Start

Initialize SmartAi with the provider configurations you plan to use:

```typescript
import { SmartAi } from '@push.rocks/smartai';

const smartAi = new SmartAi({
  openaiToken: 'your-openai-token',
  xaiToken: 'your-xai-token',
  anthropicToken: 'your-anthropic-token',
  perplexityToken: 'your-perplexity-token',
  groqToken: 'your-groq-token',
  ollama: {
    baseUrl: 'http://localhost:11434',
    model: 'llama2'
  }
});

await smartAi.start();
```

## Usage Examples

### Chat Interactions

**Synchronous Chat:**

```typescript
const response = await smartAi.openaiProvider.chat({
  systemMessage: 'You are a helpful assistant.',
  userMessage: 'What is the capital of France?',
  messageHistory: [] // Include previous conversation messages if applicable
});

console.log(response.message);
```

### Streaming Chat

**Real-Time Streaming:**

```typescript
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

// Create a transform stream for sending and receiving data
const { writable, readable } = new TransformStream();
const writer = writable.getWriter();

const message = {
  role: 'user',
  content: 'Tell me a story about a brave knight'
};

writer.write(textEncoder.encode(JSON.stringify(message) + '\n'));

// Start streaming the response
const stream = await smartAi.openaiProvider.chatStream(readable);
const reader = stream.getReader();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log('AI:', value);
}
```

### Audio Generation

Generate audio (supported by providers like OpenAI):

```typescript
const audioStream = await smartAi.openaiProvider.audio({
  message: 'Hello, this is a test of text-to-speech'
});

// Process the audio stream, for example, play it or save to a file.
```

### Document Processing

Analyze and extract key information from documents:

```typescript
// Example using OpenAI
const documentResult = await smartAi.openaiProvider.document({
  systemMessage: 'Classify the document type',
  userMessage: 'What type of document is this?',
  messageHistory: [],
  pdfDocuments: [pdfBuffer] // Uint8Array containing the PDF content
});
```

Other providers (e.g., Ollama and Anthropic) follow a similar pattern:

```typescript
// Using Ollama for document processing
const ollamaResult = await smartAi.ollamaProvider.document({
  systemMessage: 'You are a document analysis assistant',
  userMessage: 'Extract key information from this document',
  messageHistory: [],
  pdfDocuments: [pdfBuffer]
});
```

```typescript
// Using Anthropic for document processing
const anthropicResult = await smartAi.anthropicProvider.document({
  systemMessage: 'Analyze the document',
  userMessage: 'Please extract the main points',
  messageHistory: [],
  pdfDocuments: [pdfBuffer]
});
```

### Vision Processing

Analyze images with vision capabilities:

```typescript
// Using OpenAI GPT-4 Vision
const imageDescription = await smartAi.openaiProvider.vision({
  image: imageBuffer, // Uint8Array containing image data
  prompt: 'What do you see in this image?'
});

// Using Ollama for vision tasks
const ollamaImageAnalysis = await smartAi.ollamaProvider.vision({
  image: imageBuffer,
  prompt: 'Analyze this image in detail'
});

// Using Anthropic for vision analysis
const anthropicImageAnalysis = await smartAi.anthropicProvider.vision({
  image: imageBuffer,
  prompt: 'Describe the contents of this image'
});
```

## Error Handling

Always wrap API calls in try-catch blocks to manage errors effectively:

```typescript
try {
  const response = await smartAi.openaiProvider.chat({
    systemMessage: 'You are a helpful assistant.',
    userMessage: 'Hello!',
    messageHistory: []
  });
  console.log(response.message);
} catch (error: any) {
  console.error('AI provider error:', error.message);
}
```

## Development

### Running Tests

To run the test suite, use the following command:

```bash
npm run test
```

Ensure your environment is configured with the appropriate tokens and settings for the providers you are testing.

### Building the Project

Compile the TypeScript code and build the package using:

```bash
npm run build
```

This command prepares the library for distribution.

## Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository.
2. Create a feature branch:  
   ```bash
   git checkout -b feature/my-feature
   ```
3. Commit your changes with clear messages:  
   ```bash
   git commit -m 'Add new feature'
   ```
4. Push your branch to your fork:  
   ```bash
   git push origin feature/my-feature
   ```
5. Open a Pull Request with a detailed description of your changes.

## License

This project is licensed under the [MIT License](LICENSE).

## Legal Information

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and its related products or services are trademarks of Task Venture Capital GmbH and are not covered by the MIT License. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines.

### Company Information

Task Venture Capital GmbH  
Registered at District Court Bremen HRB 35230 HB, Germany  
Contact: hello@task.vc

By using this repository, you agree to the terms outlined in this section.

---

Happy coding with SmartAi!