push.rocks/smartai
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(docs): update documentation: clarify provider capabilities, add provider capabilities summary, polish examples and formatting, and remove Serena project config

Browse Source
This commit is contained in:
Jürgen Kunz
2026-01-20 01:27:52 +00:00
parent ae8d3ccf33
commit 2040b3c629
6 changed files with 169 additions and 408 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.serena/.gitignore vendored
View File

@@ -1 +0,0 @@
/cache

67
.serena/project.yml
View File

@@ -1,67 +0,0 @@
# language of the project (csharp, python, rust, java, typescript, go, cpp, or ruby)
# * For C, use cpp
# * For JavaScript, use typescript
# Special requirements:
# * csharp: Requires the presence of a .sln file in the project folder.
language: typescript
# whether to use the project's gitignore file to ignore files
# Added on 2025-04-07
ignore_all_files_in_gitignore: true
# list of additional paths to ignore
# same syntax as gitignore, so you can use * and **
# Was previously called `ignored_dirs`, please update your config if you are using that.
# Added (renamed) on 2025-04-07
ignored_paths: []
# whether the project is in read-only mode
# If set to true, all editing tools will be disabled and attempts to use them will result in an error
# Added on 2025-04-18
read_only: false
# list of tool names to exclude. We recommend not excluding any tools, see the readme for more details.
# Below is the complete list of tools for convenience.
# To make sure you have the latest list of tools, and to view their descriptions,
# execute `uv run scripts/print_tool_overview.py`.
#
# * `activate_project`: Activates a project by name.
# * `check_onboarding_performed`: Checks whether project onboarding was already performed.
# * `create_text_file`: Creates/overwrites a file in the project directory.
# * `delete_lines`: Deletes a range of lines within a file.
# * `delete_memory`: Deletes a memory from Serena's project-specific memory store.
# * `execute_shell_command`: Executes a shell command.
# * `find_referencing_code_snippets`: Finds code snippets in which the symbol at the given location is referenced.
# * `find_referencing_symbols`: Finds symbols that reference the symbol at the given location (optionally filtered by type).
# * `find_symbol`: Performs a global (or local) search for symbols with/containing a given name/substring (optionally filtered by type).
# * `get_current_config`: Prints the current configuration of the agent, including the active and available projects, tools, contexts, and modes.
# * `get_symbols_overview`: Gets an overview of the top-level symbols defined in a given file.
# * `initial_instructions`: Gets the initial instructions for the current project.
# Should only be used in settings where the system prompt cannot be set,
# e.g. in clients you have no control over, like Claude Desktop.
# * `insert_after_symbol`: Inserts content after the end of the definition of a given symbol.
# * `insert_at_line`: Inserts content at a given line in a file.
# * `insert_before_symbol`: Inserts content before the beginning of the definition of a given symbol.
# * `list_dir`: Lists files and directories in the given directory (optionally with recursion).
# * `list_memories`: Lists memories in Serena's project-specific memory store.
# * `onboarding`: Performs onboarding (identifying the project structure and essential tasks, e.g. for testing or building).
# * `prepare_for_new_conversation`: Provides instructions for preparing for a new conversation (in order to continue with the necessary context).
# * `read_file`: Reads a file within the project directory.
# * `read_memory`: Reads the memory with the given name from Serena's project-specific memory store.
# * `remove_project`: Removes a project from the Serena configuration.
# * `replace_lines`: Replaces a range of lines within a file with new content.
# * `replace_symbol_body`: Replaces the full definition of a symbol.
# * `restart_language_server`: Restarts the language server, may be necessary when edits not through Serena happen.
# * `search_for_pattern`: Performs a search for a pattern in the project.
# * `summarize_changes`: Provides instructions for summarizing the changes made to the codebase.
# * `switch_modes`: Activates modes by providing a list of their names
# * `think_about_collected_information`: Thinking tool for pondering the completeness of collected information.
# * `think_about_task_adherence`: Thinking tool for determining whether the agent is still on track with the current task.
# * `think_about_whether_you_are_done`: Thinking tool for determining whether the task is truly completed.
# * `write_memory`: Writes a named memory (for future reference) to Serena's project-specific memory store.
excluded_tools: []
# initial prompt for the project. It will always be given to the LLM upon activating the project
# (contrary to the memories, which are loaded on demand).
initial_prompt: ""
project_name: "smartai"

9
changelog.md
View File

@@ -1,5 +1,14 @@
# Changelog # Changelog
## 2026-01-20 - 0.12.1 - fix(docs)
update documentation: clarify provider capabilities, add provider capabilities summary, polish examples and formatting, and remove Serena project config
- Removed .serena/project.yml and cleaned up .serena/.gitignore
- Added Provider Capabilities Summary and expanded/clarified provider tables in readme.md and readme.hints.md
- Clarified Anthropic extended thinking details and Mistral native PDF OCR notes
- Polished example code snippets and fixed minor typos/formatting (GPT-5 mention, ElevenLabs model note, consistent punctuation)
- Updated test command references and other README usage instructions
## 2026-01-20 - 0.12.0 - feat(ollama) ## 2026-01-20 - 0.12.0 - feat(ollama)
add support for base64-encoded images in chat messages and forward them to the Ollama provider add support for base64-encoded images in chat messages and forward them to the Ollama provider

216
readme.hints.md
View File

@@ -3,7 +3,7 @@
## Dependencies ## Dependencies
- Uses `@git.zone/tstest` v3.x for testing (import from `@git.zone/tstest/tapbundle`) - Uses `@git.zone/tstest` v3.x for testing (import from `@git.zone/tstest/tapbundle`)
- `@push.rocks/smartfs` v1.x for file system operations (replaced smartfile) - `@push.rocks/smartfs` v1.x for file system operations
- `@anthropic-ai/sdk` v0.71.x with extended thinking support - `@anthropic-ai/sdk` v0.71.x with extended thinking support
- `@mistralai/mistralai` v1.x for Mistral OCR and chat capabilities - `@mistralai/mistralai` v1.x for Mistral OCR and chat capabilities
- `openai` v6.x for OpenAI API integration - `openai` v6.x for OpenAI API integration
@@ -14,12 +14,26 @@
- When extended thinking is enabled, temperature parameter must NOT be set (or set to 1) - When extended thinking is enabled, temperature parameter must NOT be set (or set to 1)
- The `streamNode()` method was removed in smartrequest v5, use `response.stream()` with `Readable.fromWeb()` instead - The `streamNode()` method was removed in smartrequest v5, use `response.stream()` with `Readable.fromWeb()` instead
## Provider Capabilities Summary
| Provider | Chat | Stream | TTS | Vision | Documents | Research | Images |
|--------------|------|--------|-----|--------|-----------|----------|--------|
| OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Anthropic | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ |
| Mistral | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ |
| ElevenLabs | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
| Ollama | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ |
| XAI | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ |
| Perplexity | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ |
| Groq | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Exo | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
## Mistral Provider Integration ## Mistral Provider Integration
### Overview ### Overview
The Mistral provider supports: The Mistral provider supports:
- **Document AI** via Mistral OCR 3 (December 2025) - native PDF processing without image conversion - **Document AI** via Mistral OCR (December 2025) - native PDF processing without image conversion
- **Chat capabilities** using Mistral's chat models (`mistral-large-latest`, etc.) - **Chat capabilities** using Mistral's chat models (`mistral-large-latest`, etc.)
### Key Advantage: Native PDF Support ### Key Advantage: Native PDF Support
@@ -41,47 +55,12 @@ const provider = new smartai.MistralProvider({
await provider.start(); await provider.start();
``` ```
### Supported Methods
| Method | Support | Notes |
|--------|---------|-------|
| `chat()` | ✅ | Standard chat completion |
| `chatStream()` | ✅ | Streaming chat responses |
| `document()` | ✅ | Native PDF OCR - no image conversion needed |
| `vision()` | ✅ | Image OCR with optional chat analysis |
| `audio()` | ❌ | Not supported - use ElevenLabs |
| `research()` | ❌ | Not supported - use Perplexity |
| `imageGenerate()` | ❌ | Not supported - use OpenAI |
| `imageEdit()` | ❌ | Not supported - use OpenAI |
### Document Processing
The `document()` method uses Mistral OCR to extract text from PDFs, then uses Mistral chat to process the user's query with the extracted content.
```typescript
const result = await provider.document({
systemMessage: 'You are a document analyst.',
userMessage: 'Summarize this document.',
pdfDocuments: [pdfBuffer],
messageHistory: [],
});
```
### API Key ### API Key
Tests require `MISTRAL_API_KEY` in `.nogit/env.json`. Tests require `MISTRAL_API_KEY` in `.nogit/env.json`.
### Pricing (as of December 2025)
- OCR: $2 per 1,000 pages ($1 with Batch API)
- Chat: Varies by model (see Mistral pricing page)
## Anthropic Extended Thinking Feature ## Anthropic Extended Thinking Feature
### Overview
The Anthropic provider now supports extended thinking by default across all methods. Extended thinking enables Claude to spend more time reasoning about complex problems before generating responses, leading to higher quality answers for difficult questions.
### Configuration ### Configuration
Extended thinking is configured at the provider level during instantiation: Extended thinking is configured at the provider level during instantiation:
@@ -97,8 +76,6 @@ const provider = new smartai.AnthropicProvider({
### Thinking Modes ### Thinking Modes
The `extendedThinking` parameter accepts four modes:
| Mode | Budget Tokens | Use Case | | Mode | Budget Tokens | Use Case |
| ---------- | ------------- | ----------------------------------------------- | | ---------- | ------------- | ----------------------------------------------- |
| `'quick'` | 2,048 | Lightweight reasoning for simple queries | | `'quick'` | 2,048 | Lightweight reasoning for simple queries |
@@ -106,141 +83,13 @@ The `extendedThinking` parameter accepts four modes:
| `'deep'` | 16,000 | Complex reasoning for difficult problems | | `'deep'` | 16,000 | Complex reasoning for difficult problems |
| `'off'` | 0 | Disable extended thinking | | `'off'` | 0 | Disable extended thinking |
**Default Behavior**: If `extendedThinking` is not specified, it defaults to `'normal'` mode (8,000 tokens). ### Implementation Details
### Supported Methods - Extended thinking is implemented via `getThinkingConfig()` private method
- When thinking is enabled, temperature must NOT be set
- Uses `claude-sonnet-4-5-20250929` model
Extended thinking is automatically applied to all Anthropic provider methods: ## Testing
- `chat()` - Synchronous chat
- `chatStream()` - Streaming chat
- `vision()` - Image analysis
- `document()` - PDF document processing
- `research()` - Web research with citations
### Token Budget Constraints
**Important**: The thinking budget must be less than `max_tokens` for the API call. The current `max_tokens` values are:
- `chatStream()`: 20,000 tokens (sufficient for all modes ✓)
- `chat()`: 20,000 tokens (sufficient for all modes ✓)
- `vision()`: 10,000 tokens (sufficient for all modes ✓)
- `document()`: 20,000 tokens (sufficient for all modes ✓)
- `research()`: 20,000 tokens for all searchDepth levels (sufficient ✓)
### Performance and Cost Implications
**Token Usage**:
- You are charged for the **full thinking tokens** generated, not just the summary
- Higher thinking budgets may result in more thorough reasoning but increased costs
- The budget is a **target**, not a strict limit - actual usage may vary
**Response Quality**:
- `'quick'`: Fast responses, basic reasoning
- `'normal'`: Good balance between quality and speed (recommended for most use cases)
- `'deep'`: Highest quality reasoning for complex problems, slower responses
**Recommendations**:
- Start with `'normal'` (default) for general usage
- Use `'deep'` for complex analytical tasks, philosophy, mathematics, or research
- Use `'quick'` for simple factual queries where deep reasoning isn't needed
- Use `'off'` only if you want traditional Claude behavior without extended thinking
### Usage Examples
#### Example 1: Default (Normal Mode)
```typescript
const provider = new smartai.AnthropicProvider({
anthropicToken: process.env.ANTHROPIC_TOKEN,
// extendedThinking defaults to 'normal'
});
await provider.start();
const response = await provider.chat({
systemMessage: 'You are a helpful assistant.',
userMessage: 'Explain the implications of quantum computing.',
messageHistory: [],
});
```
#### Example 2: Deep Thinking for Complex Analysis
```typescript
const provider = new smartai.AnthropicProvider({
anthropicToken: process.env.ANTHROPIC_TOKEN,
extendedThinking: 'deep', // 16,000 token budget
});
await provider.start();
const response = await provider.chat({
systemMessage: 'You are a philosopher and ethicist.',
userMessage: 'Analyze the trolley problem from multiple ethical frameworks.',
messageHistory: [],
});
```
#### Example 3: Quick Mode for Simple Queries
```typescript
const provider = new smartai.AnthropicProvider({
anthropicToken: process.env.ANTHROPIC_TOKEN,
extendedThinking: 'quick', // 2,048 token budget
});
await provider.start();
const response = await provider.chat({
systemMessage: 'You are a helpful assistant.',
userMessage: 'What is the capital of France?',
messageHistory: [],
});
```
#### Example 4: Disable Thinking
```typescript
const provider = new smartai.AnthropicProvider({
anthropicToken: process.env.ANTHROPIC_TOKEN,
extendedThinking: 'off', // No extended thinking
});
await provider.start();
const response = await provider.chat({
systemMessage: 'You are a helpful assistant.',
userMessage: 'Tell me a joke.',
messageHistory: [],
});
```
#### Example 5: Extended Thinking with Vision
```typescript
const provider = new smartai.AnthropicProvider({
anthropicToken: process.env.ANTHROPIC_TOKEN,
extendedThinking: 'normal',
});
await provider.start();
const imageBuffer = await fs.promises.readFile('./image.jpg');
const analysis = await provider.vision({
image: imageBuffer,
prompt: 'Analyze this image in detail and explain what you see.',
});
```
### Testing
Comprehensive tests for extended thinking are available in:
- `test/test.thinking.anthropic.ts` - Tests all thinking modes
Run tests with: Run tests with:
@@ -248,27 +97,8 @@ Run tests with:
pnpm test pnpm test
``` ```
Run specific thinking tests: Run specific tests:
```bash ```bash
npx tstest test/test.thinking.anthropic.ts --verbose npx tstest test/test.something.ts --verbose
``` ```
### API Reference
According to Anthropic's documentation:
- Extended thinking is supported on Claude Sonnet 4.5, 4, 3.7, Haiku 4.5, and Opus 4.1, 4
- The current model used is `claude-sonnet-4-5-20250929`
- Minimum thinking budget is 1,024 tokens
- Thinking budget must be less than `max_tokens`
### Implementation Details
The extended thinking feature is implemented via:
1. **Interface**: `IAnthropicProviderOptions.extendedThinking` property
2. **Helper Method**: `getThinkingConfig()` private method that maps modes to token budgets
3. **API Parameter**: Adds `thinking: { type: 'enabled', budget_tokens: number }` to all API calls
The thinking configuration is applied automatically to all API calls when the provider is instantiated.

282
readme.md
View File

@@ -6,7 +6,7 @@
[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/) [![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
SmartAI unifies the world's leading AI providers - OpenAI, Anthropic, Mistral, Perplexity, Ollama, Groq, XAI, Exo, and ElevenLabs - under a single, elegant TypeScript interface. Build AI applications at lightning speed without vendor lock-in. SmartAI unifies the world's leading AI providers — OpenAI, Anthropic, Mistral, Perplexity, Ollama, Groq, XAI, Exo, and ElevenLabs — under a single, elegant TypeScript interface. Build AI applications at lightning speed without vendor lock-in.
## Issue Reporting and Security ## Issue Reporting and Security
@@ -14,19 +14,23 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
## 🎯 Why SmartAI? ## 🎯 Why SmartAI?
- **🔌 Universal Interface**: Write once, run with any AI provider. Switch between GPT-4, Claude, Llama, or Grok with a single line change. - **🔌 Universal Interface**: Write once, run with any AI provider. Switch between GPT-5, Claude, Llama, or Grok with a single line change.
- **🛡️ Type-Safe**: Full TypeScript support with comprehensive type definitions for all operations - **🛡️ Type-Safe**: Full TypeScript support with comprehensive type definitions for all operations.
- **🌊 Streaming First**: Built for real-time applications with native streaming support - **🌊 Streaming First**: Built for real-time applications with native streaming support.
- **🎨 Multi-Modal**: Seamlessly work with text, images, audio, and documents - **🎨 Multi-Modal**: Seamlessly work with text, images, audio, and documents.
- **🏠 Local & Cloud**: Support for both cloud providers and local models via Ollama - **🏠 Local & Cloud**: Support for both cloud providers and local models via Ollama/Exo.
- **⚡ Zero Lock-In**: Your code remains portable across all AI providers - **⚡ Zero Lock-In**: Your code remains portable across all AI providers.
## 🚀 Quick Start ## 📦 Installation
```bash ```bash
npm install @push.rocks/smartai npm install @push.rocks/smartai
# or
pnpm install @push.rocks/smartai
``` ```
## 🚀 Quick Start
```typescript ```typescript
import { SmartAi } from '@push.rocks/smartai'; import { SmartAi } from '@push.rocks/smartai';
@@ -48,6 +52,8 @@ const response = await ai.openaiProvider.chat({
userMessage: 'Explain quantum computing in simple terms', userMessage: 'Explain quantum computing in simple terms',
messageHistory: [], messageHistory: [],
}); });
console.log(response.message);
``` ```
## 📊 Provider Capabilities Matrix ## 📊 Provider Capabilities Matrix
@@ -56,15 +62,15 @@ Choose the right provider for your use case:
| Provider | Chat | Streaming | TTS | Vision | Documents | Research | Images | Highlights | | Provider | Chat | Streaming | TTS | Vision | Documents | Research | Images | Highlights |
| -------------- | :--: | :-------: | :-: | :----: | :-------: | :------: | :----: | --------------------------------------------------------------- | | -------------- | :--: | :-------: | :-: | :----: | :-------: | :------: | :----: | --------------------------------------------------------------- |
| **OpenAI** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | • gpt-image-1<br>• DALL-E 3<br>• Deep research API | | **OpenAI** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | gpt-image-1 • DALL-E 3 • Deep Research API |
| **Anthropic** | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | • Claude Sonnet 4.5<br>• Superior reasoning<br>• Web search API | | **Anthropic** | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | Claude Sonnet 4.5 • Extended Thinking • Web Search API |
| **Mistral** | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | • Native PDF OCR<br>• mistral-large<br>• Fast inference | | **Mistral** | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | Native PDF OCR • mistral-large • Fast inference |
| **ElevenLabs** | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | • Premium TTS<br>• 70+ languages<br>• Natural voices | | **ElevenLabs** | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Premium TTS • 70+ languages • v3 model |
| **Ollama** | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | • 100% local<br>• Privacy-first<br>• No API costs | | **Ollama** | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | 100% local • Privacy-first • No API costs |
| **XAI** | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | • Grok models<br>• Real-time data<br>• Uncensored | | **XAI** | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | Grok 2 • Real-time data |
| **Perplexity** | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | • Web-aware<br>• Research-focused<br>• Sonar Pro models | | **Perplexity** | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | Web-aware • Research-focused • Sonar Pro |
| **Groq** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | • 10x faster<br>• LPU inference<br>• Low latency | | **Groq** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | 10x faster • LPU inference • Llama 3.3 |
| **Exo** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | • Distributed<br>• P2P compute<br>• Decentralized | | **Exo** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | Distributed • P2P compute • Decentralized |
## 🎮 Core Features ## 🎮 Core Features
@@ -73,9 +79,9 @@ Choose the right provider for your use case:
Works identically across all providers: Works identically across all providers:
```typescript ```typescript
// Use GPT-4 for complex reasoning // Use GPT-5 for complex reasoning
const gptResponse = await ai.openaiProvider.chat({ const gptResponse = await ai.openaiProvider.chat({
systemMessage: 'You are a expert physicist.', systemMessage: 'You are an expert physicist.',
userMessage: 'Explain the implications of quantum entanglement', userMessage: 'Explain the implications of quantum entanglement',
messageHistory: [], messageHistory: [],
}); });
@@ -128,20 +134,17 @@ const audioStream = await ai.openaiProvider.audio({
const elevenLabsAudio = await ai.elevenlabsProvider.audio({ const elevenLabsAudio = await ai.elevenlabsProvider.audio({
message: 'Experience the most lifelike text to speech technology.', message: 'Experience the most lifelike text to speech technology.',
voiceId: '19STyYD15bswVz51nqLf', // Optional: Samara voice voiceId: '19STyYD15bswVz51nqLf', // Optional: Samara voice
modelId: 'eleven_v3', // Optional: defaults to eleven_v3 (70+ languages, most expressive) modelId: 'eleven_v3', // Optional: defaults to eleven_v3 (70+ languages)
voiceSettings: { voiceSettings: {
// Optional: fine-tune voice characteristics // Optional: fine-tune voice characteristics
stability: 0.5, // 0-1: Speech consistency stability: 0.5, // 0-1: Speech consistency
similarity_boost: 0.8, // 0-1: Voice similarity to original similarity_boost: 0.8, // 0-1: Voice similarity to original
style: 0.0, // 0-1: Expressiveness (higher = more expressive) style: 0.0, // 0-1: Expressiveness
use_speaker_boost: true, // Enhanced clarity use_speaker_boost: true, // Enhanced clarity
}, },
}); });
// Stream directly to speakers // Stream directly to speakers or save to file
audioStream.pipe(speakerOutput);
// Or save to file
audioStream.pipe(fs.createWriteStream('welcome.mp3')); audioStream.pipe(fs.createWriteStream('welcome.mp3'));
``` ```
@@ -158,7 +161,7 @@ const gptVision = await ai.openaiProvider.vision({
prompt: 'Describe this product and suggest marketing angles', prompt: 'Describe this product and suggest marketing angles',
}); });
// Anthropic: Detailed analysis // Anthropic: Detailed analysis with extended thinking
const claudeVision = await ai.anthropicProvider.vision({ const claudeVision = await ai.anthropicProvider.vision({
image, image,
prompt: 'Identify any safety concerns or defects', prompt: 'Identify any safety concerns or defects',
@@ -179,7 +182,7 @@ Extract insights from PDFs with AI:
const contract = fs.readFileSync('contract.pdf'); const contract = fs.readFileSync('contract.pdf');
const invoice = fs.readFileSync('invoice.pdf'); const invoice = fs.readFileSync('invoice.pdf');
// Analyze documents // Analyze documents with OpenAI
const analysis = await ai.openaiProvider.document({ const analysis = await ai.openaiProvider.document({
systemMessage: 'You are a legal expert.', systemMessage: 'You are a legal expert.',
userMessage: 'Compare these documents and highlight key differences', userMessage: 'Compare these documents and highlight key differences',
@@ -187,7 +190,7 @@ const analysis = await ai.openaiProvider.document({
pdfDocuments: [contract, invoice], pdfDocuments: [contract, invoice],
}); });
// Multi-document analysis // Multi-document analysis with Anthropic
const taxDocs = [form1099, w2, receipts]; const taxDocs = [form1099, w2, receipts];
const taxAnalysis = await ai.anthropicProvider.document({ const taxAnalysis = await ai.anthropicProvider.document({
systemMessage: 'You are a tax advisor.', systemMessage: 'You are a tax advisor.',
@@ -213,6 +216,8 @@ console.log(deepResearch.answer);
console.log('Sources:', deepResearch.sources); console.log('Sources:', deepResearch.sources);
// Anthropic Web Search - Domain-filtered research // Anthropic Web Search - Domain-filtered research
import { AnthropicProvider } from '@push.rocks/smartai';
const anthropic = new AnthropicProvider({ const anthropic = new AnthropicProvider({
anthropicToken: 'sk-ant-...', anthropicToken: 'sk-ant-...',
enableWebSearch: true, enableWebSearch: true,
@@ -233,14 +238,14 @@ const perplexityResearch = await ai.perplexityProvider.research({
**Research Options:** **Research Options:**
- `searchDepth`: 'basic' | 'advanced' | 'deep' - `searchDepth`: `'basic'` | `'advanced'` | `'deep'`
- `maxSources`: Number of sources to include - `maxSources`: Number of sources to include
- `includeWebSearch`: Enable web search (OpenAI) - `includeWebSearch`: Enable web search (OpenAI)
- `background`: Run as background task (OpenAI) - `background`: Run as background task (OpenAI)
**Supported Providers:** **Supported Providers:**
- **OpenAI**: Deep Research API with specialized models (`o3-deep-research-2025-06-26`, `o4-mini-deep-research-2025-06-26`) - **OpenAI**: Deep Research API with specialized models (`o3-deep-research-*`, `o4-mini-deep-research-*`)
- **Anthropic**: Web Search API with domain filtering - **Anthropic**: Web Search API with domain filtering
- **Perplexity**: Sonar and Sonar Pro models with built-in citations - **Perplexity**: Sonar and Sonar Pro models with built-in citations
@@ -269,12 +274,12 @@ const response = await anthropic.chat({
**Thinking Modes:** **Thinking Modes:**
| Mode | Budget Tokens | Use Case | | Mode | Budget Tokens | Use Case |
|------|---------------|----------| | ---------- | ------------- | ------------------------------------------------ |
| `'quick'` | 2,048 | Lightweight reasoning for simple queries | | `'quick'` | 2,048 | Lightweight reasoning for simple queries |
| `'normal'` | 8,000 | **Default** - Balanced reasoning for most tasks | | `'normal'` | 8,000 | **Default** — Balanced reasoning for most tasks |
| `'deep'` | 16,000 | Complex reasoning for difficult problems | | `'deep'` | 16,000 | Complex reasoning for difficult problems |
| `'off'` | 0 | Disable extended thinking | | `'off'` | 0 | Disable extended thinking |
**Best Practices:** **Best Practices:**
@@ -285,16 +290,16 @@ const response = await anthropic.chat({
### 📑 Native PDF OCR (Mistral) ### 📑 Native PDF OCR (Mistral)
Mistral provides native PDF document processing via their OCR API - no image conversion required: Mistral provides native PDF document processing via their OCR API — no image conversion required:
```typescript ```typescript
import { MistralProvider } from '@push.rocks/smartai'; import { MistralProvider } from '@push.rocks/smartai';
const mistral = new MistralProvider({ const mistral = new MistralProvider({
mistralToken: 'your-api-key', mistralToken: 'your-api-key',
chatModel: 'mistral-large-latest', // Default chatModel: 'mistral-large-latest', // Default
ocrModel: 'mistral-ocr-latest', // Default ocrModel: 'mistral-ocr-latest', // Default
tableFormat: 'markdown', // 'markdown' | 'html' tableFormat: 'markdown', // 'markdown' | 'html'
}); });
await mistral.start(); await mistral.start();
@@ -311,6 +316,7 @@ const result = await mistral.document({
**Key Advantage**: Unlike other providers that convert PDFs to images first, Mistral's OCR API processes PDFs natively, potentially offering faster and more accurate text extraction for document-heavy workloads. **Key Advantage**: Unlike other providers that convert PDFs to images first, Mistral's OCR API processes PDFs natively, potentially offering faster and more accurate text extraction for document-heavy workloads.
**Supported Formats:** **Supported Formats:**
- Native PDF processing via Files API - Native PDF processing via Files API
- Image OCR (JPEG, PNG, GIF, WebP) for vision tasks - Image OCR (JPEG, PNG, GIF, WebP) for vision tasks
- Table extraction with markdown or HTML output - Table extraction with markdown or HTML output
@@ -381,14 +387,14 @@ const editedImage = await ai.openaiProvider.imageEdit({
**Image Generation Options:** **Image Generation Options:**
- `model`: 'gpt-image-1' | 'dall-e-3' | 'dall-e-2' - `model`: `'gpt-image-1'` | `'dall-e-3'` | `'dall-e-2'`
- `quality`: 'low' | 'medium' | 'high' | 'auto' - `quality`: `'low'` | `'medium'` | `'high'` | `'auto'`
- `size`: Multiple aspect ratios up to 4096×4096 - `size`: Multiple aspect ratios up to 4096×4096
- `background`: 'transparent' | 'opaque' | 'auto' - `background`: `'transparent'` | `'opaque'` | `'auto'`
- `outputFormat`: 'png' | 'jpeg' | 'webp' - `outputFormat`: `'png'` | `'jpeg'` | `'webp'`
- `outputCompression`: 0-100 for webp/jpeg - `outputCompression`: 0–100 for webp/jpeg
- `moderation`: 'low' | 'auto' - `moderation`: `'low'` | `'auto'`
- `n`: Number of images (1-10) - `n`: Number of images (1–10)
**gpt-image-1 Advantages:** **gpt-image-1 Advantages:**
@@ -424,7 +430,7 @@ await inputWriter.write('Now show me how to make it thread-safe');
```typescript ```typescript
const supportBot = new SmartAi({ const supportBot = new SmartAi({
anthropicToken: process.env.ANTHROPIC_KEY // Claude for empathetic responses anthropicToken: process.env.ANTHROPIC_KEY, // Claude for empathetic responses
}); });
async function handleCustomerQuery(query: string, history: ChatMessage[]) { async function handleCustomerQuery(query: string, history: ChatMessage[]) {
@@ -433,13 +439,13 @@ async function handleCustomerQuery(query: string, history: ChatMessage[]) {
systemMessage: `You are a helpful customer support agent. systemMessage: `You are a helpful customer support agent.
Be empathetic, professional, and solution-oriented.`, Be empathetic, professional, and solution-oriented.`,
userMessage: query, userMessage: query,
messageHistory: history messageHistory: history,
}); });
return response.message; return response.message;
} catch (error) { } catch (error) {
// Fallback to another provider if needed // Fallback to another provider if needed
return await supportBot.openaiProvider.chat({...}); return await supportBot.openaiProvider.chat({ /* ... */ });
} }
} }
``` ```
@@ -452,19 +458,16 @@ const codeReviewer = new SmartAi({
}); });
async function reviewCode(code: string, language: string) { async function reviewCode(code: string, language: string) {
const startTime = Date.now();
const review = await codeReviewer.groqProvider.chat({ const review = await codeReviewer.groqProvider.chat({
systemMessage: `You are a ${language} expert. Review code for: systemMessage: `You are a ${language} expert. Review code for:
- Security vulnerabilities - Security vulnerabilities
- Performance issues - Performance issues
- Best practices - Best practices
- Potential bugs`, - Potential bugs`,
userMessage: `Review this code:\n\n${code}`, userMessage: `Review this code:\n\n${code}`,
messageHistory: [], messageHistory: [],
}); });
console.log(`Review completed in ${Date.now() - startTime}ms`);
return review.message; return review.message;
} }
``` ```
@@ -478,14 +481,15 @@ const researcher = new SmartAi({
async function research(topic: string) { async function research(topic: string) {
// Perplexity excels at web-aware research // Perplexity excels at web-aware research
const findings = await researcher.perplexityProvider.chat({ const findings = await researcher.perplexityProvider.research({
systemMessage: query: `Research the latest developments in ${topic}`,
'You are a research assistant. Provide factual, cited information.', searchDepth: 'deep',
userMessage: `Research the latest developments in ${topic}`,
messageHistory: [],
}); });
return findings.message; return {
answer: findings.answer,
sources: findings.sources,
};
} }
``` ```
@@ -522,23 +526,26 @@ async function analyzeSensitiveDoc(pdfBuffer: Buffer) {
class SmartAIRouter { class SmartAIRouter {
constructor(private ai: SmartAi) {} constructor(private ai: SmartAi) {}
async query(message: string, requirements: { async query(
speed?: boolean; message: string,
accuracy?: boolean; requirements: {
cost?: boolean; speed?: boolean;
privacy?: boolean; accuracy?: boolean;
}) { cost?: boolean;
privacy?: boolean;
}
) {
if (requirements.privacy) { if (requirements.privacy) {
return this.ai.ollamaProvider.chat({...}); // Local only return this.ai.ollamaProvider.chat({ /* ... */ }); // Local only
} }
if (requirements.speed) { if (requirements.speed) {
return this.ai.groqProvider.chat({...}); // 10x faster return this.ai.groqProvider.chat({ /* ... */ }); // 10x faster
} }
if (requirements.accuracy) { if (requirements.accuracy) {
return this.ai.anthropicProvider.chat({...}); // Best reasoning return this.ai.anthropicProvider.chat({ /* ... */ }); // Best reasoning
} }
// Default fallback // Default fallback
return this.ai.openaiProvider.chat({...}); return this.ai.openaiProvider.chat({ /* ... */ });
} }
} }
``` ```
@@ -549,7 +556,7 @@ class SmartAIRouter {
// Don't wait for the entire response // Don't wait for the entire response
async function streamResponse(userQuery: string) { async function streamResponse(userQuery: string) {
const stream = await ai.openaiProvider.chatStream( const stream = await ai.openaiProvider.chatStream(
createInputStream(userQuery), createInputStream(userQuery)
); );
// Process tokens as they arrive // Process tokens as they arrive
@@ -566,9 +573,9 @@ async function streamResponse(userQuery: string) {
// Get the best answer from multiple AIs // Get the best answer from multiple AIs
async function consensusQuery(question: string) { async function consensusQuery(question: string) {
const providers = [ const providers = [
ai.openaiProvider.chat({...}), ai.openaiProvider.chat({ /* ... */ }),
ai.anthropicProvider.chat({...}), ai.anthropicProvider.chat({ /* ... */ }),
ai.perplexityProvider.chat({...}) ai.perplexityProvider.chat({ /* ... */ }),
]; ];
const responses = await Promise.all(providers); const responses = await Promise.all(providers);
@@ -576,21 +583,61 @@ async function consensusQuery(question: string) {
} }
``` ```
## 🛠️ Advanced Features ## 🛠️ Advanced Configuration
### Custom Streaming Transformations ### Provider-Specific Options
```typescript ```typescript
// Add real-time translation const ai = new SmartAi({
const translationStream = new TransformStream({ // OpenAI
async transform(chunk, controller) { openaiToken: 'sk-...',
const translated = await translateChunk(chunk);
controller.enqueue(translated); // Anthropic with extended thinking
anthropicToken: 'sk-ant-...',
// Perplexity for research
perplexityToken: 'pplx-...',
// Groq for speed
groqToken: 'gsk_...',
// Mistral with OCR settings
mistralToken: 'your-key',
mistral: {
chatModel: 'mistral-large-latest',
ocrModel: 'mistral-ocr-latest',
tableFormat: 'markdown',
},
// XAI (Grok)
xaiToken: 'xai-...',
// ElevenLabs TTS
elevenlabsToken: 'sk-...',
elevenlabs: {
defaultVoiceId: '19STyYD15bswVz51nqLf',
defaultModelId: 'eleven_v3',
},
// Ollama (local)
ollama: {
baseUrl: 'http://localhost:11434',
model: 'llama2',
visionModel: 'llava',
defaultOptions: {
num_ctx: 4096,
temperature: 0.7,
top_p: 0.9,
},
defaultTimeout: 120000,
},
// Exo (distributed)
exo: {
baseUrl: 'http://localhost:8080/v1',
apiKey: 'optional-key',
}, },
}); });
const responseStream = await ai.openaiProvider.chatStream(input);
const translatedStream = responseStream.pipeThrough(translationStream);
``` ```
### Error Handling & Fallbacks ### Error Handling & Fallbacks
@@ -613,84 +660,27 @@ class ResilientAI {
} }
``` ```
### Token Counting & Cost Management
```typescript
// Track usage across providers
class UsageTracker {
async trackedChat(provider: string, options: ChatOptions) {
const start = Date.now();
const response = await ai[`${provider}Provider`].chat(options);
const usage = {
provider,
duration: Date.now() - start,
inputTokens: estimateTokens(options),
outputTokens: estimateTokens(response.message),
};
await this.logUsage(usage);
return response;
}
}
```
## 📦 Installation & Setup
### Prerequisites
- Node.js 16+
- TypeScript 4.5+
- API keys for your chosen providers
### Environment Setup
```bash
# Install
npm install @push.rocks/smartai
# Set up environment variables
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export PERPLEXITY_API_KEY=pplx-...
export ELEVENLABS_API_KEY=sk-...
# ... etc
```
### TypeScript Configuration
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"lib": ["ES2022"],
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true
}
}
```
## 🎯 Choosing the Right Provider ## 🎯 Choosing the Right Provider
| Use Case | Recommended Provider | Why | | Use Case | Recommended Provider | Why |
| --------------------- | -------------------- | --------------------------------------------------------- | | --------------------- | -------------------- | --------------------------------------------------------- |
| **General Purpose** | OpenAI | Most features, stable, well-documented | | **General Purpose** | OpenAI | Most features, stable, well-documented |
| **Complex Reasoning** | Anthropic | Superior logical thinking, safer outputs | | **Complex Reasoning** | Anthropic | Superior logical thinking, extended thinking, safer |
| **Document OCR** | Mistral | Native PDF processing, no image conversion overhead | | **Document OCR** | Mistral | Native PDF processing, no image conversion overhead |
| **Research & Facts** | Perplexity | Web-aware, provides citations | | **Research & Facts** | Perplexity | Web-aware, provides citations |
| **Deep Research** | OpenAI | Deep Research API with comprehensive analysis | | **Deep Research** | OpenAI | Deep Research API with comprehensive analysis |
| **Premium TTS** | ElevenLabs | Most natural voices, 70+ languages, superior quality (v3) | | **Premium TTS** | ElevenLabs | Most natural voices, 70+ languages, v3 model |
| **Speed Critical** | Groq | 10x faster inference, sub-second responses | | **Speed Critical** | Groq | 10x faster inference, sub-second responses |
| **Privacy Critical** | Ollama | 100% local, no data leaves your servers | | **Privacy Critical** | Ollama | 100% local, no data leaves your servers |
| **Real-time Data** | XAI | Access to current information | | **Real-time Data** | XAI | Grok with access to current information |
| **Cost Sensitive** | Ollama/Exo | Free (local) or distributed compute | | **Cost Sensitive** | Ollama/Exo | Free (local) or distributed compute |
## 📈 Roadmap ## 📈 Roadmap
- [x] Research & Web Search API - [x] Research & Web Search API
- [x] Image generation support (gpt-image-1, DALL-E 3, DALL-E 2) - [x] Image generation support (gpt-image-1, DALL-E 3, DALL-E 2)
- [x] Extended thinking (Anthropic)
- [x] Native PDF OCR (Mistral)
- [ ] Streaming function calls - [ ] Streaming function calls
- [ ] Voice input processing - [ ] Voice input processing
- [ ] Fine-tuning integration - [ ] Fine-tuning integration

2
ts/00_commitinfo_data.ts
View File

@@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@push.rocks/smartai', name: '@push.rocks/smartai',
version: '0.12.0', version: '0.12.1',
description: 'SmartAi is a versatile TypeScript library designed to facilitate integration and interaction with various AI models, offering functionalities for chat, audio generation, document processing, and vision tasks.' description: 'SmartAi is a versatile TypeScript library designed to facilitate integration and interaction with various AI models, offering functionalities for chat, audio generation, document processing, and vision tasks.'
} }