936 lines
24 KiB
Markdown
936 lines
24 KiB
Markdown
# @push.rocks/smartdata 🚀
|
||
|
||
[](https://www.npmjs.com/package/@push.rocks/smartdata)
|
||
|
||
**The ultimate TypeScript-first MongoDB wrapper** that turns your database into a type-safe powerhouse. Built for modern applications that demand real-time performance, distributed coordination, and rock-solid reliability.
|
||
|
||
## 🌟 Why SmartData?
|
||
|
||
SmartData isn't just another MongoDB wrapper - it's a complete paradigm shift in how you work with databases:
|
||
|
||
- 🔒 **100% Type-Safe**: Full TypeScript with decorators, generics, and compile-time query validation
|
||
- ⚡ **Lightning Fast**: Connection pooling, cursor streaming, and intelligent indexing
|
||
- 🔄 **Real-time Ready**: MongoDB Change Streams with RxJS for reactive applications
|
||
- 🌍 **Distributed Systems**: Built-in leader election and task coordination
|
||
- 🛡️ **Security First**: NoSQL injection prevention, automatic sanitization, and secure defaults
|
||
- 🎯 **Developer Experience**: Intuitive API, powerful search, and IntelliSense that just works
|
||
|
||
## 📦 Installation
|
||
|
||
```bash
|
||
# Using pnpm (recommended)
|
||
pnpm add @push.rocks/smartdata
|
||
|
||
# Using npm
|
||
npm install @push.rocks/smartdata
|
||
|
||
# Using yarn
|
||
yarn add @push.rocks/smartdata
|
||
```
|
||
|
||
## 🚦 Requirements
|
||
|
||
- **Node.js** >= 16.x
|
||
- **MongoDB** >= 4.4
|
||
- **TypeScript** >= 4.x (for development)
|
||
|
||
## 🎯 Quick Start
|
||
|
||
### 1️⃣ Connect to Your Database
|
||
|
||
```typescript
|
||
import { SmartdataDb } from '@push.rocks/smartdata';
|
||
|
||
// Create a database instance with smart defaults
|
||
const db = new SmartdataDb({
|
||
mongoDbUrl: 'mongodb://localhost:27017/myapp',
|
||
mongoDbName: 'myapp',
|
||
mongoDbUser: 'username',
|
||
mongoDbPass: 'password',
|
||
|
||
// Optional: Advanced connection pooling
|
||
maxPoolSize: 100, // Max connections in pool
|
||
maxIdleTimeMS: 300000, // Max idle time before connection close
|
||
serverSelectionTimeoutMS: 30000 // Connection timeout
|
||
});
|
||
|
||
// Initialize with automatic retry and health monitoring
|
||
await db.init();
|
||
```
|
||
|
||
### 2️⃣ Define Your Data Models
|
||
|
||
```typescript
|
||
import {
|
||
SmartDataDbDoc,
|
||
Collection,
|
||
unI,
|
||
svDb,
|
||
index,
|
||
searchable,
|
||
} from '@push.rocks/smartdata';
|
||
import { ObjectId } from 'mongodb';
|
||
|
||
@Collection(() => db)
|
||
class User extends SmartDataDbDoc<User, User> {
|
||
@unI()
|
||
public id: string; // Unique index with automatic ID generation
|
||
|
||
@svDb()
|
||
@searchable() // Enable Lucene-style searching
|
||
public username: string;
|
||
|
||
@svDb()
|
||
@searchable()
|
||
@index({ unique: false }) // Performance index
|
||
public email: string;
|
||
|
||
@svDb()
|
||
public status: 'active' | 'inactive' | 'pending'; // Full union type support
|
||
|
||
@svDb()
|
||
public organizationId: ObjectId; // Native MongoDB types
|
||
|
||
@svDb()
|
||
public profilePicture: Buffer; // Binary data support
|
||
|
||
@svDb({
|
||
// Custom serialization for complex objects
|
||
serialize: (data) => JSON.stringify(data),
|
||
deserialize: (data) => JSON.parse(data),
|
||
})
|
||
public preferences: Record<string, any>;
|
||
|
||
@svDb()
|
||
public tags: string[]; // Array support with operators
|
||
|
||
@svDb()
|
||
public createdAt: Date = new Date();
|
||
|
||
constructor(username: string, email: string) {
|
||
super();
|
||
this.username = username;
|
||
this.email = email;
|
||
this.id = User.getNewId();
|
||
}
|
||
}
|
||
```
|
||
|
||
### 3️⃣ Perform CRUD Operations
|
||
|
||
```typescript
|
||
// ✨ Create
|
||
const user = new User('johndoe', 'john@example.com');
|
||
user.status = 'active';
|
||
user.tags = ['developer', 'typescript'];
|
||
await user.save();
|
||
|
||
// 🔍 Read - with full type safety
|
||
const foundUser = await User.getInstance({ username: 'johndoe' });
|
||
const activeUsers = await User.getInstances({ status: 'active' });
|
||
|
||
// ✏️ Update
|
||
foundUser.email = 'newemail@example.com';
|
||
foundUser.tags.push('mongodb');
|
||
await foundUser.save();
|
||
|
||
// 🗑️ Delete
|
||
await foundUser.delete();
|
||
```
|
||
|
||
## 🔥 Advanced Features
|
||
|
||
### 🎯 Type-Safe Query Filters
|
||
|
||
SmartData provides the most advanced type-safe filtering system for MongoDB, supporting all operators while maintaining full IntelliSense:
|
||
|
||
#### Basic Filtering
|
||
|
||
```typescript
|
||
// Simple equality
|
||
const john = await User.getInstances({ name: 'John Doe' });
|
||
|
||
// Multiple fields (implicit AND)
|
||
const activeAdults = await User.getInstances({
|
||
status: 'active',
|
||
age: { $gte: 18 }
|
||
});
|
||
|
||
// Union types work perfectly
|
||
const users = await User.getInstances({
|
||
status: { $in: ['active', 'pending'] } // TypeScript validates these values!
|
||
});
|
||
```
|
||
|
||
#### Deep Nested Object Filtering
|
||
|
||
SmartData supports **both** nested object notation and dot notation for querying nested fields, with intelligent merging when both are used:
|
||
|
||
```typescript
|
||
// Nested object notation - natural TypeScript syntax
|
||
const users = await User.getInstances({
|
||
metadata: {
|
||
loginCount: { $gte: 5 }
|
||
}
|
||
});
|
||
|
||
// Dot notation - MongoDB style
|
||
const sameUsers = await User.getInstances({
|
||
'metadata.loginCount': { $gte: 5 }
|
||
});
|
||
|
||
// POWERFUL: Combine both notations - operators are merged!
|
||
const filtered = await User.getInstances({
|
||
metadata: { loginCount: { $gte: 3 } }, // Object notation
|
||
'metadata.loginCount': { $lte: 10 } // Dot notation
|
||
// Result: metadata.loginCount between 3 and 10
|
||
});
|
||
|
||
// Deep nesting with full type safety
|
||
const deepQuery = await User.getInstances({
|
||
profile: {
|
||
settings: {
|
||
notifications: {
|
||
email: true,
|
||
frequency: { $in: ['daily', 'weekly'] }
|
||
}
|
||
}
|
||
}
|
||
});
|
||
|
||
// Mix styles for complex queries
|
||
const advanced = await User.getInstances({
|
||
// Object notation for structure
|
||
profile: {
|
||
age: { $gte: 21 },
|
||
verified: true
|
||
},
|
||
// Dot notation for specific overrides
|
||
'profile.settings.theme': 'dark',
|
||
'profile.lastSeen': { $gte: new Date('2024-01-01') }
|
||
});
|
||
```
|
||
|
||
#### Comparison Operators
|
||
|
||
```typescript
|
||
// Numeric comparisons with type checking
|
||
const adults = await User.getInstances({
|
||
age: { $gte: 18, $lt: 65 } // Type-safe numeric comparisons
|
||
});
|
||
|
||
// Date comparisons
|
||
const recentUsers = await User.getInstances({
|
||
createdAt: { $gte: new Date('2024-01-01') }
|
||
});
|
||
|
||
// Not equal
|
||
const nonAdmins = await User.getInstances({
|
||
role: { $ne: 'admin' }
|
||
});
|
||
```
|
||
|
||
#### Array Operations
|
||
|
||
```typescript
|
||
// Array operations with full type safety
|
||
const experts = await User.getInstances({
|
||
tags: { $all: ['typescript', 'mongodb'] }, // Must have all tags
|
||
skills: { $size: 5 } // Exactly 5 skills
|
||
});
|
||
|
||
// Array element matching
|
||
const results = await Order.getInstances({
|
||
items: {
|
||
$elemMatch: { // Match array elements
|
||
product: 'laptop',
|
||
quantity: { $gte: 2 }
|
||
}
|
||
}
|
||
});
|
||
|
||
// Check if value exists in array field
|
||
const nodeUsers = await User.getInstances({
|
||
skills: { $in: ['nodejs'] } // Has nodejs in skills array
|
||
});
|
||
```
|
||
|
||
#### Logical Operators
|
||
|
||
```typescript
|
||
// Complex nested queries with $and
|
||
const results = await Order.getInstances({
|
||
$and: [
|
||
{ status: { $in: ['pending', 'processing'] } },
|
||
{ 'items.price': { $gte: 100 } },
|
||
{ customer: { verified: true } }
|
||
]
|
||
});
|
||
|
||
// $or operator
|
||
const urgentOrHighValue = await Order.getInstances({
|
||
$or: [
|
||
{ priority: 'urgent' },
|
||
{ totalAmount: { $gte: 1000 } }
|
||
]
|
||
});
|
||
|
||
// $nor operator - none of the conditions
|
||
const excluded = await User.getInstances({
|
||
$nor: [
|
||
{ status: 'banned' },
|
||
{ role: 'guest' }
|
||
]
|
||
});
|
||
|
||
// Combine logical operators
|
||
const complex = await Order.getInstances({
|
||
$and: [
|
||
{ status: 'active' },
|
||
{
|
||
$or: [
|
||
{ priority: 'high' },
|
||
{ value: { $gte: 1000 } }
|
||
]
|
||
}
|
||
]
|
||
});
|
||
```
|
||
|
||
#### Element Operators
|
||
|
||
```typescript
|
||
// Check field existence
|
||
const withEmail = await User.getInstances({
|
||
email: { $exists: true }
|
||
});
|
||
|
||
// Check for null or missing nested fields
|
||
const noPreferences = await User.getInstances({
|
||
'profile.preferences': { $exists: false }
|
||
});
|
||
```
|
||
|
||
#### Text and Pattern Matching
|
||
|
||
```typescript
|
||
// Regex patterns
|
||
const gmailUsers = await User.getInstances({
|
||
email: { $regex: '@gmail\\.com$', $options: 'i' }
|
||
});
|
||
|
||
// Starts with pattern
|
||
const johnUsers = await User.getInstances({
|
||
name: { $regex: '^John' }
|
||
});
|
||
```
|
||
|
||
#### Security Features
|
||
|
||
```typescript
|
||
// Security: $where is automatically blocked for injection protection
|
||
// await User.getInstances({ $where: '...' }); // ❌ Throws security error
|
||
|
||
// Invalid operators are caught
|
||
// await User.getInstances({ $badOp: 'value' }); // ⚠️ Warning logged
|
||
```
|
||
|
||
#### Advanced Patterns
|
||
|
||
```typescript
|
||
// Combine multiple filtering approaches
|
||
const advancedQuery = await User.getInstances({
|
||
// Direct field matching
|
||
status: 'active',
|
||
|
||
// Nested object with operators
|
||
profile: {
|
||
age: { $gte: 18, $lte: 65 },
|
||
verified: true
|
||
},
|
||
|
||
// Dot notation for deep paths
|
||
'settings.notifications.email': true,
|
||
'metadata.lastLogin': { $gte: new Date(Date.now() - 30*24*60*60*1000) },
|
||
|
||
// Array operations
|
||
roles: { $in: ['admin', 'moderator'] },
|
||
tags: { $all: ['verified', 'premium'] },
|
||
|
||
// Logical grouping
|
||
$or: [
|
||
{ 'subscription.plan': 'premium' },
|
||
{ 'subscription.trial': true }
|
||
]
|
||
});
|
||
```
|
||
|
||
### 🔎 Powerful Search Engine
|
||
|
||
Built-in Lucene-style search with automatic indexing:
|
||
|
||
```typescript
|
||
@Collection(() => db)
|
||
class Product extends SmartDataDbDoc<Product, Product> {
|
||
@unI() public id: string;
|
||
@svDb() @searchable() public name: string;
|
||
@svDb() @searchable() public description: string;
|
||
@svDb() @searchable() public category: string;
|
||
@svDb() public price: number;
|
||
}
|
||
|
||
// Simple text search across all searchable fields
|
||
const results = await Product.search('laptop');
|
||
|
||
// Field-specific search
|
||
const electronics = await Product.search('category:Electronics');
|
||
|
||
// Wildcard searches
|
||
const matches = await Product.search('Mac*');
|
||
|
||
// Boolean operators
|
||
const query = await Product.search('laptop AND NOT gaming');
|
||
|
||
// Phrase search
|
||
const exact = await Product.search('"MacBook Pro 16"');
|
||
|
||
// Combined with filters for powerful queries
|
||
const affordable = await Product.search('laptop', {
|
||
filter: { price: { $lte: 1500 } },
|
||
validate: async (p) => p.inStock === true
|
||
});
|
||
```
|
||
|
||
### 💾 EasyStore - Type-Safe Key-Value Storage
|
||
|
||
Perfect for configuration, caching, and session management:
|
||
|
||
```typescript
|
||
interface AppConfig {
|
||
apiKey: string;
|
||
features: {
|
||
darkMode: boolean;
|
||
notifications: boolean;
|
||
};
|
||
limits: {
|
||
maxUsers: number;
|
||
maxStorage: number;
|
||
};
|
||
}
|
||
|
||
// Create a type-safe store
|
||
const config = await db.createEasyStore<AppConfig>('app-config');
|
||
|
||
// Write with full type checking
|
||
await config.writeKey('features', {
|
||
darkMode: true,
|
||
notifications: false
|
||
});
|
||
|
||
// Read with guaranteed types
|
||
const features = await config.readKey('features');
|
||
// TypeScript knows: features.darkMode is boolean ✅
|
||
|
||
// Atomic updates
|
||
await config.updateKey('limits', (current) => ({
|
||
...current,
|
||
maxUsers: current.maxUsers + 100
|
||
}));
|
||
|
||
// Delete keys
|
||
await config.deleteKey('features');
|
||
|
||
// Clear entire store
|
||
await config.wipe();
|
||
```
|
||
|
||
### 📡 Real-Time Change Streams
|
||
|
||
React to database changes instantly with RxJS integration:
|
||
|
||
```typescript
|
||
// Watch for changes with automatic reconnection
|
||
const watcher = await User.watch(
|
||
{ status: 'active' }, // Filter which documents to watch
|
||
{
|
||
fullDocument: 'updateLookup', // Get full document on updates
|
||
bufferTimeMs: 100 // Buffer changes for efficiency
|
||
}
|
||
);
|
||
|
||
// Subscribe to changes with RxJS
|
||
watcher.changeSubject.subscribe({
|
||
next: (change) => {
|
||
console.log('User changed:', change.fullDocument);
|
||
|
||
switch(change.operationType) {
|
||
case 'insert':
|
||
console.log('New user created');
|
||
break;
|
||
case 'update':
|
||
console.log('User updated');
|
||
break;
|
||
case 'delete':
|
||
console.log('User deleted');
|
||
break;
|
||
}
|
||
},
|
||
error: (err) => console.error('Watch error:', err)
|
||
});
|
||
|
||
// Advanced: Watch with aggregation pipeline
|
||
const complexWatcher = await Order.watch(
|
||
{ status: 'pending' },
|
||
{
|
||
pipeline: [
|
||
{ $match: { 'fullDocument.totalAmount': { $gte: 1000 } } },
|
||
{ $addFields: { isHighValue: true } }
|
||
]
|
||
}
|
||
);
|
||
|
||
// Clean up when done
|
||
await watcher.close();
|
||
```
|
||
|
||
### 🌐 Distributed Coordination
|
||
|
||
Build resilient distributed systems with automatic leader election:
|
||
|
||
```typescript
|
||
import { SmartdataDistributedCoordinator } from '@push.rocks/smartdata';
|
||
|
||
const coordinator = new SmartdataDistributedCoordinator(db);
|
||
|
||
// Start coordination with automatic heartbeat
|
||
await coordinator.start();
|
||
|
||
// Leader election happens automatically
|
||
coordinator.asyncExecutionScheduler.addJobHandler('maintenance', async () => {
|
||
// This code only runs on the leader instance
|
||
console.log('🎖️ Running maintenance as leader');
|
||
await performDatabaseCleanup();
|
||
});
|
||
|
||
// Check leadership status
|
||
const isLeader = coordinator.isLeader;
|
||
|
||
// Fire distributed tasks (integrates with @push.rocks/taskbuffer)
|
||
const result = await coordinator.fireDistributedTaskRequest({
|
||
taskName: 'process-payments',
|
||
taskExecutionTime: Date.now(),
|
||
requestResponseId: 'unique-id'
|
||
});
|
||
|
||
// Graceful shutdown with leadership handoff
|
||
await coordinator.stop();
|
||
```
|
||
|
||
### 🔄 Cursor Streaming for Large Datasets
|
||
|
||
Process millions of documents without memory issues:
|
||
|
||
```typescript
|
||
// Get a cursor for memory-efficient iteration
|
||
const cursor = await User.getCursor(
|
||
{ status: 'active' },
|
||
{
|
||
// Optional: Use MongoDB native cursor modifiers
|
||
modifier: (cursor) => cursor
|
||
.sort({ createdAt: -1 })
|
||
.limit(10000)
|
||
.project({ email: 1, username: 1 })
|
||
}
|
||
);
|
||
|
||
// Process one at a time
|
||
await cursor.forEach(async (user) => {
|
||
await sendEmail(user.email);
|
||
// Each document is loaded individually
|
||
});
|
||
|
||
// Or use async iteration
|
||
for await (const user of cursor) {
|
||
if (shouldStop(user)) break;
|
||
await processUser(user);
|
||
}
|
||
|
||
// Always clean up
|
||
await cursor.close();
|
||
```
|
||
|
||
### 🔐 Transaction Support
|
||
|
||
Ensure data consistency with MongoDB transactions:
|
||
|
||
```typescript
|
||
const session = db.startSession();
|
||
|
||
try {
|
||
await session.withTransaction(async () => {
|
||
// All operations in this block are atomic
|
||
const sender = await User.getInstance(
|
||
{ id: 'user-1' },
|
||
{ session } // Pass session to all operations
|
||
);
|
||
|
||
sender.balance -= 100;
|
||
await sender.save({ session });
|
||
|
||
const receiver = await User.getInstance(
|
||
{ id: 'user-2' },
|
||
{ session }
|
||
);
|
||
|
||
receiver.balance += 100;
|
||
await receiver.save({ session });
|
||
|
||
// If anything fails, everything rolls back
|
||
if (sender.balance < 0) {
|
||
throw new Error('Insufficient funds!');
|
||
}
|
||
});
|
||
|
||
console.log('✅ Transaction completed');
|
||
} finally {
|
||
await session.endSession();
|
||
}
|
||
```
|
||
|
||
### 🎨 Custom Serialization
|
||
|
||
Handle complex data types with custom serializers:
|
||
|
||
```typescript
|
||
class Document extends SmartDataDbDoc<Document, Document> {
|
||
@svDb({
|
||
// Encrypt sensitive data before storing
|
||
serialize: async (value) => await encrypt(value),
|
||
deserialize: async (value) => await decrypt(value)
|
||
})
|
||
public sensitiveData: string;
|
||
|
||
@svDb({
|
||
// Compress large JSON objects
|
||
serialize: (value) => compress(JSON.stringify(value)),
|
||
deserialize: (value) => JSON.parse(decompress(value))
|
||
})
|
||
public largePayload: any;
|
||
|
||
@svDb({
|
||
// Store Sets as arrays
|
||
serialize: (set) => Array.from(set),
|
||
deserialize: (arr) => new Set(arr)
|
||
})
|
||
public tags: Set<string>;
|
||
|
||
@svDb({
|
||
// Handle custom date formats
|
||
serialize: (date) => date?.toISOString(),
|
||
deserialize: (str) => str ? new Date(str) : null
|
||
})
|
||
public scheduledAt: Date | null;
|
||
}
|
||
```
|
||
|
||
### 🎣 Lifecycle Hooks
|
||
|
||
Add custom logic at any point in the document lifecycle:
|
||
|
||
```typescript
|
||
@Collection(() => db)
|
||
class Order extends SmartDataDbDoc<Order, Order> {
|
||
@unI() public id: string;
|
||
@svDb() public items: Array<{ product: string; quantity: number }>;
|
||
@svDb() public totalAmount: number;
|
||
@svDb() public status: string;
|
||
|
||
// Called before saving (create or update)
|
||
async beforeSave() {
|
||
// Recalculate total
|
||
this.totalAmount = this.items.reduce((sum, item) =>
|
||
sum + (item.price * item.quantity), 0
|
||
);
|
||
|
||
// Validate
|
||
if (this.totalAmount < 0) {
|
||
throw new Error('Invalid order total');
|
||
}
|
||
}
|
||
|
||
// Called after successful save
|
||
async afterSave() {
|
||
// Send notifications
|
||
await notificationService.orderUpdated(this.id);
|
||
|
||
// Update cache
|
||
await cache.set(`order:${this.id}`, this);
|
||
}
|
||
|
||
// Called before deletion
|
||
async beforeDelete() {
|
||
// Archive order
|
||
await archive.store(this);
|
||
|
||
// Check if deletion is allowed
|
||
if (this.status === 'completed') {
|
||
throw new Error('Cannot delete completed orders');
|
||
}
|
||
}
|
||
|
||
// Called after successful deletion
|
||
async afterDelete() {
|
||
// Clean up related data
|
||
await cache.delete(`order:${this.id}`);
|
||
|
||
// Log deletion
|
||
console.log(`Order ${this.id} deleted at ${new Date()}`);
|
||
}
|
||
}
|
||
```
|
||
|
||
### 🚀 Performance Optimization
|
||
|
||
#### Intelligent Indexing
|
||
|
||
```typescript
|
||
@Collection(() => db)
|
||
class HighPerformanceDoc extends SmartDataDbDoc<HighPerformanceDoc, HighPerformanceDoc> {
|
||
@unI() // Unique index
|
||
public id: string;
|
||
|
||
@index() // Single field index
|
||
public userId: string;
|
||
|
||
@index({ sparse: true }) // Sparse index for optional fields
|
||
public deletedAt?: Date;
|
||
|
||
@index({
|
||
unique: false,
|
||
background: true, // Non-blocking index creation
|
||
expireAfterSeconds: 86400 // TTL index
|
||
})
|
||
public sessionToken: string;
|
||
|
||
// Compound indexes for complex queries
|
||
static async createIndexes() {
|
||
await this.collection.createIndex(
|
||
{ userId: 1, createdAt: -1 }, // Compound index
|
||
{ name: 'user_activity_idx' }
|
||
);
|
||
|
||
// Text index for search
|
||
await this.collection.createIndex(
|
||
{ title: 'text', content: 'text' },
|
||
{ weights: { title: 10, content: 5 } }
|
||
);
|
||
}
|
||
}
|
||
```
|
||
|
||
#### Connection Pool Management
|
||
|
||
```typescript
|
||
const db = new SmartdataDb({
|
||
mongoDbUrl: 'mongodb://localhost:27017',
|
||
mongoDbName: 'myapp',
|
||
|
||
// Connection pool optimization
|
||
maxPoolSize: 100, // Maximum connections
|
||
minPoolSize: 10, // Minimum connections to maintain
|
||
maxIdleTimeMS: 300000, // Close idle connections after 5 minutes
|
||
waitQueueTimeoutMS: 5000, // Max time to wait for available connection
|
||
|
||
// Server selection
|
||
serverSelectionTimeoutMS: 30000, // Timeout for selecting a server
|
||
heartbeatFrequencyMS: 10000, // How often to check server status
|
||
|
||
// Socket settings
|
||
socketTimeoutMS: 360000, // Socket timeout (6 minutes)
|
||
family: 4, // Force IPv4
|
||
});
|
||
```
|
||
|
||
## 🎯 Best Practices
|
||
|
||
### 1. Always Use TypeScript
|
||
|
||
SmartData is built for TypeScript. Using JavaScript means missing out on:
|
||
- Compile-time query validation
|
||
- IntelliSense for MongoDB operators
|
||
- Type-safe document updates
|
||
- Automatic refactoring support
|
||
|
||
### 2. Design Your Indexes Thoughtfully
|
||
|
||
```typescript
|
||
// ✅ Good: Index fields you query frequently
|
||
@index() public userId: string;
|
||
@index() public status: string;
|
||
|
||
// ❌ Bad: Over-indexing
|
||
@index() public everyField: string; // Don't index everything!
|
||
```
|
||
|
||
### 3. Use Transactions for Critical Operations
|
||
|
||
```typescript
|
||
// ✅ Good: Atomic operations for financial transactions
|
||
await session.withTransaction(async () => {
|
||
await debitAccount(fromAccount, amount, { session });
|
||
await creditAccount(toAccount, amount, { session });
|
||
});
|
||
|
||
// ❌ Bad: Multiple operations without transactions
|
||
await debitAccount(fromAccount, amount); // What if this fails?
|
||
await creditAccount(toAccount, amount);
|
||
```
|
||
|
||
### 4. Leverage Cursors for Large Datasets
|
||
|
||
```typescript
|
||
// ✅ Good: Stream processing
|
||
const cursor = await LargeCollection.getCursor({});
|
||
await cursor.forEach(async (doc) => {
|
||
await processOne(doc);
|
||
});
|
||
|
||
// ❌ Bad: Loading everything into memory
|
||
const allDocs = await LargeCollection.getInstances({}); // Could OOM!
|
||
```
|
||
|
||
### 5. Implement Proper Error Handling
|
||
|
||
```typescript
|
||
// ✅ Good: Graceful error handling
|
||
try {
|
||
const user = await User.getInstance({ id: userId });
|
||
if (!user) {
|
||
throw new Error('User not found');
|
||
}
|
||
await processUser(user);
|
||
} catch (error) {
|
||
logger.error('Failed to process user:', error);
|
||
await notificationService.alertAdmin(error);
|
||
}
|
||
|
||
// ❌ Bad: Ignoring errors
|
||
const user = await User.getInstance({ id: userId });
|
||
await processUser(user); // What if user is null?
|
||
```
|
||
|
||
## 🔧 Troubleshooting
|
||
|
||
### Connection Issues
|
||
|
||
```typescript
|
||
// Enable debug logging
|
||
const db = new SmartdataDb({
|
||
mongoDbUrl: 'mongodb://localhost:27017',
|
||
mongoDbName: 'myapp',
|
||
// Add connection event handlers
|
||
});
|
||
|
||
db.mongoClient.on('error', (err) => {
|
||
console.error('MongoDB connection error:', err);
|
||
});
|
||
|
||
db.mongoClient.on('timeout', () => {
|
||
console.error('MongoDB connection timeout');
|
||
});
|
||
|
||
// Check connection status
|
||
const isConnected = db.status === 'connected';
|
||
```
|
||
|
||
### Memory Leaks
|
||
|
||
Always clean up resources:
|
||
|
||
```typescript
|
||
// Watchers
|
||
const watcher = await User.watch({});
|
||
// ... use watcher
|
||
await watcher.close(); // Always close!
|
||
|
||
// Cursors
|
||
const cursor = await User.getCursor({});
|
||
// ... use cursor
|
||
await cursor.close(); // Always close!
|
||
|
||
// Sessions
|
||
const session = db.startSession();
|
||
// ... use session
|
||
await session.endSession(); // Always end!
|
||
```
|
||
|
||
### Performance Issues
|
||
|
||
```typescript
|
||
// Profile slow queries
|
||
await db.mongoDb.setProfilingLevel('all');
|
||
|
||
// Check query execution
|
||
const stats = await User.collection.mongoDbCollection
|
||
.find({ complex: 'query' })
|
||
.explain('executionStats');
|
||
|
||
console.log('Query execution time:', stats.executionTimeMillis);
|
||
console.log('Documents examined:', stats.totalDocsExamined);
|
||
console.log('Index used:', stats.executionStats.executionStages.indexName);
|
||
```
|
||
|
||
## 📚 API Reference
|
||
|
||
### Core Classes
|
||
|
||
- `SmartdataDb` - Database connection and management
|
||
- `SmartDataDbDoc` - Base class for all documents
|
||
- `SmartdataCollection` - Collection management
|
||
- `SmartdataDbCursor` - Cursor for streaming operations
|
||
- `SmartdataDbWatcher` - Change stream watcher
|
||
- `SmartdataDistributedCoordinator` - Distributed coordination
|
||
|
||
### Decorators
|
||
|
||
- `@Collection(dbGetter)` - Define collection for a document class
|
||
- `@unI()` - Create unique index
|
||
- `@index(options)` - Create regular index
|
||
- `@svDb(options)` - Mark field as saveable with optional serialization
|
||
- `@searchable()` - Enable text search on field
|
||
|
||
### Key Methods
|
||
|
||
- `getInstances(filter)` - Find multiple documents
|
||
- `getInstance(filter)` - Find single document
|
||
- `getCursor(filter)` - Get cursor for streaming
|
||
- `watch(filter, options)` - Watch for changes
|
||
- `search(query, options)` - Lucene-style search
|
||
- `save(options)` - Save document
|
||
- `delete(options)` - Delete document
|
||
- `updateFromDb()` - Refresh from database
|
||
|
||
## 🌍 Community & Support
|
||
|
||
- 📖 **Documentation**: [https://code.foss.global/push.rocks/smartdata](https://code.foss.global/push.rocks/smartdata)
|
||
- 💬 **Issues**: [GitLab Issues](https://code.foss.global/push.rocks/smartdata/issues)
|
||
- 📧 **Email**: hello@task.vc
|
||
|
||
## License and Legal Information
|
||
|
||
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
|
||
|
||
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
|
||
|
||
### Trademarks
|
||
|
||
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
|
||
|
||
### Company Information
|
||
|
||
Task Venture Capital GmbH
|
||
Registered at District court Bremen HRB 35230 HB, Germany
|
||
|
||
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
|
||
|
||
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works. |