feat: Add comprehensive query filters guide and enhance type safety for MongoDB queries
- Introduced a detailed guide on query filters in the README, covering basic filtering, comparison operators, array operators, logical operators, element operators, and advanced filtering patterns. - Implemented a type-safe filtering system in `classes.doc.ts` with `MongoFilterCondition` and `MongoFilter` types to support MongoDB operators while maintaining nested type safety. - Enhanced error handling for invalid operators and conditions in the filtering logic. - Added extensive tests for various filtering scenarios, including basic, comparison, array, logical, and complex filters, ensuring robust functionality and performance. - Implemented security measures to prevent the use of dangerous operators like `$where` and validate operator usage.
This commit is contained in:
332
readme.md
332
readme.md
@@ -132,6 +132,338 @@ await foundUser.save();
|
||||
await foundUser.delete();
|
||||
```
|
||||
|
||||
## 🔍 Query Filters Guide
|
||||
|
||||
SmartData provides a comprehensive and type-safe filtering system that supports all MongoDB query operators while preventing dangerous operations like `$where` for security.
|
||||
|
||||
### Basic Filtering
|
||||
|
||||
```typescript
|
||||
// Simple equality - find users with exact name
|
||||
const johns = await User.getInstances({ name: 'John Doe' });
|
||||
|
||||
// Multiple conditions (implicit AND)
|
||||
const activeAdults = await User.getInstances({
|
||||
status: 'active',
|
||||
age: { $gte: 18 }
|
||||
});
|
||||
|
||||
// Nested object fields - use dot notation
|
||||
const recentUsers = await User.getInstances({
|
||||
'metadata.lastLogin': { $gte: new Date('2024-01-01') },
|
||||
'metadata.loginCount': { $gt: 5 }
|
||||
});
|
||||
```
|
||||
|
||||
### Comparison Operators
|
||||
|
||||
All standard MongoDB comparison operators are supported:
|
||||
|
||||
```typescript
|
||||
// Greater than / Greater than or equal
|
||||
const adults = await User.getInstances({ age: { $gte: 18 } });
|
||||
const seniors = await User.getInstances({ age: { $gt: 65 } });
|
||||
|
||||
// Less than / Less than or equal
|
||||
const youth = await User.getInstances({ age: { $lt: 25 } });
|
||||
const under30 = await User.getInstances({ age: { $lte: 30 } });
|
||||
|
||||
// Not equal
|
||||
const notPending = await User.getInstances({ status: { $ne: 'pending' } });
|
||||
|
||||
// Combine multiple comparisons
|
||||
const middleAged = await User.getInstances({
|
||||
age: { $gte: 30, $lt: 50 }
|
||||
});
|
||||
```
|
||||
|
||||
### Array Operators
|
||||
|
||||
SmartData fully supports MongoDB's array query operators:
|
||||
|
||||
```typescript
|
||||
// $in - Match any value in array
|
||||
const adminsAndMods = await User.getInstances({
|
||||
role: { $in: ['admin', 'moderator'] }
|
||||
});
|
||||
|
||||
// $nin - Match none of the values
|
||||
const regularUsers = await User.getInstances({
|
||||
role: { $nin: ['admin', 'moderator'] }
|
||||
});
|
||||
|
||||
// $all - Array contains all specified elements
|
||||
const fullStackDevs = await User.getInstances({
|
||||
skills: { $all: ['javascript', 'nodejs', 'mongodb'] }
|
||||
});
|
||||
|
||||
// $size - Array has specific length
|
||||
const usersWithThreeSkills = await User.getInstances({
|
||||
skills: { $size: 3 }
|
||||
});
|
||||
|
||||
// $elemMatch - Match array elements with multiple conditions
|
||||
const ordersWithExpensiveItems = await Order.getInstances({
|
||||
items: {
|
||||
$elemMatch: {
|
||||
product: 'laptop',
|
||||
quantity: { $gte: 2 },
|
||||
price: { $gt: 1000 }
|
||||
}
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Working with Arrays
|
||||
|
||||
```typescript
|
||||
// Find documents where array field contains a value
|
||||
const javascriptUsers = await User.getInstances({
|
||||
tags: { $in: ['javascript'] } // Users with 'javascript' in tags array
|
||||
});
|
||||
|
||||
// Find documents where array field contains ALL values
|
||||
const expertUsers = await User.getInstances({
|
||||
certifications: { $all: ['mongodb', 'nodejs', 'typescript'] }
|
||||
});
|
||||
|
||||
// Query nested arrays of objects
|
||||
const highValueOrders = await Order.getInstances({
|
||||
'items.price': { $gte: 100 } // Orders with any item priced >= 100
|
||||
});
|
||||
|
||||
// Complex array element matching
|
||||
const bulkElectronicsOrders = await Order.getInstances({
|
||||
items: {
|
||||
$elemMatch: {
|
||||
category: 'electronics',
|
||||
quantity: { $gte: 5 },
|
||||
discount: { $exists: true }
|
||||
}
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Logical Operators
|
||||
|
||||
Combine multiple conditions with logical operators:
|
||||
|
||||
```typescript
|
||||
// $or - Match any condition
|
||||
const urgentOrHighValue = await Order.getInstances({
|
||||
$or: [
|
||||
{ priority: 'urgent' },
|
||||
{ totalAmount: { $gte: 1000 } }
|
||||
]
|
||||
});
|
||||
|
||||
// $and - Match all conditions (usually implicit, but useful for complex queries)
|
||||
const premiumActiveUsers = await User.getInstances({
|
||||
$and: [
|
||||
{ subscription: 'premium' },
|
||||
{ status: 'active' },
|
||||
{ lastLogin: { $gte: new Date(Date.now() - 30*24*60*60*1000) } }
|
||||
]
|
||||
});
|
||||
|
||||
// $nor - Match none of the conditions
|
||||
const availableProducts = await Product.getInstances({
|
||||
$nor: [
|
||||
{ status: 'discontinued' },
|
||||
{ inventory: { $lte: 0 } }
|
||||
]
|
||||
});
|
||||
|
||||
// $not - Negate a condition
|
||||
const nonEmptyArrays = await User.getInstances({
|
||||
tags: { $not: { $size: 0 } }
|
||||
});
|
||||
|
||||
// Nested logical operations
|
||||
const complexQuery = await User.getInstances({
|
||||
$or: [
|
||||
{
|
||||
$and: [
|
||||
{ role: 'admin' },
|
||||
{ department: 'IT' }
|
||||
]
|
||||
},
|
||||
{
|
||||
$and: [
|
||||
{ role: 'manager' },
|
||||
{ yearsExperience: { $gte: 5 } }
|
||||
]
|
||||
}
|
||||
]
|
||||
});
|
||||
```
|
||||
|
||||
### Element Operators
|
||||
|
||||
Check for field existence and types:
|
||||
|
||||
```typescript
|
||||
// $exists - Check if field exists
|
||||
const usersWithAvatar = await User.getInstances({
|
||||
'profile.avatar': { $exists: true }
|
||||
});
|
||||
|
||||
const usersWithoutPhone = await User.getInstances({
|
||||
phoneNumber: { $exists: false }
|
||||
});
|
||||
|
||||
// $type - Check field type (MongoDB BSON types)
|
||||
const usersWithStringId = await User.getInstances({
|
||||
id: { $type: 'string' } // or use type number: 2 for string
|
||||
});
|
||||
```
|
||||
|
||||
### Regular Expressions
|
||||
|
||||
Use regex for pattern matching:
|
||||
|
||||
```typescript
|
||||
// Case-insensitive search
|
||||
const johnUsers = await User.getInstances({
|
||||
name: { $regex: '^john', $options: 'i' }
|
||||
});
|
||||
|
||||
// Email domain matching
|
||||
const gmailUsers = await User.getInstances({
|
||||
email: { $regex: '@gmail\.com$' }
|
||||
});
|
||||
|
||||
// Complex pattern matching
|
||||
const validPhones = await User.getInstances({
|
||||
phone: { $regex: '^\+1-\d{3}-\d{3}-\d{4}$' }
|
||||
});
|
||||
```
|
||||
|
||||
### Advanced Filtering Patterns
|
||||
|
||||
```typescript
|
||||
// Combine multiple operator types
|
||||
const targetUsers = await User.getInstances({
|
||||
$and: [
|
||||
{ age: { $gte: 25, $lte: 45 } },
|
||||
{ status: { $in: ['active', 'premium'] } },
|
||||
{
|
||||
$or: [
|
||||
{ 'subscription.type': 'annual' },
|
||||
{ 'subscription.credits': { $gte: 100 } }
|
||||
]
|
||||
},
|
||||
{ tags: { $all: ['verified', 'trusted'] } }
|
||||
]
|
||||
});
|
||||
|
||||
// Filter with post-processing validation
|
||||
const results = await Product.search('laptop', {
|
||||
filter: {
|
||||
price: { $lte: 2000 },
|
||||
brand: { $in: ['Apple', 'Dell', 'Lenovo'] }
|
||||
},
|
||||
validate: async (product) => {
|
||||
// Additional custom validation
|
||||
return product.warrantyYears >= 2 && product.inStock;
|
||||
}
|
||||
});
|
||||
|
||||
// Transaction-safe filtering
|
||||
const session = db.startSession();
|
||||
await session.withTransaction(async () => {
|
||||
const users = await User.getInstances(
|
||||
{ status: 'pending' },
|
||||
{ session } // Pass session for transaction
|
||||
);
|
||||
// Process users within transaction
|
||||
});
|
||||
```
|
||||
|
||||
### Performance Tips
|
||||
|
||||
1. **Use Indexes**: Create indexes on frequently filtered fields:
|
||||
```typescript
|
||||
@index() public status: string; // Regular index
|
||||
@unI() public email: string; // Unique index
|
||||
```
|
||||
|
||||
2. **Limit Results**: Use cursors for large datasets:
|
||||
```typescript
|
||||
const cursor = await User.getCursor({ status: 'active' });
|
||||
await cursor.forEach(async (user) => {
|
||||
// Process one at a time
|
||||
});
|
||||
```
|
||||
|
||||
3. **Projection**: Retrieve only needed fields (using MongoDB native):
|
||||
```typescript
|
||||
const collection = User.getCollection();
|
||||
const results = await collection.mongoDbCollection
|
||||
.find({ status: 'active' })
|
||||
.project({ name: 1, email: 1 })
|
||||
.toArray();
|
||||
```
|
||||
|
||||
### Security Considerations
|
||||
|
||||
SmartData automatically prevents dangerous operations:
|
||||
|
||||
```typescript
|
||||
// ❌ These will throw errors:
|
||||
await User.getInstances({
|
||||
$where: 'this.age > 25' // Blocked: JavaScript execution
|
||||
});
|
||||
|
||||
await User.getInstances({
|
||||
'user.name': { 'bad.key': 'value' } // Blocked: dots in keys
|
||||
});
|
||||
|
||||
// ✅ Safe alternatives:
|
||||
await User.getInstances({
|
||||
age: { $gt: 25 } // Use operators instead
|
||||
});
|
||||
|
||||
await User.getInstances({
|
||||
'user.name': 'value' // Use dot notation for nested fields
|
||||
});
|
||||
```
|
||||
|
||||
### Common Pitfalls and Solutions
|
||||
|
||||
```typescript
|
||||
// ❌ Wrong: Array operators need arrays
|
||||
await User.getInstances({
|
||||
role: { $in: 'admin' } // Error: $in requires array
|
||||
});
|
||||
|
||||
// ✅ Correct:
|
||||
await User.getInstances({
|
||||
role: { $in: ['admin'] }
|
||||
});
|
||||
|
||||
// ❌ Wrong: Direct array comparison (exact match)
|
||||
await User.getInstances({
|
||||
tags: 'javascript' // Only matches if tags = ['javascript'] exactly
|
||||
});
|
||||
|
||||
// ✅ Correct: Check if array contains element
|
||||
await User.getInstances({
|
||||
tags: { $in: ['javascript'] } // Matches if 'javascript' is in tags
|
||||
});
|
||||
|
||||
// ❌ Wrong: Checking multiple array elements incorrectly
|
||||
await User.getInstances({
|
||||
skills: 'javascript',
|
||||
skills: 'nodejs' // Object can't have duplicate keys!
|
||||
});
|
||||
|
||||
// ✅ Correct: Use $all for multiple elements
|
||||
await User.getInstances({
|
||||
skills: { $all: ['javascript', 'nodejs'] }
|
||||
});
|
||||
```
|
||||
|
||||
## 🔥 Advanced Features
|
||||
|
||||
### 🔎 Powerful Search Engine
|
||||
|
||||
Reference in New Issue
Block a user