feat(search): Enhance search functionality with robust Lucene query transformation and reliable fallback mechanisms

readme.md

@@ -18,6 +18,7 @@ A powerful TypeScript-first MongoDB wrapper that provides advanced features for
 - **Enhanced Cursors**: Chainable, type-safe cursor API with memory efficient data processing
 - **Type Conversion**: Automatic handling of MongoDB types like ObjectId and Binary data
 - **Serialization Hooks**: Custom serialization and deserialization of document properties
+- **Powerful Search Capabilities**: Lucene-like query syntax with field-specific search, advanced operators, and fallback mechanisms
 
 ## Requirements
 
@@ -64,7 +65,7 @@ await db.init();
 Data models in `@push.rocks/smartdata` are classes that represent collections and documents in your MongoDB database. Use decorators such as `@Collection`, `@unI`, and `@svDb` to define your data models.
 
 ```typescript
-import { SmartDataDbDoc, Collection, unI, svDb, oid, bin, index } from '@push.rocks/smartdata';
+import { SmartDataDbDoc, Collection, unI, svDb, oid, bin, index, searchable } from '@push.rocks/smartdata';
 import { ObjectId } from 'mongodb';
 
 @Collection(() => db)  // Associate this model with the database instance
@@ -73,9 +74,11 @@ class User extends SmartDataDbDoc<User, User> {
   public id: string = 'unique-user-id'; // Mark 'id' as a unique index
   
   @svDb()
+  @searchable() // Mark 'username' as searchable
   public username: string;  // Mark 'username' to be saved in DB
   
   @svDb()
+  @searchable() // Mark 'email' as searchable
   @index() // Create a regular index for this field
   public email: string;  // Mark 'email' to be saved in DB
   
@@ -169,6 +172,73 @@ await user.delete();  // Delete the user from the database
 
 ## Advanced Features
 
+### Search Functionality
+SmartData provides powerful search capabilities with a Lucene-like query syntax and robust fallback mechanisms:
+
+```typescript
+// Define a model with searchable fields
+@Collection(() => db)
+class Product extends SmartDataDbDoc<Product, Product> {
+  @unI()
+  public id: string = 'product-id';
+  
+  @svDb()
+  @searchable() // Mark this field as searchable
+  public name: string;
+  
+  @svDb()
+  @searchable() // Mark this field as searchable
+  public description: string;
+  
+  @svDb()
+  @searchable() // Mark this field as searchable
+  public category: string;
+  
+  @svDb()
+  public price: number;
+}
+
+// Get all fields marked as searchable for a class
+const searchableFields = getSearchableFields('Product'); // ['name', 'description', 'category']
+
+// Basic search across all searchable fields
+const iPhoneProducts = await Product.searchWithLucene('iPhone');
+
+// Field-specific search
+const electronicsProducts = await Product.searchWithLucene('category:Electronics');
+
+// Search with wildcards
+const macProducts = await Product.searchWithLucene('Mac*');
+
+// Search in specific fields with partial words
+const laptopResults = await Product.searchWithLucene('description:laptop');
+
+// Search is case-insensitive
+const results1 = await Product.searchWithLucene('electronics');
+const results2 = await Product.searchWithLucene('Electronics');
+// results1 and results2 will contain the same documents
+
+// Using boolean operators (requires text index in MongoDB)
+const wirelessOrLaptop = await Product.searchWithLucene('wireless OR laptop');
+
+// Negative searches
+const electronicsNotSamsung = await Product.searchWithLucene('Electronics NOT Samsung');
+
+// Phrase searches
+const exactPhrase = await Product.searchWithLucene('"high-speed blender"');
+
+// Grouping with parentheses
+const complexQuery = await Product.searchWithLucene('(wireless OR bluetooth) AND Electronics');
+```
+
+The search functionality includes:
+- `@searchable()` decorator for marking fields as searchable
+- `getSearchableFields()` to retrieve all searchable fields for a class
+- `search()` method for basic search (requires MongoDB text index)
+- `searchWithLucene()` method with robust fallback mechanisms
+- Support for field-specific searches, wildcards, and boolean operators
+- Automatic fallback to regex-based search if MongoDB text search fails
+
 ### EasyStore
 EasyStore provides a simple key-value storage system with automatic persistence:
 
@@ -442,9 +512,16 @@ class Order extends SmartDataDbDoc<Order, Order> {
 - Set appropriate connection pool sizes based on your application's needs
 
 ### Document Design
-- Use appropriate decorators (`@svDb`, `@unI`, `@index`) to optimize database operations
+- Use appropriate decorators (`@svDb`, `@unI`, `@index`, `@searchable`) to optimize database operations
 - Implement type-safe models by properly extending `SmartDataDbDoc`
 - Consider using interfaces to define document structures separately from implementation
+- Mark fields that need to be searched with the `@searchable()` decorator
+
+### Search Optimization
+- Create MongoDB text indexes for collections that need advanced search operations
+- Use `searchWithLucene()` for robust searches with fallback mechanisms
+- Prefer field-specific searches when possible for better performance
+- Use simple term queries instead of boolean operators if you don't have text indexes
 
 ### Performance Optimization
 - Use cursors for large datasets instead of loading all documents into memory