5.15.0

changelog.md

@@ -1,5 +1,72 @@
 # Changelog
 
+## 2025-04-24 - 5.15.0 - feat(svDb)
+Enhance svDb decorator to support custom serialization and deserialization options
+
+- Added an optional options parameter to the svDb decorator to accept serialize/deserialize functions
+- Updated instance creation logic (updateFromDb) to apply custom deserialization if provided
+- Updated createSavableObject to use custom serialization when available
+
+## 2025-04-23 - 5.14.1 - fix(db operations)
+Update transaction API to consistently pass optional session parameters across database operations
+
+- Revised transaction support in readme to use startSession without await and showcased session usage in getInstance and save calls
+- Updated methods in classes.collection.ts to accept an optional session parameter for findOne, getCursor, findAll, insert, update, delete, and getCount
+- Enhanced SmartDataDbDoc save and delete methods to propagate session parameters
+- Improved overall consistency of transactional APIs across the library
+
+## 2025-04-23 - 5.14.0 - feat(doc)
+Implement support for beforeSave, afterSave, beforeDelete, and afterDelete lifecycle hooks in document save and delete operations to allow custom logic execution during these critical moments.
+
+- Calls beforeSave hook if defined before performing insert or update.
+- Calls afterSave hook after a document is saved.
+- Calls beforeDelete hook before deletion and afterDelete hook afterward.
+- Ensures _updatedAt timestamp is refreshed during save operations.
+
+## 2025-04-22 - 5.13.1 - fix(search)
+Improve search query parsing for implicit AND queries by preserving quoted substrings and better handling free terms, quoted phrases, and field:value tokens.
+
+- Replace previous implicit AND logic with tokenization that preserves quoted substrings
+- Support both free term and field:value tokens with wildcards inside quotes
+- Ensure errors are thrown for non-searchable fields in field-specific queries
+
+## 2025-04-22 - 5.13.0 - feat(search)
+Improve search query handling and update documentation
+
+- Added 'codex.md' providing a high-level project overview and detailed search API documentation.
+- Enhanced search parsing in SmartDataDbDoc to support combined free-term and quoted field phrase queries.
+- Introduced a new fallback branch in the search method to handle free term with quoted field input.
+- Updated tests in test/test.search.ts to cover new combined query scenarios and ensure robust behavior.
+
+## 2025-04-22 - 5.12.2 - fix(search)
+Fix handling of quoted wildcard patterns in field-specific search queries and add tests for location-based wildcard phrase searches
+
+- Strip surrounding quotes from wildcard patterns in field queries to correctly transform them to regex
+- Introduce new tests in test/test.search.ts to validate exact quoted and unquoted wildcard searches on a location field
+
+## 2025-04-22 - 5.12.1 - fix(search)
+Improve implicit AND logic for mixed free term and field queries in search and enhance wildcard field handling.
+
+- Updated regex for field:value parsing to capture full value with wildcards.
+- Added explicit handling for free terms by converting to regex across searchable fields.
+- Improved error messaging for attempts to search non-searchable fields.
+- Extended tests to cover combined free term and wildcard field searches, including error cases.
+
+## 2025-04-22 - 5.12.0 - feat(doc/search)
+Enhance search functionality with filter and validate options for advanced query control
+
+- Added 'filter' option to merge additional MongoDB query constraints in search
+- Introduced 'validate' hook to post-process and filter fetched documents
+- Refactored underlying execQuery function to support additional search options
+- Updated tests to cover new search scenarios and fallback mechanisms
+
+## 2025-04-22 - 5.11.4 - fix(search)
+Implement implicit AND logic for mixed simple term and field:value queries in search
+
+- Added a new branch to detect and handle search queries that mix field:value pairs with plain terms without explicit operators
+- Builds an implicit $and filter when query parts contain colon(s) but lack explicit boolean operators or quotes
+- Ensures proper parsing and improved robustness of search filters
+
 ## 2025-04-22 - 5.11.3 - fix(lucene adapter and search tests)
 Improve range query parsing in Lucene adapter and expand search test coverage
 

codex.md

@@ -0,0 +1,77 @@
+# SmartData Project Overview
+
+This document provides a high-level overview of the SmartData library (`@push.rocks/smartdata`), its architecture, core components, and key features—including recent enhancements to the search API.
+
+## 1. Project Purpose
+- A TypeScript‑first wrapper around MongoDB that supplies:
+  - Strongly‑typed document & collection classes
+  - Decorator‑based schema definition (no external schema files)
+  - Advanced search capabilities with Lucene‑style queries
+  - Built‑in support for real‑time data sync, distributed coordination, and key‑value EasyStore
+
+## 2. Core Concepts & Components
+- **SmartDataDb**: Manages the MongoDB connection, pooling, and initialization of collections.
+- **SmartDataDbDoc**: Base class for all document models; provides CRUD, upsert, and cursor APIs.
+- **Decorators**:
+  - `@Collection`: Associates a class with a MongoDB collection
+  - `@svDb()`: Marks a field as persisted to the DB
+  - `@unI()`: Marks a field as a unique index
+  - `@index()`: Adds a regular index
+  - `@searchable()`: Marks a field for inclusion in text searches or regex queries
+- **SmartdataCollection**: Wraps a MongoDB collection; auto‑creates indexes based on decorators.
+- **Lucene Adapter**: Parses a Lucene query string into an AST and transforms it to a MongoDB filter object.
+- **EasyStore**: A simple, schema‑less key‑value store built on top of MongoDB for sharing ephemeral data.
+- **Distributed Coordinator**: Leader election and task‑distribution API for building resilient, multi‑instance systems.
+- **Watcher**: Listens to change streams for real‑time updates and integrates with RxJS.
+
+## 3. Search API
+SmartData provides a unified `.search(query[, opts])` method on all models with `@searchable()` fields:
+
+- **Supported Syntax**:
+  1. Exact field:value (e.g. `field:Value`)
+  2. Quoted phrases (e.g. `"exact phrase"` or `'exact phrase'`)
+  3. Wildcards: `*` (zero or more chars) and `?` (single char)
+  4. Boolean operators: `AND`, `OR`, `NOT`
+  5. Grouping: parenthesis `(A OR B) AND C`
+  6. Range queries: `[num TO num]`, `{num TO num}`
+  7. Multi‑term unquoted: terms AND’d across all searchable fields
+  8. Empty query returns all documents
+
+- **Fallback Mechanisms**:
+  1. Text index based `$text` search (if supported)
+  2. Field‑scoped and multi‑field regex queries
+  3. In‑memory filtering for complex or unsupported cases
+
+### New Security & Extensibility Hooks
+The `.search(query, opts?)` signature now accepts a `SearchOptions<T>` object:
+```ts
+interface SearchOptions<T> {
+  filter?: Record<string, any>;        // Additional MongoDB filter AND‑merged
+  validate?: (doc: T) => boolean;      // Post‑fetch hook to drop results
+}
+```
+- **filter**: Enforces mandatory constraints (e.g. multi‑tenant isolation) directly in the Mongo query.
+- **validate**: An async function that runs after fetching; return `false` to exclude a document.
+
+## 4. Testing Strategy
+- Unit tests in `test/test.search.ts` cover basic search functionality and new options:
+  - Exact, wildcard, phrase, boolean and grouping cases
+  - Implicit AND and mixed free‑term + field searches
+  - Edge cases (non‑searchable fields, quoted wildcards, no matches)
+  - `filter` and `validate` tests ensure security hooks work as intended
+- Advanced search scenarios are covered in `test/test.search.advanced.ts`.
+
+## 5. Usage Example
+```ts
+// Basic search
+const prods = await Product.search('wireless earbuds');
+
+// Scoped search (only your organization’s items)
+const myItems = await Product.search('book', { filter: { ownerId } });
+
+// Post‑search validation (only cheap items)
+const cheapItems = await Product.search('', { validate: p => p.price < 50 });
+```
+
+---
+Last updated: 2025-04-22

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@push.rocks/smartdata",
-  "version": "5.11.3",
+  "version": "5.15.0",
   "private": false,
   "description": "An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.",
   "main": "dist_ts/index.js",

readme.md

@@ -225,6 +225,10 @@ await Product.search('TypeScript Aufgabe');
 
 // 7: Empty query returns all documents
 await Product.search('');
+// 8: Scoped search with additional filter (e.g. multi-tenant isolation)
+await Product.search('book', { filter: { ownerId: currentUserId } });
+// 9: Post-search validation hook to drop unwanted results (e.g. price check)
+await Product.search('', { validate: (p) => p.price < 100 });
 ```
 
 The search functionality includes:
@@ -405,19 +409,23 @@ class Product extends SmartDataDbDoc<Product, Product> {
 
 ### Transaction Support
 
-Use MongoDB transactions for atomic operations:
+Use MongoDB transactions for atomic operations. SmartData now exposes `startSession()` and accepts an optional session in all fetch and write APIs:
 
 ```typescript
-const session = await db.startSession();
+// start a client session (no await)
+const session = db.startSession();
 try {
+  // wrap operations in a transaction
   await session.withTransaction(async () => {
-    const user = await User.getInstance({ id: 'user-id' }, { session });
+    // pass session as second arg to getInstance
+    const user = await User.getInstance({ id: 'user-id' }, session);
     user.balance -= 100;
+    // pass session in save opts
     await user.save({ session });
 
-    const recipient = await User.getInstance({ id: 'recipient-id' }, { session });
+    const recipient = await User.getInstance({ id: 'recipient-id' }, session);
     recipient.balance += 100;
-    await user.save({ session });
+    await recipient.save({ session });
   });
 } finally {
   await session.endSession();
@@ -514,6 +522,11 @@ class Order extends SmartDataDbDoc<Order, Order> {
       throw new Error('Order cannot be deleted');
     }
   }
+  // Called after deleting the document
+  async afterDelete() {
+    // Cleanup or audit actions
+    await auditLogDeletion(this.id);
+  }
 }
 ```
 

test/test.search.ts

@@ -9,6 +9,8 @@ import { searchable } from '../ts/classes.doc.js';
 // Set up database connection
 let smartmongoInstance: smartmongo.SmartMongo;
 let testDb: smartdata.SmartdataDb;
+// Class for location-based wildcard/phrase tests
+let LocationDoc: any;
 
 // Define a test class with searchable fields using the standard SmartDataDbDoc
 @smartdata.Collection(() => testDb)
@@ -276,6 +278,123 @@ tap.test('should support wildcard plain term with question mark pattern', async
   expect(names).toEqual(['Galaxy S21', 'iPhone 12']);
 });
 
+// Filter and Validation tests
+tap.test('should apply filter option to restrict results', async () => {
+  // search term 'book' across all fields but restrict to Books category
+  const bookFiltered = await Product.search('book', { filter: { category: 'Books' } });
+  expect(bookFiltered.length).toEqual(2);
+  bookFiltered.forEach((p) => expect(p.category).toEqual('Books'));
+});
+tap.test('should apply validate hook to post-filter results', async () => {
+  // return only products with price > 500
+  const expensive = await Product.search('', { validate: (p) => p.price > 500 });
+  expect(expensive.length).toBeGreaterThan(0);
+  expensive.forEach((p) => expect(p.price).toBeGreaterThan(500));
+});
+
+// Tests for quoted and wildcard field-specific phrases
+tap.test('setup location test products', async () => {
+  @smartdata.Collection(() => testDb)
+  class LD extends smartdata.SmartDataDbDoc<LD, LD> {
+    @smartdata.unI() public id: string = smartunique.shortId();
+    @smartdata.svDb() @searchable() public location: string;
+    constructor(loc: string) { super(); this.location = loc; }
+  }
+  // Assign to outer variable for subsequent tests
+  LocationDoc = LD;
+  const locations = ['Berlin', 'Frankfurt am Main', 'Frankfurt am Oder', 'London'];
+  for (const loc of locations) {
+    await new LocationDoc(loc).save();
+  }
+});
+tap.test('should search exact quoted field phrase', async () => {
+  const results = await (LocationDoc as any).search('location:"Frankfurt am Main"');
+  expect(results.length).toEqual(1);
+  expect(results[0].location).toEqual('Frankfurt am Main');
+});
+tap.test('should search wildcard quoted field phrase', async () => {
+  const results = await (LocationDoc as any).search('location:"Frankfurt am *"');
+  const names = results.map((d: any) => d.location).sort();
+  expect(names).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
+});
+tap.test('should search unquoted wildcard field', async () => {
+  const results = await (LocationDoc as any).search('location:Frankfurt*');
+  const names = results.map((d: any) => d.location).sort();
+  expect(names).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
+});
+
+// Combined free-term + field phrase/wildcard tests
+let CombinedDoc: any;
+tap.test('setup combined docs for free-term and location tests', async () => {
+  @smartdata.Collection(() => testDb)
+  class CD extends smartdata.SmartDataDbDoc<CD, CD> {
+    @smartdata.unI() public id: string = smartunique.shortId();
+    @smartdata.svDb() @searchable() public name: string;
+    @smartdata.svDb() @searchable() public location: string;
+    constructor(name: string, location: string) { super(); this.name = name; this.location = location; }
+  }
+  CombinedDoc = CD;
+  const docs = [
+    new CombinedDoc('TypeScript', 'Berlin'),
+    new CombinedDoc('TypeScript', 'Frankfurt am Main'),
+    new CombinedDoc('TypeScript', 'Frankfurt am Oder'),
+    new CombinedDoc('JavaScript', 'Berlin'),
+  ];
+  for (const d of docs) await d.save();
+});
+tap.test('should search free term and exact quoted field phrase', async () => {
+  const res = await CombinedDoc.search('TypeScript location:"Berlin"');
+  expect(res.length).toEqual(1);
+  expect(res[0].location).toEqual('Berlin');
+});
+tap.test('should not match free term with non-matching quoted field phrase', async () => {
+  const res = await CombinedDoc.search('TypeScript location:"Frankfurt d"');
+  expect(res.length).toEqual(0);
+});
+tap.test('should search free term with quoted wildcard field phrase', async () => {
+  const res = await CombinedDoc.search('TypeScript location:"Frankfurt am *"');
+  const locs = res.map((r: any) => r.location).sort();
+  expect(locs).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
+});
+// Quoted exact field phrase without wildcard should return no matches if no exact match
+tap.test('should not match location:"Frankfurt d"', async () => {
+  const results = await (LocationDoc as any).search('location:"Frankfurt d"');
+  expect(results.length).toEqual(0);
+});
+
+// Combined free-term and field wildcard tests
+tap.test('should combine free term and wildcard field search', async () => {
+  const results = await Product.search('book category:Book*');
+  expect(results.length).toEqual(2);
+  results.forEach((p) => expect(p.category).toEqual('Books'));
+});
+tap.test('should not match when free term matches but wildcard field does not', async () => {
+  const results = await Product.search('book category:Kitchen*');
+  expect(results.length).toEqual(0);
+});
+
+// Non-searchable field should cause an error for combined queries
+tap.test('should throw when combining term with non-searchable field', async () => {
+  let error: Error;
+  try {
+    await Product.search('book location:Berlin');
+  } catch (e) {
+    error = e as Error;
+  }
+  expect(error).toBeTruthy();
+  expect(error.message).toMatch(/not searchable/);
+});
+tap.test('should throw when combining term with non-searchable wildcard field', async () => {
+  let error: Error;
+  try {
+    await Product.search('book location:Berlin*');
+  } catch (e) {
+    error = e as Error;
+  }
+  expect(error).toBeTruthy();
+  expect(error.message).toMatch(/not searchable/);
+});
+
 // Close database connection
 tap.test('close database connection', async () => {
   await testDb.mongoDb.dropDatabase();

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartdata',
-  version: '5.11.3',
+  version: '5.15.0',
   description: 'An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.'
 }

ts/classes.collection.ts

@@ -222,29 +222,34 @@ export class SmartdataCollection<T> {
   /**
    * finds an object in the DbCollection
    */
-  public async findOne(filterObject: any): Promise<any> {
+  public async findOne(
+    filterObject: any,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ): Promise<any> {
     await this.init();
-    const cursor = this.mongoDbCollection.find(filterObject);
+    // Use MongoDB driver's findOne with optional session
-    const result = await cursor.next();
+    return this.mongoDbCollection.findOne(filterObject, { session: opts?.session });
-    cursor.close();
-    return result;
   }
 
   public async getCursor(
     filterObjectArg: any,
     dbDocArg: typeof SmartDataDbDoc,
+    opts?: { session?: plugins.mongodb.ClientSession }
   ): Promise<SmartdataDbCursor<any>> {
     await this.init();
-    const cursor = this.mongoDbCollection.find(filterObjectArg);
+    const cursor = this.mongoDbCollection.find(filterObjectArg, { session: opts?.session });
     return new SmartdataDbCursor(cursor, dbDocArg);
   }
 
   /**
    * finds an object in the DbCollection
    */
-  public async findAll(filterObject: any): Promise<any[]> {
+  public async findAll(
+    filterObject: any,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ): Promise<any[]> {
     await this.init();
-    const cursor = this.mongoDbCollection.find(filterObject);
+    const cursor = this.mongoDbCollection.find(filterObject, { session: opts?.session });
     const result = await cursor.toArray();
     cursor.close();
     return result;
@@ -276,7 +281,10 @@ export class SmartdataCollection<T> {
   /**
    * create an object in the database
    */
-  public async insert(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> {
+  public async insert(
+    dbDocArg: T & SmartDataDbDoc<T, unknown>,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ): Promise<any> {
     await this.init();
     await this.checkDoc(dbDocArg);
     this.markUniqueIndexes(dbDocArg.uniqueIndexes);
@@ -287,14 +295,17 @@ export class SmartdataCollection<T> {
     }
     
     const saveableObject = await dbDocArg.createSavableObject();
-    const result = await this.mongoDbCollection.insertOne(saveableObject);
+    const result = await this.mongoDbCollection.insertOne(saveableObject, { session: opts?.session });
     return result;
   }
 
   /**
    * inserts object into the DbCollection
    */
-  public async update(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> {
+  public async update(
+    dbDocArg: T & SmartDataDbDoc<T, unknown>,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ): Promise<any> {
     await this.init();
     await this.checkDoc(dbDocArg);
     const identifiableObject = await dbDocArg.createIdentifiableObject();
@@ -309,21 +320,27 @@ export class SmartdataCollection<T> {
     const result = await this.mongoDbCollection.updateOne(
       identifiableObject,
       { $set: updateableObject },
-      { upsert: true },
+      { upsert: true, session: opts?.session },
     );
     return result;
   }
 
-  public async delete(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> {
+  public async delete(
+    dbDocArg: T & SmartDataDbDoc<T, unknown>,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ): Promise<any> {
     await this.init();
     await this.checkDoc(dbDocArg);
     const identifiableObject = await dbDocArg.createIdentifiableObject();
-    await this.mongoDbCollection.deleteOne(identifiableObject);
+    await this.mongoDbCollection.deleteOne(identifiableObject, { session: opts?.session });
   }
 
-  public async getCount(filterObject: any) {
+  public async getCount(
+    filterObject: any,
+    opts?: { session?: plugins.mongodb.ClientSession }
+  ) {
     await this.init();
-    return this.mongoDbCollection.countDocuments(filterObject);
+    return this.mongoDbCollection.countDocuments(filterObject, { session: opts?.session });
   }
 
   /**

ts/classes.db.ts

@@ -63,6 +63,12 @@ export class SmartdataDb {
     this.status = 'disconnected';
     logger.log('info', `disconnected from database ${this.smartdataOptions.mongoDbName}`);
   }
+  /**
+   * Start a MongoDB client session for transactions
+   */
+  public startSession(): plugins.mongodb.ClientSession {
+    return this.mongoDbClient.startSession();
+  }
 
   // handle table to class distribution
 

ts/classes.doc.ts

@@ -5,6 +5,25 @@ import { SmartdataDbCursor } from './classes.cursor.js';
 import { type IManager, SmartdataCollection } from './classes.collection.js';
 import { SmartdataDbWatcher } from './classes.watcher.js';
 import { SmartdataLuceneAdapter } from './classes.lucene.adapter.js';
+/**
+ * Search options for `.search()`:
+ * - filter: additional MongoDB query to AND-merge
+ * - validate: post-fetch validator, return true to keep a doc
+ */
+export interface SearchOptions<T> {
+  /**
+   * Additional MongoDB filter to AND‐merge into the query
+   */
+  filter?: Record<string, any>;
+  /**
+   * Post‐fetch validator; return true to keep each doc
+   */
+  validate?: (doc: T) => Promise<boolean> | boolean;
+  /**
+   * Optional MongoDB session for transactional operations
+   */
+  session?: plugins.mongodb.ClientSession;
+}
 
 export type TDocCreation = 'db' | 'new' | 'mixed';
 
@@ -20,16 +39,34 @@ export function globalSvDb() {
   };
 }
 
+/**
+ * Options for custom serialization/deserialization of a field.
+ */
+export interface SvDbOptions {
+  /** Function to serialize the field value before saving to DB */
+  serialize?: (value: any) => any;
+  /** Function to deserialize the field value after reading from DB */
+  deserialize?: (value: any) => any;
+}
+
 /**
  * saveable - saveable decorator to be used on class properties
  */
-export function svDb() {
+export function svDb(options?: SvDbOptions) {
   return (target: SmartDataDbDoc<unknown, unknown>, key: string) => {
     console.log(`called svDb() on >${target.constructor.name}.${key}<`);
     if (!target.saveableProperties) {
       target.saveableProperties = [];
     }
     target.saveableProperties.push(key);
+    // attach custom serializer/deserializer options to the class constructor
+    const ctor = target.constructor as any;
+    if (!ctor._svDbOptions) {
+      ctor._svDbOptions = {};
+    }
+    if (options) {
+      ctor._svDbOptions[key] = options;
+    }
   };
 }
 
@@ -170,7 +207,12 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
     const newInstance = new this();
     (newInstance as any).creationStatus = 'db';
     for (const key of Object.keys(mongoDbNativeDocArg)) {
-      newInstance[key] = mongoDbNativeDocArg[key];
+      const rawValue = mongoDbNativeDocArg[key];
+      const optionsMap = (this as any)._svDbOptions || {};
+      const opts = optionsMap[key];
+      newInstance[key] = opts && typeof opts.deserialize === 'function'
+        ? opts.deserialize(rawValue)
+        : rawValue;
     }
     return newInstance;
   }
@@ -184,8 +226,13 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
   public static async getInstances<T>(
     this: plugins.tsclass.typeFest.Class<T>,
     filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
+    opts?: { session?: plugins.mongodb.ClientSession }
   ): Promise<T[]> {
-    const foundDocs = await (this as any).collection.findAll(convertFilterForMongoDb(filterArg));
+    // Pass session through to findAll for transactional queries
+    const foundDocs = await (this as any).collection.findAll(
+      convertFilterForMongoDb(filterArg),
+      { session: opts?.session },
+    );
     const returnArray = [];
     for (const foundDoc of foundDocs) {
       const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc);
@@ -203,8 +250,13 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
   public static async getInstance<T>(
     this: plugins.tsclass.typeFest.Class<T>,
     filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
+    opts?: { session?: plugins.mongodb.ClientSession }
   ): Promise<T> {
-    const foundDoc = await (this as any).collection.findOne(convertFilterForMongoDb(filterArg));
+    // Retrieve one document, with optional session for transactions
+    const foundDoc = await (this as any).collection.findOne(
+      convertFilterForMongoDb(filterArg),
+      { session: opts?.session },
+    );
     if (foundDoc) {
       const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc);
       return newInstance;
@@ -227,14 +279,19 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
    * get cursor
    * @returns
    */
+  /**
+   * Get a cursor for streaming results, with optional session
+   */
   public static async getCursor<T>(
     this: plugins.tsclass.typeFest.Class<T>,
     filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
+    opts?: { session?: plugins.mongodb.ClientSession }
   ) {
     const collection: SmartdataCollection<T> = (this as any).collection;
     const cursor: SmartdataDbCursor<T> = await collection.getCursor(
       convertFilterForMongoDb(filterArg),
       this as any as typeof SmartDataDbDoc,
+      { session: opts?.session },
     );
     return cursor;
   }
@@ -318,15 +375,42 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
     const ctor = this as any;
     return Array.isArray(ctor.searchableFields) ? ctor.searchableFields : [];
   }
+  /**
+   * Execute a query with optional hard filter and post-fetch validation
+   */
+  private static async execQuery<T>(
+    this: plugins.tsclass.typeFest.Class<T>,
+    baseFilter: Record<string, any>,
+    opts?: SearchOptions<T>
+  ): Promise<T[]> {
+    let mongoFilter = baseFilter || {};
+    if (opts?.filter) {
+      mongoFilter = { $and: [mongoFilter, opts.filter] };
+    }
+    // Fetch with optional session for transactions
+    // Fetch within optional session
+    let docs: T[] = await (this as any).getInstances(mongoFilter, { session: opts?.session });
+    if (opts?.validate) {
+      const out: T[] = [];
+      for (const d of docs) {
+        if (await opts.validate(d)) out.push(d);
+      }
+      docs = out;
+    }
+    return docs;
+  }
 
   /**
    * Search documents by text or field:value syntax, with safe regex fallback
+   * Supports additional filtering and post-fetch validation via opts
    * @param query A search term or field:value expression
+   * @param opts Optional filter and validate hooks
    * @returns Array of matching documents
    */
   public static async search<T>(
     this: plugins.tsclass.typeFest.Class<T>,
     query: string,
+    opts?: SearchOptions<T>,
   ): Promise<T[]> {
     const searchableFields = (this as any).getSearchableFields();
     if (searchableFields.length === 0) {
@@ -335,7 +419,8 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
     // empty query -> return all
     const q = query.trim();
     if (!q) {
-      return await (this as any).getInstances({});
+      // empty query: fetch all, apply opts
+      return await (this as any).execQuery({}, opts);
     }
     // simple exact field:value (no spaces, no wildcards, no quotes)
     // simple exact field:value (no spaces, wildcards, quotes)
@@ -346,30 +431,35 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
       if (!searchableFields.includes(field)) {
         throw new Error(`Field '${field}' is not searchable for class ${this.name}`);
       }
-      return await (this as any).getInstances({ [field]: value });
+      // simple field:value search
+      return await (this as any).execQuery({ [field]: value }, opts);
     }
     // quoted phrase across all searchable fields: exact match of phrase
     const quoted = q.match(/^"(.+)"$|^'(.+)'$/);
     if (quoted) {
       const phrase = quoted[1] || quoted[2] || '';
-      // build regex that matches the exact phrase (allowing flexible whitespace)
       const parts = phrase.split(/\s+/).map((t) => escapeForRegex(t));
       const pattern = parts.join('\\s+');
       const orConds = searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } }));
-      return await (this as any).getInstances({ $or: orConds });
+      return await (this as any).execQuery({ $or: orConds }, opts);
     }
     // wildcard field:value (supports * and ?) -> direct regex on that field
     const wildcardField = q.match(/^(\w+):(.+[*?].*)$/);
     if (wildcardField) {
       const field = wildcardField[1];
-      const pattern = wildcardField[2];
+      // Support quoted wildcard patterns: strip surrounding quotes
+      let pattern = wildcardField[2];
+      if ((pattern.startsWith('"') && pattern.endsWith('"')) ||
+          (pattern.startsWith("'") && pattern.endsWith("'"))) {
+        pattern = pattern.slice(1, -1);
+      }
       if (!searchableFields.includes(field)) {
         throw new Error(`Field '${field}' is not searchable for class ${this.name}`);
       }
       // escape regex special chars except * and ?, then convert wildcards
       const escaped = pattern.replace(/([.+^${}()|[\\]\\])/g, '\\$1');
       const regexPattern = escaped.replace(/\*/g, '.*').replace(/\?/g, '.');
-      return await (this as any).getInstances({ [field]: { $regex: regexPattern, $options: 'i' } });
+      return await (this as any).execQuery({ [field]: { $regex: regexPattern, $options: 'i' } }, opts);
     }
     // wildcard plain term across all fields (supports * and ?)
     if (!q.includes(':') && (q.includes('*') || q.includes('?'))) {
@@ -377,13 +467,60 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
       const escaped = q.replace(/([.+^${}()|[\\]\\])/g, '\\$1');
       const pattern = escaped.replace(/\*/g, '.*').replace(/\?/g, '.');
       const orConds = searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } }));
-      return await (this as any).getInstances({ $or: orConds });
+      return await (this as any).execQuery({ $or: orConds }, opts);
+    }
+    // implicit AND for multiple tokens: free terms, quoted phrases, and field:values
+    {
+      // Split query into tokens, preserving quoted substrings
+      const rawTokens = q.match(/(?:[^\s"']+|"[^"]*"|'[^']*')+/g) || [];
+      // Only apply when more than one token and no boolean operators or grouping
+      if (
+        rawTokens.length > 1 &&
+        !/(\bAND\b|\bOR\b|\bNOT\b|\(|\))/i.test(q) &&
+        !/\[|\]/.test(q)
+      ) {
+        const andConds: any[] = [];
+        for (let token of rawTokens) {
+          // field:value token
+          const fv = token.match(/^(\w+):(.+)$/);
+          if (fv) {
+            const field = fv[1];
+            let value = fv[2];
+            if (!searchableFields.includes(field)) {
+              throw new Error(`Field '${field}' is not searchable for class ${this.name}`);
+            }
+            // Strip surrounding quotes if present
+            if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) {
+              value = value.slice(1, -1);
+            }
+            // Wildcard search?
+            if (value.includes('*') || value.includes('?')) {
+              const escaped = value.replace(/([.+^${}()|[\\]\\])/g, '\\$1');
+              const pattern = escaped.replace(/\*/g, '.*').replace(/\?/g, '.');
+              andConds.push({ [field]: { $regex: pattern, $options: 'i' } });
+            } else {
+              andConds.push({ [field]: value });
+            }
+          } else if ((token.startsWith('"') && token.endsWith('"')) || (token.startsWith("'") && token.endsWith("'"))) {
+            // Quoted free phrase across all fields
+            const phrase = token.slice(1, -1);
+            const parts = phrase.split(/\s+/).map((t) => escapeForRegex(t));
+            const pattern = parts.join('\\s+');
+            andConds.push({ $or: searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } })) });
+          } else {
+            // Free term across all fields
+            const esc = escapeForRegex(token);
+            andConds.push({ $or: searchableFields.map((f) => ({ [f]: { $regex: esc, $options: 'i' } })) });
+          }
+        }
+        return await (this as any).execQuery({ $and: andConds }, opts);
+      }
     }
     // detect advanced Lucene syntax: field:value, wildcards, boolean, grouping
     const luceneSyntax = /(\w+:[^\s]+)|\*|\?|\bAND\b|\bOR\b|\bNOT\b|\(|\)/;
     if (luceneSyntax.test(q)) {
       const filter = (this as any).createSearchFilter(q);
-      return await (this as any).getInstances(filter);
+      return await (this as any).execQuery(filter, opts);
     }
     // multi-term unquoted -> AND of regex across fields for each term
     const terms = q.split(/\s+/);
@@ -393,12 +530,12 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
         const ors = searchableFields.map((f) => ({ [f]: { $regex: esc, $options: 'i' } }));
         return { $or: ors };
       });
-      return await (this as any).getInstances({ $and: andConds });
+      return await (this as any).execQuery({ $and: andConds }, opts);
     }
     // single term -> regex across all searchable fields
     const esc = escapeForRegex(q);
     const orConds = searchableFields.map((f) => ({ [f]: { $regex: esc, $options: 'i' } }));
-    return await (this as any).getInstances({ $or: orConds });
+    return await (this as any).execQuery({ $or: orConds }, opts);
   }
 
 
@@ -459,35 +596,52 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
   constructor() {}
 
   /**
-   * saves this instance but not any connected items
+   * saves this instance (optionally within a transaction)
-   * may lead to data inconsistencies, but is faster
    */
-  public async save() {
+  public async save(opts?: { session?: plugins.mongodb.ClientSession }) {
+    // allow hook before saving
+    if (typeof (this as any).beforeSave === 'function') {
+      await (this as any).beforeSave();
+    }
     // tslint:disable-next-line: no-this-assignment
     const self: any = this;
     let dbResult: any;
-
+    // update timestamp
     this._updatedAt = new Date().toISOString();
-
+    // perform insert or update
     switch (this.creationStatus) {
       case 'db':
-        dbResult = await this.collection.update(self);
+        dbResult = await this.collection.update(self, { session: opts?.session });
         break;
       case 'new':
-        dbResult = await this.collection.insert(self);
+        dbResult = await this.collection.insert(self, { session: opts?.session });
         this.creationStatus = 'db';
         break;
       default:
         console.error('neither new nor in db?');
     }
+    // allow hook after saving
+    if (typeof (this as any).afterSave === 'function') {
+      await (this as any).afterSave();
+    }
     return dbResult;
   }
 
   /**
-   * deletes a document from the database
+   * deletes a document from the database (optionally within a transaction)
    */
-  public async delete() {
+  public async delete(opts?: { session?: plugins.mongodb.ClientSession }) {
-    await this.collection.delete(this);
+    // allow hook before deleting
+    if (typeof (this as any).beforeDelete === 'function') {
+      await (this as any).beforeDelete();
+    }
+    // perform deletion
+    const result = await this.collection.delete(this, { session: opts?.session });
+    // allow hook after delete
+    if (typeof (this as any).afterDelete === 'function') {
+      await (this as any).afterDelete();
+    }
+    return result;
   }
 
   /**
@@ -514,7 +668,12 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
   public async updateFromDb() {
     const mongoDbNativeDoc = await this.collection.findOne(await this.createIdentifiableObject());
     for (const key of Object.keys(mongoDbNativeDoc)) {
-      this[key] = mongoDbNativeDoc[key];
+      const rawValue = mongoDbNativeDoc[key];
+      const optionsMap = (this.constructor as any)._svDbOptions || {};
+      const opts = optionsMap[key];
+      this[key] = opts && typeof opts.deserialize === 'function'
+        ? opts.deserialize(rawValue)
+        : rawValue;
     }
   }
 
@@ -524,8 +683,14 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
   public async createSavableObject(): Promise<TImplements> {
     const saveableObject: unknown = {}; // is not exposed to outside, so any is ok here
     const saveableProperties = [...this.globalSaveableProperties, ...this.saveableProperties];
+    // apply custom serialization if configured
+    const optionsMap = (this.constructor as any)._svDbOptions || {};
     for (const propertyNameString of saveableProperties) {
-      saveableObject[propertyNameString] = this[propertyNameString];
+      const rawValue = (this as any)[propertyNameString];
+      const opts = optionsMap[propertyNameString];
+      (saveableObject as any)[propertyNameString] = opts && typeof opts.serialize === 'function'
+        ? opts.serialize(rawValue)
+        : rawValue;
     }
     return saveableObject as TImplements;
   }