Compare commits

...

10 Commits

Author SHA1 Message Date
2b8b0e5bdd 5.14.1
Some checks failed
Default (tags) / security (push) Successful in 38s
Default (tags) / test (push) Successful in 3m3s
Default (tags) / release (push) Failing after 51s
Default (tags) / metadata (push) Successful in 58s
2025-04-23 17:28:49 +00:00
3ae2a7fcf5 fix(db operations): Update transaction API to consistently pass optional session parameters across database operations 2025-04-23 17:28:49 +00:00
0806d3749b 5.14.0
Some checks failed
Default (tags) / security (push) Successful in 37s
Default (tags) / test (push) Successful in 3m7s
Default (tags) / release (push) Failing after 50s
Default (tags) / metadata (push) Successful in 57s
2025-04-23 09:03:15 +00:00
f5d5e20a97 feat(doc): Implement support for beforeSave, afterSave, beforeDelete, and afterDelete lifecycle hooks in document save and delete operations to allow custom logic execution during these critical moments. 2025-04-23 09:03:15 +00:00
db2767010d 5.13.1
Some checks failed
Default (tags) / security (push) Successful in 33s
Default (tags) / test (push) Successful in 3m2s
Default (tags) / release (push) Failing after 50s
Default (tags) / metadata (push) Successful in 59s
2025-04-22 20:42:11 +00:00
e2dc094afd fix(search): Improve search query parsing for implicit AND queries by preserving quoted substrings and better handling free terms, quoted phrases, and field:value tokens. 2025-04-22 20:42:11 +00:00
39d2957b7d 5.13.0
Some checks failed
Default (tags) / security (push) Successful in 39s
Default (tags) / test (push) Successful in 3m5s
Default (tags) / release (push) Failing after 52s
Default (tags) / metadata (push) Successful in 57s
2025-04-22 20:34:23 +00:00
490524516e feat(search): Improve search query handling and update documentation 2025-04-22 20:34:23 +00:00
ccd4b9e1ec 5.12.2
Some checks failed
Default (tags) / security (push) Successful in 39s
Default (tags) / test (push) Successful in 3m7s
Default (tags) / release (push) Failing after 52s
Default (tags) / metadata (push) Successful in 1m2s
2025-04-22 20:09:21 +00:00
9c6d6d9f2c fix(search): Fix handling of quoted wildcard patterns in field-specific search queries and add tests for location-based wildcard phrase searches 2025-04-22 20:09:21 +00:00
9 changed files with 348 additions and 66 deletions

View File

@ -1,5 +1,42 @@
# Changelog # Changelog
## 2025-04-23 - 5.14.1 - fix(db operations)
Update transaction API to consistently pass optional session parameters across database operations
- Revised transaction support in readme to use startSession without await and showcased session usage in getInstance and save calls
- Updated methods in classes.collection.ts to accept an optional session parameter for findOne, getCursor, findAll, insert, update, delete, and getCount
- Enhanced SmartDataDbDoc save and delete methods to propagate session parameters
- Improved overall consistency of transactional APIs across the library
## 2025-04-23 - 5.14.0 - feat(doc)
Implement support for beforeSave, afterSave, beforeDelete, and afterDelete lifecycle hooks in document save and delete operations to allow custom logic execution during these critical moments.
- Calls beforeSave hook if defined before performing insert or update.
- Calls afterSave hook after a document is saved.
- Calls beforeDelete hook before deletion and afterDelete hook afterward.
- Ensures _updatedAt timestamp is refreshed during save operations.
## 2025-04-22 - 5.13.1 - fix(search)
Improve search query parsing for implicit AND queries by preserving quoted substrings and better handling free terms, quoted phrases, and field:value tokens.
- Replace previous implicit AND logic with tokenization that preserves quoted substrings
- Support both free term and field:value tokens with wildcards inside quotes
- Ensure errors are thrown for non-searchable fields in field-specific queries
## 2025-04-22 - 5.13.0 - feat(search)
Improve search query handling and update documentation
- Added 'codex.md' providing a high-level project overview and detailed search API documentation.
- Enhanced search parsing in SmartDataDbDoc to support combined free-term and quoted field phrase queries.
- Introduced a new fallback branch in the search method to handle free term with quoted field input.
- Updated tests in test/test.search.ts to cover new combined query scenarios and ensure robust behavior.
## 2025-04-22 - 5.12.2 - fix(search)
Fix handling of quoted wildcard patterns in field-specific search queries and add tests for location-based wildcard phrase searches
- Strip surrounding quotes from wildcard patterns in field queries to correctly transform them to regex
- Introduce new tests in test/test.search.ts to validate exact quoted and unquoted wildcard searches on a location field
## 2025-04-22 - 5.12.1 - fix(search) ## 2025-04-22 - 5.12.1 - fix(search)
Improve implicit AND logic for mixed free term and field queries in search and enhance wildcard field handling. Improve implicit AND logic for mixed free term and field queries in search and enhance wildcard field handling.

77
codex.md Normal file
View File

@ -0,0 +1,77 @@
# SmartData Project Overview
This document provides a high-level overview of the SmartData library (`@push.rocks/smartdata`), its architecture, core components, and key features—including recent enhancements to the search API.
## 1. Project Purpose
- A TypeScriptfirst wrapper around MongoDB that supplies:
- Stronglytyped document & collection classes
- Decoratorbased schema definition (no external schema files)
- Advanced search capabilities with Lucenestyle queries
- Builtin support for realtime data sync, distributed coordination, and keyvalue EasyStore
## 2. Core Concepts & Components
- **SmartDataDb**: Manages the MongoDB connection, pooling, and initialization of collections.
- **SmartDataDbDoc**: Base class for all document models; provides CRUD, upsert, and cursor APIs.
- **Decorators**:
- `@Collection`: Associates a class with a MongoDB collection
- `@svDb()`: Marks a field as persisted to the DB
- `@unI()`: Marks a field as a unique index
- `@index()`: Adds a regular index
- `@searchable()`: Marks a field for inclusion in text searches or regex queries
- **SmartdataCollection**: Wraps a MongoDB collection; autocreates indexes based on decorators.
- **Lucene Adapter**: Parses a Lucene query string into an AST and transforms it to a MongoDB filter object.
- **EasyStore**: A simple, schemaless keyvalue store built on top of MongoDB for sharing ephemeral data.
- **Distributed Coordinator**: Leader election and taskdistribution API for building resilient, multiinstance systems.
- **Watcher**: Listens to change streams for realtime updates and integrates with RxJS.
## 3. Search API
SmartData provides a unified `.search(query[, opts])` method on all models with `@searchable()` fields:
- **Supported Syntax**:
1. Exact field:value (e.g. `field:Value`)
2. Quoted phrases (e.g. `"exact phrase"` or `'exact phrase'`)
3. Wildcards: `*` (zero or more chars) and `?` (single char)
4. Boolean operators: `AND`, `OR`, `NOT`
5. Grouping: parenthesis `(A OR B) AND C`
6. Range queries: `[num TO num]`, `{num TO num}`
7. Multiterm unquoted: terms ANDd across all searchable fields
8. Empty query returns all documents
- **Fallback Mechanisms**:
1. Text index based `$text` search (if supported)
2. Fieldscoped and multifield regex queries
3. Inmemory filtering for complex or unsupported cases
### New Security & Extensibility Hooks
The `.search(query, opts?)` signature now accepts a `SearchOptions<T>` object:
```ts
interface SearchOptions<T> {
filter?: Record<string, any>; // Additional MongoDB filter ANDmerged
validate?: (doc: T) => boolean; // Postfetch hook to drop results
}
```
- **filter**: Enforces mandatory constraints (e.g. multitenant isolation) directly in the Mongo query.
- **validate**: An async function that runs after fetching; return `false` to exclude a document.
## 4. Testing Strategy
- Unit tests in `test/test.search.ts` cover basic search functionality and new options:
- Exact, wildcard, phrase, boolean and grouping cases
- Implicit AND and mixed freeterm + field searches
- Edge cases (nonsearchable fields, quoted wildcards, no matches)
- `filter` and `validate` tests ensure security hooks work as intended
- Advanced search scenarios are covered in `test/test.search.advanced.ts`.
## 5. Usage Example
```ts
// Basic search
const prods = await Product.search('wireless earbuds');
// Scoped search (only your organizations items)
const myItems = await Product.search('book', { filter: { ownerId } });
// Postsearch validation (only cheap items)
const cheapItems = await Product.search('', { validate: p => p.price < 50 });
```
---
Last updated: 2025-04-22

View File

@ -1,6 +1,6 @@
{ {
"name": "@push.rocks/smartdata", "name": "@push.rocks/smartdata",
"version": "5.12.1", "version": "5.14.1",
"private": false, "private": false,
"description": "An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.", "description": "An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.",
"main": "dist_ts/index.js", "main": "dist_ts/index.js",

View File

@ -409,19 +409,23 @@ class Product extends SmartDataDbDoc<Product, Product> {
### Transaction Support ### Transaction Support
Use MongoDB transactions for atomic operations: Use MongoDB transactions for atomic operations. SmartData now exposes `startSession()` and accepts an optional session in all fetch and write APIs:
```typescript ```typescript
const session = await db.startSession(); // start a client session (no await)
const session = db.startSession();
try { try {
// wrap operations in a transaction
await session.withTransaction(async () => { await session.withTransaction(async () => {
const user = await User.getInstance({ id: 'user-id' }, { session }); // pass session as second arg to getInstance
const user = await User.getInstance({ id: 'user-id' }, session);
user.balance -= 100; user.balance -= 100;
// pass session in save opts
await user.save({ session }); await user.save({ session });
const recipient = await User.getInstance({ id: 'recipient-id' }, { session }); const recipient = await User.getInstance({ id: 'recipient-id' }, session);
recipient.balance += 100; recipient.balance += 100;
await user.save({ session }); await recipient.save({ session });
}); });
} finally { } finally {
await session.endSession(); await session.endSession();
@ -518,6 +522,11 @@ class Order extends SmartDataDbDoc<Order, Order> {
throw new Error('Order cannot be deleted'); throw new Error('Order cannot be deleted');
} }
} }
// Called after deleting the document
async afterDelete() {
// Cleanup or audit actions
await auditLogDeletion(this.id);
}
} }
``` ```

View File

@ -9,6 +9,8 @@ import { searchable } from '../ts/classes.doc.js';
// Set up database connection // Set up database connection
let smartmongoInstance: smartmongo.SmartMongo; let smartmongoInstance: smartmongo.SmartMongo;
let testDb: smartdata.SmartdataDb; let testDb: smartdata.SmartdataDb;
// Class for location-based wildcard/phrase tests
let LocationDoc: any;
// Define a test class with searchable fields using the standard SmartDataDbDoc // Define a test class with searchable fields using the standard SmartDataDbDoc
@smartdata.Collection(() => testDb) @smartdata.Collection(() => testDb)
@ -290,6 +292,76 @@ tap.test('should apply validate hook to post-filter results', async () => {
expensive.forEach((p) => expect(p.price).toBeGreaterThan(500)); expensive.forEach((p) => expect(p.price).toBeGreaterThan(500));
}); });
// Tests for quoted and wildcard field-specific phrases
tap.test('setup location test products', async () => {
@smartdata.Collection(() => testDb)
class LD extends smartdata.SmartDataDbDoc<LD, LD> {
@smartdata.unI() public id: string = smartunique.shortId();
@smartdata.svDb() @searchable() public location: string;
constructor(loc: string) { super(); this.location = loc; }
}
// Assign to outer variable for subsequent tests
LocationDoc = LD;
const locations = ['Berlin', 'Frankfurt am Main', 'Frankfurt am Oder', 'London'];
for (const loc of locations) {
await new LocationDoc(loc).save();
}
});
tap.test('should search exact quoted field phrase', async () => {
const results = await (LocationDoc as any).search('location:"Frankfurt am Main"');
expect(results.length).toEqual(1);
expect(results[0].location).toEqual('Frankfurt am Main');
});
tap.test('should search wildcard quoted field phrase', async () => {
const results = await (LocationDoc as any).search('location:"Frankfurt am *"');
const names = results.map((d: any) => d.location).sort();
expect(names).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
});
tap.test('should search unquoted wildcard field', async () => {
const results = await (LocationDoc as any).search('location:Frankfurt*');
const names = results.map((d: any) => d.location).sort();
expect(names).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
});
// Combined free-term + field phrase/wildcard tests
let CombinedDoc: any;
tap.test('setup combined docs for free-term and location tests', async () => {
@smartdata.Collection(() => testDb)
class CD extends smartdata.SmartDataDbDoc<CD, CD> {
@smartdata.unI() public id: string = smartunique.shortId();
@smartdata.svDb() @searchable() public name: string;
@smartdata.svDb() @searchable() public location: string;
constructor(name: string, location: string) { super(); this.name = name; this.location = location; }
}
CombinedDoc = CD;
const docs = [
new CombinedDoc('TypeScript', 'Berlin'),
new CombinedDoc('TypeScript', 'Frankfurt am Main'),
new CombinedDoc('TypeScript', 'Frankfurt am Oder'),
new CombinedDoc('JavaScript', 'Berlin'),
];
for (const d of docs) await d.save();
});
tap.test('should search free term and exact quoted field phrase', async () => {
const res = await CombinedDoc.search('TypeScript location:"Berlin"');
expect(res.length).toEqual(1);
expect(res[0].location).toEqual('Berlin');
});
tap.test('should not match free term with non-matching quoted field phrase', async () => {
const res = await CombinedDoc.search('TypeScript location:"Frankfurt d"');
expect(res.length).toEqual(0);
});
tap.test('should search free term with quoted wildcard field phrase', async () => {
const res = await CombinedDoc.search('TypeScript location:"Frankfurt am *"');
const locs = res.map((r: any) => r.location).sort();
expect(locs).toEqual(['Frankfurt am Main', 'Frankfurt am Oder']);
});
// Quoted exact field phrase without wildcard should return no matches if no exact match
tap.test('should not match location:"Frankfurt d"', async () => {
const results = await (LocationDoc as any).search('location:"Frankfurt d"');
expect(results.length).toEqual(0);
});
// Combined free-term and field wildcard tests // Combined free-term and field wildcard tests
tap.test('should combine free term and wildcard field search', async () => { tap.test('should combine free term and wildcard field search', async () => {
const results = await Product.search('book category:Book*'); const results = await Product.search('book category:Book*');

View File

@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@push.rocks/smartdata', name: '@push.rocks/smartdata',
version: '5.12.1', version: '5.14.1',
description: 'An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.' description: 'An advanced library for NoSQL data organization and manipulation using TypeScript with support for MongoDB, data validation, collections, and custom data types.'
} }

View File

@ -222,29 +222,34 @@ export class SmartdataCollection<T> {
/** /**
* finds an object in the DbCollection * finds an object in the DbCollection
*/ */
public async findOne(filterObject: any): Promise<any> { public async findOne(
filterObject: any,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<any> {
await this.init(); await this.init();
const cursor = this.mongoDbCollection.find(filterObject); // Use MongoDB driver's findOne with optional session
const result = await cursor.next(); return this.mongoDbCollection.findOne(filterObject, { session: opts?.session });
cursor.close();
return result;
} }
public async getCursor( public async getCursor(
filterObjectArg: any, filterObjectArg: any,
dbDocArg: typeof SmartDataDbDoc, dbDocArg: typeof SmartDataDbDoc,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<SmartdataDbCursor<any>> { ): Promise<SmartdataDbCursor<any>> {
await this.init(); await this.init();
const cursor = this.mongoDbCollection.find(filterObjectArg); const cursor = this.mongoDbCollection.find(filterObjectArg, { session: opts?.session });
return new SmartdataDbCursor(cursor, dbDocArg); return new SmartdataDbCursor(cursor, dbDocArg);
} }
/** /**
* finds an object in the DbCollection * finds an object in the DbCollection
*/ */
public async findAll(filterObject: any): Promise<any[]> { public async findAll(
filterObject: any,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<any[]> {
await this.init(); await this.init();
const cursor = this.mongoDbCollection.find(filterObject); const cursor = this.mongoDbCollection.find(filterObject, { session: opts?.session });
const result = await cursor.toArray(); const result = await cursor.toArray();
cursor.close(); cursor.close();
return result; return result;
@ -276,7 +281,10 @@ export class SmartdataCollection<T> {
/** /**
* create an object in the database * create an object in the database
*/ */
public async insert(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> { public async insert(
dbDocArg: T & SmartDataDbDoc<T, unknown>,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<any> {
await this.init(); await this.init();
await this.checkDoc(dbDocArg); await this.checkDoc(dbDocArg);
this.markUniqueIndexes(dbDocArg.uniqueIndexes); this.markUniqueIndexes(dbDocArg.uniqueIndexes);
@ -287,14 +295,17 @@ export class SmartdataCollection<T> {
} }
const saveableObject = await dbDocArg.createSavableObject(); const saveableObject = await dbDocArg.createSavableObject();
const result = await this.mongoDbCollection.insertOne(saveableObject); const result = await this.mongoDbCollection.insertOne(saveableObject, { session: opts?.session });
return result; return result;
} }
/** /**
* inserts object into the DbCollection * inserts object into the DbCollection
*/ */
public async update(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> { public async update(
dbDocArg: T & SmartDataDbDoc<T, unknown>,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<any> {
await this.init(); await this.init();
await this.checkDoc(dbDocArg); await this.checkDoc(dbDocArg);
const identifiableObject = await dbDocArg.createIdentifiableObject(); const identifiableObject = await dbDocArg.createIdentifiableObject();
@ -309,21 +320,27 @@ export class SmartdataCollection<T> {
const result = await this.mongoDbCollection.updateOne( const result = await this.mongoDbCollection.updateOne(
identifiableObject, identifiableObject,
{ $set: updateableObject }, { $set: updateableObject },
{ upsert: true }, { upsert: true, session: opts?.session },
); );
return result; return result;
} }
public async delete(dbDocArg: T & SmartDataDbDoc<T, unknown>): Promise<any> { public async delete(
dbDocArg: T & SmartDataDbDoc<T, unknown>,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<any> {
await this.init(); await this.init();
await this.checkDoc(dbDocArg); await this.checkDoc(dbDocArg);
const identifiableObject = await dbDocArg.createIdentifiableObject(); const identifiableObject = await dbDocArg.createIdentifiableObject();
await this.mongoDbCollection.deleteOne(identifiableObject); await this.mongoDbCollection.deleteOne(identifiableObject, { session: opts?.session });
} }
public async getCount(filterObject: any) { public async getCount(
filterObject: any,
opts?: { session?: plugins.mongodb.ClientSession }
) {
await this.init(); await this.init();
return this.mongoDbCollection.countDocuments(filterObject); return this.mongoDbCollection.countDocuments(filterObject, { session: opts?.session });
} }
/** /**

View File

@ -63,6 +63,12 @@ export class SmartdataDb {
this.status = 'disconnected'; this.status = 'disconnected';
logger.log('info', `disconnected from database ${this.smartdataOptions.mongoDbName}`); logger.log('info', `disconnected from database ${this.smartdataOptions.mongoDbName}`);
} }
/**
* Start a MongoDB client session for transactions
*/
public startSession(): plugins.mongodb.ClientSession {
return this.mongoDbClient.startSession();
}
// handle table to class distribution // handle table to class distribution

View File

@ -11,8 +11,18 @@ import { SmartdataLuceneAdapter } from './classes.lucene.adapter.js';
* - validate: post-fetch validator, return true to keep a doc * - validate: post-fetch validator, return true to keep a doc
*/ */
export interface SearchOptions<T> { export interface SearchOptions<T> {
/**
* Additional MongoDB filter to ANDmerge into the query
*/
filter?: Record<string, any>; filter?: Record<string, any>;
/**
* Postfetch validator; return true to keep each doc
*/
validate?: (doc: T) => Promise<boolean> | boolean; validate?: (doc: T) => Promise<boolean> | boolean;
/**
* Optional MongoDB session for transactional operations
*/
session?: plugins.mongodb.ClientSession;
} }
export type TDocCreation = 'db' | 'new' | 'mixed'; export type TDocCreation = 'db' | 'new' | 'mixed';
@ -193,8 +203,13 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
public static async getInstances<T>( public static async getInstances<T>(
this: plugins.tsclass.typeFest.Class<T>, this: plugins.tsclass.typeFest.Class<T>,
filterArg: plugins.tsclass.typeFest.PartialDeep<T>, filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<T[]> { ): Promise<T[]> {
const foundDocs = await (this as any).collection.findAll(convertFilterForMongoDb(filterArg)); // Pass session through to findAll for transactional queries
const foundDocs = await (this as any).collection.findAll(
convertFilterForMongoDb(filterArg),
{ session: opts?.session },
);
const returnArray = []; const returnArray = [];
for (const foundDoc of foundDocs) { for (const foundDoc of foundDocs) {
const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc); const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc);
@ -212,8 +227,13 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
public static async getInstance<T>( public static async getInstance<T>(
this: plugins.tsclass.typeFest.Class<T>, this: plugins.tsclass.typeFest.Class<T>,
filterArg: plugins.tsclass.typeFest.PartialDeep<T>, filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
opts?: { session?: plugins.mongodb.ClientSession }
): Promise<T> { ): Promise<T> {
const foundDoc = await (this as any).collection.findOne(convertFilterForMongoDb(filterArg)); // Retrieve one document, with optional session for transactions
const foundDoc = await (this as any).collection.findOne(
convertFilterForMongoDb(filterArg),
{ session: opts?.session },
);
if (foundDoc) { if (foundDoc) {
const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc); const newInstance: T = (this as any).createInstanceFromMongoDbNativeDoc(foundDoc);
return newInstance; return newInstance;
@ -236,14 +256,19 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
* get cursor * get cursor
* @returns * @returns
*/ */
/**
* Get a cursor for streaming results, with optional session
*/
public static async getCursor<T>( public static async getCursor<T>(
this: plugins.tsclass.typeFest.Class<T>, this: plugins.tsclass.typeFest.Class<T>,
filterArg: plugins.tsclass.typeFest.PartialDeep<T>, filterArg: plugins.tsclass.typeFest.PartialDeep<T>,
opts?: { session?: plugins.mongodb.ClientSession }
) { ) {
const collection: SmartdataCollection<T> = (this as any).collection; const collection: SmartdataCollection<T> = (this as any).collection;
const cursor: SmartdataDbCursor<T> = await collection.getCursor( const cursor: SmartdataDbCursor<T> = await collection.getCursor(
convertFilterForMongoDb(filterArg), convertFilterForMongoDb(filterArg),
this as any as typeof SmartDataDbDoc, this as any as typeof SmartDataDbDoc,
{ session: opts?.session },
); );
return cursor; return cursor;
} }
@ -339,7 +364,9 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
if (opts?.filter) { if (opts?.filter) {
mongoFilter = { $and: [mongoFilter, opts.filter] }; mongoFilter = { $and: [mongoFilter, opts.filter] };
} }
let docs: T[] = await (this as any).getInstances(mongoFilter); // Fetch with optional session for transactions
// Fetch within optional session
let docs: T[] = await (this as any).getInstances(mongoFilter, { session: opts?.session });
if (opts?.validate) { if (opts?.validate) {
const out: T[] = []; const out: T[] = [];
for (const d of docs) { for (const d of docs) {
@ -397,7 +424,12 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
const wildcardField = q.match(/^(\w+):(.+[*?].*)$/); const wildcardField = q.match(/^(\w+):(.+[*?].*)$/);
if (wildcardField) { if (wildcardField) {
const field = wildcardField[1]; const field = wildcardField[1];
const pattern = wildcardField[2]; // Support quoted wildcard patterns: strip surrounding quotes
let pattern = wildcardField[2];
if ((pattern.startsWith('"') && pattern.endsWith('"')) ||
(pattern.startsWith("'") && pattern.endsWith("'"))) {
pattern = pattern.slice(1, -1);
}
if (!searchableFields.includes(field)) { if (!searchableFields.includes(field)) {
throw new Error(`Field '${field}' is not searchable for class ${this.name}`); throw new Error(`Field '${field}' is not searchable for class ${this.name}`);
} }
@ -414,38 +446,53 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
const orConds = searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } })); const orConds = searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } }));
return await (this as any).execQuery({ $or: orConds }, opts); return await (this as any).execQuery({ $or: orConds }, opts);
} }
// implicit AND: combine free terms and field:value terms (with or without wildcards) // implicit AND for multiple tokens: free terms, quoted phrases, and field:values
const parts = q.split(/\s+/); {
const hasColon = parts.some((t) => t.includes(':')); // Split query into tokens, preserving quoted substrings
const rawTokens = q.match(/(?:[^\s"']+|"[^"]*"|'[^']*')+/g) || [];
// Only apply when more than one token and no boolean operators or grouping
if ( if (
parts.length > 1 && hasColon && rawTokens.length > 1 &&
!q.includes(' AND ') && !q.includes(' OR ') && !q.includes(' NOT ') && !/(\bAND\b|\bOR\b|\bNOT\b|\(|\))/i.test(q) &&
!q.includes('(') && !q.includes(')') && !/\[|\]/.test(q)
!q.includes('[') && !q.includes(']')
) { ) {
const andConds = parts.map((term) => { const andConds: any[] = [];
const m = term.match(/^(\w+):(.+)$/); for (let token of rawTokens) {
if (m) { // field:value token
const field = m[1]; const fv = token.match(/^(\w+):(.+)$/);
const value = m[2]; if (fv) {
const field = fv[1];
let value = fv[2];
if (!searchableFields.includes(field)) { if (!searchableFields.includes(field)) {
throw new Error(`Field '${field}' is not searchable for class ${this.name}`); throw new Error(`Field '${field}' is not searchable for class ${this.name}`);
} }
// Strip surrounding quotes if present
if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) {
value = value.slice(1, -1);
}
// Wildcard search?
if (value.includes('*') || value.includes('?')) { if (value.includes('*') || value.includes('?')) {
// wildcard field search
const escaped = value.replace(/([.+^${}()|[\\]\\])/g, '\\$1'); const escaped = value.replace(/([.+^${}()|[\\]\\])/g, '\\$1');
const pattern = escaped.replace(/\*/g, '.*').replace(/\?/g, '.'); const pattern = escaped.replace(/\*/g, '.*').replace(/\?/g, '.');
return { [field]: { $regex: pattern, $options: 'i' } }; andConds.push({ [field]: { $regex: pattern, $options: 'i' } });
} else {
andConds.push({ [field]: value });
}
} else if ((token.startsWith('"') && token.endsWith('"')) || (token.startsWith("'") && token.endsWith("'"))) {
// Quoted free phrase across all fields
const phrase = token.slice(1, -1);
const parts = phrase.split(/\s+/).map((t) => escapeForRegex(t));
const pattern = parts.join('\\s+');
andConds.push({ $or: searchableFields.map((f) => ({ [f]: { $regex: pattern, $options: 'i' } })) });
} else {
// Free term across all fields
const esc = escapeForRegex(token);
andConds.push({ $or: searchableFields.map((f) => ({ [f]: { $regex: esc, $options: 'i' } })) });
} }
// exact field:value
return { [field]: value };
} }
// free term -> regex across all searchable fields
const esc = escapeForRegex(term);
return { $or: searchableFields.map((f) => ({ [f]: { $regex: esc, $options: 'i' } })) };
});
return await (this as any).execQuery({ $and: andConds }, opts); return await (this as any).execQuery({ $and: andConds }, opts);
} }
}
// detect advanced Lucene syntax: field:value, wildcards, boolean, grouping // detect advanced Lucene syntax: field:value, wildcards, boolean, grouping
const luceneSyntax = /(\w+:[^\s]+)|\*|\?|\bAND\b|\bOR\b|\bNOT\b|\(|\)/; const luceneSyntax = /(\w+:[^\s]+)|\*|\?|\bAND\b|\bOR\b|\bNOT\b|\(|\)/;
if (luceneSyntax.test(q)) { if (luceneSyntax.test(q)) {
@ -526,35 +573,52 @@ export class SmartDataDbDoc<T extends TImplements, TImplements, TManager extends
constructor() {} constructor() {}
/** /**
* saves this instance but not any connected items * saves this instance (optionally within a transaction)
* may lead to data inconsistencies, but is faster
*/ */
public async save() { public async save(opts?: { session?: plugins.mongodb.ClientSession }) {
// allow hook before saving
if (typeof (this as any).beforeSave === 'function') {
await (this as any).beforeSave();
}
// tslint:disable-next-line: no-this-assignment // tslint:disable-next-line: no-this-assignment
const self: any = this; const self: any = this;
let dbResult: any; let dbResult: any;
// update timestamp
this._updatedAt = new Date().toISOString(); this._updatedAt = new Date().toISOString();
// perform insert or update
switch (this.creationStatus) { switch (this.creationStatus) {
case 'db': case 'db':
dbResult = await this.collection.update(self); dbResult = await this.collection.update(self, { session: opts?.session });
break; break;
case 'new': case 'new':
dbResult = await this.collection.insert(self); dbResult = await this.collection.insert(self, { session: opts?.session });
this.creationStatus = 'db'; this.creationStatus = 'db';
break; break;
default: default:
console.error('neither new nor in db?'); console.error('neither new nor in db?');
} }
// allow hook after saving
if (typeof (this as any).afterSave === 'function') {
await (this as any).afterSave();
}
return dbResult; return dbResult;
} }
/** /**
* deletes a document from the database * deletes a document from the database (optionally within a transaction)
*/ */
public async delete() { public async delete(opts?: { session?: plugins.mongodb.ClientSession }) {
await this.collection.delete(this); // allow hook before deleting
if (typeof (this as any).beforeDelete === 'function') {
await (this as any).beforeDelete();
}
// perform deletion
const result = await this.collection.delete(this, { session: opts?.session });
// allow hook after delete
if (typeof (this as any).afterDelete === 'function') {
await (this as any).afterDelete();
}
return result;
} }
/** /**