smartdata/changelog.md

# Changelog

## 2025-04-22 - 5.13.1 - fix(search)
Improve search query parsing for implicit AND queries by preserving quoted substrings and better handling free terms, quoted phrases, and field:value tokens.

- Replace previous implicit AND logic with tokenization that preserves quoted substrings
- Support both free term and field:value tokens with wildcards inside quotes
- Ensure errors are thrown for non-searchable fields in field-specific queries

## 2025-04-22 - 5.13.0 - feat(search)
Improve search query handling and update documentation

- Added 'codex.md' providing a high-level project overview and detailed search API documentation.
- Enhanced search parsing in SmartDataDbDoc to support combined free-term and quoted field phrase queries.
- Introduced a new fallback branch in the search method to handle free term with quoted field input.
- Updated tests in test/test.search.ts to cover new combined query scenarios and ensure robust behavior.

## 2025-04-22 - 5.12.2 - fix(search)
Fix handling of quoted wildcard patterns in field-specific search queries and add tests for location-based wildcard phrase searches

- Strip surrounding quotes from wildcard patterns in field queries to correctly transform them to regex
- Introduce new tests in test/test.search.ts to validate exact quoted and unquoted wildcard searches on a location field

## 2025-04-22 - 5.12.1 - fix(search)
Improve implicit AND logic for mixed free term and field queries in search and enhance wildcard field handling.

- Updated regex for field:value parsing to capture full value with wildcards.
- Added explicit handling for free terms by converting to regex across searchable fields.
- Improved error messaging for attempts to search non-searchable fields.
- Extended tests to cover combined free term and wildcard field searches, including error cases.

## 2025-04-22 - 5.12.0 - feat(doc/search)
Enhance search functionality with filter and validate options for advanced query control

- Added 'filter' option to merge additional MongoDB query constraints in search
- Introduced 'validate' hook to post-process and filter fetched documents
- Refactored underlying execQuery function to support additional search options
- Updated tests to cover new search scenarios and fallback mechanisms

## 2025-04-22 - 5.11.4 - fix(search)
Implement implicit AND logic for mixed simple term and field:value queries in search

- Added a new branch to detect and handle search queries that mix field:value pairs with plain terms without explicit operators
- Builds an implicit $and filter when query parts contain colon(s) but lack explicit boolean operators or quotes
- Ensures proper parsing and improved robustness of search filters

## 2025-04-22 - 5.11.3 - fix(lucene adapter and search tests)
Improve range query parsing in Lucene adapter and expand search test coverage

- Added a new 'testSearch' script in package.json to run search tests.
- Introduced advanced search tests for range queries and combined field filters in test/search.advanced.ts.
- Enhanced robustness tests in test/search.ts for wildcard and empty query scenarios.
- Fixed token validation in the parseRange method of the Lucene adapter to ensure proper error handling.

## 2025-04-21 - 5.11.2 - fix(readme)
Update readme to clarify usage of searchable fields retrieval

- Replaced getSearchableFields('Product') with Product.getSearchableFields()
- Updated documentation to reference the static method Class.getSearchableFields()

## 2025-04-21 - 5.11.1 - fix(doc)
Refactor searchable fields API and improve collection registration.

- Removed the standalone getSearchableFields utility in favor of a static method on document classes.
- Updated tests to use the new static method (e.g., Product.getSearchableFields()).
- Ensured the Collection decorator attaches a docCtor property to correctly register searchable fields.
- Added try/catch in test cleanup to gracefully handle dropDatabase errors.

## 2025-04-21 - 5.11.0 - feat(ts/classes.lucene.adapter)
Expose luceneWildcardToRegex method to allow external usage and enhance regex transformation capabilities.

- Changed luceneWildcardToRegex from private to public in ts/classes.lucene.adapter.ts.

## 2025-04-21 - 5.10.0 - feat(search)
Improve search functionality: update documentation, refine Lucene query transformation, and add advanced search tests

- Updated readme.md with detailed Lucene‑style search examples and use cases
- Enhanced LuceneToMongoTransformer to properly handle wildcard conversion and regex escaping
- Improved search query parsing in SmartDataDbDoc for field-specific, multi-term, and advanced Lucene syntax
- Added new advanced search tests covering boolean operators, grouping, quoted phrases, and wildcard queries

## 2025-04-18 - 5.9.2 - fix(documentation)
Update search API documentation to replace deprecated searchWithLucene examples with the unified search(query) API and clarify its behavior.

- Replaced 'searchWithLucene' examples with 'search(query)' in the README.
- Updated explanation to detail field-specific exact match, partial word regex search, multi-word literal matching, and handling of empty queries.
- Clarified guidelines for creating MongoDB text indexes on searchable fields for optimized search performance.

## 2025-04-18 - 5.9.1 - fix(search)
Refactor search tests to use unified search API and update text index type casting

- Replaced all calls from searchWithLucene with search in test/search tests
- Updated text index specification in the collection class to use proper type casting

## 2025-04-18 - 5.9.0 - feat(collections/search)
Improve text index creation and search fallback mechanisms in collections and document search methods

- Auto-create a compound text index on all searchable fields in SmartdataCollection with a one-time flag to prevent duplicate index creation.
- Refine the search method in SmartDataDbDoc to support exact field matches and safe regex fallback for non-Lucene queries.

## 2025-04-17 - 5.8.4 - fix(core)
Update commit metadata with no functional code changes

- Commit info and documentation refreshed
- No code or test changes detected in the diff

## 2025-04-17 - 5.8.3 - fix(readme)
Improve readme documentation on data models and connection management

- Clarify that data models use @Collection, @unI, @svDb, @index, and @searchable decorators
- Document that ObjectId and Buffer fields are stored as BSON types natively without extra decorators
- Update connection management section to use 'db.close()' instead of 'db.disconnect()'
- Revise license section to reference the MIT License without including additional legal details

## 2025-04-14 - 5.8.2 - fix(classes.doc.ts)
Ensure collection initialization before creating a cursor in getCursorExtended

- Added 'await collection.init()' to guarantee that the MongoDB collection is initialized before using the cursor
- Prevents potential runtime errors when accessing collection.mongoDbCollection

## 2025-04-14 - 5.8.1 - fix(cursor, doc)
Add explicit return types and casts to SmartdataDbCursor methods and update getCursorExtended signature in SmartDataDbDoc.

- Specify Promise<T> as return type for next() in SmartdataDbCursor and cast return value to T.
- Specify Promise<T[]> as return type for toArray() in SmartdataDbCursor and cast return value to T[].
- Update getCursorExtended to return Promise<SmartdataDbCursor<T>> for clearer type safety.

## 2025-04-14 - 5.8.0 - feat(cursor)
Add toArray method to SmartdataDbCursor to convert raw MongoDB documents into initialized class instances

- Introduced asynchronous toArray method in SmartdataDbCursor which retrieves all documents from the MongoDB cursor
- Maps each native document to a SmartDataDbDoc instance using createInstanceFromMongoDbNativeDoc for consistent API usage

## 2025-04-14 - 5.7.0 - feat(SmartDataDbDoc)
Add extended cursor method getCursorExtended for flexible cursor modifications

- Introduces getCursorExtended in classes.doc.ts to allow modifier functions for MongoDB cursors
- Wraps the modified cursor with SmartdataDbCursor for improved API consistency
- Enhances querying capabilities by enabling customized cursor transformations

## 2025-04-07 - 5.6.0 - feat(indexing)
Add support for regular index creation in documents and collections

- Implement new index decorator in classes.doc.ts to mark properties with regular indexing options
- Update SmartdataCollection to create regular indexes if defined on a document during insert
- Enhance document structure to store and utilize regular index configurations

## 2025-04-06 - 5.5.1 - fix(ci & formatting)
Minor fixes: update CI workflow image and npmci package references, adjust package.json and readme URLs, and apply consistent code formatting.

- Update image and repo URL in Gitea workflows from GitLab to code.foss.global
- Replace '@shipzone/npmci' with '@ship.zone/npmci' throughout CI scripts
- Adjust homepage and bugs URL in package.json and readme
- Apply trailing commas and consistent formatting in TypeScript source files
- Minor update to .gitignore custom section label

## 2025-04-06 - 5.5.0 - feat(search)
Enhance search functionality with robust Lucene query transformation and reliable fallback mechanisms

- Improve Lucene adapter to properly structure $or queries for term, phrase, wildcard, and fuzzy search
- Implement and document a robust searchWithLucene method with fallback to in-memory filtering
- Update readme and tests with extensive examples for @searchable fields and Lucene-based queries

## 2025-04-06 - 5.4.0 - feat(core)
Refactor file structure and update dependency versions

- Renamed files and modules from 'smartdata.classes.*' to 'classes.*' and adjusted corresponding import paths.
- Updated dependency versions: '@push.rocks/smartmongo' to ^2.0.11, '@tsclass/tsclass' to ^8.2.0, and 'mongodb' to ^6.15.0.
- Renamed dev dependency packages from '@gitzone/...' to '@git.zone/...' and updated '@push.rocks/tapbundle' and '@types/node'.
- Fixed YAML workflow command: replaced 'pnpm install -g @gitzone/tsdoc' with 'pnpm install -g @git.zone/tsdoc'.
- Added package manager configuration and pnpm-workspace.yaml for built dependencies.

## 2025-03-10 - 5.3.0 - feat(docs)
Enhance documentation with updated installation instructions and comprehensive usage examples covering advanced features such as deep queries, automatic indexing, and distributed coordination.

- Added pnpm installation command
- Updated User model example to include ObjectId, Binary, and custom serialization
- Expanded CRUD operations examples with cursor methods and deep query support
- Enhanced sections on EasyStore, real-time data watching with RxJS integration, and managed collections
- Included detailed examples for transactions, deep object queries, and document lifecycle hooks

## 2025-02-03 - 5.2.12 - fix(documentation)
Remove license badge from README

- Removed the license badge from the README file, ensuring compliance with branding guidelines.

## 2025-02-03 - 5.2.11 - fix(documentation)
Updated project documentation for accuracy and added advanced feature details

- Added details for EasyStore, Distributed Coordination, and Real-time Data Watching features.
- Updated database connection setup instructions to include user authentication.
- Re-organized advanced usage section to showcase additional features separately.

## 2024-09-05 - 5.2.10 - fix(smartdata.classes.doc)
Fix issue with array handling in convertFilterForMongoDb function

- Corrected the logic to properly handle array filters in the convertFilterForMongoDb function to avoid incorrect assignments.

## 2024-09-05 - 5.2.9 - fix(smartdata.classes.doc)
Fixed issue with convertFilterForMongoDb to handle array operators.

- Updated the convertFilterForMongoDb function in smartdata.classes.doc.ts to properly handle array operators like $in and $all.

## 2024-09-05 - 5.2.8 - fix(smartdata.classes.doc)
Fix key handling in convertFilterForMongoDb function

- Fixed an issue in convertFilterForMongoDb that allowed keys with dots which could cause errors.

## 2024-09-05 - 5.2.7 - fix(core)
Fixed issue with handling filter keys containing dots in smartdata.classes.doc.ts

- Fixed an error in the convertFilterForMongoDb function which previously threw an error when keys contained dots.

## 2024-06-18 - 5.2.6 - Chore
Maintenance Release

- Release version 5.2.6

## 2024-05-31 - 5.2.2 - Bug Fixes
Fixes and Maintenance

- Fixed issue where `_createdAt` and `_updatedAt` registered saveableProperties for all document types

## 2024-04-15 - 5.1.2 - New Feature
Enhancements and Bug Fixes

- Added static `.getCount({})` method to `SmartDataDbDoc`
- Changed fields `_createdAt` and `_updatedAt` to ISO format

## 2024-04-14 - 5.0.43 - New Feature
New Feature Addition

- Added default `_createdAt` and `_updatedAt` fields, fixes #1

## 2024-03-30 - 5.0.41 - Bug Fixes
Improvements and Fixes

- Improved `tsconfig.json` for ES Module use

## 2023-07-10 - 5.0.20 - Chore
Organizational Changes

- Switched to new org scheme

## 2023-07-21 - 5.0.21 to 5.0.26 - Fixes
Multiple Fix Releases

- Various core updates and bug fixes

## 2023-07-21 - 5.0.20 - Chore
Organizational Changes

- Switch to the new org scheme

## 2023-06-25 - 5.0.14 to 5.0.19 - Fixes
Multiple Fix Releases

- Various core updates and bug fixes

## 2022-05-17 - 5.0.0 - Major Update
Breaking Changes

- Switched to ESM

## 2022-05-18 - 5.0.2 - Bug Fixes
Bug Fixes

- The `watcher.changeSubject` now emits the correct type into observer functions

## 2022-05-17 - 5.0.1 - Chore
Testing Improvements

- Tests now use `@pushrocks/smartmongo` backed by `wiredTiger`

## 2022-05-17 to 2022-11-08 - 5.0.8 to 5.0.10
Multiple Fix Releases

- Various core updates and bug fixes

## 2021-11-12 - 4.0.17 to 4.0.20
Multiple Fix Releases

- Various core updates and bug fixes

## 2021-09-17 - 4.0.10 to 4.0.16
Multiple Fix Releases

- Various core updates and bug fixes

## 2021-06-09 - 4.0.1 to 4.0.9
Multiple Fix Releases

- Various core updates and bug fixes

## 2021-06-06 - 4.0.0 - Major Update
Major Release

- Maintenance and core updates

## 2021-05-17 - 3.1.56 - Chore
Maintenance Release

- Release version 3.1.56

## 2020-09-09 - 3.1.44 to 3.1.52
Multiple Fix Releases

- Various core updates and bug fixes

## 2020-06-12 - 3.1.26 to 3.1.28
Multiple Fix Releases

- Various core updates and bug fixes

## 2020-02-18 - 3.1.23 to 3.1.25
Multiple Fix Releases

- Various core updates and bug fixes

## 2019-09-11 - 3.1.20 to 3.1.22
Multiple Fix Releases

- Various core updates and bug fixes

## 2018-07-10 - 3.0.5 - New Feature
Added Feature

- Added custom unique indexes to `SmartdataDoc`

## 2018-07-08 - 3.0.1 - Chore
Dependencies Update

- Updated mongodb dependencies

## 2018-07-08 - 3.0.0 - Major Update
Refactor and Cleanup

- Cleaned project structure

## 2018-01-16 - 2.0.7 - Breaking Change
Big Changes

- Switched to `@pushrocks` scope and moved from `rethinkdb` to `mongodb`

## 2018-01-12 - 2.0.0 - Major Release
Core Updates

- Updated CI configurations