77 lines
3.7 KiB
Markdown
77 lines
3.7 KiB
Markdown
# SmartData Project Overview
|
||
|
||
This document provides a high-level overview of the SmartData library (`@push.rocks/smartdata`), its architecture, core components, and key features—including recent enhancements to the search API.
|
||
|
||
## 1. Project Purpose
|
||
- A TypeScript‑first wrapper around MongoDB that supplies:
|
||
- Strongly‑typed document & collection classes
|
||
- Decorator‑based schema definition (no external schema files)
|
||
- Advanced search capabilities with Lucene‑style queries
|
||
- Built‑in support for real‑time data sync, distributed coordination, and key‑value EasyStore
|
||
|
||
## 2. Core Concepts & Components
|
||
- **SmartDataDb**: Manages the MongoDB connection, pooling, and initialization of collections.
|
||
- **SmartDataDbDoc**: Base class for all document models; provides CRUD, upsert, and cursor APIs.
|
||
- **Decorators**:
|
||
- `@Collection`: Associates a class with a MongoDB collection
|
||
- `@svDb()`: Marks a field as persisted to the DB
|
||
- `@unI()`: Marks a field as a unique index
|
||
- `@index()`: Adds a regular index
|
||
- `@searchable()`: Marks a field for inclusion in text searches or regex queries
|
||
- **SmartdataCollection**: Wraps a MongoDB collection; auto‑creates indexes based on decorators.
|
||
- **Lucene Adapter**: Parses a Lucene query string into an AST and transforms it to a MongoDB filter object.
|
||
- **EasyStore**: A simple, schema‑less key‑value store built on top of MongoDB for sharing ephemeral data.
|
||
- **Distributed Coordinator**: Leader election and task‑distribution API for building resilient, multi‑instance systems.
|
||
- **Watcher**: Listens to change streams for real‑time updates and integrates with RxJS.
|
||
|
||
## 3. Search API
|
||
SmartData provides a unified `.search(query[, opts])` method on all models with `@searchable()` fields:
|
||
|
||
- **Supported Syntax**:
|
||
1. Exact field:value (e.g. `field:Value`)
|
||
2. Quoted phrases (e.g. `"exact phrase"` or `'exact phrase'`)
|
||
3. Wildcards: `*` (zero or more chars) and `?` (single char)
|
||
4. Boolean operators: `AND`, `OR`, `NOT`
|
||
5. Grouping: parenthesis `(A OR B) AND C`
|
||
6. Range queries: `[num TO num]`, `{num TO num}`
|
||
7. Multi‑term unquoted: terms AND’d across all searchable fields
|
||
8. Empty query returns all documents
|
||
|
||
- **Fallback Mechanisms**:
|
||
1. Text index based `$text` search (if supported)
|
||
2. Field‑scoped and multi‑field regex queries
|
||
3. In‑memory filtering for complex or unsupported cases
|
||
|
||
### New Security & Extensibility Hooks
|
||
The `.search(query, opts?)` signature now accepts a `SearchOptions<T>` object:
|
||
```ts
|
||
interface SearchOptions<T> {
|
||
filter?: Record<string, any>; // Additional MongoDB filter AND‑merged
|
||
validate?: (doc: T) => boolean; // Post‑fetch hook to drop results
|
||
}
|
||
```
|
||
- **filter**: Enforces mandatory constraints (e.g. multi‑tenant isolation) directly in the Mongo query.
|
||
- **validate**: An async function that runs after fetching; return `false` to exclude a document.
|
||
|
||
## 4. Testing Strategy
|
||
- Unit tests in `test/test.search.ts` cover basic search functionality and new options:
|
||
- Exact, wildcard, phrase, boolean and grouping cases
|
||
- Implicit AND and mixed free‑term + field searches
|
||
- Edge cases (non‑searchable fields, quoted wildcards, no matches)
|
||
- `filter` and `validate` tests ensure security hooks work as intended
|
||
- Advanced search scenarios are covered in `test/test.search.advanced.ts`.
|
||
|
||
## 5. Usage Example
|
||
```ts
|
||
// Basic search
|
||
const prods = await Product.search('wireless earbuds');
|
||
|
||
// Scoped search (only your organization’s items)
|
||
const myItems = await Product.search('book', { filter: { ownerId } });
|
||
|
||
// Post‑search validation (only cheap items)
|
||
const cheapItems = await Product.search('', { validate: p => p.price < 50 });
|
||
```
|
||
|
||
---
|
||
Last updated: 2025-04-22 |