SmartData Project Overview

This document provides a high-level overview of the SmartData library (@push.rocks/smartdata), its architecture, core components, and key features—including recent enhancements to the search API.

1. Project Purpose

A TypeScript‑first wrapper around MongoDB that supplies:
- Strongly‑typed document & collection classes
- Decorator‑based schema definition (no external schema files)
- Advanced search capabilities with Lucene‑style queries
- Built‑in support for real‑time data sync, distributed coordination, and key‑value EasyStore

2. Core Concepts & Components

SmartDataDb: Manages the MongoDB connection, pooling, and initialization of collections.
SmartDataDbDoc: Base class for all document models; provides CRUD, upsert, and cursor APIs.
Decorators:
- @Collection: Associates a class with a MongoDB collection
- @svDb(): Marks a field as persisted to the DB
- @unI(): Marks a field as a unique index
- @index(): Adds a regular index
- @searchable(): Marks a field for inclusion in text searches or regex queries
SmartdataCollection: Wraps a MongoDB collection; auto‑creates indexes based on decorators.
Lucene Adapter: Parses a Lucene query string into an AST and transforms it to a MongoDB filter object.
EasyStore: A simple, schema‑less key‑value store built on top of MongoDB for sharing ephemeral data.
Distributed Coordinator: Leader election and task‑distribution API for building resilient, multi‑instance systems.
Watcher: Listens to change streams for real‑time updates and integrates with RxJS.

3. Search API

SmartData provides a unified .search(query[, opts]) method on all models with @searchable() fields:

Supported Syntax:
1. Exact field:value (e.g. field:Value)
2. Quoted phrases (e.g. "exact phrase" or 'exact phrase')
3. Wildcards: * (zero or more chars) and ? (single char)
4. Boolean operators: AND, OR, NOT
5. Grouping: parenthesis (A OR B) AND C
6. Range queries: [num TO num], {num TO num}
7. Multi‑term unquoted: terms AND’d across all searchable fields
8. Empty query returns all documents
Fallback Mechanisms:
1. Text index based $text search (if supported)
2. Field‑scoped and multi‑field regex queries
3. In‑memory filtering for complex or unsupported cases

New Security & Extensibility Hooks

The .search(query, opts?) signature now accepts a SearchOptions<T> object:

interface SearchOptions<T> {
  filter?: Record<string, any>;        // Additional MongoDB filter AND‑merged
  validate?: (doc: T) => boolean;      // Post‑fetch hook to drop results
}

filter: Enforces mandatory constraints (e.g. multi‑tenant isolation) directly in the Mongo query.
validate: An async function that runs after fetching; return false to exclude a document.

4. Testing Strategy

Unit tests in test/test.search.ts cover basic search functionality and new options:
- Exact, wildcard, phrase, boolean and grouping cases
- Implicit AND and mixed free‑term + field searches
- Edge cases (non‑searchable fields, quoted wildcards, no matches)
- filter and validate tests ensure security hooks work as intended
Advanced search scenarios are covered in test/test.search.advanced.ts.

5. Usage Example

// Basic search
const prods = await Product.search('wireless earbuds');

// Scoped search (only your organization’s items)
const myItems = await Product.search('book', { filter: { ownerId } });

// Post‑search validation (only cheap items)
const cheapItems = await Product.search('', { validate: p => p.price < 50 });

Last updated: 2025-04-22

3.7 KiB Raw Blame History Unescape Escape