For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install

pnpm add @modelprofile.com/flexharness

Node.js 24 or newer is required.

Overview

FlexHarness owns session state, audit messages, successful model context, permission decisions, event delivery, cancellation, and persistence. Model selection and tool execution remain application-defined extension points. The package depends on SmartAgent but does not expose AI SDK or SmartAI imports as part of its API.

Core Setup

import {
  FlexHarness,
  JsonFileFlexHarnessStore,
  type IFlexResolvedModel,
  type TFlexAgentToolSet,
} from '@modelprofile.com/flexharness';

interface IProjectScope {
  projectRoot: string;
}

const harness = new FlexHarness<IProjectScope>({
  scopeResolver: {
    async resolveScope(scopeId) {
      const project = await projectRegistry.get(scopeId);
      return {
        // Aliases that resolve to this same key share sessions and save ordering.
        storageKey: project.accountAndProjectKey,
        scope: { projectRoot: project.root },
      };
    },
  },
  modelResolver: {
    async resolveModel({ scope, modelHint, signal }): Promise<IFlexResolvedModel> {
      const configured = await modelRegistry.resolve({ scope, modelHint, signal });
      return {
        model: configured.model,
        identity: {
          provider: configured.providerId,
          model: configured.modelId,
          displayName: configured.label,
        },
        providerOptions: configured.providerOptions,
      };
    },
  },
  toolProvider: {
    async provideTools(context) {
      const tools: TFlexAgentToolSet = await createProjectTools({
        root: context.scope.projectRoot,
        signal: context.signal,
        requestPermission: context.requestPermission,
      });
      return {
        tools,
        close: async () => closeProjectTools(tools),
      };
    },
  },
  store: new JsonFileFlexHarnessStore({
    directory: '/var/lib/my-app/model-sessions',
  }),
  toolOutputLimits: {
    maxDepth: 12,
    maxBytes: 256 * 1024,
  },
  callbackLimits: {
    maxEvents: 10_000,
    maxOutputBytes: 1024 * 1024,
    maxParts: 2_000,
  },
});

modelRegistry, projectRegistry, createProjectTools, and closeProjectTools in this example are application-owned integrations. FlexHarness passes the same run AbortSignal to the model resolver and tool provider.

Sessions And Prompts

const session = await harness.createSession('project:billing', {
  title: 'Invoice import',
});

const result = await harness.prompt(
  'project:billing',
  session.sessionId,
  [
    { type: 'text', text: 'Extract the invoice totals.' },
    {
      type: 'file',
      data: invoicePdfBase64,
      mediaType: 'application/pdf',
      name: 'invoice.pdf',
    },
  ],
  { modelHint: 'document-model', maxSteps: 12 },
);

console.log(result.assistantMessage.parts);
console.log(result.usage);

TFlexPrompt is deliberately JSON-safe. It accepts a string or an ordered array of:

{ type: 'text', text }
{ type: 'image', data, mediaType?, name? }
{ type: 'file', data, mediaType, name? }

Attachment data is a string containing base64, a data URL, or a remote URL. Public input never requires Buffer or URL objects. Remote URL strings are converted only at the private SmartAgent invocation boundary.

Attachment payloads are never copied into public audit messages or events. Public attachment parts contain metadata only:

{
  type: 'attachment',
  partId: '...',
  attachmentType: 'file',
  source: 'inline-base64', // or data-url / remote-url
  sizeBytes: 48231,        // omitted when it cannot be determined
  mediaType: 'application/pdf',
  name: 'invoice.pdf',
}

When the turn succeeds, the original string remains only in private persisted model history so a later model turn can receive the attachment again. Failed, cancelled, resolver-failed, cleanup-failed, and persistence-failed turns do not add it to future context.

The main session methods are:

await harness.listSessions(scopeId);
await harness.createSession(scopeId, { sessionId, title });
await harness.getSession(scopeId, sessionId);
await harness.getMessages(scopeId, sessionId);
await harness.updateSession(scopeId, sessionId, { title: 'Renamed', archived: true });
await harness.updateSession(scopeId, sessionId, { title: null, archived: false });
await harness.deleteSession(scopeId, sessionId);
await harness.prompt(scopeId, sessionId, prompt, options);
await harness.abort(scopeId, sessionId, 'Cancelled by the user');
await harness.listPendingPermissions(scopeId, sessionId);
await harness.respondToPermission(scopeId, sessionId, permissionId, 'once');
await harness.dispose();

Only one run may be active in a session. Different sessions can run concurrently. The run ID and audit messages are reserved and persisted before asynchronous model or tool resolution begins.

updateSession() supports title replacement, explicit title clearing with null, and archive state through archived. Archived sessions expose archivedAt. Update and deletion are rejected while the session has an active run or pending permission. Deletion removes the complete persisted session, including messages, private model history, and remembered permission grants.

abort() returns true only while cancellation is still accepted. Terminal persistence is the run's commit point; once it starts, abort() returns false and the already-fixed terminal outcome completes while the session remains busy.

History And Audit Behavior

Successful model context is accumulated as:

Previous successful model history.
The normalized current user message.
SmartAgent's result messages.

A failed or cancelled prompt remains visible through getMessages(), with failed or cancelled status, but is not included in future model context. Model history is held only in the store snapshot and is not exposed by the session or message APIs.

Public audit history is safe to send to controllers: attachment parts contain source and size metadata, never inline base64, data URLs, or remote URL payloads. Private model history retains those values solely for subsequent model turns.

Sessions expose idle, running, waiting_permission, failed, and cancelled status. Persisted running and waiting_permission states normalize to idle after process restart; incomplete messages and parts normalize to cancelled.

Permissions

Tools request permission through the run-scoped provider context:

await context.requestPermission({
  kind: 'filesystem.write',
  description: 'Write generated files into the project',
  toolCallId,
  rememberKey: 'filesystem.write:project-output',
  metadata: { target: 'generated/' },
});

Pending requests are runtime-only and queryable with listPendingPermissions(). A controller answers with:

once: allow this request.
always: allow and remember the request's rememberKey for this session.
reject: reject the tool execution.

always is invalid when the request has no rememberKey. Remembered decisions are persisted before the waiting tool resolves. If persistence fails, the key is rolled back and the request remains pending so the response can be retried. Concurrent response attempts are serialized and exactly one successful response settles a request.

Tool Output Safety

FlexHarness wraps every provided tool execute method before SmartAgent receives it. Direct outputs and every AsyncIterable yield are converted into bounded JSON-safe values. Circular references, functions, symbols, bigint values, dates, URLs, and binary values receive deterministic descriptions or records. Thrown errors and iterator failures remain errors.

toolOutputLimits in the complete setup above bounds traversal depth and encoded bytes. The normalizer enforces its byte allowance incrementally: oversized strings are replaced before entering output, and arrays/objects stop reading entries once only truncation metadata fits.

Streaming callbacks use run-local synchronous state rather than one persistence promise per delta. Adjacent text and reasoning deltas coalesce. callbackLimits bounds callback events, accumulated output bytes, and part count; overflow aborts internally with FlexHarnessCallbackOverflowError and the turn is recorded as failed. Reservation and terminal finalization are the normal persistence checkpoints, with permission state changes as explicit additional checkpoints.

normalizeJsonValue() is also exported for integrations that need the same conversion independently.

Events

const unsubscribe = harness.subscribe((event) => {
  switch (event.type) {
    case 'part.delta':
      renderDelta(event.sessionId, event.messageId, event.partId, event.delta);
      break;
    case 'permission.requested':
      showPermission(event.request);
      break;
    case 'session.updated':
      renderSession(event.session);
      break;
    case 'session.deleted':
      removeSession(event.sessionId);
      break;
    case 'run.finished':
      markRunFinished(event.runId, event.status);
      break;
  }
});

unsubscribe();

Events are discriminated, sequenced, deeply immutable snapshots. Listener exceptions are isolated from runs and other listeners. Events contain public IDs and snapshots only; they do not expose the resolved scope object, storage key, model object, or provider options.

Stores

InMemoryFlexHarnessStore provides revision-based compare-and-swap behavior for tests and ephemeral processes.

JsonFileFlexHarnessStore stores one sha256(storageKey).json file per resolved key. It provides:

Snapshot schema version 1 and optimistic revisions.
Static process-wide queues shared by all store instances for the same absolute file.
Revision re-reads inside the queue before every save.
Atomic temporary-file write and rename.
Directory mode 0700 and file mode 0600, including existing paths.
Stale temporary-file cleanup and strict snapshot validation.

The JSON file store is explicitly not cross-process safe. Use a custom IFlexHarnessStore backed by a database or another cross-process CAS mechanism when several processes write the same storage key.

Store conflicts are surfaced as FlexHarnessStoreConflictError; malformed, wrong-schema, or non-JSON snapshots are surfaced as FlexHarnessStoreFormatError. FlexHarness does not merge conflicts.

Every harness mutation snapshots the persistent session state inside its per-storage queue. If the mutation itself or store.save() fails, the in-memory revision and sessions are restored before the queue settles. Runtime run controllers, pending permission objects, and queue identity are preserved. CAS conflicts therefore expose neither an uncommitted create/update/delete nor an automatic merge.

Shutdown

Model and tool resolution share the run signal. Synchronous throws are observed as resolver failures; the first failure aborts that signal and finalizes immediately without waiting for an unresponsive sibling. A detached tool provider that resolves later is observed and its handle is closed; disposal waits for that settlement and reports a late close failure.

Tool-handle close settles before a turn can be successful. Final persistence is attempted even when model execution or close fails. Cleanup failure prevents history append. If final persistence also fails, the stored reserved snapshot remains unchanged while the current in-memory audit is terminalized and terminal events are emitted exactly once.

dispose() is asynchronous and idempotent. It marks the harness closed, prevents operations waiting on persistence from reserving a run, aborts cancellable active runs, rejects pending permissions, waits for committing runs, all run finalizers, and state save tails, then clears listeners and loaded state caches. Multiple run or cleanup failures are reported through FlexHarnessRunError.

Cancellation is cooperative: model resolvers, tool providers, runners, tools, and cleanup functions must observe the supplied AbortSignal and settle their work. dispose() deliberately waits for owned work instead of abandoning resources. A process host that needs a hard shutdown deadline must enforce that deadline outside FlexHarness and terminate only the process it owns.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md

@modelprofile.com/flexharness

Issue Reporting and Security