The package also provides an isolated browser-side entry at @push.rocks/smartbrowser/web. Its LiveBrowserVideoRenderer presents native WebRTC video, while LiveBrowserCanvasRenderer displays SmartPuppeteer image frames in a canvas and maps local input back to transport-neutral browser commands without bundling Puppeteer or Node.js code into the web application. DevToolsFrontend embeds the official Chrome DevTools frontend using an authenticated transport supplied by the application.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install

Install the package with pnpm:

pnpm add @push.rocks/smartbrowser

The Node.js entry requires Node.js 22.12 or newer and a Chromium-compatible browser. It uses SmartPuppeteer 2.9, Puppeteer 25, and SmartPDF 5. SmartPuppeteer detects common local, CI, and container environments and configures Chromium accordingly.

Usage

@push.rocks/smartbrowser provides a high-level SmartBrowser class that wraps Puppeteer for common browser automation tasks: generating PDFs, capturing screenshots, and evaluating JavaScript on web pages.

Getting Started

Import and initialize a SmartBrowser instance:

import { SmartBrowser } from '@push.rocks/smartbrowser';

const smartBrowser = new SmartBrowser();
await smartBrowser.start();

Generating a PDF from a Webpage

Generate a full-page PDF from any URL. The result includes a Buffer with the PDF contents:

const pdfResult = await smartBrowser.pdfFromPage('https://example.com');
console.log(pdfResult.buffer); // PDF file buffer
console.log(pdfResult.name);   // Generated name identifier

The PDF generation is powered by @push.rocks/smartpdf, which is lazily initialized on first use. This means the SmartPdf server is only started when you actually call pdfFromPage(), keeping resource usage minimal.

Capturing a Screenshot of a Webpage

Capture a PNG screenshot of any webpage:

const screenshotResult = await smartBrowser.screenshotFromPage('https://example.com');
console.log(screenshotResult.buffer); // Screenshot buffer (PNG)
console.log(screenshotResult.name);   // Short unique identifier
console.log(screenshotResult.id);     // Identifier with extension

Evaluating JavaScript on a Webpage

Run arbitrary JavaScript inside a page context and retrieve the result:

const pageTitle = await smartBrowser.evaluateOnPage('https://example.com', async () => {
  return document.title;
});
console.log(pageTitle); // "Example Domain"

The evaluateOnPage method supports generic return types:

const metrics = await smartBrowser.evaluateOnPage<{ width: number; height: number }>(
  'https://example.com',
  async () => {
    return {
      width: window.innerWidth,
      height: window.innerHeight,
    };
  }
);
console.log(metrics.width, metrics.height);

Pages are automatically closed after evaluation, even if an error occurs.

Accessing the Underlying Puppeteer Browser

For advanced use cases, you can access the Puppeteer browser instance directly:

const page = await smartBrowser.headlessBrowser.newPage();
await page.goto('https://example.com');
// ... custom Puppeteer operations
await page.close();

You can also import the smartpuppeteer module directly for lower-level browser management:

import { smartpuppeteer } from '@push.rocks/smartbrowser';

const browser = await smartpuppeteer.getEnvAwareBrowserInstance();

Embedded Chrome DevTools

The backend export getDevToolsFrontendAssets() supplies serveDirectory, urlPrefix, entrypointUrl, and chromeMajor. Serve the directory at its full URL path under urlPrefix, with GET/HEAD access and same-origin framing enabled. Authenticate the separate CDP transport before admitting commands or delivering events. Keep the main application's framing policy separate. The assets include the official Chrome 151 frontend, lazy panels, translations, and third-party notices; no external DevTools website or debugging port is required.

import { DevToolsFrontend } from '@push.rocks/smartbrowser/web';

export async function mountDevTools(
  iframe: HTMLIFrameElement,
  entrypointUrl: string,
  send: (message: string) => Promise<void>,
  onClose: (reason: string) => void,
) {
  const frontend = new DevToolsFrontend({ iframe, entrypointUrl, send, onClose });
  await frontend.start();
  frontend.showPanel('console');
  return frontend;
}

Connect send to a selected-tab SmartPuppeteer openDevTools connection through your authenticated transport. Resolve send on bounded transport admission, before CDP execution finishes. Execute backend commands concurrently so Debugger.resume can run while an evaluation is paused. Feed every backend message to await frontend.dispatch(message) in order, including messages received during start(). Await dispatch before reading the next message to preserve backpressure. The protocol permits 1 MiB commands and 8 MiB responses/events. Oversized messages, stalled receivers, and invalid handshakes close the inspector explicitly.

showPanel accepts elements, console, network, or sources. close() releases the MessagePort, rejects pending deliveries, and clears the iframe. Applications must also close the backend attachment on view revocation, tab replacement, or transport loss. The iframe handshake is bound to its origin, window, and per-mount nonce; this does not replace backend authorization. Browser-wide operations and native host features remain subject to the managed browser's policy.

The pinned revision and SHA-256 resource manifest are shipped beside the frontend. To rebuild from official Chromium sources, run node scripts/build.devtools.mjs from the repository. This uses the pinned upstream GN/Ninja build and collects its shipping manifest. Building the package or installing it does not download a build toolchain. Frontend preferences use the smartbrowser.devtools. storage prefix.

Website exceptions arrive through renderer onError with code remote_page_error and an optional tabId. Display them in the inspected website's context; they do not indicate that the browser transport failed. Runtime failures retain remote_browser_error.

Live Browser Video

LiveBrowserVideoRenderer presents a SmartPuppeteer video stream directly in an HTMLVideoElement. It shares the canvas renderer's bounded input queue, held-input cleanup, viewport negotiation, and suspend/resume lifecycle. There is no JavaScript image decoding or canvas copy in the video path.

import { LiveBrowserVideoRenderer, type ILiveBrowserVideoClient } from '@push.rocks/smartbrowser/web';

export async function mountVideo(video: HTMLVideoElement, client: ILiveBrowserVideoClient) {
  const renderer = new LiveBrowserVideoRenderer({ video, client, resizeTarget: video.parentElement! });
  await renderer.start();
  return renderer;
}

The client provides the same state and input operations as ILiveBrowserCanvasClient, replacing frame acknowledgements with openVideoPeer(options), answerVideoPeer(negotiationId, description, options), closeVideoPeer(options), and getVideoStatistics(options). Every asynchronous operation receives an abort signal and must settle promptly when cancelled. The host authenticates and owns the peer through the viewer's lease; callers do not choose peer IDs or ICE credentials. Subscribe to state/error events without JPEG frames. Keep the signaling transport alive until stop() completes, and close the host lease even when transport cleanup fails.

start() installs listeners and begins negotiation without waiting for video, allowing applications to activate their signaling transport afterward. Input remains blocked until the browser presents a frame for the exact active tab, stream generation and viewport revision. Source changes clear the old picture immediately. suspend()/resume() close and negotiate a fresh peer. Hiding the document releases its peer; showing it opens a new one. Native video uses muted autoplay, inline playback, and object-fit: contain; pointer coordinates exclude the surrounding letterbox.

ICE configuration comes from the host offer. With the default empty ICE server list, media connects directly over a reachable LAN or VPN without a relay or external STUN service. Signaling continues to use the application's authenticated transport. The receiver requests the lowest supported jitter-buffer target; the browser retains its network-dependent minimum. Chromium owns decoding, congestion control and frame scheduling.

The Rust/NVENC backend also supplies state.videoSource. Forward that object intact: its rtpTimestampFloor prevents buffered frames from a previous viewport or navigation from enabling input, including when the dimensions have not changed. Odd native dimensions use frameAlignment: 2; GPU CSS compositing crops the encoder's alignment pixel and centers the visible viewport. Pointer mapping uses that same rectangle through resizes. No frame pixels pass through JavaScript. Mouse input includes the original DOM timestampMs; transport adapters must preserve it, while synthesized releases omit it.

getStatistics() includes the existing cumulative input/frame counters and an optional video snapshot: connection state, received bytes, bitrate, decoded/dropped frames, FPS, round-trip time, jitter-buffer delay, codec, decoder, selected candidate type and host encoder statistics when available. Sampling is sequential at one-second intervals. Hardware acceleration is reported only when the browser supplies it. Media negotiation and connection failures use video_negotiation_failed / video_connection_failed; applications should recover or close the view explicitly. There is no automatic JPEG fallback.

Live Browser Canvas

Import the browser-only renderer from @push.rocks/smartbrowser/web. The Node.js root entry is intentionally separate and must not be imported into a frontend bundle.

The renderer receives a caller-provided ILiveBrowserCanvasClient. An application adapter implements that interface using its authenticated transport and keeps the latest ILiveBrowserState available through getState(). The adapter must emit frame events with strictly increasing frame.sequence values for each onEvent() subscription/renderer run. SmartBrowser does not prescribe TypedSocket, WebSocket framing, base64 conversion, authentication, authorization, or session ownership.

import {
  LiveBrowserCanvasRenderer,
  type ILiveBrowserCanvasClient,
} from '@push.rocks/smartbrowser/web';

export async function mountLiveBrowser(client: ILiveBrowserCanvasClient) {
  const viewport = document.querySelector<HTMLElement>('[data-live-browser-viewport]')!;
  const canvas = viewport.querySelector<HTMLCanvasElement>('canvas')!;

  const renderer = new LiveBrowserCanvasRenderer({
    canvas,
    client,
    // Observe a stable CSS-sized element, not the canvas backing bitmap.
    resizeTarget: viewport,
    onError: (error) => console.error(error.code, error.message),
  });

  await renderer.start();
  return async () => {
    // Stop the renderer before closing its transport so cancellation reaches the adapter.
    await renderer.stop();
  };
}

A minimal host keeps CSS sizing independent from the canvas's encoded backing dimensions:

<div data-live-browser-viewport style="width: 100%; height: 600px; overflow: hidden">
  <canvas style="display: block; width: 100%; height: 100%"></canvas>
</div>

ILiveBrowserCanvasClient exposes the renderer-facing subset of the canonical SmartPuppeteer live-session API:

Cached state and events: getState() and onEvent()
Frame flow control: acknowledgeFrame()
Viewport synchronization: setViewport()
Input: dispatchMouse(), dispatchWheel(), dispatchKey(), and insertText()

Every asynchronous client method receives a required second ILiveBrowserCanvasOperationOptions argument containing signal: AbortSignal. Adapters must pass that signal through to their transport operation and reject promptly when it aborts. The renderer aborts it at the operation deadline and when the current run is suspended or stopped. There is no compatibility path for adapters that omit cancellation.

The renderer keeps at most one frame decoding and one newest frame queued. For an active run it validates the complete static frame protocol before requiring frame.sequence to be strictly greater than that run's high-water sequence. Duplicate or out-of-order frames are terminal frame_render_failed protocol errors and are not acknowledged. A resumed run creates a new subscription and resets the high-water sequence by design.

Every frame that passes validation is acknowledged immediately at receipt, before it is decoded, so upstream flow control no longer waits for local decode and presentation. Only the newest queued frame is decoded; a frame superseded while another frame decodes is skipped without decoding. The newest decoded bitmap is presented once per animation frame (a 16 ms timer stands in while the document is hidden), and the canvas backing store is reallocated only when the frame dimensions change. On a canvas without an existing rendering context the renderer claims an ImageBitmapRenderingContext and presents frames with transferFromImageBitmap(); a canvas that already owns a 2D context keeps the 2D drawing path, which is also what applications need when they read pixels back from the canvas.

The renderer makes exactly one acknowledgement attempt for each valid frame, at receipt, while its run remains active; a frame that is later dropped, found stale, superseded, or fails decoding is not acknowledged again. SmartPuppeteer may fulfill an acknowledgement with { accepted: false } for stale, retired, duplicate, or identity-mismatched frames. That result settles the renderer's local attempt without suspending or retrying, and it does not prove upstream retirement. Only an acknowledgement operation that throws, rejects, or times out suspends the run; reaching acknowledgement capacity does the same. Input identity always comes from the frame actually displayed, so tab, generation, viewport, and active-stream changes block stale clicks and keystrokes. A rejected non-timeout input dispatch clears the ambiguous frame and requires a newly rendered matching frame; a timeout suspends the run and requires explicit resume plus a fresh frame. Encoded frame.width and frame.height set the canvas backing bitmap, while pointer coordinates map through the displayed canvas rectangle into the frame's logical CSS viewport.

Call suspend() when the adapter observes a transport interruption, then call resume() after the adapter has established a fresh transport. Suspension aborts active work, removes listeners, drops queued input and pressed-input recovery, and clears state and the displayed frame. resume() creates a new renderer run generation, re-subscribes to events, reads state again, and keeps input blocked until a new matching frame is displayed. Old input and release transitions are never replayed into the new generation. Operation timeouts request this same suspended lifecycle, so one transient acknowledgement timeout does not permanently poison the renderer instance.

Input commands are dispatched in queue order with up to four commands in flight. Wheel input is sent immediately when capacity exists. While all slots are occupied, adjacent wheel events with the same document, viewport and modifiers accumulate their deltas; completion of an input operation releases capacity without waiting for an animation frame. Later input flushes accumulated wheel deltas first. Adjacent unpressed pointer moves may coalesce to the newest position; held-button movements remain discrete so drag paths are preserved. Under pressure only queued hover positions may be discarded. Wheel deltas never cross an input or modifier barrier. The queue allows 128 discrete commands and 32 coalescable commands. When no eligible hover can free a full queue, the renderer reports input_queue_capacity_exceeded and rejects the new command.

Malformed frame identities, metadata, viewport values, formats, MIME pairings, dimensions, pixel areas, byte lengths, and deterministic image decode or dimension-integrity failures are terminal to the current run. Decoded dimensions are checked before supersession and currentness, so a mismatched decoded bitmap remains terminal even when a newer frame arrived during decoding. These failures are reported as frame_render_failed and require an explicit resume(); the renderer does not retry them automatically. The renderer only schedules rendering, acknowledgements, viewport synchronization, and direct input. Navigation scheduling remains the responsibility of the server runtime and application adapter.

The renderer's fixed internal protocol ceilings are 16 pending frame acknowledgement operations; 4 in-flight input commands, 128 queued discrete input commands, and 32 queued coalescable input commands; frame dimensions of at most 12,288 on either axis and 8,294,400 pixels; encoded frame data of at most 34,226,176 bytes; and viewports of at most 4,096 x 4,096 CSS pixels, device scale factor 3, and 8,294,400 physical pixels (ceil(width * deviceScaleFactor) * ceil(height * deviceScaleFactor)).

Browser-native createImageBitmap() work is not abortable. After a wrapper timeout or run interruption, the renderer closes a bitmap that resolves late and waits for all prior raw decode jobs to settle before start() or resume() starts another run. Consequently, start() or resume() can remain pending indefinitely if the browser platform never settles a prior createImageBitmap() call.

Pointer and wheel input are captured from the canvas. Keyboard and best-effort compositionend input are captured from focusTarget, which defaults to the canvas and receives focus on pointer down. The renderer temporarily makes an unfocusable focus target focusable and restores its prior tabindex on stop. Applications with a dedicated text or IME control can call renderer.insertText(text) explicitly.

When resizeTarget is supplied, resize updates are deduplicated, serialized, and fenced by viewport revision. Sizes observed through ResizeObserver are debounced for about 100 ms (trailing), so a drag resize produces one setViewport() call for the final size; syncViewport() and window resize events measure immediately. Positive fractional dimensions are rounded to at least one CSS pixel, oversized dimensions are reduced proportionally to the 4,096 x 4,096 ceiling, and the device scale factor is clamped to 0.25 through 3 before being reduced further when necessary to stay within the physical-pixel ceiling. The target must have stable CSS dimensions that do not depend on canvas.width or canvas.height; this prevents intrinsic canvas updates from causing resize feedback.

Shared browser hosts may return { viewport, viewportRevision } from setViewport() to acknowledge the effective viewport after combining viewer preferences. The renderer accepts a smaller viewport or an unchanged revision, remembers its own requested size separately, and enables input only when a current displayed frame matches the accepted revision. A frame received before the viewport response can satisfy that fence. A newer authoritative viewport revision supersedes a pending shared resize. Existing clients returning void retain the single-view contract: apply the requested size and advance the viewport revision. Input remains blocked while the request settles; a still-current image stays visible when the aggregate size does not change.

ILiveBrowserCanvasRendererOptions supports:

canvas and client: required rendering and transport-adapter dependencies.
focusTarget: optional keyboard and composition event target; defaults to canvas.
resizeTarget: optional stable CSS-sized element observed for remote viewport updates.
getDeviceScaleFactor: optional scale provider; defaults to window.devicePixelRatio and is useful when the application controls remote scaling explicitly.
operationTimeoutMs: deadline for client acknowledgements, viewport updates, and input operations; defaults to 10 seconds. Reaching it aborts the operation signal and suspends the run.
frameDecodeTimeoutMs: image decode wrapper deadline; defaults to operationTimeoutMs. Reaching it suspends the run and closes the decoded bitmap if it completes late, but cannot force the browser's native decode job to settle.
onError: receives typed ILiveBrowserCanvasError values without interrupting renderer cleanup.
onFrameRendered: called after a current frame has been drawn.

The renderer exposes start(), suspend(), resume(), stop(), insertText(), syncViewport(), getStatistics(), and the isRunning and isSuspended getters. getStatistics() returns an ILiveBrowserCanvasRendererStatistics snapshot with cumulative framesReceived, framesDecoded, framesSkipped, inputCommandsEnqueued (every accepted input submission, including ones later merged), inputCommandsCoalesced (submissions merged into another command or dropped under pressure), the live inputCommandsInFlight, and lastInputRoundTripMs for the most recently completed input dispatch. lastInputQueueMs reports its local queue wait and lastInputTotalMs includes that wait through the operation reply. These values do not measure visible feedback. start() begins a stopped renderer, while resume() is required for a suspended renderer. syncViewport() requests a fresh measurement of the configured resizeTarget; it is a no-op when no target is configured or the renderer is not running. An explicitly stopped renderer can be started again. The /web entry also exports ILiveBrowserCanvasClient, ILiveBrowserCanvasOperationOptions, ILiveBrowserCanvasRendererOptions, ILiveBrowserCanvasRendererStatistics, ILiveBrowserCanvasError, TLiveBrowserCanvasErrorCode, and the canonical SmartPuppeteer live-browser contract types.

LiveBrowserCanvasRenderer is not a security boundary. The application adapter must authenticate viewers, authorize control, restrict navigation, enforce browser-session ownership, and apply network/egress policy before forwarding commands. The renderer displays webpage viewport pixels only. It does not provide native Chrome UI, audio, extensions, file transfer, clipboard, camera, microphone, or touch emulation.

Shutting Down

Always stop the browser instance when done to free resources:

await smartBrowser.stop();

This cleanly shuts down the SmartPdf server (if it was initialized) and closes the browser.

Full Example

import { SmartBrowser } from '@push.rocks/smartbrowser';

async function main() {
  const smartBrowser = new SmartBrowser();
  await smartBrowser.start();

  // Generate a PDF
  const pdfResult = await smartBrowser.pdfFromPage('https://example.com');
  console.log('PDF size:', pdfResult.buffer.length, 'bytes');

  // Take a screenshot
  const screenshot = await smartBrowser.screenshotFromPage('https://example.com');
  console.log('Screenshot size:', screenshot.buffer.length, 'bytes');

  // Evaluate JavaScript
  const title = await smartBrowser.evaluateOnPage('https://example.com', async () => {
    return document.title;
  });
  console.log('Page title:', title);

  await smartBrowser.stop();
}

main();

Native video retains its peer across document navigation and viewport resizing. Input waits for current document state and a video presentation matching the accepted viewport proportions. The native host replaces capture tracks when dimensions change, preserving the negotiated connection. Pending page dialogs block renderer input; the embedding UI owns the dialog response. A client can reject obsolete input with an Error whose code is stale_input; the renderer retires that obsolete input without treating navigation as a connection failure. dialog_pending retains held keys and buttons, and their release is sent after the dialog closes.

Video statistics sample the receiver independently of remote sender requests. presentedFramesPerSecond measures frames submitted to the compositor; framesPerSecond retains the WebRTC decoded-frame rate. sampledAt uses the viewer's performance.now() clock, allowing consumers to distinguish an old sample from a fresh zero. Optional captureToDisplayMs estimates capture-to-presentation latency from WebRTC frame metadata, while frameAgeMs reports the age of the latest presented capture. Interval measurements include decodeMs, processingMs, jitterBufferMs, jitterBufferTargetMs, and jitterBufferMinimumMs. These measurements can overlap and should not be summed. Network roundTripMs is separate from display latency. Codec, transport protocol, and sender acceleration information remain available.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md

@push.rocks/smartbrowser