A modern, cross-platform HTTP/HTTPS request library for Node.js and browsers with a unified API, supporting form data, file uploads, JSON, binary data, streams, and unix sockets.

Install

# Using npm
npm install @push.rocks/smartrequest --save

# Using pnpm  
pnpm add @push.rocks/smartrequest

# Using yarn
yarn add @push.rocks/smartrequest

Key Features

🚀 Modern Fetch-like API - Familiar response methods (.json(), .text(), .arrayBuffer(), .stream())
🌐 Cross-Platform - Works in both Node.js and browsers with a unified API
🔌 Unix Socket Support - Connect to local services like Docker (Node.js only)
📦 Form Data & File Uploads - Built-in support for multipart/form-data
🔁 Pagination Support - Multiple strategies (offset, cursor, Link headers)
⚡ Keep-Alive Connections - Efficient connection pooling in Node.js
🛡️ TypeScript First - Full type safety and IntelliSense support
🎯 Zero Magic Defaults - Explicit configuration following fetch API principles
📡 Streaming Support - Handle large files and real-time data
🔧 Highly Configurable - Timeouts, retries, headers, and more

Architecture

SmartRequest v3.0 features a multi-layer architecture that provides consistent behavior across platforms:

Core Base - Abstract classes and unified types shared across implementations
Core Node - Node.js implementation using native http/https modules
Core Fetch - Browser implementation using the Fetch API
Core - Dynamic implementation selection based on environment
Client - High-level fluent API for everyday use

Usage

@push.rocks/smartrequest provides a clean, type-safe API inspired by the native fetch API but with additional features needed for modern applications.

Basic Usage

import { SmartRequest } from '@push.rocks/smartrequest';

// Simple GET request
async function fetchUserData(userId: number) {
  const response = await SmartRequest.create()
    .url(`https://jsonplaceholder.typicode.com/users/${userId}`)
    .get();

  // Use the fetch-like response API
  const userData = await response.json();
  console.log(userData); // The parsed JSON response
}

// POST request with JSON body
async function createPost(title: string, body: string, userId: number) {
  const response = await SmartRequest.create()
    .url('https://jsonplaceholder.typicode.com/posts')
    .json({ title, body, userId })
    .post();

  const createdPost = await response.json();
  console.log(createdPost); // The created post
}

Direct Core API Usage

For advanced use cases, you can use the Core API directly:

import { CoreRequest } from '@push.rocks/smartrequest';

async function directCoreRequest() {
  const request = new CoreRequest('https://api.example.com/data', {
    method: 'GET',
    headers: {
      'Accept': 'application/json'
    }
  });
  
  const response = await request.fire();
  const data = await response.json();
  return data;
}

Setting Headers and Query Parameters

import { SmartRequest } from '@push.rocks/smartrequest';

async function searchRepositories(query: string, perPage: number = 10) {
  const response = await SmartRequest.create()
    .url('https://api.github.com/search/repositories')
    .header('Accept', 'application/vnd.github.v3+json')
    .query({
      q: query,
      per_page: perPage.toString()
    })
    .get();

  const data = await response.json();
  return data.items;
}

Handling Timeouts and Retries

import { SmartRequest } from '@push.rocks/smartrequest';

async function fetchWithRetry(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .timeout(5000) // 5 seconds timeout
    .retry(3) // Retry up to 3 times on failure
    .get();

  return await response.json();
}

Setting Request Options

Use the options() method to set any request options supported by the underlying implementation:

import { SmartRequest } from '@push.rocks/smartrequest';

// Set various options
const response = await SmartRequest.create()
  .url('https://api.example.com/data')
  .options({
    keepAlive: true,           // Enable connection reuse (Node.js)
    timeout: 10000,            // 10 second timeout
    hardDataCuttingTimeout: 15000, // 15 second hard timeout
    // Platform-specific options are also supported
  })
  .get();

Working with Different Response Types

The API provides a fetch-like interface for handling different response types:

import { SmartRequest } from '@push.rocks/smartrequest';

// JSON response (default)
async function fetchJson(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .get();
  
  return await response.json(); // Parses JSON automatically
}

// Text response
async function fetchText(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .get();
  
  return await response.text(); // Returns response as string
}

// Binary data
async function downloadImage(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .accept('binary') // Optional: hints to server we want binary
    .get();

  const buffer = await response.arrayBuffer();
  return Buffer.from(buffer); // Convert ArrayBuffer to Buffer if needed
}

// Streaming response (Web Streams API)
async function streamLargeFile(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .get();

  // Get a web-style ReadableStream (works in both Node.js and browsers)
  const stream = response.stream();
  
  if (stream) {
    const reader = stream.getReader();
    
    try {
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        console.log(`Received ${value.length} bytes of data`);
      }
    } finally {
      reader.releaseLock();
    }
  }
}

// Node.js specific stream (only in Node.js environment)
async function streamWithNodeApi(url: string) {
  const response = await SmartRequest.create()
    .url(url)
    .get();

  // Only available in Node.js, throws error in browser
  const nodeStream = response.streamNode();
  
  nodeStream.on('data', (chunk) => {
    console.log(`Received ${chunk.length} bytes of data`);
  });

  return new Promise((resolve, reject) => {
    nodeStream.on('end', resolve);
    nodeStream.on('error', reject);
  });
}

Response Object Methods

The response object provides these methods:

json<T>(): Promise<T> - Parse response as JSON
text(): Promise<string> - Get response as text
arrayBuffer(): Promise<ArrayBuffer> - Get response as ArrayBuffer
stream(): ReadableStream<Uint8Array> | null - Get web-style ReadableStream (cross-platform)
streamNode(): NodeJS.ReadableStream - Get Node.js stream (Node.js only, throws in browser)
raw(): Response | http.IncomingMessage - Get the underlying platform response

Each body method can only be called once per response, similar to the fetch API.

Advanced Features

Form Data with File Uploads

import { SmartRequest } from '@push.rocks/smartrequest';
import * as fs from 'fs';

async function uploadMultipleFiles(files: Array<{name: string, path: string}>) {
  const formFields = files.map(file => ({
    name: 'files',
    value: fs.readFileSync(file.path),
    filename: file.name,
    contentType: 'application/octet-stream'
  }));

  const response = await SmartRequest.create()
    .url('https://api.example.com/upload')
    .formData(formFields)
    .post();
  
  return await response.json();
}

Unix Socket Support (Node.js only)

import { SmartRequest } from '@push.rocks/smartrequest';

// Connect to a service via Unix socket
async function queryViaUnixSocket() {
  const response = await SmartRequest.create()
    .url('http://unix:/var/run/docker.sock:/v1.24/containers/json')
    .get();
  
  return await response.json();
}

Pagination Support

The library includes built-in support for various pagination strategies:

import { SmartRequest } from '@push.rocks/smartrequest';

// Offset-based pagination (page & limit)
async function fetchAllUsers() {
  const client = SmartRequest.create()
    .url('https://api.example.com/users')
    .withOffsetPagination({
      pageParam: 'page',
      limitParam: 'limit',
      startPage: 1,
      pageSize: 20,
      totalPath: 'meta.total'
    });

  // Get first page with pagination info
  const firstPage = await client.getPaginated();
  console.log(`Found ${firstPage.items.length} users on first page`);
  console.log(`Has more pages: ${firstPage.hasNextPage}`);

  if (firstPage.hasNextPage) {
    // Get next page
    const secondPage = await firstPage.getNextPage();
    console.log(`Found ${secondPage.items.length} more users`);
  }

  // Or get all pages at once (use with caution for large datasets)
  const allUsers = await client.getAllPages();
  console.log(`Retrieved ${allUsers.length} users in total`);
}

// Cursor-based pagination
async function fetchAllPosts() {
  const allPosts = await SmartRequest.create()
    .url('https://api.example.com/posts')
    .withCursorPagination({
      cursorParam: 'cursor',
      cursorPath: 'meta.nextCursor',
      hasMorePath: 'meta.hasMore'
    })
    .getAllPages();

  console.log(`Retrieved ${allPosts.length} posts in total`);
}

// Link header-based pagination (GitHub API style)
async function fetchAllIssues(repo: string) {
  const paginatedResponse = await SmartRequest.create()
    .url(`https://api.github.com/repos/${repo}/issues`)
    .header('Accept', 'application/vnd.github.v3+json')
    .withLinkPagination()
    .getPaginated();

  return paginatedResponse.getAllPages();
}

Keep-Alive Connections (Node.js)

import { SmartRequest } from '@push.rocks/smartrequest';

// Enable keep-alive for better performance with multiple requests
async function performMultipleRequests() {
  // Note: keepAlive is NOT enabled by default
  const response1 = await SmartRequest.create()
    .url('https://api.example.com/endpoint1')
    .options({ keepAlive: true })
    .get();

  const response2 = await SmartRequest.create()
    .url('https://api.example.com/endpoint2')
    .options({ keepAlive: true })
    .get();

  // Connections are pooled and reused when keepAlive is enabled
  return [await response1.json(), await response2.json()];
}

Rate Limiting (429 Too Many Requests) Handling

The library includes built-in support for handling HTTP 429 (Too Many Requests) responses with intelligent backoff:

import { SmartRequest } from '@push.rocks/smartrequest';

// Simple usage - handle 429 with defaults
async function fetchWithRateLimitHandling() {
  const response = await SmartRequest.create()
    .url('https://api.example.com/data')
    .handle429Backoff()  // Automatically retry on 429
    .get();

  return await response.json();
}

// Advanced usage with custom configuration
async function fetchWithCustomRateLimiting() {
  const response = await SmartRequest.create()
    .url('https://api.example.com/data')
    .handle429Backoff({
      maxRetries: 5,              // Try up to 5 times (default: 3)
      respectRetryAfter: true,    // Honor Retry-After header (default: true)
      maxWaitTime: 30000,         // Max 30 seconds wait (default: 60000)
      fallbackDelay: 2000,        // 2s initial delay if no Retry-After (default: 1000)
      backoffFactor: 2,           // Exponential backoff multiplier (default: 2)
      onRateLimit: (attempt, waitTime) => {
        console.log(`Rate limited. Attempt ${attempt}, waiting ${waitTime}ms`);
      }
    })
    .get();

  return await response.json();
}

// Example: API client with rate limit handling
class RateLimitedApiClient {
  private async request(path: string) {
    return SmartRequest.create()
      .url(`https://api.example.com${path}`)
      .handle429Backoff({
        maxRetries: 3,
        onRateLimit: (attempt, waitTime) => {
          console.log(`API rate limit hit. Waiting ${waitTime}ms before retry ${attempt}`);
        }
      });
  }

  async fetchData(id: string) {
    const response = await this.request(`/data/${id}`).get();
    return response.json();
  }
}

The rate limiting feature:

Automatically detects 429 responses and retries with backoff
Respects the Retry-After header when present (supports both seconds and HTTP date formats)
Uses exponential backoff when no Retry-After header is provided
Allows custom callbacks for monitoring rate limit events
Caps maximum wait time to prevent excessive delays

Platform-Specific Features

Browser-Specific Options

When running in a browser, you can use browser-specific fetch options:

const response = await SmartRequest.create()
  .url('https://api.example.com/data')
  .options({
    credentials: 'include', // Include cookies
    mode: 'cors',          // CORS mode
    cache: 'no-cache',     // Cache mode
    referrerPolicy: 'no-referrer'
  })
  .get();

Node.js-Specific Options

When running in Node.js, you can use Node-specific options:

import { Agent } from 'https';

const response = await SmartRequest.create()
  .url('https://api.example.com/data')
  .options({
    agent: new Agent({ keepAlive: true }), // Custom agent
    socketPath: '/var/run/api.sock',       // Unix socket
  })
  .get();

Complete Example: Building a REST API Client

Here's a complete example of building a typed API client:

import { SmartRequest, type CoreResponse } from '@push.rocks/smartrequest';

interface User {
  id: number;
  name: string;
  email: string;
}

interface Post {
  id: number;
  title: string;
  body: string;
  userId: number;
}

class BlogApiClient {
  private baseUrl = 'https://jsonplaceholder.typicode.com';
  
  private async request(path: string) {
    return SmartRequest.create()
      .url(`${this.baseUrl}${path}`)
      .header('Accept', 'application/json');
  }
  
  async getUser(id: number): Promise<User> {
    const response = await this.request(`/users/${id}`).get();
    return response.json<User>();
  }
  
  async createPost(post: Omit<Post, 'id'>): Promise<Post> {
    const response = await this.request('/posts')
      .json(post)
      .post();
    return response.json<Post>();
  }
  
  async deletePost(id: number): Promise<void> {
    const response = await this.request(`/posts/${id}`).delete();
    
    if (!response.ok) {
      throw new Error(`Failed to delete post: ${response.statusText}`);
    }
  }
  
  async getAllPosts(userId?: number): Promise<Post[]> {
    const client = this.request('/posts');
    
    if (userId) {
      client.query({ userId: userId.toString() });
    }
    
    const response = await client.get();
    return response.json<Post[]>();
  }
}

// Usage
const api = new BlogApiClient();
const user = await api.getUser(1);
const posts = await api.getAllPosts(user.id);

Error Handling

import { SmartRequest } from '@push.rocks/smartrequest';

async function fetchWithErrorHandling(url: string) {
  try {
    const response = await SmartRequest.create()
      .url(url)
      .timeout(5000)
      .retry(2)
      .get();
    
    // Check if request was successful
    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }
    
    // Handle different content types
    const contentType = response.headers['content-type'];
    
    if (contentType?.includes('application/json')) {
      return await response.json();
    } else if (contentType?.includes('text/')) {
      return await response.text();
    } else {
      return await response.arrayBuffer();
    }
  } catch (error) {
    if (error.code === 'ECONNREFUSED') {
      console.error('Connection refused - is the server running?');
    } else if (error.code === 'ETIMEDOUT') {
      console.error('Request timed out');
    } else if (error.name === 'AbortError') {
      console.error('Request was aborted');
    } else {
      console.error('Request failed:', error.message);
    }
    throw error;
  }
}

Migrating from v2.x to v3.x

Version 3.0 brings significant architectural improvements and a more consistent API:

Legacy API Removed: The function-based API (getJson, postJson, etc.) has been removed. Use SmartRequest instead.
Unified Response API: All responses now use the same fetch-like interface regardless of platform.
Stream Changes: The stream() method now returns a web-style ReadableStream on all platforms. Use streamNode() for Node.js streams.
Cross-Platform by Default: The library now works in browsers out of the box with automatic platform detection.

License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the license file within this repository.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

readme.md

@push.rocks/smartrequest