Files

Juergen Kunz c9c35169fc fix(tests): Fix tests and documentation: adjust test server routes and expectations, add timeout/fallback routes, and refresh README

2025-10-20 13:41:03 +00:00

17 KiB

Raw Blame History

@push.rocks/webrequest

Modern, fetch-compatible web request library with intelligent HTTP caching, retry strategies, and advanced fault tolerance.

Features

🌐 Fetch-Compatible API - Drop-in replacement for native fetch() with enhanced features
💾 Intelligent HTTP Caching - Respects Cache-Control, ETag, Last-Modified, and Expires headers (RFC 7234)
🔄 Multiple Cache Strategies - network-first, cache-first, stale-while-revalidate, network-only, cache-only
🔁 Advanced Retry System - Configurable retry with exponential/linear/constant backoff
🎯 Request/Response Interceptors - Middleware pattern for transforming requests and responses
🚫 Request Deduplication - Automatically deduplicate simultaneous identical requests
📘 TypeScript Generics - Type-safe response parsing with webrequest.getJson<T>()
🛡️ Better Fault Tolerance - Multi-endpoint fallback with retry strategies
⏱️ Timeout Support - Configurable request timeouts with AbortController

Installation

pnpm install @push.rocks/webrequest
# or
npm install @push.rocks/webrequest

This package requires a modern JavaScript environment with ESM and TypeScript support.

Quick Start

Basic Fetch-Compatible Usage

import { webrequest } from '@push.rocks/webrequest';

// Use exactly like fetch()
const response = await webrequest('https://api.example.com/data');
const data = await response.json();

// With options (fetch-compatible + enhanced)
const response = await webrequest('https://api.example.com/data', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ key: 'value' }),
  timeout: 30000,
  retry: true
});

JSON Convenience Methods

import { webrequest } from '@push.rocks/webrequest';

// GET JSON with type safety
interface User {
  id: number;
  name: string;
  email: string;
}

const user = await webrequest.getJson<User>('https://api.example.com/user/1');
// user is typed as User

// POST JSON
const result = await webrequest.postJson('https://api.example.com/users', {
  name: 'John Doe',
  email: 'john@example.com'
});

// Other convenience methods
await webrequest.putJson(url, data);
await webrequest.patchJson(url, data);
await webrequest.deleteJson(url);

Cache Strategies

Network-First (Default)

Always fetch from network, fall back to cache on failure. Respects HTTP caching headers.

const data = await webrequest.getJson('https://api.example.com/data', {
  cacheStrategy: 'network-first'
});

Cache-First

Check cache first, only fetch from network if not cached or stale.

const data = await webrequest.getJson('https://api.example.com/data', {
  cacheStrategy: 'cache-first',
  cacheMaxAge: 60000 // 60 seconds
});

Stale-While-Revalidate

Return cached data immediately, update in background.

const data = await webrequest.getJson('https://api.example.com/data', {
  cacheStrategy: 'stale-while-revalidate'
});

Network-Only and Cache-Only

// Always fetch from network, never cache
const data = await webrequest.getJson(url, {
  cacheStrategy: 'network-only'
});

// Only use cache, never fetch from network
const data = await webrequest.getJson(url, {
  cacheStrategy: 'cache-only'
});

HTTP Header-Based Caching

The library automatically respects HTTP caching headers:

// Server returns: Cache-Control: max-age=3600, ETag: "abc123"
const response = await webrequest('https://api.example.com/data', {
  cacheStrategy: 'network-first'
});

// Subsequent requests automatically send:
// If-None-Match: "abc123"
// Server returns 304 Not Modified - cache is used

Custom Cache Keys

const response = await webrequest('https://api.example.com/search?q=test', {
  cacheStrategy: 'cache-first',
  cacheKey: (request) => {
    const url = new URL(request.url);
    return `search:${url.searchParams.get('q')}`;
  }
});

Retry Strategies

Basic Retry

const response = await webrequest('https://api.example.com/data', {
  retry: true // Uses defaults: 3 attempts, exponential backoff
});

Advanced Retry Configuration

const response = await webrequest('https://api.example.com/data', {
  retry: {
    maxAttempts: 5,
    backoff: 'exponential', // or 'linear', 'constant'
    initialDelay: 1000,      // 1 second
    maxDelay: 30000,         // 30 seconds
    retryOn: [408, 429, 500, 502, 503, 504], // Status codes to retry
    onRetry: (attempt, error, nextDelay) => {
      console.log(`Retry attempt ${attempt}, waiting ${nextDelay}ms`);
    }
  }
});

Multi-Endpoint Fallback

const response = await webrequest('https://api1.example.com/data', {
  fallbackUrls: [
    'https://api2.example.com/data',
    'https://api3.example.com/data'
  ],
  retry: {
    maxAttempts: 3,
    backoff: 'exponential'
  }
});

Request/Response Interceptors

Global Interceptors

import { webrequest } from '@push.rocks/webrequest';

// Add authentication to all requests
webrequest.addRequestInterceptor((request) => {
  const headers = new Headers(request.headers);
  headers.set('Authorization', `Bearer ${getToken()}`);
  return new Request(request, { headers });
});

// Log all responses
webrequest.addResponseInterceptor((response) => {
  console.log(`${response.status} ${response.url}`);
  return response;
});

// Handle errors globally
webrequest.addErrorInterceptor((error) => {
  console.error('Request failed:', error);
  throw error;
});

Per-Request Interceptors

const response = await webrequest('https://api.example.com/data', {
  interceptors: {
    request: [(req) => {
      console.log('Sending:', req.url);
      return req;
    }],
    response: [(res) => {
      console.log('Received:', res.status);
      return res;
    }],
    error: [(err) => {
      console.error('Error:', err);
      throw err;
    }]
  }
});

Request Deduplication

Automatically prevent duplicate simultaneous requests:

// Only one actual network request is made
const [res1, res2, res3] = await Promise.all([
  webrequest('https://api.example.com/data', { deduplicate: true }),
  webrequest('https://api.example.com/data', { deduplicate: true }),
  webrequest('https://api.example.com/data', { deduplicate: true }),
]);

// All three get the same response

Client API with Default Options

For more control, use WebrequestClient to set default options:

import { WebrequestClient } from '@push.rocks/webrequest';

const apiClient = new WebrequestClient({
  logging: true,
  timeout: 30000,
  cacheStrategy: 'network-first',
  retry: {
    maxAttempts: 3,
    backoff: 'exponential'
  }
});

// Add global interceptors to this client
apiClient.addRequestInterceptor((request) => {
  const headers = new Headers(request.headers);
  headers.set('X-API-Key', process.env.API_KEY);
  return new Request(request, { headers });
});

// All requests through this client use the configured defaults
const data = await apiClient.getJson('https://api.example.com/data');

// Standard fetch-compatible API also available
const response = await apiClient.request('https://api.example.com/data');

Advanced Features

Timeout

const response = await webrequest('https://api.example.com/data', {
  timeout: 5000 // 5 seconds
});

Custom Headers

const response = await webrequest('https://api.example.com/data', {
  headers: {
    'Authorization': 'Bearer token123',
    'X-Custom-Header': 'value'
  }
});

Cache Management

// Clear all cached responses
await webrequest.clearCache();

// Clear specific cache entry
await webrequest.clearCache('https://api.example.com/data');

TypeScript Support

Full TypeScript support with generics for type-safe responses:

interface ApiResponse<T> {
  data: T;
  status: 'success' | 'error';
  message?: string;
}

interface User {
  id: number;
  name: string;
  email: string;
}

// Fully typed response
const response = await webrequest.getJson<ApiResponse<User>>(
  'https://api.example.com/user/1'
);

// response.data.id, response.data.name, response.data.email are all typed

API Reference

Main Function

webrequest(input: string | Request | URL, options?: IWebrequestOptions): Promise<Response>

Convenience Methods

webrequest.getJson<T>(url: string, options?: IWebrequestOptions): Promise<T>
webrequest.postJson<T>(url: string, body: any, options?: IWebrequestOptions): Promise<T>
webrequest.putJson<T>(url: string, body: any, options?: IWebrequestOptions): Promise<T>
webrequest.patchJson<T>(url: string, body: any, options?: IWebrequestOptions): Promise<T>
webrequest.deleteJson<T>(url: string, options?: IWebrequestOptions): Promise<T>

Global Methods

webrequest.addRequestInterceptor(interceptor: TRequestInterceptor): void
webrequest.addResponseInterceptor(interceptor: TResponseInterceptor): void
webrequest.addErrorInterceptor(interceptor: TErrorInterceptor): void
webrequest.clearCache(url?: string): Promise<void>

Options Interface

interface IWebrequestOptions extends Omit<RequestInit, 'cache'> {
  // Standard fetch options
  method?: string;
  headers?: HeadersInit;
  body?: BodyInit;

  // Enhanced options
  cache?: 'default' | 'no-store' | 'reload' | 'no-cache' | 'force-cache' | 'only-if-cached';
  cacheStrategy?: 'network-first' | 'cache-first' | 'stale-while-revalidate' | 'network-only' | 'cache-only';
  cacheMaxAge?: number; // milliseconds
  cacheKey?: (request: Request) => string;

  retry?: boolean | IRetryOptions;
  fallbackUrls?: string[];
  timeout?: number; // milliseconds

  interceptors?: {
    request?: TRequestInterceptor[];
    response?: TResponseInterceptor[];
    error?: TErrorInterceptor[];
  };

  deduplicate?: boolean;
  logging?: boolean;
}

Migration from v3

Version 4.0 is a complete rewrite of @push.rocks/webrequest with breaking changes. The v3 API has been completely removed - all v3 code must be migrated to v4.

What's New in v4

Fetch-Compatible API: Drop-in replacement for native fetch() with enhanced features
Intelligent HTTP Caching: Respects Cache-Control, ETag, Last-Modified, and Expires headers (RFC 7234)
Multiple Cache Strategies: network-first, cache-first, stale-while-revalidate, network-only, cache-only
Advanced Retry System: Configurable retry with exponential/linear/constant backoff
Request/Response Interceptors: Middleware pattern for transforming requests and responses
Request Deduplication: Automatically deduplicate simultaneous identical requests
TypeScript Generics: Type-safe response parsing with webrequest.getJson<T>()
Better Fault Tolerance: Enhanced multi-endpoint fallback with retry strategies

Breaking Changes

Removed WebRequest class - Use webrequest function or WebrequestClient class instead
Cache API changed - Boolean useCache replaced with explicit cacheStrategy options
Multi-endpoint API changed - requestMultiEndpoint() replaced with fallbackUrls option

Migration Examples

Basic Fetch-Compatible Usage

// v3
import { WebRequest } from '@push.rocks/webrequest';
const client = new WebRequest();
const response = await client.request('https://api.example.com/data', {
  method: 'GET'
});
const data = await response.json();

// v4 - Fetch-compatible
import { webrequest } from '@push.rocks/webrequest';
const response = await webrequest('https://api.example.com/data');
const data = await response.json();

JSON Convenience Methods

// v3
const client = new WebRequest();
const data = await client.getJson('https://api.example.com/data', true);

// v4 - Function API
import { webrequest } from '@push.rocks/webrequest';
const data = await webrequest.getJson('https://api.example.com/data', {
  cacheStrategy: 'cache-first'
});

// v4 - Client API (similar to v3)
import { WebrequestClient } from '@push.rocks/webrequest';
const client = new WebrequestClient({ logging: true });
const data = await client.getJson('https://api.example.com/data', {
  cacheStrategy: 'cache-first'
});

Caching

// v3 - Boolean flag
const data = await client.getJson(url, true); // useCache = true

// v4 - Explicit strategies
const data = await webrequest.getJson(url, {
  cacheStrategy: 'cache-first',
  cacheMaxAge: 60000 // 60 seconds
});

// v4 - HTTP header-based caching (automatic)
const data = await webrequest.getJson(url, {
  cacheStrategy: 'network-first' // Respects Cache-Control headers
});

Multi-Endpoint Fallback

// v3
const client = new WebRequest();
const response = await client.requestMultiEndpoint(
  ['https://api1.example.com/data', 'https://api2.example.com/data'],
  { method: 'GET' }
);

// v4
import { webrequest } from '@push.rocks/webrequest';
const response = await webrequest('https://api1.example.com/data', {
  fallbackUrls: ['https://api2.example.com/data'],
  retry: {
    maxAttempts: 3,
    backoff: 'exponential'
  }
});

Timeout

// v3
const response = await client.request(url, {
  method: 'GET',
  timeoutMs: 30000
});

// v4
const response = await webrequest(url, {
  timeout: 30000 // milliseconds
});

Examples

Complete Example with All Features

import { webrequest } from '@push.rocks/webrequest';

async function fetchUserData(userId: string) {
  interface User {
    id: string;
    name: string;
    email: string;
  }

  const user = await webrequest.getJson<User>(
    `https://api.example.com/users/${userId}`,
    {
      // Caching
      cacheStrategy: 'stale-while-revalidate',
      cacheMaxAge: 300000, // 5 minutes

      // Retry
      retry: {
        maxAttempts: 3,
        backoff: 'exponential',
        retryOn: [500, 502, 503, 504]
      },

      // Fallback
      fallbackUrls: [
        'https://api-backup.example.com/users/${userId}'
      ],

      // Timeout
      timeout: 10000, // 10 seconds

      // Deduplication
      deduplicate: true,

      // Per-request interceptor
      interceptors: {
        request: [(req) => {
          console.log(`Fetching user ${userId}`);
          return req;
        }]
      }
    }
  );

  return user;
}

Building a Typed API Client

import { WebrequestClient } from '@push.rocks/webrequest';

class ApiClient {
  private client: WebrequestClient;

  constructor(private baseUrl: string, private apiKey: string) {
    this.client = new WebrequestClient({
      timeout: 30000,
      cacheStrategy: 'network-first',
      retry: {
        maxAttempts: 3,
        backoff: 'exponential'
      }
    });

    // Add auth interceptor
    this.client.addRequestInterceptor((request) => {
      const headers = new Headers(request.headers);
      headers.set('Authorization', `Bearer ${this.apiKey}`);
      headers.set('Content-Type', 'application/json');
      return new Request(request, { headers });
    });
  }

  async getUser(id: string): Promise<User> {
    return this.client.getJson<User>(`${this.baseUrl}/users/${id}`);
  }

  async createUser(data: CreateUserData): Promise<User> {
    return this.client.postJson<User>(`${this.baseUrl}/users`, data);
  }

  async updateUser(id: string, data: UpdateUserData): Promise<User> {
    return this.client.putJson<User>(`${this.baseUrl}/users/${id}`, data);
  }

  async deleteUser(id: string): Promise<void> {
    await this.client.deleteJson(`${this.baseUrl}/users/${id}`);
  }
}

// Usage
const api = new ApiClient('https://api.example.com', process.env.API_KEY);
const user = await api.getUser('123');

License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the license file within this repository.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

17 KiB Raw Blame History

@push.rocks/webrequest

Features

Installation

Quick Start

Basic Fetch-Compatible Usage

JSON Convenience Methods

Cache Strategies

Network-First (Default)

Cache-First

Stale-While-Revalidate

Network-Only and Cache-Only

HTTP Header-Based Caching

Custom Cache Keys

Retry Strategies

Basic Retry

Advanced Retry Configuration

Multi-Endpoint Fallback

Request/Response Interceptors

Global Interceptors

Per-Request Interceptors

Request Deduplication

Client API with Default Options

Advanced Features

Timeout

Custom Headers

Cache Management

TypeScript Support

API Reference

Main Function

Convenience Methods

Global Methods

Options Interface

Migration from v3

What's New in v4

Breaking Changes

Migration Examples

Basic Fetch-Compatible Usage

JSON Convenience Methods

Caching

Multi-Endpoint Fallback

Timeout

Examples

Complete Example with All Features

Building a Typed API Client

License and Legal Information

Trademarks

Company Information

17 KiB

Raw Blame History