2023-07-27 16:16:37 +02:00
2020-10-25 22:12:38 +00:00
2024-04-14 18:19:14 +02:00

@push.rocks/smartsitemap

🗺️ Modern sitemap generation and management for TypeScript — Create XML sitemaps from RSS feeds, YAML configs, or programmatically. Perfect for SEO optimization and search engine indexing.

Install

# Using pnpm (recommended)
pnpm install @push.rocks/smartsitemap

# Using npm
npm install @push.rocks/smartsitemap

# Using yarn
yarn add @push.rocks/smartsitemap

Features

Multiple Sitemap Types

  • 📰 News Sitemaps — Generate Google News-compatible sitemaps from RSS feeds
  • 🌐 Website Sitemaps — Standard XML sitemaps with customizable update frequencies
  • 🔄 Dynamic Generation — Create sitemaps from various sources (RSS, YAML, arrays)

🚀 Developer-Friendly

  • 💪 Full TypeScript Support — Complete type definitions and interfaces
  • 🎯 Simple API — Intuitive methods for common sitemap operations
  • 📦 Zero Configuration — Works out of the box with sensible defaults
  • 🔍 Sitemap Parsing — Read and parse existing sitemaps from URLs

🎨 Flexible Input Sources

  • RSS/Atom feeds (URL or string)
  • YAML configuration files
  • URL info arrays
  • Article objects

Quick Start

import { SmartSitemap } from '@push.rocks/smartsitemap';

const sitemap = new SmartSitemap();

// Generate a news sitemap from RSS feed
const newsSitemap = await sitemap.createSitemapNewsFromFeedUrl(
  'https://yoursite.com/rss/'
);

console.log(newsSitemap);

Usage Examples

📰 Creating News Sitemaps

From RSS Feed URL

Perfect for blogs, news sites, and content platforms with RSS feeds:

import { SmartSitemap } from '@push.rocks/smartsitemap';

const sitemap = new SmartSitemap();

// Fetch RSS feed and generate Google News sitemap
const newsSitemap = await sitemap.createSitemapNewsFromFeedUrl(
  'https://coffee.link/rss/'
);

console.log(newsSitemap);

Output:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:news="http://www.google.com/schemas/sitemap-news/0.9">
  <url>
    <loc>https://coffee.link/article-url/</loc>
    <news:news>
      <news:publication>
        <news:language>en</news:language>
        <news:name>some name</news:name>
      </news:publication>
      <news:keywords></news:keywords>
      <news:publication_date>2025-11-18T16:57:15.000Z</news:publication_date>
      <news:title>Article Title</news:title>
    </news:news>
  </url>
</urlset>

From RSS Feed String

When you already have the RSS XML content:

import { SmartSitemap } from '@push.rocks/smartsitemap';

const rssFeedXml = `
  <?xml version="1.0" encoding="UTF-8"?>
  <rss version="2.0">
    <!-- Your RSS feed content -->
  </rss>
`;

const sitemap = new SmartSitemap();
const newsSitemap = await sitemap.createSitemapNewsFromAFeedStringArg(rssFeedXml);

console.log(newsSitemap);

From Article Array

Create news sitemaps programmatically from your CMS or database:

import { SmartSitemap } from '@push.rocks/smartsitemap';
import type { IArticle } from '@tsclass/tsclass';

const articles: IArticle[] = [
  {
    title: 'Breaking News Article',
    content: 'Article content here...',
    url: 'https://yoursite.com/breaking-news',
    // Add other required IArticle properties
  },
  // More articles...
];

const sitemap = new SmartSitemap();
const newsSitemap = await sitemap.createSitemapNewsFromArticleArray(articles);

console.log(newsSitemap);

🌐 Creating Website Sitemaps

From YAML Configuration

Define your sitemap structure in YAML for easy maintenance:

import { SmartSitemap } from '@push.rocks/smartsitemap';

const yamlConfig = `
daily:
  - https://yoursite.com/
  - https://yoursite.com/blog
  - https://yoursite.com/products
`;

const sitemap = new SmartSitemap();
const websiteSitemap = await sitemap.createSitemapFromYmlString(yamlConfig);

console.log(websiteSitemap);

Output:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yoursite.com/</loc>
    <lastmod>2025-11-19T12:00:00.000Z</lastmod>
    <changefreq>daily</changefreq>
  </url>
  <!-- More URLs... -->
</urlset>

From URL Info Array

Fine-grained control over each URL's properties:

import { SmartSitemap, IUrlInfo } from '@push.rocks/smartsitemap';

const urlInfos: IUrlInfo[] = [
  {
    url: 'https://yoursite.com/',
    timestamp: Date.now(),
    frequency: 'daily', // How often the page changes
  },
  {
    url: 'https://yoursite.com/about',
    timestamp: Date.now() - 86400000, // Yesterday
    frequency: 'monthly',
  },
  {
    url: 'https://yoursite.com/blog',
    timestamp: Date.now(),
    frequency: 'weekly',
  },
];

const sitemap = new SmartSitemap();
const websiteSitemap = await sitemap.createSitemapFromUrlInfoArray(urlInfos);

console.log(websiteSitemap);

Available Frequencies:

  • 'never' — Archived content
  • 'daily' — Updated every day
  • 'weekly' — Updated weekly
  • 'monthly' — Updated monthly
  • 'yearly' — Updated annually

🔍 Parsing Existing Sitemaps

Read and parse sitemaps from remote URLs:

import { SmartSitemap } from '@push.rocks/smartsitemap';

const sitemap = new SmartSitemap();

// Parse a sitemap from URL
const parsedSitemap = await sitemap.parseSitemapUrl(
  'https://www.theverge.com/sitemaps/google_news'
);

console.log(parsedSitemap);
// {
//   urlset: {
//     url: [
//       { loc: '...', lastmod: '...', changefreq: '...' },
//       // ...
//     ]
//   }
// }

Parse Sitemap String

If you already have the sitemap XML:

import { SmartSitemap } from '@push.rocks/smartsitemap';

const sitemapXml = `
  <?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <!-- Sitemap content -->
  </urlset>
`;

const sitemap = new SmartSitemap();
const parsed = await sitemap.parseSitemap(sitemapXml);

console.log(parsed);

Advanced Usage

Working with Lower-Level Classes

For more control, use the individual sitemap classes directly:

SitemapWebsite Class

import { SitemapWebsite, IUrlInfo } from '@push.rocks/smartsitemap';

const websiteSitemap = new SitemapWebsite();

// Add URLs one by one
websiteSitemap.addUrl({
  url: 'https://yoursite.com/page1',
  timestamp: Date.now(),
  frequency: 'daily',
});

websiteSitemap.addUrl({
  url: 'https://yoursite.com/page2',
  timestamp: Date.now(),
  frequency: 'weekly',
});

// Export to XML
const xml = websiteSitemap.exportSitemapXml();
console.log(xml);

SitemapNews Class

import { SitemapNews } from '@push.rocks/smartsitemap';

const newsSitemap = new SitemapNews({});

// Add from RSS feed URL
await newsSitemap.readAndAddFromRssFeedUrl('https://yoursite.com/rss');

// Or from RSS string
await newsSitemap.readAndAddFromRssFeedString(rssFeedXml);

// Or from articles
await newsSitemap.readAndParseArticles(articles);

// Export to XML
const xml = newsSitemap.exportSitemapXml();
console.log(xml);

Combining Multiple Sources

Create comprehensive sitemaps by combining different data sources:

import { SitemapWebsite } from '@push.rocks/smartsitemap';

const sitemap = new SitemapWebsite();

// Add static pages
sitemap.addUrl({
  url: 'https://yoursite.com/',
  timestamp: Date.now(),
  frequency: 'daily',
});

// Add dynamic content from database
const blogPosts = await fetchBlogPostsFromDB();
for (const post of blogPosts) {
  sitemap.addUrl({
    url: post.url,
    timestamp: post.updatedAt,
    frequency: 'weekly',
  });
}

// Add pages from CMS
const cmsPages = await fetchPagesFromCMS();
for (const page of cmsPages) {
  sitemap.addUrl({
    url: page.url,
    timestamp: page.lastModified,
    frequency: page.updateFrequency,
  });
}

const xml = sitemap.exportSitemapXml();

TypeScript Types

IUrlInfo

interface IUrlInfo {
  url: string;
  timestamp: number; // Unix timestamp in milliseconds
  frequency?: TUpdateFrequency;
}

type TUpdateFrequency = 'never' | 'daily' | 'weekly' | 'monthly' | 'yearly';

IParsedSiteMap

interface IParsedSiteMap {
  urlset: {
    url:
      | { loc: string; lastmod: string; changefreq: string }
      | { loc: string; lastmod: string; changefreq: string }[]
      | {
          loc: string;
          'news:news': {
            'news:publication': [];
            'news:keywords': string;
            'news:publication_date': string;
            'news:title': string;
          };
        }[];
  };
}

IRssItem

interface IRssItem {
  [key: string]: any;
  link?: string;
  guid?: string;
  title?: string;
  pubDate?: string;
  creator?: string;
  content?: string;
  isoDate?: string;
  categories?: string[];
  contentSnippet?: string;
  enclosure?: any;
}

Real-World Integration Examples

Express.js Server

import express from 'express';
import { SmartSitemap } from '@push.rocks/smartsitemap';

const app = express();
const sitemap = new SmartSitemap();

// Serve sitemap dynamically
app.get('/sitemap.xml', async (req, res) => {
  const urlInfos = await getUrlsFromDatabase();
  const xml = await sitemap.createSitemapFromUrlInfoArray(urlInfos);

  res.header('Content-Type', 'application/xml');
  res.send(xml);
});

// News sitemap from RSS
app.get('/news-sitemap.xml', async (req, res) => {
  const xml = await sitemap.createSitemapNewsFromFeedUrl(
    'https://yoursite.com/rss'
  );

  res.header('Content-Type', 'application/xml');
  res.send(xml);
});

app.listen(3000);

Static Site Generator

import { SmartSitemap } from '@push.rocks/smartsitemap';
import { writeFileSync } from 'fs';

async function generateSitemap() {
  const sitemap = new SmartSitemap();

  // Get all pages from your static site
  const pages = [
    { url: 'https://yoursite.com/', frequency: 'daily' as const },
    { url: 'https://yoursite.com/about', frequency: 'monthly' as const },
    // ... more pages
  ];

  const urlInfos = pages.map(page => ({
    url: page.url,
    timestamp: Date.now(),
    frequency: page.frequency,
  }));

  const xml = await sitemap.createSitemapFromUrlInfoArray(urlInfos);

  // Write to public directory
  writeFileSync('./public/sitemap.xml', xml);
  console.log('✅ Sitemap generated!');
}

generateSitemap();

Content Management System

import { SmartSitemap, IUrlInfo } from '@push.rocks/smartsitemap';

class CMSSitemapGenerator {
  private sitemap = new SmartSitemap();

  async generateCompleteSitemap(): Promise<string> {
    const urlInfos: IUrlInfo[] = [];

    // Add all content types
    urlInfos.push(...await this.getPageUrls());
    urlInfos.push(...await this.getBlogUrls());
    urlInfos.push(...await this.getProductUrls());

    return this.sitemap.createSitemapFromUrlInfoArray(urlInfos);
  }

  private async getPageUrls(): Promise<IUrlInfo[]> {
    const pages = await this.cms.getPages();
    return pages.map(page => ({
      url: page.url,
      timestamp: page.updatedAt,
      frequency: 'weekly',
    }));
  }

  private async getBlogUrls(): Promise<IUrlInfo[]> {
    const posts = await this.cms.getBlogPosts();
    return posts.map(post => ({
      url: post.url,
      timestamp: post.publishedAt,
      frequency: 'monthly',
    }));
  }

  private async getProductUrls(): Promise<IUrlInfo[]> {
    const products = await this.cms.getProducts();
    return products.map(product => ({
      url: product.url,
      timestamp: product.lastModified,
      frequency: 'daily',
    }));
  }
}

Best Practices

1. Update Frequencies

Choose appropriate update frequencies for better crawling:

// Homepage and main sections: daily
{ url: 'https://yoursite.com/', frequency: 'daily' }

// Blog posts: weekly
{ url: 'https://yoursite.com/blog/post', frequency: 'weekly' }

// Static pages: monthly or yearly
{ url: 'https://yoursite.com/about', frequency: 'monthly' }

// Archived content: never
{ url: 'https://yoursite.com/archive/2020', frequency: 'never' }

2. Sitemap Size Limits

Google recommends keeping sitemaps under 50,000 URLs and 50MB. Split large sitemaps:

import { SitemapWebsite } from '@push.rocks/smartsitemap';

async function generateSitemaps(allUrls: IUrlInfo[]) {
  const chunkSize = 50000;
  const chunks = [];

  for (let i = 0; i < allUrls.length; i += chunkSize) {
    chunks.push(allUrls.slice(i, i + chunkSize));
  }

  const sitemaps = await Promise.all(
    chunks.map(async (chunk, index) => {
      const sitemap = new SitemapWebsite();
      chunk.forEach(url => sitemap.addUrl(url));
      const xml = sitemap.exportSitemapXml();

      // Save as sitemap-1.xml, sitemap-2.xml, etc.
      return { filename: `sitemap-${index + 1}.xml`, content: xml };
    })
  );

  return sitemaps;
}

3. Cache Generated Sitemaps

Don't regenerate on every request:

import { SmartSitemap } from '@push.rocks/smartsitemap';

class SitemapCache {
  private cache: string | null = null;
  private lastGenerated: number = 0;
  private cacheDuration = 3600000; // 1 hour

  async getSitemap(): Promise<string> {
    const now = Date.now();

    if (!this.cache || now - this.lastGenerated > this.cacheDuration) {
      const sitemap = new SmartSitemap();
      const urls = await this.fetchUrls();
      this.cache = await sitemap.createSitemapFromUrlInfoArray(urls);
      this.lastGenerated = now;
    }

    return this.cache;
  }

  invalidate() {
    this.cache = null;
  }

  private async fetchUrls() {
    // Fetch from your data source
    return [];
  }
}

4. Submit to Search Engines

After generating your sitemap, submit it to search engines:

// In your robots.txt
`
User-agent: *
Allow: /
Sitemap: https://yoursite.com/sitemap.xml
Sitemap: https://yoursite.com/news-sitemap.xml
`

Or ping search engines programmatically:

async function submitSitemap(sitemapUrl: string) {
  const encodedUrl = encodeURIComponent(sitemapUrl);

  // Google
  await fetch(`https://www.google.com/ping?sitemap=${encodedUrl}`);

  // Bing
  await fetch(`https://www.bing.com/ping?sitemap=${encodedUrl}`);
}

submitSitemap('https://yoursite.com/sitemap.xml');

API Reference

SmartSitemap

Main class for sitemap operations.

Methods

createSitemapNewsFromFeedUrl(feedUrl: string): Promise<string>

  • Generate a Google News sitemap from an RSS/Atom feed URL
  • Returns XML string

createSitemapNewsFromAFeedStringArg(feedString: string): Promise<string>

  • Generate a Google News sitemap from RSS/Atom feed XML string
  • Returns XML string

createSitemapNewsFromArticleArray(articles: IArticle[]): Promise<string>

  • Generate a Google News sitemap from an array of article objects
  • Returns XML string

createSitemapFromYmlString(yamlString: string): Promise<string>

  • Generate a standard sitemap from YAML configuration
  • Returns XML string

createSitemapFromUrlInfoArray(urlInfos: IUrlInfo[]): Promise<string>

  • Generate a standard sitemap from URL info array
  • Returns XML string

parseSitemapUrl(url: string): Promise<IParsedSiteMap>

  • Parse a sitemap from a URL
  • Returns parsed sitemap object

parseSitemap(sitemapXml: string): Promise<IParsedSiteMap>

  • Parse a sitemap from XML string
  • Returns parsed sitemap object

SitemapWebsite

Lower-level class for website sitemaps.

Methods

addUrl(urlInfo: IUrlInfo): void

  • Add a URL to the sitemap

exportSitemapXml(): string

  • Export sitemap as XML string

SitemapNews

Lower-level class for news sitemaps.

Methods

readAndAddFromRssFeedUrl(url: string): Promise<void>

  • Add news items from RSS feed URL

readAndAddFromRssFeedString(feedString: string): Promise<void>

  • Add news items from RSS feed string

readAndParseArticles(articles: IArticle[]): Promise<void>

  • Add news items from article array

exportSitemapXml(): string

  • Export news sitemap as XML string

Why @push.rocks/smartsitemap?

  • Simple & Intuitive — Clean API that just works
  • Type-Safe — Full TypeScript support with complete type definitions
  • Flexible — Multiple input sources and output formats
  • Battle-Tested — Used in production across multiple projects
  • Modern — ESM-first, async/await, latest standards
  • SEO-Friendly — Generates valid XML sitemaps that search engines love

Part of the @push.rocks ecosystem:

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the license file within this repository.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

Company Information

Task Venture Capital GmbH Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

Description
A module for generating and managing sitemaps, supporting dynamic sitemap generation from feeds.
Readme 643 KiB
Languages
TypeScript 100%