fix(CodeFeed): Improve commit fetching concurrency and add tagged-only commit filtering along with updated documentation and tests

changelog.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 2025-04-25 - 1.7.1 - fix(CodeFeed)
+Improve commit fetching concurrency and add tagged-only commit filtering along with updated documentation and tests
+
+- Updated readme examples to clarify default and options usage, including caching and tagged-only filtering
+- Increased non-exclusive concurrency from 5 to 20 in fetchAllCommitsFromInstance
+- Added tagged-only filtering logic for both cached and non-cached commit results
+- Modified tests to enable tagged-only mode and require npm check
+
 ## 2025-04-25 - 1.7.0 - feat(core)
 Enhance commit fetching with caching, concurrency improvements, and dependency upgrades
 

readme.md

@@ -25,13 +25,30 @@ To get started, import the `CodeFeed` class from the module:
 import { CodeFeed } from '@foss.global/codefeed';
 ```
 
-Then, create an instance of `CodeFeed`. You'll need the base URL of your Gitea instance and optionally an API token if your repositories require authentication:
+Then, create an instance of `CodeFeed`. You'll need the base URL of your Gitea instance and optionally an API token if your repositories require authentication.
 
 ```typescript
-const codeFeed = new CodeFeed('https://your-gitea-instance-url.com', 'your-api-token');
+// default: fetch commits since 7 days ago, no caching or npm checks, include all commits
+const codeFeed = new CodeFeed(
+  'https://your-gitea-instance-url.com',
+  'your-api-token'
+);
+// with options: cache commits in-memory for 30 days, disable npm lookups, return only tagged commits
+const thirtyDays = 30 * 24 * 60 * 60 * 1000;
+const codeFeedStateful = new CodeFeed(
+  'https://your-gitea-instance-url.com',
+  'your-api-token',
+  undefined, // defaults to 7 days ago
+  {
+    enableCache: true,
+    cacheWindowMs: thirtyDays,
+    enableNpmCheck: false,
+    taggedOnly: true,
+  }
+);
 ```
 
-The constructor can also accept a `lastRunTimestamp` which indicates the last time a sync was performed. If not provided, it defaults to 24 hours prior to the current time.
+The constructor can also accept a `lastRunTimestamp` which indicates the last time a sync was performed. If not provided, it defaults to one week (7 days) prior to the current time.
 
 ### Fetching Commits
 
@@ -48,7 +65,12 @@ One of the core functionalities of CodeFeed is fetching commits from a Gitea ins
 })();
 ```
 
-This method scans all organizations and repositories, filters commits tagged within the last 24 hours, and enriches them with metadata like changelogs or npm publication status.
+This method scans all organizations and repositories, fetches all commits since the constructor’s `lastRunTimestamp` (default: one week ago), and enriches them with metadata like:
+- Git tags (to detect releases)
+- npm publication status (when enabled)
+- parsed changelog entries (when available)
+
+When `taggedOnly` is enabled, only commits marked as release tags are returned. When `enableCache` is enabled, previously fetched commits are kept in memory (up to `cacheWindowMs`), and only new commits are fetched on subsequent calls.
 
 Each commit object in the resulting array conforms to the `ICommitResult` interface, containing details such as:
 - `baseUrl`
@@ -112,15 +134,6 @@ const changelog = codeFeed.getChangelogForVersion('1.0.0');
 console.log('Changelog for version 1.0.0:', changelog);
 ```
 
-### Reacting to Repository Activity
-
-The method `hasNewActivity` checks for recent changes within an organization or a repository. This is particularly useful for setting up alerting systems or continuous integration triggers:
-
-```typescript
-const hasActivity = await codeFeed.hasNewActivity({ orgName: 'orgName', repoName: 'repoName' });
-console.log('New activity detected:', hasActivity);
-```
-
 ### Conclusion
 
 The `@foss.global/codefeed` module provides robust capabilities for extracting and managing feed data related to code developments in Gitea environments. Through systematic setup and leveraging API-driven methods, it becomes a valuable tool for developers aiming to keep track of software progress and changes efficiently. The integration hooks like changelog and npm verification further enrich its utility, offering consolidated insights into each commit's journey from codebase to published package.

test/test.ts

@@ -16,7 +16,7 @@ tap.test('first test', async () => {
     'https://code.foss.global',
     token,
     oneYearAgo,
-    { enableCache: true, cacheWindowMs: oneYearMs, enableNpmCheck: false }
+    { enableCache: true, cacheWindowMs: oneYearMs, enableNpmCheck: true, taggedOnly: true }
   );
   expect(testCodeFeed).toBeInstanceOf(codefeed.CodeFeed);
 });

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@foss.global/codefeed',
-  version: '1.7.0',
+  version: '1.7.1',
   description: 'The @foss.global/codefeed module is designed for generating feeds from Gitea repositories, enhancing development workflows by processing commit data and repository activities.'
 }

ts/index.ts

@@ -14,6 +14,8 @@ export class CodeFeed {
   private cache: plugins.interfaces.ICommitResult[] = [];
   // enable or disable npm publishedOnNpm checks (true by default)
   private enableNpmCheck: boolean = true;
+  // return only tagged commits (false by default)
+  private enableTaggedOnly: boolean = false;
 
   constructor(
     baseUrl: string,
@@ -23,6 +25,7 @@ export class CodeFeed {
       enableCache?: boolean;
       cacheWindowMs?: number;
       enableNpmCheck?: boolean;
+      taggedOnly?: boolean;
     }
   ) {
     this.baseUrl = baseUrl;
@@ -33,6 +36,7 @@ export class CodeFeed {
     this.enableCache = options?.enableCache ?? false;
     this.cacheWindowMs = options?.cacheWindowMs;
     this.enableNpmCheck = options?.enableNpmCheck ?? true;
+    this.enableTaggedOnly = options?.taggedOnly ?? false;
     this.cache = [];
     // npm registry instance for version lookups
     this.npmRegistry = new plugins.smartnpm.NpmRegistry();
@@ -45,7 +49,7 @@ export class CodeFeed {
   public async fetchAllCommitsFromInstance(): Promise<plugins.interfaces.ICommitResult[]> {
     // Controlled concurrency with AsyncExecutionStack
     const stack = new plugins.lik.AsyncExecutionStack();
-    stack.setNonExclusiveMaxConcurrency(5);
+    stack.setNonExclusiveMaxConcurrency(20);
     // determine since timestamp for this run (stateful caching)
     let effectiveSince = this.lastRunTimestamp;
     if (this.enableCache && this.cache.length > 0) {
@@ -176,9 +180,16 @@ export class CodeFeed {
       this.lastRunTimestamp = new Date().toISOString();
       // sort descending by timestamp
       this.cache.sort((a, b) => b.timestamp.localeCompare(a.timestamp));
+      // apply tagged-only filter if requested
+      if (this.enableTaggedOnly) {
+        return this.cache.filter((c) => c.tagged === true);
+      }
       return this.cache;
     }
-    // otherwise, return only newly fetched commits
+    // no caching: apply tagged-only filter if requested
+    if (this.enableTaggedOnly) {
+      return newResults.filter((c) => c.tagged === true);
+    }
     return newResults;
   }