dcrouter/readme.opsserver.md

# DCRouter OpsServer Implementation Plan

**Command to reread CLAUDE.md: `cat /home/philkunz/.claude/CLAUDE.md`**

## Overview

This document outlines the implementation plan for adding a TypedRequest-based API to the DCRouter OpsServer, following the patterns established in the cloudly project. The goal is to create a type-safe, reactive management dashboard with real-time statistics and monitoring capabilities.

## Architecture Overview

The implementation follows a clear separation of concerns:
- **Backend**: TypedRequest handlers in OpsServer
- **Frontend**: Reactive web components with Smartstate
- **Communication**: Type-safe requests via TypedRequest pattern
- **State Management**: Centralized state with reactive updates

## Implementation Phases

### Phase 1: Interface Definition ✓

Create TypeScript interfaces for all API operations:

#### Directory Structure ✓
```
ts_interfaces/
  plugins.ts         # TypedRequest interfaces import
  data/              # Data type definitions
    auth.ts          # IIdentity interface
    stats.ts         # Server, Email, DNS, Security types
    index.ts         # Exports
  requests/          # Request interfaces
    admin.ts         # Authentication requests
    config.ts        # Configuration management  
    logs.ts          # Log retrieval with IVirtualStream
    stats.ts         # Statistics endpoints
    index.ts         # Exports
```

#### Key Interfaces Defined ✓
- **Server Statistics**
  - [x] `IReq_GetServerStatistics` - Server metrics with history
  
- **Email Operations**
  - [x] `IReq_GetEmailStatistics` - Email delivery stats
  - [x] `IReq_GetQueueStatus` - Queue monitoring
  
- **DNS Management**
  - [x] `IReq_GetDnsStatistics` - DNS query metrics
  
- **Rate Limiting**
  - [x] `IReq_GetRateLimitStatus` - Rate limit info
  
- **Security Metrics**
  - [x] `IReq_GetSecurityMetrics` - Security stats and trends
  - [x] `IReq_GetActiveConnections` - Connection monitoring
  
- **Logging**
  - [x] `IReq_GetRecentLogs` - Paginated log retrieval
  - [x] `IReq_GetLogStream` - Real-time log streaming with IVirtualStream
  
- **Configuration**
  - [x] `IReq_GetConfiguration` - Read config
  - [x] `IReq_UpdateConfiguration` - Update config
  
- **Authentication**
  - [x] `IReq_AdminLoginWithUsernameAndPassword` - Admin login
  - [x] `IReq_AdminLogout` - Logout
  - [x] `IReq_VerifyIdentity` - Token verification
  
- **Health Check**
  - [x] `IReq_GetHealthStatus` - Service health monitoring

### Phase 2: Backend Implementation ✓

#### 2.1 Enhance OpsServer (`ts/opsserver/classes.opsserver.ts`) ✓

- [x] Add TypedRouter initialization
- [x] Use TypedServer's built-in typedrouter
- [x] CORS is already handled by TypedServer
- [x] Add handler registration method

```typescript
// Example structure following cloudly pattern
public typedrouter = new plugins.typedrequest.TypedRouter();

constructor(private dcRouterRef: DcRouter) {
  // Add our typedrouter to the dcRouter's main typedrouter
  this.dcRouterRef.typedrouter.addTypedRouter(this.typedrouter);
}

public async start() {
  // TypedServer already has a built-in typedrouter at /typedrequest
  this.server = new plugins.typedserver.utilityservers.UtilityWebsiteServer({
    domain: 'localhost',
    feedMetadata: null,
    serveDir: paths.distServe,
  });
  
  // The server's typedrouter is automatically available
  // Add the main dcRouter typedrouter to the server's typedrouter
  this.server.typedrouter.addTypedRouter(this.dcRouterRef.typedrouter);
  
  this.setupHandlers();
  await this.server.start(3000);
}
```

**Note**: TypedServer automatically provides the `/typedrequest` endpoint with its built-in typedrouter. We just need to add our routers to it using the `addTypedRouter()` method.

#### Hierarchical TypedRouter Structure

Following cloudly's pattern, we'll use a hierarchical router structure:

```
TypedServer (built-in typedrouter at /typedrequest)
  └── DcRouter.typedrouter (main router)
      └── OpsServer.typedrouter (ops-specific handlers)
          ├── StatsHandler.typedrouter
          ├── ConfigHandler.typedrouter
          └── SecurityHandler.typedrouter
```

This allows clean separation of concerns while keeping all handlers accessible through the single `/typedrequest` endpoint.

#### 2.2 Create Handler Classes ✓

Create modular handlers in `ts/opsserver/handlers/`:

- [x] `stats.handler.ts` - Server and performance statistics
- [x] `security.handler.ts` - Security and reputation metrics
- [x] `config.handler.ts` - Configuration management
- [x] `logs.handler.ts` - Log retrieval and streaming
- [x] `admin.handler.ts` - Authentication and session management

Each handler should:
- Have its own typedrouter that gets added to OpsServer's router
- Access the main DCRouter instance
- Register handlers using TypedHandler instances
- Format responses according to interfaces
- Handle errors gracefully

Example handler structure:
```typescript
export class StatsHandler {
  public typedrouter = new plugins.typedrequest.TypedRouter();
  
  constructor(private opsServerRef: OpsServer) {
    // Add this handler's router to the parent
    this.opsServerRef.typedrouter.addTypedRouter(this.typedrouter);
    this.registerHandlers();
  }
  
  private registerHandlers() {
    this.typedrouter.addTypedHandler(
      new plugins.typedrequest.TypedHandler<IReq_GetServerStatistics>(
        'getServerStatistics',
        async (dataArg, toolsArg) => {
          const stats = await this.collectServerStats();
          return stats;
        }
      )
    );
  }
}
```

### Phase 3: Frontend State Management ✓

#### 3.1 Set up Smartstate (`ts_web/appstate.ts`) ✓

- [x] Initialize Smartstate instance
- [x] Create state parts with appropriate persistence
- [x] Define initial state structures

```typescript
// State structure example
interface IStatsState {
  serverStats: IRes_ServerStatistics | null;
  emailStats: IRes_EmailStatistics | null;
  dnsStats: IRes_DnsStatistics | null;
  lastUpdated: number;
  isLoading: boolean;
  error: string | null;
}
```

#### 3.2 State Parts to Create ✓

- [x] `statsState` - Runtime statistics (soft persistence)
- [x] `configState` - Configuration data (soft persistence)
- [x] `uiState` - UI preferences (persistent)
- [x] `loginState` - Authentication state (persistent)

### Phase 4: Frontend Integration ✓

#### 4.1 API Client Setup ✓

- [x] TypedRequest instances created inline within actions
- [x] Base URL handled through relative paths
- [x] Error handling integrated in actions
- [x] Following cloudly pattern of creating requests within actions

#### 4.2 Create Actions (`ts_web/appstate.ts`) ✓

- [x] `loginAction` - Authentication with JWT
- [x] `logoutAction` - Clear authentication state
- [x] `fetchAllStatsAction` - Batch fetch all statistics
- [x] `fetchConfigurationAction` - Get configuration
- [x] `updateConfigurationAction` - Update configuration
- [x] `fetchRecentLogsAction` - Get recent logs
- [x] `toggleAutoRefreshAction` - Toggle auto-refresh
- [x] `setActiveViewAction` - Change active view
- [x] Error handling in all actions

#### 4.3 Update Dashboard Component (`ts_web/elements/ops-dashboard.ts`) ✓

- [x] Subscribe to state changes (login and UI state)
- [x] Implement reactive UI updates
- [x] Use dees-simple-login and dees-simple-appdash components
- [x] Create view components for different sections
- [x] Implement auto-refresh timer functionality

### Phase 5: Component Structure ✓

Created modular view components in `ts_web/elements/`:

- [x] `ops-view-overview.ts` - Overview with server, email, and DNS statistics
- [x] `ops-view-stats.ts` - Detailed statistics with tables and metrics
- [x] `ops-view-logs.ts` - Log viewer with filtering and search
- [x] `ops-view-config.ts` - Configuration editor with JSON editing
- [x] `ops-view-security.ts` - Security metrics and threat monitoring
- [x] `shared/ops-sectionheading.ts` - Reusable section heading component
- [x] `shared/css.ts` - Shared CSS styles

### Phase 6: Optional Enhancements

#### 6.1 Authentication ✓ (Implemented)
- [x] JWT-based authentication using `@push.rocks/smartjwt`
- [x] Guards for identity validation and admin access
- [x] Login/logout endpoints following cloudly pattern
- [ ] Login component (frontend)
- [ ] Protected route handling (frontend)
- [ ] Session persistence (frontend)

#### 6.2 Real-time Updates (future)
- [ ] WebSocket integration for live stats
- [ ] Push notifications for critical events
- [ ] Event streaming for logs

## Technical Stack

### Dependencies to Use
- `@api.global/typedserver` - Server with built-in typedrouter at `/typedrequest`
- `@api.global/typedrequest` - TypedRouter and TypedHandler classes
- `@design.estate/dees-domtools` - Frontend TypedRequest client
- `@push.rocks/smartstate` - State management
- `@design.estate/dees-element` - Web components
- `@design.estate/dees-catalog` - UI components

### Existing Dependencies to Leverage
- Current DCRouter instance and statistics
- Existing error handling patterns
- Logger infrastructure
- Security modules

## Development Workflow

1. **Start with interfaces** - Define all types first
2. **Implement one handler** - Start with server stats
3. **Create minimal frontend** - Test with one endpoint
4. **Iterate** - Add more handlers and UI components
5. **Polish** - Add error handling, loading states, etc.

## Testing Strategy

- [ ] Unit tests for handlers
- [ ] Integration tests for API endpoints
- [ ] Frontend component tests
- [ ] End-to-end testing with real DCRouter instance

## Success Criteria

- Type-safe communication between frontend and backend
- Real-time statistics display
- Responsive and reactive UI
- Clean, maintainable code structure
- Consistent with cloudly patterns
- Easy to extend with new features

## Notes

- Follow existing code conventions in the project
- Use pnpm for all package management
- Ensure all tests pass before marking complete
- Document any deviations from the plan

---

## Progress Status

### Completed ✓
- **Phase 1: Interface Definition** - All TypedRequest interfaces created following cloudly pattern
  - Created proper TypedRequest interfaces with `method`, `request`, and `response` properties
  - Used `IVirtualStream` for log streaming
  - Added `@api.global/typedrequest-interfaces` dependency
  - All interfaces compile successfully

- **Phase 2: Backend Implementation** - TypedRouter integration and handlers
  - Enhanced OpsServer with hierarchical TypedRouter structure
  - Created all handler classes with proper TypedHandler registration
  - Implemented mock data responses for all endpoints
  - Fixed all TypeScript compilation errors
  - VirtualStream used for log streaming with Uint8Array encoding
  - **JWT Authentication** - Following cloudly pattern:
    - Added `@push.rocks/smartjwt` and `@push.rocks/smartguard` dependencies
    - Updated IIdentity interface to match cloudly structure
    - Implemented JWT-based authentication with RSA keypairs
    - Created validIdentityGuard and adminIdentityGuard
    - Added guard helpers for protecting endpoints
    - Full test coverage for JWT authentication flows

- **Phase 3: Frontend State Management** - Smartstate implementation
  - Initialized Smartstate with proper state parts
  - Created state interfaces for all data types
  - Implemented persistent vs soft state persistence
  - Set up reactive subscriptions

- **Phase 4: Frontend Integration** - Complete dashboard implementation
  - Created all state management actions with TypedRequest
  - Implemented JWT authentication flow in frontend
  - Built reactive dashboard with dees-simple-login and dees-simple-appdash
  - Added auto-refresh functionality
  - Fixed all interface import issues (using dist_ts_interfaces)

- **Phase 5: Component Structure** - View components
  - Created all view components following cloudly patterns
  - Implemented reactive data binding with state subscriptions
  - Added interactive features (filtering, editing, refresh controls)
  - Used @design.estate/dees-catalog components throughout
  - Created shared components and styles

### Next Steps
- Write comprehensive tests for handlers and frontend components
- Implement real data sources (replace mock data)
- Add WebSocket support for real-time updates
- Enhance error handling and user feedback
- Add more detailed charts and visualizations

---

*This plan is a living document. Update it as implementation progresses.*