mailer/test/readme.testmigration.md

# Test Migration Tracker: dcrouter → mailer (Deno)

This document tracks the migration of SMTP/mail tests from `../dcrouter` (Node.js/tap) to `./test` (Deno native).

## Source & Destination

**Source**: `/mnt/data/lossless/serve.zone/dcrouter/test/`
- Framework: @git.zone/tstest/tapbundle (Node.js)
- Test files: ~100+ test files
- Assertions: expect().toBeTruthy(), expect().toEqual()
- Network: Node.js net module

**Destination**: `/mnt/data/lossless/serve.zone/mailer/test/`
- Framework: Deno.test (native)
- Assertions: assert(), assertEquals(), assertMatch() from @std/assert
- Network: Deno.connect(), Deno.connectTls()

## Migration Status

### Legend
- ✅ **Ported** - Test migrated and passing
- 🔄 **In Progress** - Currently being migrated
- 📋 **Planned** - Identified for migration
- ⏸️ **Deferred** - Low priority, will port later
- ❌ **Skipped** - Not applicable or obsolete

---

## Test Categories

### 1. Connection Management (CM)

Tests for SMTP connection handling, TLS support, and connection lifecycle.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| **CM-01** | (dcrouter TLS tests) | `test/suite/smtpserver_connection/test.cm-01.tls-connection.test.ts` | **✅ Ported** | 8/8 | STARTTLS capability, TLS upgrade, certificate handling |
| CM-02 | TBD | `test/suite/smtpserver_connection/test.cm-02.multiple-connections.test.ts` | 📋 Planned | - | Concurrent connection testing |
| CM-03 | TBD | `test/suite/smtpserver_connection/test.cm-03.connection-timeout.test.ts` | 📋 Planned | - | Timeout and idle connection handling |
| CM-06 | TBD | `test/suite/smtpserver_connection/test.cm-06.starttls-upgrade.test.ts` | 📋 Planned | - | Full STARTTLS lifecycle |
| CM-10 | TBD | `test/suite/smtpserver_connection/test.cm-10.plain-connection.test.ts` | ⏸️ Deferred | - | Basic plain connection (covered by CMD tests) |

---

### 2. SMTP Commands (CMD)

Tests for SMTP protocol command implementation.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| **CMD-01** | (dcrouter EHLO tests) | `test/suite/smtpserver_commands/test.cmd-01.ehlo-command.test.ts` | **✅ Ported** | 5/5 | EHLO capabilities, hostname validation, pipelining |
| **CMD-02** | (dcrouter MAIL FROM tests) | `test/suite/smtpserver_commands/test.cmd-02.mail-from.test.ts` | **✅ Ported** | 6/6 | Sender validation, SIZE parameter, sequence enforcement |
| **CMD-03** | (dcrouter RCPT TO tests) | `test/suite/smtpserver_commands/test.cmd-03.rcpt-to.test.ts` | **✅ Ported** | 7/7 | Recipient validation, multiple recipients, RSET |
| **CMD-04** | (dcrouter DATA tests) | `test/suite/smtpserver_commands/test.cmd-04.data-command.test.ts` | **✅ Ported** | 7/7 | Email content, dot-stuffing, large messages |
| **CMD-06** | (dcrouter RSET tests) | `test/suite/smtpserver_commands/test.cmd-06.rset-command.test.ts` | **✅ Ported** | 8/8 | Transaction reset, recipient clearing, idempotent |
| **CMD-13** | (dcrouter QUIT tests) | `test/suite/smtpserver_commands/test.cmd-13.quit-command.test.ts` | **✅ Ported** | 7/7 | Graceful disconnect, idempotent behavior |

---

### 3. Email Processing (EP)

Tests for email content handling, parsing, and delivery.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| **EP-01** | (dcrouter EP-01 tests) | `test/suite/smtpserver_email-processing/test.ep-01.basic-email-sending.test.ts` | **✅ Ported** | 7/7 | Complete SMTP flow, MIME, HTML, custom headers, minimal email |
| EP-02 | TBD | `test/suite/smtpserver_email-processing/test.ep-02.invalid-address.test.ts` | 📋 Planned | - | Email address validation |
| EP-04 | TBD | `test/suite/smtpserver_email-processing/test.ep-04.large-email.test.ts` | 📋 Planned | - | Large attachment handling |
| EP-05 | TBD | `test/suite/smtpserver_email-processing/test.ep-05.mime-handling.test.ts` | 📋 Planned | - | MIME multipart messages |

---

### 4. Security (SEC)

Tests for security features and protections.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| SEC-01 | TBD | `test/suite/smtpserver_security/test.sec-01.authentication.test.ts` | 📋 Planned | - | SMTP AUTH mechanisms |
| SEC-03 | TBD | `test/suite/smtpserver_security/test.sec-03.dkim.test.ts` | 📋 Planned | - | DKIM signing/verification |
| SEC-04 | TBD | `test/suite/smtpserver_security/test.sec-04.spf.test.ts` | 📋 Planned | - | SPF record checking |
| **SEC-06** | (dcrouter SEC-06 tests) | `test/suite/smtpserver_security/test.sec-06.ip-reputation.test.ts` | **✅ Ported** | 7/7 | IP reputation infrastructure, legitimate traffic acceptance |
| SEC-08 | TBD | `test/suite/smtpserver_security/test.sec-08.rate-limiting.test.ts` | 📋 Planned | - | Connection/command rate limits |
| SEC-10 | TBD | `test/suite/smtpserver_security/test.sec-10.header-injection.test.ts` | 📋 Planned | - | Header injection prevention |

---

### 5. Error Handling (ERR)

Tests for proper error handling and recovery.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| **ERR-01** | (dcrouter ERR-01 tests) | `test/suite/smtpserver_error-handling/test.err-01.syntax-errors.test.ts` | **✅ Ported** | 10/10 | Invalid commands, missing brackets, wrong sequences, long commands, malformed addresses |
| **ERR-02** | (dcrouter ERR-02 tests) | `test/suite/smtpserver_error-handling/test.err-02.invalid-sequence.test.ts` | **✅ Ported** | 10/10 | MAIL before EHLO, RCPT before MAIL, DATA before RCPT, multiple EHLO, commands after QUIT, sequence recovery |
| ERR-05 | TBD | `test/suite/smtpserver_error-handling/test.err-05.resource-exhaustion.test.ts` | 📋 Planned | - | Memory/connection limits |
| ERR-07 | TBD | `test/suite/smtpserver_error-handling/test.err-07.exception-handling.test.ts` | 📋 Planned | - | Unexpected errors, crashes |

---

### 6. Performance (PERF)

Tests for server performance under load.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| PERF-01 | TBD | `test/suite/smtpserver_performance/test.perf-01.throughput.test.ts` | ⏸️ Deferred | - | Message throughput testing |
| PERF-02 | TBD | `test/suite/smtpserver_performance/test.perf-02.concurrent.test.ts` | ⏸️ Deferred | - | Concurrent connection handling |

---

### 7. Reliability (REL)

Tests for reliability and fault tolerance.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| REL-01 | TBD | `test/suite/smtpserver_reliability/test.rel-01.recovery.test.ts` | ⏸️ Deferred | - | Error recovery, retries |
| REL-02 | TBD | `test/suite/smtpserver_reliability/test.rel-02.persistence.test.ts` | ⏸️ Deferred | - | Queue persistence |

---

### 8. Edge Cases (EDGE)

Tests for uncommon scenarios and edge cases.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| EDGE-01 | TBD | `test/suite/smtpserver_edge-cases/test.edge-01.empty-data.test.ts` | ⏸️ Deferred | - | Empty messages, null bytes |
| EDGE-02 | TBD | `test/suite/smtpserver_edge-cases/test.edge-02.unicode.test.ts` | ⏸️ Deferred | - | Unicode in commands/data |

---

### 9. RFC Compliance (RFC)

Tests for RFC 5321/5322 compliance.

| Test ID | Source File | Destination File | Status | Tests | Notes |
|---------|-------------|------------------|--------|-------|-------|
| RFC-01 | TBD | `test/suite/smtpserver_rfc-compliance/test.rfc-01.smtp.test.ts` | ⏸️ Deferred | - | RFC 5321 compliance |
| RFC-02 | TBD | `test/suite/smtpserver_rfc-compliance/test.rfc-02.message-format.test.ts` | ⏸️ Deferred | - | RFC 5322 compliance |

---

## Progress Summary

### Overall Statistics
- **Total test files identified**: ~100+
- **Files ported**: 11/100+ (11%)
- **Total tests ported**: 82/~500+ (16%)
- **Tests passing**: 82/82 (100%)

### By Priority

#### High Priority (Phase 1: Core SMTP Functionality)
- ✅ CMD-01: EHLO Command (5 tests)
- ✅ CMD-02: MAIL FROM (6 tests)
- ✅ CMD-03: RCPT TO (7 tests)
- ✅ CMD-04: DATA (7 tests)
- ✅ CMD-13: QUIT (7 tests)
- ✅ CM-01: TLS Connection (8 tests)
- ✅ EP-01: Basic Email Sending (7 tests)

**Phase 1 Progress**: 7/7 complete (100%) ✅ **COMPLETE**

#### High Priority (Phase 2: Security & Validation)
- 📋 SEC-01: Authentication
- ✅ SEC-06: IP Reputation (7 tests)
- 📋 SEC-08: Rate Limiting
- 📋 SEC-10: Header Injection
- ✅ ERR-01: Syntax Errors (10 tests)
- ✅ ERR-02: Invalid Sequence (10 tests)

**Phase 2 Progress**: 3/6 complete (50%)

#### Medium Priority (Phase 3: Advanced Features)
- 📋 SEC-03: DKIM
- 📋 SEC-04: SPF
- 📋 EP-04: Large Emails
- 📋 EP-05: MIME Handling
- 📋 CM-02: Multiple Connections
- 📋 CM-06: STARTTLS Upgrade
- ✅ CMD-06: RSET Command (8 tests)

**Phase 3 Progress**: 1/7 complete (14%)

---

## Key Conversion Patterns

### Framework Changes
```typescript
// BEFORE (dcrouter - tap)
tap.test('should accept EHLO', async (t) => {
  expect(response).toBeTruthy();
  expect(response).toEqual('250 OK');
});

// AFTER (mailer - Deno)
Deno.test({
  name: 'CMD-01: EHLO - accepts valid hostname',
  async fn() {
    assert(response);
    assertEquals(response, '250 OK');
  },
  sanitizeResources: false,
  sanitizeOps: false,
});
```

### Network I/O Changes
```typescript
// BEFORE (dcrouter - Node.js)
import * as net from 'net';
const socket = net.connect({ port, host });

// AFTER (mailer - Deno)
const conn = await Deno.connect({
  hostname: host,
  port,
  transport: 'tcp',
});
```

### Assertion Changes
```typescript
// BEFORE (dcrouter)
expect(response).toBeTruthy()
expect(value).toEqual(expected)
expect(text).toMatch(/pattern/)

// AFTER (mailer)
assert(response)
assertEquals(value, expected)
assertMatch(text, /pattern/)
```

---

## Next Steps

### Immediate (Phase 1 completion)
- [ ] EP-01: Basic Email Sending test

### Phase 2 (Security & Validation)
- [ ] SEC-01: Authentication
- [ ] SEC-06: IP Reputation
- [ ] SEC-08: Rate Limiting
- [ ] SEC-10: Header Injection Prevention
- [ ] ERR-01: Syntax Error Handling
- [ ] ERR-02: Invalid Sequence Handling

### Phase 3 (Advanced Features)
- [ ] CMD-06: RSET Command
- [ ] SEC-03: DKIM Processing
- [ ] SEC-04: SPF Checking
- [ ] EP-04: Large Email Handling
- [ ] EP-05: MIME Handling
- [ ] CM-02: Multiple Concurrent Connections
- [ ] CM-06: Full STARTTLS Upgrade

### Phase 4 (Complete Coverage)
- [ ] All performance tests (PERF-*)
- [ ] All reliability tests (REL-*)
- [ ] All edge case tests (EDGE-*)
- [ ] All RFC compliance tests (RFC-*)
- [ ] SMTP client tests (if applicable)

---

## Migration Checklist Template

When porting a new test file, use this checklist:

- [ ] Identify source test file in dcrouter
- [ ] Create destination test file with proper naming
- [ ] Convert tap.test() to Deno.test()
- [ ] Update imports (.js → .ts, @std/assert)
- [ ] Convert expect() to assert/assertEquals/assertMatch
- [ ] Replace Node.js net with Deno.connect()
- [ ] Add sanitizeResources: false, sanitizeOps: false
- [ ] Preserve all test logic and validations
- [ ] Run tests and verify all passing
- [ ] Update this migration tracker
- [ ] Update test/readme.md with new tests

---

## Infrastructure Files

### Created for Deno Migration

| File | Purpose | Status |
|------|---------|--------|
| `test/helpers/utils.ts` | Deno-native SMTP protocol utilities | ✅ Complete |
| `test/helpers/server.loader.ts` | Test server lifecycle management | ✅ Complete |
| `test/helpers/smtp.client.ts` | SMTP client test utilities | ✅ Complete |
| `test/fixtures/test-key.pem` | Self-signed TLS private key | ✅ Complete |
| `test/fixtures/test-cert.pem` | Self-signed TLS certificate | ✅ Complete |
| `test/readme.md` | Test suite documentation | ✅ Complete |
| `test/readme.testmigration.md` | This migration tracker | ✅ Complete |

---

## Notes

- **Test Ports**: Each test file uses a unique port to avoid conflicts (CMD-01: 25251, CMD-02: 25252, etc.)
- **Type Checking**: Tests run with `--no-check` flag due to existing TypeScript errors in mailer codebase
- **TLS Testing**: Self-signed certificates used; some TLS handshake timeouts are expected and acceptable
- **Test Isolation**: Each test file has setup/cleanup tests for server lifecycle
- **Coverage Goal**: Aim for >90% test coverage before production deployment

---

Last Updated: 2025-10-28