Juergen Kunz
10e14af85b
- Added validation types for EN16931 compliance in `validation.types.ts`, including interfaces for `ValidationResult`, `ValidationOptions`, and `ValidationReport`. - Introduced `VATCategoriesValidator` in `vat-categories.validator.ts` to validate VAT categories according to EN16931 rules, including detailed checks for standard, zero-rated, exempt, reverse charge, intra-community, export, and out-of-scope services. - Enhanced `IEInvoiceMetadata` interface in `en16931-metadata.ts` to include additional fields required for full standards compliance, such as delivery information, payment information, allowances, and charges. - Implemented helper methods for VAT calculations and validation logic to ensure accurate compliance with EN16931 standards.
230 lines
8.9 KiB
Markdown
230 lines
8.9 KiB
Markdown
# E-Invoice Standards Compliance Implementation Plan
|
||
|
||
## Executive Summary
|
||
Current compliance: ~75% of required rules (up from ~70%)
|
||
Target: 100% compliance with EN16931, XRechnung, Peppol BIS 3.0, and Factur-X profiles
|
||
|
||
**Latest Update (2025-01-11)**:
|
||
- Completed Saxon-JS Schematron integration with official EN16931 rules
|
||
- Implemented comprehensive VAT category validator (all BR-S-*, BR-Z-*, BR-E-*, BR-AE-*, BR-K-*, BR-G-*, BR-O-* rules)
|
||
- Added conformance test harness with official test samples
|
||
- Created BR coverage matrix generation
|
||
|
||
## Scale of Work
|
||
- EN16931 core: ~120-150 business rules
|
||
- XRechnung CIUS: 100-200+ format-specific constraints
|
||
- Peppol BIS 3.0: Additional Schematron layer
|
||
- Factur-X profiles: Profile-specific cardinalities
|
||
- **Total: 300-500+ validations needed**
|
||
|
||
## Implementation Roadmap
|
||
|
||
### Phase 0: Baseline Infrastructure (Week 1) ✅ COMPLETE
|
||
- [x] Create rule registry with all EN16931, XRechnung, Peppol rule IDs (partial - EN16931 done)
|
||
- [x] Build coverage tracking system (ValidationReport with coverage metrics)
|
||
- [x] Set up validation result data model (ValidationResult interface with BT/BG references)
|
||
- [x] Implement Schematron engine integration ✅ (Saxon-JS with official rules)
|
||
|
||
### Phase 1: Core EN16931 Business Rules (Weeks 1-2) ✅ PARTIALLY COMPLETE
|
||
- [x] Create EN16931BusinessRulesValidator class
|
||
- [x] Implement calculation rules (BR-CO-*)
|
||
- BR-CO-10: Sum of invoice lines = Line extension amount ✅
|
||
- BR-CO-13: Tax exclusive = Lines - Allowances + Charges ✅
|
||
- BR-CO-15: Tax inclusive = Tax exclusive + VAT ✅
|
||
- BR-CO-14: Invoice total VAT amount ✅
|
||
- BR-CO-16: Amount due for payment ✅
|
||
- [x] Implement VAT rules (BR-S-*, BR-Z-*, partial)
|
||
- BR-S-01 to BR-S-03: Standard rated VAT ✅
|
||
- BR-Z-01: Zero rated VAT ✅
|
||
- [x] Add document level rules (BR-01 to BR-65) - ~25 rules implemented
|
||
- BR-01 to BR-11: Mandatory fields ✅
|
||
- BR-16: Invoice lines ✅
|
||
- [x] Add line level rules (BR-21 to BR-30) - All implemented ✅
|
||
|
||
### Phase 2: Calculation Engine (Weeks 2-3) ✅ PARTIALLY COMPLETE
|
||
- [ ] Build canonical semantic model (BT/BG fields)
|
||
- [ ] Create UBL/CII adapters to semantic model
|
||
- [x] Implement calculation verification:
|
||
- Line totals (quantity × price) ✅
|
||
- Tax base per category ✅
|
||
- Header allowance/charge distribution ✅
|
||
- Rounding and tolerance handling ✅ (ISO 4217 currency-aware)
|
||
- [x] Handle edge cases:
|
||
- Mixed VAT categories ✅
|
||
- Reverse charge (partial)
|
||
- Multi-currency ✅ (ISO 4217 support)
|
||
|
||
### Phase 3: XRechnung CIUS (Week 4)
|
||
- [ ] Integrate XRechnung Schematron pack
|
||
- [ ] Implement Leitweg-ID validation (pattern: [0-9]{2,3}-[0-9]{1,12}-[0-9]{2,30})
|
||
- [ ] Enforce mandatory buyer reference (BT-10)
|
||
- [ ] Add German-specific payment terms validation
|
||
- [ ] IBAN/BIC validation for SEPA
|
||
|
||
### Phase 4: Peppol BIS 3.0 (Week 5)
|
||
- [ ] Add Peppol Schematron layer
|
||
- [ ] Implement endpoint ID validation (0088:xxxxxxxxx)
|
||
- [ ] Add document type ID validation
|
||
- [ ] Party identification scheme validation
|
||
- [ ] Process ID validation
|
||
|
||
### Phase 5: Factur-X Profiles (Week 6)
|
||
- [ ] Implement profile detection
|
||
- [ ] Add profile-specific validators:
|
||
- MINIMUM: Only BT-1, BT-2, BT-3
|
||
- BASIC: Core fields
|
||
- EN16931: Full compliance
|
||
- EXTENDED: Additional structured data
|
||
- [ ] Profile-based field cardinality enforcement
|
||
|
||
### Phase 6: Code List Validators ✅ COMPLETE
|
||
- [x] ISO 4217 currency codes (BR-CL-03, BR-CL-04) ✅
|
||
- [x] ISO 3166 country codes (BR-CL-14, BR-CL-15, BR-CL-16) ✅
|
||
- [x] UN/ECE 4461 payment means codes (BR-CL-16) ✅
|
||
- [x] UNTDID 1001 document type codes (BR-CL-01) ✅
|
||
- [x] VAT category codes (UNCL5305 - BR-CL-10) ✅
|
||
- [x] UNECE Rec 20 unit codes (BR-CL-23) ✅
|
||
|
||
## Technical Architecture
|
||
|
||
### Layered Validation Approach:
|
||
1. **Schema validation**: XSD for UBL/CII
|
||
2. **Schematron packs**: EN16931, CIUS, code lists
|
||
3. **Programmatic engine**: Calculations and relationships
|
||
|
||
### API Design:
|
||
```typescript
|
||
interface ValidationOptions {
|
||
format?: 'ubl' | 'cii';
|
||
profile?: 'EN16931' | 'XRechnung_3.0' | 'Peppol_BIS_3.0' | 'FacturX_Basic';
|
||
tolerance?: number; // Default 0.01
|
||
strictMode?: boolean;
|
||
}
|
||
|
||
interface ValidationResult {
|
||
ruleId: string;
|
||
severity: 'error' | 'warning' | 'info';
|
||
message: string;
|
||
location?: string; // XPath
|
||
context?: any;
|
||
}
|
||
```
|
||
|
||
### Security Considerations:
|
||
- Disable DTD/XXE in XML parsing
|
||
- Enforce document size limits
|
||
- Sandbox XSLT execution
|
||
- Validate only trusted rule packs
|
||
|
||
## Success Criteria
|
||
- Pass official test suites for each standard
|
||
- 100% coverage of mandatory rules
|
||
- Performance: <100ms for full validation
|
||
- Clear error messages with rule IDs and locations
|
||
|
||
## Resources Needed
|
||
- EN16931 Schematron from official sources
|
||
- XRechnung artifacts for current version
|
||
- Peppol BIS 3.0 Schematron
|
||
- Factur-X profile documentation
|
||
- Official test invoices for each standard
|
||
|
||
## Risk Mitigation
|
||
- Version pinning for rule packs
|
||
- Snapshot testing for regression detection
|
||
- Configurable tolerances for calculations
|
||
- Layer precedence for conflicting rules
|
||
|
||
## Immediate Next Steps
|
||
1. ~~Set up Saxon-JS for Schematron integration~~ ✅ Complete
|
||
2. ~~Download official EN16931 Schematron files~~ ✅ Complete
|
||
3. ~~Create hybrid validation pipeline~~ ✅ Complete
|
||
4. ~~Implement ISO 4217 currency-aware rounding~~ ✅ Complete
|
||
5. ~~Complete remaining VAT category rules~~ ✅ Complete
|
||
6. ~~Add conformance test harness~~ ✅ Complete
|
||
7. Implement decimal arithmetic library (next priority)
|
||
8. Add XRechnung CIUS support
|
||
9. Implement PEPPOL BIS 3.0 overlay
|
||
|
||
## Accomplishments (2025-01-11)
|
||
|
||
### Implemented Components:
|
||
1. **EN16931BusinessRulesValidator** (`ts/formats/validation/en16931.business-rules.validator.ts`)
|
||
- ~40 business rules implemented
|
||
- Document, calculation, VAT, and line-level validation
|
||
- Currency-aware calculation verification with ISO 4217 support
|
||
|
||
2. **CodeListValidator** (`ts/formats/validation/codelist.validator.ts`)
|
||
- All major code lists validated
|
||
- Currency, country, tax category, payment means, unit codes
|
||
- Context-aware validation with exemption reasons
|
||
|
||
3. **Enhanced ValidationResult Interface** (`ts/formats/validation/validation.types.ts`)
|
||
- Business Term (BT) and Business Group (BG) references
|
||
- Semantic model mapping
|
||
- Code list metadata
|
||
- Remediation hints
|
||
|
||
4. **Extended Metadata Support** (`ts/interfaces/en16931-metadata.ts`)
|
||
- Comprehensive EN16931 metadata fields
|
||
- Delivery addresses, payment accounts
|
||
- Allowances and charges structure
|
||
|
||
5. **Feature Flag Integration**
|
||
- Gradual rollout capability
|
||
- Backward compatibility maintained
|
||
- Separate flags for EN16931 and code lists
|
||
|
||
6. **ISO 4217 Currency-Aware Rounding** (`ts/formats/utils/currency.utils.ts`)
|
||
- Complete ISO 4217 currency minor units database
|
||
- 7 rounding modes (HALF_UP, HALF_DOWN, HALF_EVEN, UP, DOWN, CEILING, FLOOR)
|
||
- CurrencyCalculator class for EN16931 calculations
|
||
- Replaces flat 0.01 tolerance with currency-specific tolerances
|
||
|
||
7. **Saxon-JS Schematron Integration** (`ts/formats/validation/schematron.*.ts`)
|
||
- Saxon-JS for XSLT 3.0 processing
|
||
- Official EN16931 Schematron files downloaded (v1.3.14)
|
||
- Worker thread pool for non-blocking validation
|
||
- Hybrid validator combining TypeScript and Schematron
|
||
- Automatic format detection (UBL/CII)
|
||
- SVRL parsing and result integration
|
||
|
||
### Test Coverage:
|
||
- Unit tests for validators (`test/test.en16931-validators.ts`)
|
||
- Currency utilities tests (`test/test.currency-utils.ts`) - 7/7 passing
|
||
- Schematron infrastructure tests (`test/test.schematron-validator.ts`) - 9/9 passing
|
||
- Integration with existing test suite
|
||
- 496/497 tests passing overall (added 16 new tests)
|
||
|
||
8. **VAT Categories Validator** (`ts/formats/validation/vat-categories.validator.ts`)
|
||
- Complete implementation of all VAT category business rules
|
||
- BR-S-* (Standard rate), BR-Z-* (Zero rated), BR-E-* (Exempt)
|
||
- BR-AE-* (Reverse charge), BR-K-* (Intra-community), BR-G-* (Export)
|
||
- BR-O-* (Out of scope services)
|
||
- Cross-category validation rules
|
||
|
||
9. **Conformance Test Harness** (`ts/formats/validation/conformance.harness.ts`)
|
||
- Automated testing against official samples
|
||
- BR coverage matrix generation
|
||
- HTML coverage reports
|
||
- Support for PEPPOL and CEN test suites
|
||
- Performance metrics collection
|
||
|
||
10. **Test Sample Downloader** (`scripts/download-test-samples.ts`)
|
||
- Automated download from official repositories
|
||
- PEPPOL BIS 3.0 examples
|
||
- CEN TC434 test files
|
||
- Metadata tracking
|
||
|
||
11. **XML to EInvoice Converter** (`ts/formats/converters/xml-to-einvoice.converter.ts`)
|
||
- Basic UBL and CII parsing
|
||
- Integration with conformance testing
|
||
|
||
### Next Priority Items:
|
||
1. ~~Set up Saxon-JS for Schematron integration~~ ✅ COMPLETE
|
||
2. ~~Integrate official EN16931 Schematron from ConnectingEurope~~ ✅ COMPLETE
|
||
3. ~~Complete remaining VAT category rules~~ ✅ COMPLETE
|
||
4. ~~Add conformance test harness with official test packs~~ ✅ COMPLETE
|
||
5. Implement decimal arithmetic for even more precision
|
||
6. Add XRechnung CIUS layer
|
||
7. Implement PEPPOL BIS 3.0 support |