# Development of Validator Configuration XRechnung

This repository contains an ANT `build.xml` which allows downloading all necessary tools and artefacts for creating this validator configuration for XRechnung. It also allows testing the configuration against a few UBL and UN/CEFACT documents and creates a release zip file.

## Dependencies overview

### Compile time

The following dependencies are downloaded automatically from within the build script:

* UBL Schema
* CII Schema
* CEN Schematron
* Saxon HE 9.9.x

### Testing

* KoSIT XML Validator
* XML Mutate
* XRechnung Testsuite

## Build from scratch

We recommend `ant` version 1.10.x or newer (but should work with > 1.8.x).

The main `ant` targets for developing are:

* `clean`
* `compile` (here it is assembling all resources for validation)
* `test`
* and `dist` (creating the distribution artefact)

However, because of the complex dependencies, you may only expect `compile` target to work without any customizations.

### Test dependencies on the fly

If you build own local custom versions of dependencies such as XRechnung Schematron you can customize the ant build at runtime:

The following example demonstrates how to use a different CEN Schematron release:

```shell
ant clean dist \
  -Dcen.rules.version=1.3.12
```

The following example demonstrates how to use a different CEN Schematron release, downloaded from the CEN GitHub repository:

```shell
ant clean dist \
  -Dcen.rules.version=1.3.12 \
```

To test with the latest SNAPSHOT versions of the CEN rules, add the following property to the Ant call:

```shell
ant clean dist \
  -Dcen.rules.version=master
```

Note: `master` is symbolic name for latest code from Git master branch.

Note: please make sure that after using a custom CEN rule version, to call `ant clean` afterwards to ensure the default version is restored correctly.

### Development properties file

In order to configure more complex adoption to the local development needs, you have to load a set of different properties from a file.

We provide the `development.build.properties.example` file for the most common properties to be set different than default. It contains some documentation.

You have to copy the file to e.g. `development.build.properties` and you have to explicitly provide the property file location at CLI for your development (otherwise tests will always fail or not be executed at all).

```shell
ant -propertyfile ${your.own.property.file.name}
```

## Test Approach

### Installing XML Mutate

`XML Mutate` needs to be build and setup manually - see https://projekte.kosit.org/kosit/xml-mutate for installation details.

Additionally a custom property needs to point to the XML Mutate binary and the version
* Property `xmute.download.url.prefix` indicates the URL to the cloned repository (as e.g. in `file:///../xml-mutate`)
* Property `xmute.version.full` indicates the full version number of the created binary JAR file (defaults to `0.5`).

Example Ant call (for Windows users):

```
ant clean dist \
  -Dxmute.download.url.prefix=file:///C:/dev/git3p/xml-mutate \
  -Dxmute.version.full=0.5
```

Hint: we recommend that you place the properties `xmute.download.url.prefix` and `xmute.version.full` in the "development properties file" (see above).

### Configuration Report Tests

All test instances in `src/test/instances` are designed to trigger certain kinds of report outputs. This is to test the report output on a technical level.

These reports are then tested using `src/test/assertions.xml`.

Ant call:

```shell
ant test-validator-assertions
```

### XRechnung Testsuite

This test phase runs the KoSIT validator against all XRechnung Testsuite instances and expects all instances to be valid.

Ant call:

```shell
ant create-validator-reports-from-testsuite
```

### CEN Unit Tests

The correctness of the whole validation process depends mostly on Schematron rules as implemented by CEN.
Hence, we started to also test aspects of those rules.

Currently, test instances are in `src/test/cen-unit-test` and tested using `XML Mutate`. We use positive and negative tests to verify the expected behavior of the CEN rules.

Filenames follow this convention:

`${cii,ubl}-${schematron-rule-id}-${description}.xml`

If more than one rule is tested than omit `${schematron-rule-id}`.

Ant call:

```shell
ant test-integration
```

### XRechnung Integration Tests

Finally, the generated test instances -- which are generated by XML-Mutate from the previous CEN Unit Tests build step -- are run with the fully configured Validator Configuration.

This allows to also test the correct behavior of the whole integrated validation procedure in case of wrong input also including effects of changing severity of validation rules on the calculation of the final assessment (recommendation).

The integration test are coded in `src/test/assertions-integration-tests.xml`

Ant call:

```shell
ant test-integration
```

### Unexpected Behaviour of CEN Rules

We provide tests to check on unexpected behaviour of CEN Schematron rules, i.e. the rules in the current state have wrong semantics and hence produce wrong validation results.

Now, we formulate the mutator instructions in a way that we test for the future target state. 

Therefore, each single test will fail as long as the current state of CEN rules does not match the target state.

In order to track the discrepancy between current state and target state, we established following conventions for mutator descriptions:

* `description="expected-to-pass: {DESCRIPTION}"` for a rule which in the current state does **NOT pass** but is expected to pass (i.e. **should pass**) in the future.

* `description="expected-to-fail: {DESCRIPTION}"` for a rule which in the current state does **pass** but is expected to fail (i.e. **should NOT pass**) in the future.

The test instances are in `src/test/unexpected` and tested using `XML Mutate`.

Ant call:
```shell
ant test-cen-unexpected-behaviour
```

This ant task is expected to fail. If not called manually, this task will be skipped.

## Distribution

The `ant` target `dist` creates the distribution zip archive for releases including several targets for testing.

If you want to skip tests call `ant dist-only`.


## Release

### Checklist

* Are all issues scheduled for the release solved?
* Is everything merged to master branch?
* Does the configuration refer to the correct version of XRechnung Specification?
* Does the scenario match refer to the correct XRechnung Specification?
* Make sure that XRechnung Schematron and testsuite are already released and used by this release version.
* Make sure that CHANGELOG.md is up to date.

### Prepare

* Make sure you committed and pushed everything.
* Create the distribution 
   * Do **not** use your development properties file.
   * Use the `clean` target to build and test everything from scratch

  This requires setting some properties at command line:

  ```
  ant -Dxmute.download.url.prefix='file:/home/renzo/projects/xml-mutate/' clean dist
  ```

* Tag the last commit according to the following naming rule: 
   `release-${date-of-scheduled-release-e.g. 2024-10-31}`
  e.g.
  `git tag release-2024-10-31 && git push origin release-2024-10-31`

### Publish

* Draft a new release at https://github.com/itplr-kosit/validator-configuration-xrechnung/releases/new
  * Choose the git tag
* Add a release title of the following scheme: `Validator Configuration 2024-10-31 compatible with XRechnung 3.0.x`.
* Copy & paste the high quality changelog entries for this release from CHANGELOG.md.
* Upload distribution zip and tick mark this release as a `pre-release`.


### Post-Release

* Change the version of of Validator Configuration XRechnung in `build.xml` to the next release and commit.

You are done :smile:

