@ship.zone/ci-compiler is the reference compiler of the ship.zone CI standard, @ship.zone/ci-spec. It owns the deterministic parts of the standard that coordinators and runners share: RFC 8785 canonical JSON and digests, the specification version rule, the strict reading of ci_actions.yml and its check against the schema and the workflow rules, branch and tag pattern matching, validation of compiled runner jobs, and candidate image index assembly. Run compilation follows in later releases.

Every function is pure. The only I/O is loading the normative JSON Schemas from the installed @ship.zone/ci-spec once, when the module loads; no function reads a clock, randomness, the environment, the filesystem, or the network, and equal inputs give byte-equal outputs.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install

pnpm add @ship.zone/ci-compiler

Specification Version

The compiler implements exactly one specification version: the version of the @ship.zone/ci-spec it pins exactly. A coordinator or runner that uses the compiler pins the same @ship.zone/ci-spec version, so the version it negotiates is the version it compiles and checks against.

import { acceptCiSpec, implementedCiSpecVersion, lowerCiSpec } from '@ship.zone/ci-compiler';

implementedCiSpecVersion; // '2.1.1'
acceptCiSpec(implementedCiSpecVersion, '2.0.0'); // 'accepted'
acceptCiSpec(implementedCiSpecVersion, '2.2.0'); // 'newer'
acceptCiSpec(implementedCiSpecVersion, '1.2.0'); // 'major-mismatch'
acceptCiSpec(implementedCiSpecVersion, '2.0'); // 'malformed'
lowerCiSpec('2.1.0', '2.0.5'); // '2.0.5'

acceptCiSpec and lowerCiSpec throw CiCompilerUsageError for a version that must be canonical and is not; lowerCiSpec also throws for versions of different major versions, which never form a session.

Canonical JSON and Digests

import {
  canonicalizeCiJson,
  digestCiCompiledPlan,
  digestCiJson,
  projectCiCompiledPlan,
  sha256CiHex,
} from '@ship.zone/ci-compiler';

canonicalizeCiJson({ b: 1, a: [true, null] }); // '{"a":[true,null],"b":1}'
digestCiJson(requestBody); // canonicalRequestDigest: lowercase SHA-256 of the RFC 8785 bytes
projectCiCompiledPlan(job); // the job without compiledPlanDigest, every secrets value replaced by true
digestCiCompiledPlan(job); // compiledPlanDigest
sha256CiHex(bytes); // lowercase hex SHA-256 of bytes or of a UTF-8 string

Canonicalization throws CiCanonicalJsonError for a value without an RFC 8785 serialization: a lone UTF-16 surrogate in a string or member name, a non-finite number, a value that is not JSON, an array with holes, or a cycle.

Workflow Check

checkCiWorkflow reads ci_actions.yml bytes as a coordinator reads them from the source commit and checks the five layers of ci-actions, Compilation Failures, that depend only on the bytes: source, yaml, version, schema, and workflow. Every run of one commit fails these layers alike. It returns the typed document with the admitted trigger kinds of every job, or the failure of the first layer with a violation:

import { checkCiWorkflow } from '@ship.zone/ci-compiler';

const result = checkCiWorkflow(bytes);
if (result.ok) {
  result.value.spec; // '2.0.0'
  result.value.workflowDigest; // lowercase hex SHA-256 of the bytes as read
  result.value.document; // ICiWorkflow
  result.value.mappingKeyOrder.get('/jobs/test/matrix'); // matrix dimensions in declaration order
  result.value.admittedKinds.get('test'); // ['push', 'tag']: the run kinds that do not skip the job
} else {
  // result.failure: { code, message, location?, issues? }
}

Layer	Codes	Rule
`source`	`workflow_too_large`, `workflow_encoding_invalid`	at most 262,144 bytes, a leading byte order mark included, and well-formed UTF-8
`yaml`	`workflow_yaml_document_count`, `workflow_yaml_directive`, `workflow_yaml_duplicate_key`, `workflow_yaml_key_type`, `workflow_yaml_anchor`, `workflow_yaml_merge_key`, `workflow_yaml_tag`, `workflow_number_invalid`, `workflow_yaml_syntax`	YAML 1.2 with the core schema: one document as YAML 1.2 counts them, no directive, unique string keys, no anchor, alias, merge key, or explicit tag, and numbers that are finite and, when integral, within ±9,007,199,254,740,991; the character, byte order mark, escape, and nesting depth rules and any other parser error or warning are `workflow_yaml_syntax`
`version`	`spec_incompatible`	a canonical `spec` that is `newer` or `major-mismatch` for the implemented version
`schema`	`workflow_schema_invalid`	`schemas/ci_actions.schema.json`; `issues` carries the schema errors, sorted, at most 32
`workflow`	the 20 codes from `trigger_pattern_invalid` to `publish_reference_invalid`	the rules ci-actions applies in every run, whatever its kind; see below

The yaml layer follows ci-actions, Parsing, on the syntax tree of the yaml package:

A C0 control character other than tab, line feed, and carriage return fails anywhere, a quoted scalar included. DEL, C1 control characters other than NEL, U+FFFE, U+FFFF, and U+FEFF are content inside a single- or double-quoted scalar token and fail anywhere else.
Runs of U+FEFF at the start of a line are byte order marks, not content, unless the line lies within a document's content: after the line on which the document begins and before the end of its last content token. The file's own leading byte order mark is the first of them. Any other U+FEFF outside a quoted scalar fails.
In a double-quoted scalar, a \u high surrogate escape directly followed by a \u low surrogate escape is one character; every other surrogate escape, the \U form included, fails.
Collections nest at most 64 deep, the root collection at depth 1; a deeper collection fails and is not read.

Within the yaml layer a rule of the subset is reported before a syntax violation, and among the rules the first in the text. Positions in messages count the characters of the text without its byte order marks. Integer tokens keep their exact value, float tokens are converted to binary64, negative zero becomes zero, and quoted scalars are always strings. The document is built from the YAML syntax tree in declaration order. JavaScript enumerates integer-like object keys first, so mappingKeyOrder carries the declaration order of every mapping by JSON Pointer; matrix expansion follows it. The function throws CiCompilerUsageError when its argument is not a Uint8Array.

Workflow Rules

The workflow layer checks its rules in the order of the Compilation Failures table and reports the first code whose rule the workflow violates, wherever in the file the violation is. location points at the first violation of that rule, with jobs in declaration order.

Code	Rule
`trigger_pattern_invalid`	a branch or tag filter outside the pattern grammar
`needs_unknown`	`needs` names an undeclared job
`needs_cycle`	`needs` forms a cycle, a self-dependency included
`expansion_limit_exceeded`	more than 256 expanded jobs: one per matrix combination or build platform, skipped jobs included
`needs_trigger_wider`	a job admits a kind a job it needs does not admit
`trigger_kind_unused`	a kind in `on` that no job admits
`permission_escalation`	a job permission above the workflow permission, `images: publish` counting as `candidate`
`permission_insufficient`	an artifact without `artifacts: write`, a cache without `caches: read`, a read-write cache without `caches: read-write`, or a build job without `images: candidate`
`environment_conflict`	a `SHIPZONE_CI_*` name in an environment map or as a secret target, two secret references with one target, or a target that is a name of a step's merged environment
`environment_limit_exceeded`	a step's merged environment names, secret targets, and reserved names number more than 128
`command_limit_exceeded`	a command argument above 16,384 UTF-8 bytes or a command above 131,072
`transfer_name_duplicate`	two artifacts or two caches of a job with one name
`cache_path_overlap`	two cache paths of a job whose segments, without empty and `.` segments, are a prefix of each other
`resources_invalid`	a declared `sharedMemoryBytes` above a declared `memoryBytes`
`network_host_duplicate`	an egress host listed twice
`npm_read_invalid`	an `npmRead` registry listed twice, a scope under two registries, a registry host and port the job's egress allowlist does not permit, or, with `npmRead`, `NPM_CONFIG_USERCONFIG` in a step environment or as a secret target, or a build secret with id `npmrc`
`build_input_conflict`	a build secret naming no job secret target, a target no build secret names, or a build argument named like a target or build secret id
`candidate_reference_invalid`	a candidate of a job that is not a build job, or of a build job the referencing job does not need
`candidate_platform_missing`	a pinned `runner.platform` or a consuming build platform the producing build job does not list
`publish_reference_invalid`	an image publication of a job that is not a build job, an artifact reference to a matrix job or to an artifact not declared with `required: true` and `when: success`, or a referenced job that does not admit `tag`

Branch and Tag Patterns

matchCiRefPattern decides whether a branch or tag filter matches a short ref name under the grammar of ci-actions, Triggers: anchored, case-sensitive, over Unicode scalar values, with *, **, ?, and the escapes \*, \?, and \\. It runs in time proportional to the product of the two lengths and throws CiCompilerUsageError for a pattern that checkCiWorkflow rejects with trigger_pattern_invalid.

import { matchCiRefPattern } from '@ship.zone/ci-compiler';

matchCiRefPattern('release/*', 'release/1'); // true
matchCiRefPattern('release/*', 'release/next/1'); // false
matchCiRefPattern('release/**', 'release/next/1'); // true

Compiled Runner Jobs

checkCiRunnerJob decides whether a value is a coherent compiled runner job with its secret values inserted (runner protocol, Job Coherence), as the coordinator checks it before enqueueing and the runner before acceptance:

import { checkCiRunnerJob } from '@ship.zone/ci-compiler';

const result = checkCiRunnerJob(JSON.parse(body));
if (!result.ok) {
  // result.failure: { code, message, location?, issues? }
}

It checks the rows of the Job Coherence table in order and reports the job failure code of the first row the job violates; the codes are the specification's runnerJobFailureCodes:

Code	Rule
`spec_incompatible`	a canonical `spec` that is `newer` or `major-mismatch` for the implemented version
`job_schema_invalid`	`schemas/runner-job.schema.json`; `issues` carries the schema errors, sorted, at most 32
`job_secret_incoherent`	in an `oci-image` job, a build secret names no `secrets` member, or a member is named by no build secret
`job_limit_incoherent`	source metadata, a declared artifact or cache maximum, or a declaration count above `requirements.limits`
`job_resources_incoherent`	`resources.sharedMemoryBytes` above `resources.memoryBytes`
`job_environment_incoherent`	a step without `SHIPZONE_CI_MATRIX_JSON` and `SHIPZONE_CI_INPUTS_JSON` as RFC 8785 JSON object text, another `SHIPZONE_CI_*` name, a secret target with that prefix, or step environment names, secret targets, and reserved names that repeat or number more than 128
`job_command_limit_exceeded`	a step command argument above 16,384 UTF-8 bytes, which the schema bounds only in characters, or a step command above 131,072 UTF-8 bytes
`job_secret_inadmissible`	a secret value shorter than 8 UTF-8 bytes
`job_digest_mismatch`	`compiledPlanDigest` is not the digest of the plan projection

A planned job, whose secrets values are true, fails the schema: values enter a job only for a lease. Matching a job against a runner's capability snapshot and checking lease coherence are coordinator and runner concerns outside the job and are not part of this check. No failure message contains a secret value.

Candidate Index Assembly

assembleCiCandidateIndex builds the multi-platform OCI image index of a build job from the accepted image records of its per-platform build jobs, in the declared platform order:

import { assembleCiCandidateIndex } from '@ship.zone/ci-compiler';

const candidate = assembleCiCandidateIndex(['linux/amd64', 'linux/arm64/v8'], imageRecords);
candidate.index; // { schemaVersion: 2, mediaType, manifests }
candidate.bytes; // the RFC 8785 bytes to push
candidate.digest; // 'sha256:…'

It throws CiCompilerUsageError unless every platform is a distinct linux platform with exactly one image record and every record belongs to a platform.

Conformance

The tests run the conformance data of the installed @ship.zone/ci-spec: every case of version-cases.json and digest-cases.json, every case of runner-job-cases.json with its exact job failure code, which covers every runner-jobs fixture, the job of every runner-messages.json lease, and every compiled job of compile-cases.json, whose plan digest is recomputed and which is checked with placeholder secret values. Every workflow of compile-cases.json and every ci-actions fixture is checked by checkCiWorkflow: a case of the source, yaml, version, schema, or workflow layer fails with its exact code, every other case passes with its workflow digest, and every valid fixture yields the values of the YAML in declaration order. Every patternCases vector is matched by matchCiRefPattern and, when invalid, fails a workflow in each filter with trigger_pattern_invalid.

Verification

pnpm run build
pnpm run test:types
pnpm test

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md

@ship.zone/ci-compiler