mirror of
https://github.com/NixOS/nix.git
synced 2025-11-08 19:46:02 +01:00
Add JSON Schema infrastructure, use for Derivation
For manual, and testing formats
This commit is contained in:
Robert Hensing
John Ericson
parent
d87a06af7a
commit
01b001d5ba
25 changed files with 465 additions and 124 deletions
|
|
@ -115,6 +115,7 @@ manual = custom_target(
|
|||
builtins_md,
|
||||
rl_next_generated,
|
||||
summary_rl_next,
|
||||
json_schema_generated_files,
|
||||
nix_input,
|
||||
],
|
||||
output : [
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@
|
|||
rsync,
|
||||
nix-cli,
|
||||
changelog-d,
|
||||
json-schema-for-humans,
|
||||
officialRelease,
|
||||
|
||||
# Configuration Options
|
||||
|
|
@ -55,6 +56,7 @@ mkMesonDerivation (finalAttrs: {
|
|||
jq
|
||||
python3
|
||||
rsync
|
||||
json-schema-for-humans
|
||||
changelog-d
|
||||
]
|
||||
++ lib.optionals (!officialRelease) [
|
||||
|
|
|
|||
|
|
@ -117,6 +117,7 @@
|
|||
- [Architecture and Design](architecture/architecture.md)
|
||||
- [Formats and Protocols](protocols/index.md)
|
||||
- [JSON Formats](protocols/json/index.md)
|
||||
- [Hash](protocols/json/hash.md)
|
||||
- [Store Object Info](protocols/json/store-object-info.md)
|
||||
- [Derivation](protocols/json/derivation.md)
|
||||
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
# Process JSON schema documentation
|
||||
subdir('protocols')
|
||||
|
||||
summary_rl_next = custom_target(
|
||||
command : [
|
||||
bash,
|
||||
|
|
|
|||
|
|
@ -1,120 +1,7 @@
|
|||
# Derivation JSON Format
|
||||
{{#include derivation-v3-fixed.md}}
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
> This JSON format is currently
|
||||
> [**experimental**](@docroot@/development/experimental-features.md#xp-feature-nix-command)
|
||||
> and subject to change.
|
||||
<!--
|
||||
## Raw Schema
|
||||
|
||||
The JSON serialization of a
|
||||
[derivations](@docroot@/glossary.md#gloss-store-derivation)
|
||||
is a JSON object with the following fields:
|
||||
|
||||
* `name`:
|
||||
The name of the derivation.
|
||||
This is used when calculating the store paths of the derivation's outputs.
|
||||
|
||||
* `version`:
|
||||
Must be `3`.
|
||||
This is a guard that allows us to continue evolving this format.
|
||||
The choice of `3` is fairly arbitrary, but corresponds to this informal version:
|
||||
|
||||
- Version 0: A-Term format
|
||||
|
||||
- Version 1: Original JSON format, with ugly `"r:sha256"` inherited from A-Term format.
|
||||
|
||||
- Version 2: Separate `method` and `hashAlgo` fields in output specs
|
||||
|
||||
- Version 3: Drop store dir from store paths, just include base name.
|
||||
|
||||
Note that while this format is experimental, the maintenance of versions is best-effort, and not promised to identify every change.
|
||||
|
||||
* `outputs`:
|
||||
Information about the output paths of the derivation.
|
||||
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:
|
||||
|
||||
* `path`:
|
||||
The output path, if it is known in advanced.
|
||||
Otherwise, `null`.
|
||||
|
||||
|
||||
* `method`:
|
||||
For an output which will be [content addressed], a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
|
||||
Valid method strings are:
|
||||
|
||||
- [`flat`](@docroot@/store/store-object/content-address.md#method-flat)
|
||||
- [`nar`](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||
- [`text`](@docroot@/store/store-object/content-address.md#method-text)
|
||||
- [`git`](@docroot@/store/store-object/content-address.md#method-git)
|
||||
|
||||
Otherwise, `null`.
|
||||
|
||||
* `hashAlgo`:
|
||||
For an output which will be [content addressed], the name of the hash algorithm used.
|
||||
Valid algorithm strings are:
|
||||
|
||||
- `blake3`
|
||||
- `md5`
|
||||
- `sha1`
|
||||
- `sha256`
|
||||
- `sha512`
|
||||
|
||||
* `hash`:
|
||||
For fixed-output derivations, the expected content hash in base-16.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "outputs": {
|
||||
> "out": {
|
||||
> "method": "nar",
|
||||
> "hashAlgo": "sha256",
|
||||
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
|
||||
> }
|
||||
> }
|
||||
> ```
|
||||
|
||||
* `inputSrcs`:
|
||||
A list of store paths on which this derivation depends.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "inputSrcs": [
|
||||
> "47y241wqdhac3jm5l7nv0x4975mb1975-separate-debug-info.sh",
|
||||
> "56d0w71pjj9bdr363ym3wj1zkwyqq97j-fix-pop-var-context-error.patch"
|
||||
> ]
|
||||
> ```
|
||||
|
||||
* `inputDrvs`:
|
||||
A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "inputDrvs": {
|
||||
> "6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
|
||||
> "fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
|
||||
> }
|
||||
> ```
|
||||
|
||||
specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
|
||||
|
||||
* `system`:
|
||||
The system type on which this derivation is to be built
|
||||
(e.g. `x86_64-linux`).
|
||||
|
||||
* `builder`:
|
||||
The absolute path of the program to be executed to run the build.
|
||||
Typically this is the `bash` shell
|
||||
(e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
|
||||
|
||||
* `args`:
|
||||
The command-line arguments passed to the `builder`.
|
||||
|
||||
* `env`:
|
||||
The environment passed to the `builder`.
|
||||
|
||||
* `structuredAttrs`:
|
||||
[Structured Attributes](@docroot@/store/derivation/index.md#structured-attrs), only defined if the derivation contains them.
|
||||
Structured attributes are JSON, and thus embedded as-is.
|
||||
[JSON Schema for Derivation v3](schema/derivation-v3.json)
|
||||
-->
|
||||
|
|
|
|||
|
|
@ -0,0 +1,14 @@
|
|||
# For some reason, backticks in the JSON schema are being escaped rather
|
||||
# than being kept as intentional code spans. This removes all backtick
|
||||
# escaping, which is an ugly solution, but one that is fine, because we
|
||||
# are not using backticks for any other purpose.
|
||||
s/\\`/`/g
|
||||
|
||||
# The way that semi-external references are rendered (i.e. ones to
|
||||
# sibling schema files, as opposed to separate website ones, is not nice
|
||||
# for humans. Replace it with a nice relative link within the manual
|
||||
# instead.
|
||||
#
|
||||
# As we have more such relative links, more replacements of this nature
|
||||
# should appear below.
|
||||
s^\(./hash-v1.yaml\)\?#/$defs/algorithm^[JSON format for `Hash`](./hash.html#algorithm)^g
|
||||
7
doc/manual/source/protocols/json/hash.md
Normal file
7
doc/manual/source/protocols/json/hash.md
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
{{#include hash-v1-fixed.md}}
|
||||
|
||||
<!--
|
||||
## Raw Schema
|
||||
|
||||
[JSON Schema for Hash v1](schema/hash-v1.json)
|
||||
-->
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# Configuration file for json-schema-for-humans
|
||||
#
|
||||
# https://github.com/coveooss/json-schema-for-humans/blob/main/docs/examples/examples_md_default/Configuration.md
|
||||
|
||||
template_name: md
|
||||
show_toc: true
|
||||
# impure timestamp and distracting
|
||||
with_footer: false
|
||||
recursive_detection_depth: 3
|
||||
show_breadcrumbs: false
|
||||
description_is_markdown: true
|
||||
template_md_options:
|
||||
properties_table_columns:
|
||||
- Property
|
||||
- Type
|
||||
- Pattern
|
||||
- Title/Description
|
||||
74
doc/manual/source/protocols/json/meson.build
Normal file
74
doc/manual/source/protocols/json/meson.build
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# Tests in: ../../../../src/json-schema-checks
|
||||
|
||||
fs = import('fs')
|
||||
|
||||
# Find json-schema-for-humans if available
|
||||
json_schema_for_humans = find_program('generate-schema-doc', required : false)
|
||||
|
||||
# Configuration for json-schema-for-humans
|
||||
json_schema_config = files('json-schema-for-humans-config.yaml')
|
||||
|
||||
schemas = [
|
||||
'hash-v1',
|
||||
'derivation-v3',
|
||||
]
|
||||
|
||||
schema_files = files()
|
||||
foreach schema_name : schemas
|
||||
schema_files += files('schema' / schema_name + '.yaml')
|
||||
endforeach
|
||||
|
||||
|
||||
schema_outputs = []
|
||||
foreach schema_name : schemas
|
||||
schema_outputs += schema_name + '.md'
|
||||
endforeach
|
||||
|
||||
json_schema_generated_files = []
|
||||
|
||||
# Generate markdown documentation from JSON schema
|
||||
# Note: output must be just a filename, not a path
|
||||
gen_file = custom_target(
|
||||
schema_name + '-schema-docs.tmp',
|
||||
command : [
|
||||
json_schema_for_humans,
|
||||
'--config-file',
|
||||
json_schema_config,
|
||||
meson.current_source_dir() / 'schema',
|
||||
meson.current_build_dir(),
|
||||
],
|
||||
input : schema_files + [
|
||||
json_schema_config,
|
||||
],
|
||||
output : schema_outputs,
|
||||
capture : false,
|
||||
build_by_default : true,
|
||||
)
|
||||
|
||||
idx = 0
|
||||
if json_schema_for_humans.found()
|
||||
foreach schema_name : schemas
|
||||
#schema_file = 'schema' / schema_name + '.yaml'
|
||||
|
||||
# There is one so-so hack, and one horrible hack being done here.
|
||||
sedded_file = custom_target(
|
||||
schema_name + '-schema-docs',
|
||||
command : [
|
||||
'sed',
|
||||
'-f',
|
||||
# Out of line to avoid https://github.com/mesonbuild/meson/issues/1564
|
||||
files('fixup-json-schema-generated-doc.sed'),
|
||||
'@INPUT@',
|
||||
],
|
||||
capture : true,
|
||||
input : gen_file[idx],
|
||||
output : schema_name + '-fixed.md',
|
||||
)
|
||||
idx += 1
|
||||
json_schema_generated_files += [ sedded_file ]
|
||||
endforeach
|
||||
else
|
||||
warning(
|
||||
'json-schema-for-humans not found, skipping JSON schema documentation generation',
|
||||
)
|
||||
endif
|
||||
164
doc/manual/source/protocols/json/schema/derivation-v3.yaml
Normal file
164
doc/manual/source/protocols/json/schema/derivation-v3.yaml
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
"$schema": http://json-schema.org/draft-04/schema#
|
||||
"$id": https://nix.dev/manual/nix/latest/protocols/json/schema/derivation-v3.json
|
||||
title: Derivation
|
||||
description: |
|
||||
Experimental JSON representation of a Nix derivation (version 3).
|
||||
|
||||
This schema describes the JSON representation of Nix's `Derivation` type.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
> This JSON format is currently
|
||||
> [**experimental**](@docroot@/development/experimental-features.md#xp-feature-nix-command)
|
||||
> and subject to change.
|
||||
|
||||
type: object
|
||||
required:
|
||||
- name
|
||||
- version
|
||||
- outputs
|
||||
- inputSrcs
|
||||
- inputDrvs
|
||||
- system
|
||||
- builder
|
||||
- args
|
||||
- env
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
description: |
|
||||
The name of the derivation.
|
||||
Used when calculating store paths for the derivation’s outputs.
|
||||
|
||||
version:
|
||||
const: 3
|
||||
description: |
|
||||
Must be `3`.
|
||||
This is a guard that allows us to continue evolving this format.
|
||||
The choice of `3` is fairly arbitrary, but corresponds to this informal version:
|
||||
|
||||
- Version 0: A-Term format
|
||||
|
||||
- Version 1: Original JSON format, with ugly `"r:sha256"` inherited from A-Term format.
|
||||
|
||||
- Version 2: Separate `method` and `hashAlgo` fields in output specs
|
||||
|
||||
- Version 3: Drop store dir from store paths, just include base name.
|
||||
|
||||
Note that while this format is experimental, the maintenance of versions is best-effort, and not promised to identify every change.
|
||||
|
||||
outputs:
|
||||
type: object
|
||||
description: |
|
||||
Information about the output paths of the derivation.
|
||||
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "outputs": {
|
||||
> "out": {
|
||||
> "method": "nar",
|
||||
> "hashAlgo": "sha256",
|
||||
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
|
||||
> }
|
||||
> }
|
||||
> ```
|
||||
additionalProperties:
|
||||
"$ref": "#/$defs/output"
|
||||
|
||||
inputSrcs:
|
||||
type: array
|
||||
description: |
|
||||
List of store paths on which this derivation depends.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "inputSrcs": [
|
||||
> "47y241wqdhac3jm5l7nv0x4975mb1975-separate-debug-info.sh",
|
||||
> "56d0w71pjj9bdr363ym3wj1zkwyqq97j-fix-pop-var-context-error.patch"
|
||||
> ]
|
||||
> ```
|
||||
items:
|
||||
type: string
|
||||
|
||||
inputDrvs:
|
||||
type: object
|
||||
description: |
|
||||
Mapping of derivation paths to lists of output names they provide.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> ```json
|
||||
> "inputDrvs": {
|
||||
> "6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
|
||||
> "fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
|
||||
> }
|
||||
> ```
|
||||
>
|
||||
> specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
|
||||
|
||||
system:
|
||||
type: string
|
||||
description: |
|
||||
The system type on which this derivation is to be built
|
||||
(e.g. `x86_64-linux`).
|
||||
|
||||
builder:
|
||||
type: string
|
||||
description: |
|
||||
Absolute path of the program used to perform the build.
|
||||
Typically this is the `bash` shell
|
||||
(e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
|
||||
|
||||
args:
|
||||
type: array
|
||||
description: |
|
||||
Command-line arguments passed to the `builder`.
|
||||
items:
|
||||
type: string
|
||||
|
||||
env:
|
||||
type: object
|
||||
description: |
|
||||
Environment variables passed to the `builder`.
|
||||
additionalProperties:
|
||||
type: string
|
||||
|
||||
structuredAttrs:
|
||||
description: |
|
||||
[Structured Attributes](@docroot@/store/derivation/index.md#structured-attrs), only defined if the derivation contains them.
|
||||
Structured attributes are JSON, and thus embedded as-is.
|
||||
type: object
|
||||
additionalProperties: true
|
||||
|
||||
"$defs":
|
||||
output:
|
||||
type: object
|
||||
properties:
|
||||
path:
|
||||
type: string
|
||||
description: |
|
||||
The output path, if known in advance.
|
||||
|
||||
method:
|
||||
type: string
|
||||
enum: [flat, nar, text, git]
|
||||
description: |
|
||||
For an output which will be [content addressed](@docroot@/store/derivation/outputs/content-address.md), a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
|
||||
|
||||
Valid method strings are:
|
||||
|
||||
- [`flat`](@docroot@/store/store-object/content-address.md#method-flat)
|
||||
- [`nar`](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||
- [`text`](@docroot@/store/store-object/content-address.md#method-text)
|
||||
- [`git`](@docroot@/store/store-object/content-address.md#method-git)
|
||||
|
||||
hashAlgo:
|
||||
"$ref": "./hash-v1.yaml#/$defs/algorithm"
|
||||
|
||||
hash:
|
||||
type: string
|
||||
description: |
|
||||
For fixed-output derivations, the expected content hash in base-16.
|
||||
29
doc/manual/source/protocols/json/schema/hash-v1.yaml
Normal file
29
doc/manual/source/protocols/json/schema/hash-v1.yaml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
"$schema": http://json-schema.org/draft-04/schema#
|
||||
"$id": https://nix.dev/manual/nix/latest/protocols/json/schema/hash-v1.json
|
||||
title: Hash
|
||||
description: |
|
||||
A cryptographic hash value used throughout Nix for content addressing and integrity verification.
|
||||
|
||||
This schema describes the JSON representation of Nix's `Hash` type.
|
||||
|
||||
TODO Work in progress
|
||||
type: object
|
||||
properties:
|
||||
algorithm:
|
||||
"$ref": "#/$defs/algorithm"
|
||||
required:
|
||||
- algorithm
|
||||
additionalProperties: false
|
||||
"$defs":
|
||||
algorithm:
|
||||
type: string
|
||||
enum:
|
||||
- blake3
|
||||
- md5
|
||||
- sha1
|
||||
- sha256
|
||||
- sha512
|
||||
description: |
|
||||
The hash algorithm used to compute the hash value.
|
||||
|
||||
`blake3` is currently experimental and requires the [`blake-hashing`](@docroot@/development/experimental-features.md#xp-feature-blake-hashing) experimental feature.
|
||||
2
doc/manual/source/protocols/meson.build
Normal file
2
doc/manual/source/protocols/meson.build
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
# Process JSON schema documentation
|
||||
subdir('json')
|
||||
Loading…
Add table
Add a link
Reference in a new issue