1
0
Fork 0
Schema analysis, linting, and transformation for YAML
Go to file
renovate[bot] 6e1a875cb8 update: update typescript-eslint monorepo to v2.18.0 (#245)
Co-authored-by: WhiteSource Renovate <renovatebot@gmail.com>
2020-01-27 18:02:41 -06:00
.github adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
config lint(tests): clean up magic numbers 2019-11-19 06:03:48 -06:00
docs feat: validate rules while loading 2019-11-18 06:28:06 -06:00
rules fix(rules): remove min length from rule desc 2019-11-18 06:28:06 -06:00
scripts adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
src lint(tests): clean up magic numbers 2019-11-19 06:03:48 -06:00
test lint(tests): clean up magic numbers 2019-11-19 06:03:48 -06:00
vendor adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
.codeclimate.yml feat(build): replace tslint with eslint 2019-11-09 17:52:16 -06:00
.dockerignore adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
.gitignore adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
.gitlab-ci.yml feat(build): add sonar job 2019-11-10 09:59:58 -06:00
.mdlrc fix(build): actually fix mdl config 2019-06-30 16:51:10 -05:00
.npmignore adopt everything but rollup config (#118) 2019-11-09 21:47:19 -06:00
.npmrc fix(build): use npm mirror 2019-09-30 08:11:17 -05:00
CHANGELOG.md chore(release): 0.8.0 2019-11-10 07:28:45 -06:00
Dockerfile.alpine add: alpine dockerfile 2019-11-01 05:42:03 -05:00
Dockerfile.stretch fix(build): tag images with architecture 2019-11-01 05:42:03 -05:00
LICENSE.md docs: add repo url, fix example command 2019-06-15 18:33:50 -05:00
Makefile fix(docs): add rule options to readme 2019-11-09 22:48:15 -06:00
README.md fix(docs): put contents in section, use yarn-global target 2019-11-10 19:04:07 -06:00
package.json update: update typescript-eslint monorepo to v2.18.0 (#245) 2020-01-27 18:02:41 -06:00
renovate.json feat: extend renovate presets 2019-06-30 16:23:00 -05:00
run.mk fix(build): clean up after container builds 2019-11-03 13:13:24 -06:00
tsconfig.json feat: project, build, and bundle 2019-06-15 15:20:04 -05:00
yarn.lock update: update typescript-eslint monorepo to v2.18.0 (#245) 2020-01-27 18:02:41 -06:00

README.md

SALTY Dog

Rule-based JSON/YAML validator using JSON schemas. Capable of filtering elements to validate partial documents, supports multiple documents per stream or file, inserting defaults, and other magic.

Getting Started

salty-dog is distributed as a package and container, and can be installed or pulled:

> docker pull ssube/salty-dog:master
> yarn add -D salty-dog
> yarn global add salty-dog

Note: while the container is the preferred way of running salty-dog, it has a serious limitation: docker run combines stdout and stderr, making it impossible to separate logs and the output document. Writing either the logs or dest to a file works around this.

To download, validate, and apply a Kubernetes resource:

> curl https://raw.githubusercontent.com/ssube/k8s-shards/master/roles/apps/gitlab/server/templates/ingress.yml | \
    salty-dog \
      --rules rules/kubernetes.yml \
      --source - \
      --tag kubernetes | \
    kubectl apply --dry-run -f -

...
{"name":"salty-dog","hostname":"cerberus","pid":7860,"level":30,"msg":"all rules passed","time":"2019-06-16T02:04:37.797Z","v":0}
ingress.extensions/gitlab created (dry run)

Contents

Status

Pipeline status Lines of Code Test coverage MIT license FOSSA Status

Open bug count Open issue count Closed issue count

Renovate badge Dependency status Dev dependency status Known vulnerabilities

Maintainability score Technical debt ratio Quality issues Language grade: JavaScript Total alerts

Releases

github release link github release version github commits since release

npm package link npm release version Typescript definitions

docker image link docker image size

Build

Local Build

This project is written in Typescript and requires make, node, and yarn to build.

> git clone git@github.com:ssube/salty-dog.git
> cd salty-dog
> make

After building, run with node out/index.js or install globally with make yarn-global.

make targets are provided for some example arguments:

> curl https://raw.githubusercontent.com/ssube/k8s-shards/master/roles/apps/gitlab/server/templates/ingress.yml | \
    make run-stream 2> >($(yarn bin)/bunyan) > >(kubectl apply --dry-run -f -)

...
[2019-06-16T03:23:56.645Z]  INFO: salty-dog/8015 on cerberus: all rules passed
ingress.extensions/gitlab created (dry run)

Docker Build

This method does not require the usual dependencies to be installed, only docker itself.

Build with Docker:

# Stretch
docker run --rm -v "$(pwd):/salty-dog" -w /salty-dog node:11-stretch make
docker build -t salty-dog:stretch -f Dockerfile.stretch .

# Alpine
docker run --rm -v "$(pwd):/salty-dog" -w /salty-dog node:11-alpine sh -c "apk add build-base && make"
docker build -t salty-dog:alpine -f Dockerfile.alpine .

Install

Docker Install

To run with Docker: docker run --rm ssube/salty-dog:master

The latest semi-stable image is ssube/salty-dog:master. All tags are listed here.

The Docker container is published for each branch and git tag, tagged with the git tag (or branch slug).

Rules are baked into the image in /salty-dog/rules. To use custom rules, mount them with -v $(pwd)/rules:/salty-dog/rules:ro and load with --rules /rules/foo.yml.

Yarn Install

Global

To install with Yarn as a global CLI tool: yarn global add salty-dog

To run with Node:

> export PATH="${PATH}:$(yarn global bin)"
> salty-dog --help

Project

To install with Yarn for a single project: yarn add -D salty-dog

To run with Node:

> export PATH="${PATH}:$(yarn bin)"
> salty-dog --help

Usage

Logs

salty-dog uses node-bunyan for logging and prints JSON logs. These are not the easiest to read, and can be pretty-printed by redirecting stderr through bunyan itself or jq:

> cat resource.yml | salty-dog --rules rules/kubernetes.yml --tag kubernetes 2> >(bunyan)

...
[2019-06-15T23:53:34.223Z]  INFO: salty-dog/19839 on cerberus: all rules passed

> cat resource.yml | salty-dog --rules rules/kubernetes.yml --tag kubernetes 2> >(jq)

...
{
  "name": "salty-dog",
  "hostname": "cerberus",
  "pid": 19839,
  "level": 30,
  "msg": "all rules passed",
  "time": "2019-06-15T23:53:34.223Z",
  "v": 0
}

Using jq allows for additional filtering and formatting. For example, >(jq 'select(.level > 30)') will only print warnings and errors (log level is also part of the configuration file).

To print the last line's message and error messages: >(tail -1 | jq '[.msg, ((.errors // [])[] | .msg)]')

> cat test/examples/kubernetes-resources-high.yml | salty-dog \
    --rules rules/kubernetes.yml \
    --tag kubernetes 2> >(tail -1 | jq '[.msg, ((.errors // [])[] | .msg)]')

[
  "all rules passed"
]

> cat test/examples/kubernetes-resources-some.yml | salty-dog \
    --rules rules/kubernetes.yml \
    --tag kubernetes 2> >(tail -1 | jq '[.msg, ((.errors // [])[] | .msg)]')

[
  "some rules failed",
  ".resources.limits should have required property 'memory' at $.spec.template.spec.containers[*] for kubernetes-resources",
  ".metadata should have required property 'labels' at $ for kubernetes-labels"
]

Modes

salty-dog can run in a few different modes: check mode will report errors, fix mode will attempt to modify the input document, and list mode will print the active set of rules.

Check Mode

By default, salty-dog will validate the structure and contents of the --source document. If all rules pass, the document will be printed to --dest.

> cat examples/kubernetes-resources-pass.yml | salty-dog \
    --rules rules/kubernetes.yml \
    --tag kubernetes

...
[2019-06-15T23:53:34.223Z]  INFO: salty-dog/19839 on cerberus: all rules passed

> cat examples/kubernetes-resources-fail.yml | salty-dog \
    --rules rules/kubernetes.yml \
    --tag kubernetes

...
[2019-06-15T23:56:04.764Z] ERROR: salty-dog/22211 on cerberus: some rules failed (errors=1)

The --source and --dest default to stdin and stdout, respectively, but a path may be provided:

> salty-dog \
    --rules rules/kubernetes.yml \
    --tag kubernetes \
    --source examples/kubernetes-resources-pass.yml \
    --dest /tmp/kubernetes-resource.yml

...
[2019-06-15T23:53:34.223Z]  INFO: salty-dog/19839 on cerberus: all rules passed

Fix Mode

salty-dog can also add default values to missing properties in fix mode. If a rule does not immediately pass with the --source document, but defaults are provided in the schema, the defaults will be inserted before printing to --dest.

> salty-dog fix \
    --source examples/kubernetes-resources-some.yml \
    --rules rules/kubernetes.yml \
    --tag kubernetes
Default Values

Properties that appear in the schema with a default provided will be added to each element as it is checked. Rules apply in order, as do their defaults.

Coercing Values

Properties that appear in the document with a different type than they have in the schema may be coerced, if the value is compatible with the schema type. The full matrix of valid type coercions is documented by Ajv.

List Mode

salty-dog can list the active set of rules, to help debug tags and inclusion. Both --source and --dest are ignored in list mode.

> salty-dog list \
    --rules rules/kubernetes.yml \
    --tag kubernetes

...
[2019-06-30T18:39:11.930Z]  INFO: salty-dog/26330 on cerberus: listing active rules
    rules: [
      {
        "desc": "resource limits are too low",
        "level": "debug",
        "name": "kubernetes-resources-minimum-cpu",

...
    ]

Rules

Rules combine a jsonpath expression and JSON schema to select and validate the document.

The rule's select expression is used to select nodes that should be validated, which are filtered, then checked.

The structure of rule files and the rules within them are documented here.

Enable Rules

All rules are disabled by default and must be enabled by name, level, or tag.

To enable a single rule by name, --include-name foo-rule.

To enable a group of rules by level, --include-level warn.

To enable a group of rules by tag, --include-tag foo.

Load Rules

Rules can be loaded from a file, module, or path.

To load a file by name, --rule-file foo.yml. This will accept any extension.

To load a module, --rule-module foo. The required module exports are documented here.

To load a path, --rule-path foo/. This will recursively load any files matching *.+(json|yaml|yml).

Validate Rules

To validate the rules in the rules/ directory using the meta-rules:

> make test-rules

...
{"name":"salty-dog","hostname":"cerberus","pid":29403,"level":30,"msg":"all rules passed","time":"2019-06-16T00:56:55.132Z","v":0}

License

FOSSA Status