313 lines
12 KiB
Markdown
313 lines
12 KiB
Markdown
# Salty Dog
|
|
|
|
`salty-dog` is a tool to validate JSON and YAML data using JSON schema rules. It can filter elements and validate
|
|
select parts of the document, supports multiple documents in the same stream or file, and can insert defaults during
|
|
validation.
|
|
|
|
[Check out the getting started guide](./docs/getting-started.md).
|
|
|
|
## Contents
|
|
|
|
- [Salty Dog](#salty-dog)
|
|
- [Contents](#contents)
|
|
- [Status](#status)
|
|
- [Releases](#releases)
|
|
- [Build](#build)
|
|
- [Usage](#usage)
|
|
- [Using The Container](#using-the-container)
|
|
- [Using The Package](#using-the-package)
|
|
- [Project Package](#project-package)
|
|
- [Global Package](#global-package)
|
|
- [Usage Modes](#usage-modes)
|
|
- [Check Mode](#check-mode)
|
|
- [Fix Mode](#fix-mode)
|
|
- [Default Values](#default-values)
|
|
- [Coercing Values](#coercing-values)
|
|
- [List Mode](#list-mode)
|
|
- [Formatting Logs](#formatting-logs)
|
|
- [Rules](#rules)
|
|
- [Loading Rule Sources](#loading-rule-sources)
|
|
- [Using Rule Tags](#using-rule-tags)
|
|
- [Validating Rules](#validating-rules)
|
|
- [License](#license)
|
|
|
|
## Status
|
|
|
|
[](https://git.apextoaster.com/ssube/salty-dog/commits/master)
|
|
[](https://github.com/ssube/salty-dog/blob/master/LICENSE.md)
|
|
[](https://renovatebot.com)
|
|
[](https://snyk.io/test/github/ssube/salty-dog)
|
|
|
|
[](https://github.com/ssube/salty-dog/issues?q=is%3Aopen+is%3Aissue+label%3Atype%2Fbug)
|
|
[](https://github.com/ssube/salty-dog/issues?q=is%3Aopen+is%3Aissue)
|
|
[](https://github.com/ssube/salty-dog/issues?q=is%3Aissue+is%3Aclosed)
|
|
|
|
[](https://sonarcloud.io/dashboard?id=ssube_salty-dog)
|
|
[](https://codecov.io/gh/ssube/salty-dog)
|
|
[](https://codeclimate.com/github/ssube/salty-dog/trends/technical_debt)
|
|
|
|
[](https://codeclimate.com/github/ssube/salty-dog/maintainability)
|
|
[](https://codeclimate.com/github/ssube/salty-dog/issues)
|
|
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fssube%2Fsalty-dog?ref=badge_shield)
|
|
|
|
## Releases
|
|
|
|
[](https://github.com/ssube/salty-dog/releases)
|
|
[](https://github.com/ssube/salty-dog/releases)
|
|
[](https://github.com/ssube/salty-dog/compare/v0.9.2...master)
|
|
|
|
[](https://www.npmjs.com/package/salty-dog)
|
|
[](https://www.npmjs.com/package/salty-dog)
|
|
[](https://www.npmjs.com/package/salty-dog)
|
|
|
|
[](https://hub.docker.com/r/ssube/salty-dog)
|
|
[](https://microbadger.com/images/ssube/salty-dog:master)
|
|
|
|
## Build
|
|
|
|
`salty-dog` is written in Typescript and requires `make`, `node`, and `yarn` to build. It can be built locally or in
|
|
a container.
|
|
|
|
[Please see the build docs](./docs/index.md#build) for more details.
|
|
|
|
## Usage
|
|
|
|
`salty-dog` is distributed as a docker container and an npm package.
|
|
|
|
While the container is the preferred way of running `salty-dog`, it has one limitation: `docker run` combines
|
|
`stdout` and `stderr`, making it impossible to separate logs and the output document. Writing either the logs or dest
|
|
to a file works around this.
|
|
|
|
### Using The Container
|
|
|
|
To run the Docker container: `docker run --rm ssube/salty-dog:master`
|
|
|
|
The latest semi-stable image is published to `ssube/salty-dog:master`. Containers are published based on both Alpine
|
|
Linux and Debian (currently Stretch). All of the [available tags are listed here](https://hub.docker.com/r/ssube/salty-dog/tags).
|
|
|
|
Rules are provided in the image at `/salty-dog/rules`. To use custom rules in the container, mount them with
|
|
`-v $(pwd)/rules:/salty-dog/rules:ro` and load them with `--rules /rules/foo.yml`.
|
|
|
|
The `ssube/salty-dog` container image can be run normally or interactively.
|
|
|
|
To validate a file or input normally:
|
|
|
|
```shell
|
|
> docker run --rm ssube/salty-dog:master --help
|
|
```
|
|
|
|
You can also launch a shell within the container, using local rules:
|
|
|
|
```shell
|
|
> docker run \
|
|
--rm \
|
|
-it \
|
|
--entrypoint bash \
|
|
ssube/salty-dog:master
|
|
```
|
|
|
|
### Using The Package
|
|
|
|
`salty-dog` is also published as [an npm package](https://www.npmjs.com/package/salty-dog) with a binary, so it can
|
|
be used as a CLI command or programmatically.
|
|
|
|
#### Project Package
|
|
|
|
To install `salty-dog` for the current project:
|
|
|
|
```shell
|
|
> yarn add -D salty-dog
|
|
> $(yarn bin)/salty-dog --help
|
|
```
|
|
|
|
#### Global Package
|
|
|
|
It is also possible to install `salty-dog` globally, rather than within a project. However, this is
|
|
not recommended.
|
|
|
|
```shell
|
|
> yarn global add salty-dog
|
|
> export PATH="${PATH}:$(yarn global bin)"
|
|
> salty-dog --help
|
|
```
|
|
|
|
### Usage Modes
|
|
|
|
`salty-dog` can run in a few different modes: `check` mode will report errors, `fix` mode will attempt to modify the
|
|
input document, and `list` mode will print the active set of rules.
|
|
|
|
#### Check Mode
|
|
|
|
By default, `salty-dog` will validate the structure and contents of the `--source` document. If all rules pass, the
|
|
document will be printed to `--dest`.
|
|
|
|
```shell
|
|
> cat examples/kubernetes-resources-pass.yml | salty-dog \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes
|
|
|
|
...
|
|
[2019-06-15T23:53:34.223Z] INFO: salty-dog/19839 on cerberus: all rules passed
|
|
|
|
> cat examples/kubernetes-resources-fail.yml | salty-dog \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes
|
|
|
|
...
|
|
[2019-06-15T23:56:04.764Z] ERROR: salty-dog/22211 on cerberus: some rules failed (errors=1)
|
|
|
|
```
|
|
|
|
The `--source` and `--dest` default to stdin and stdout, respectively, but a path may be provided:
|
|
|
|
```shell
|
|
> salty-dog \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes \
|
|
--source examples/kubernetes-resources-pass.yml \
|
|
--dest /tmp/kubernetes-resource.yml
|
|
|
|
...
|
|
[2019-06-15T23:53:34.223Z] INFO: salty-dog/19839 on cerberus: all rules passed
|
|
```
|
|
|
|
#### Fix Mode
|
|
|
|
`salty-dog` can also add default values to missing properties in `fix` mode. If a rule does not immediately pass
|
|
with the `--source` document, but defaults are provided in the schema, the defaults will be inserted before printing to
|
|
`--dest`.
|
|
|
|
```shell
|
|
> salty-dog fix \
|
|
--source examples/kubernetes-resources-some.yml \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes
|
|
```
|
|
|
|
##### Default Values
|
|
|
|
Properties that appear in the schema with a `default` provided will be added to each element as it is checked. Rules
|
|
apply in order, as do their defaults.
|
|
|
|
##### Coercing Values
|
|
|
|
Properties that appear in the document with a different `type` than they have in the schema may be coerced, if the
|
|
value is compatible with the schema type. [The full matrix of valid type coercions](https://ajv.js.org/coercion.html)
|
|
is documented by Ajv.
|
|
|
|
#### List Mode
|
|
|
|
`salty-dog` can list the active set of rules, to help debug tags and inclusion. Both `--source` and `--dest` are
|
|
ignored in `list` mode.
|
|
|
|
```shell
|
|
> salty-dog list \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes
|
|
|
|
...
|
|
[2019-06-30T18:39:11.930Z] INFO: salty-dog/26330 on cerberus: listing active rules
|
|
rules: [
|
|
{
|
|
"desc": "resource limits are too low",
|
|
"level": "debug",
|
|
"name": "kubernetes-resources-minimum-cpu",
|
|
|
|
...
|
|
]
|
|
```
|
|
|
|
### Formatting Logs
|
|
|
|
`salty-dog` uses [node-bunyan](https://github.com/trentm/node-bunyan) for logging and prints structured JSON output.
|
|
Logs can be pretty-printed by redirecting `stderr` through `bunyan` itself or `jq`, both of which are installed in
|
|
the `salty-dog` container:
|
|
|
|
```shell
|
|
> cat resource.yml | salty-dog --rules rules/kubernetes.yml --tag kubernetes 2> >(bunyan)
|
|
|
|
...
|
|
[2019-06-15T23:53:34.223Z] INFO: salty-dog/19839 on cerberus: all rules passed
|
|
|
|
> cat resource.yml | salty-dog --rules rules/kubernetes.yml --tag kubernetes 2> >(jq)
|
|
|
|
...
|
|
{
|
|
"name": "salty-dog",
|
|
"hostname": "cerberus",
|
|
"pid": 19839,
|
|
"level": 30,
|
|
"msg": "all rules passed",
|
|
"time": "2019-06-15T23:53:34.223Z",
|
|
"v": 0
|
|
}
|
|
```
|
|
|
|
Using `jq` allows for additional filtering and formatting. For example, `jq 'select(.level > 30)'` will only print
|
|
warnings and errors (the minimum log level to print can be set in the configuration file).
|
|
|
|
To print the last line's message and error messages: `tail -1 | jq '[.msg, try (.errors[] | .msg)]'`
|
|
|
|
```shell
|
|
> cat test/examples/kubernetes-resources-high.yml | salty-dog \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes 2> >(tail -1 | jq '[.msg, try (.errors[] | .msg)]')
|
|
|
|
[
|
|
"all rules passed"
|
|
]
|
|
|
|
> cat test/examples/kubernetes-resources-some.yml | salty-dog \
|
|
--rules rules/kubernetes.yml \
|
|
--tag kubernetes 2> >(tail -1 | jq '[.msg, try (.errors[] | .msg)]')
|
|
|
|
[
|
|
"some rules failed",
|
|
".resources.limits should have required property 'memory' at $.spec.template.spec.containers[*] for kubernetes-resources",
|
|
".metadata should have required property 'labels' at $ for kubernetes-labels"
|
|
]
|
|
```
|
|
|
|
## Rules
|
|
|
|
Rules combine a jsonpath expression and JSON schema to select and validate the document.
|
|
|
|
The rule's `select` expression is used to select nodes that should be validated, which are `filter`ed, then `check`ed.
|
|
|
|
The structure of rule files and the rules within them [are documented here](docs/rules.md).
|
|
|
|
### Loading Rule Sources
|
|
|
|
Rules can be loaded from a file, module, or path.
|
|
|
|
To load a file by name, `--rule-file foo.yml`. This will accept any extension.
|
|
|
|
To load a module, `--rule-module foo`. The required module exports [are documented here](./docs/rules.md#from-module).
|
|
|
|
To load a path, `--rule-path foo/`. This will recursively load any files matching `*.+(json|yaml|yml)`.
|
|
|
|
### Using Rule Tags
|
|
|
|
All rules are disabled by default and must be enabled by name, level, or tag.
|
|
|
|
To enable a single rule by name, `--include-name foo-rule`.
|
|
|
|
To enable a group of rules by level, `--include-level warn`.
|
|
|
|
To enable a group of rules by tag, `--include-tag foo`.
|
|
|
|
### Validating Rules
|
|
|
|
To validate the rules in the `rules/` directory using the meta-rules:
|
|
|
|
```shell
|
|
> make test-rules
|
|
|
|
...
|
|
{"name":"salty-dog","hostname":"cerberus","pid":29403,"level":30,"msg":"all rules passed","time":"2019-06-16T00:56:55.132Z","v":0}
|
|
```
|
|
|
|
## License
|
|
|
|
[](https://app.fossa.io/projects/git%2Bgithub.com%2Fssube%2Fsalty-dog?ref=badge_large)
|