238 lines
6.5 KiB
Markdown
238 lines
6.5 KiB
Markdown
# Deploy Lock
|
|
|
|
This is a tool to lock a cluster or service, in order to prevent people from deploying changes during test automation or
|
|
restarting pods during an infrastructure incident.
|
|
|
|
## Features
|
|
|
|
- in-memory data store, mostly for testing
|
|
- DynamoDB data store
|
|
- lock paths and recursive checking
|
|
- infer lock data from CI variables
|
|
|
|
## Contents
|
|
|
|
- [Deploy Lock](#deploy-lock)
|
|
- [Features](#features)
|
|
- [Contents](#contents)
|
|
- [Concepts](#concepts)
|
|
- [Lock Path](#lock-path)
|
|
- [Lock Data](#lock-data)
|
|
- [Usage](#usage)
|
|
- [Command-line Interface](#command-line-interface)
|
|
- [Basic Options](#basic-options)
|
|
- [Lock Data Options](#lock-data-options)
|
|
- [Storage Backend Options](#storage-backend-options)
|
|
- [Admission Controller Options](#admission-controller-options)
|
|
- [REST API](#rest-api)
|
|
- [Endpoints](#endpoints)
|
|
|
|
## Concepts
|
|
|
|
|
|
|
|
### Lock Path
|
|
|
|
Briefly describe paths.
|
|
|
|
[More details here.](./docs/concepts.md#deploy-path)
|
|
|
|
### Lock Data
|
|
|
|
Each lock must contain the following fields:
|
|
|
|
```typescript
|
|
interface Lock {
|
|
type: 'automation' | 'deploy' | 'freeze' | 'incident' | 'maintenance';
|
|
path: string;
|
|
author: string;
|
|
links: Map<string, string>;
|
|
// often duplicates of path, but useful for cross-project locks
|
|
source: string;
|
|
|
|
// Timestamps, calculated from --duration and --until
|
|
created_at: number;
|
|
updated_at: number;
|
|
expires_at: number;
|
|
|
|
// CI fields, optional
|
|
ci?: {
|
|
project: string;
|
|
ref: string;
|
|
commit: string;
|
|
pipeline: string;
|
|
job: string;
|
|
}
|
|
}
|
|
```
|
|
|
|
If `$CI` is not set, the `ci` sub-struct will not be present.
|
|
|
|
## Usage
|
|
|
|
### Command-line Interface
|
|
|
|
```shell
|
|
> deploy-lock check apps/staging/a/auth-app # is equivalent to
|
|
> deploy-lock check apps && deploy-lock check apps/staging && deploy-lock check apps/staging/a && deploy-lock check apps/staging/a/auth-app
|
|
> deploy-lock check apps/staging/a/auth-app --recursive=false # only checks the leaf node
|
|
|
|
> deploy-lock list
|
|
> deploy-lock list apps/staging # list all locks within the apps/staging path
|
|
|
|
> deploy-lock lock apps/staging --type automation --duration 60m
|
|
> deploy-lock lock apps/staging/a/auth-app --type deploy --duration 5m
|
|
> deploy-lock lock apps/staging --until 2022-12-31T12:00 # local TZ, unless Z specified
|
|
|
|
> deploy-lock prune # prune all expired locks
|
|
> deploy-lock prune apps/staging # prune expired locks within the path
|
|
> deploy-lock prune apps/staging --now future-date # prune locks that will expire by --now
|
|
|
|
> deploy-lock unlock apps/staging --type automation # unlock type must match lock type
|
|
```
|
|
|
|
#### Basic Options
|
|
|
|
- command
|
|
- one of `check`, `list`, `lock`, `prune`, `unlock`
|
|
- `<path>`
|
|
- positional, required
|
|
- string
|
|
- lock path
|
|
- always lowercase (forced in code)
|
|
- `/^[-a-z\/]+$/`
|
|
- `--now`
|
|
- number, optional
|
|
- defaults to current epoch time
|
|
- `--recursive`
|
|
- boolean
|
|
- recursively check locks
|
|
- defaults to true for `check`
|
|
- defaults to false for `lock`, `unlock`
|
|
- `--type`
|
|
- string, enum
|
|
- type of lock
|
|
- one of `automation`, `deploy`, `freeze`, `incident`, or `maintenance`
|
|
|
|
#### Lock Data Options
|
|
|
|
- `--allow`
|
|
- array, strings
|
|
- check types to allow while this lock is active
|
|
- this does not allow creating other locks at the same path, it only changes whether the check returns allowed
|
|
- `--author`
|
|
- string
|
|
- defaults to `$GITLAB_USER_EMAIL` if `$GITLAB_CI` is set
|
|
- defaults to `$USER` otherwise
|
|
- `--duration`
|
|
- string
|
|
- duration of lock, relative to now
|
|
- may be given in epoch seconds (`\d+`), as an ISO-8601 date, or a human interval (`30m`)
|
|
- mutually exclusive with `--until`
|
|
- `--link`
|
|
- array, strings
|
|
- `--source`
|
|
- string
|
|
- each component:
|
|
- first
|
|
- defaults to `$CLUSTER_NAME` if set
|
|
- defaults to `path.split.0` otherwise
|
|
- second
|
|
- defaults to `$DEPLOY_ENV` if set
|
|
- defaults to `path.split.1` otherwise
|
|
- third
|
|
- defaults to `$DEPLOY_TARGET` if set
|
|
- defaults to `path.split.2` otherwise
|
|
- fourth
|
|
- defaults to `--ci-project` if set
|
|
- defaults to `$CI_PROJECT_PATH` if set
|
|
- defaults to `path.split.3` otherwise
|
|
- fifth
|
|
- defaults to `--ci-ref` if set
|
|
- defaults to `$CI_COMMIT_REF_SLUG` if set
|
|
- defaults to `path.split.4` otherwise
|
|
- `--until`
|
|
- string, timestamp
|
|
- duration of lock, absolute
|
|
- may be given in epoch seconds (`\d+`) or as an ISO-8601 date (intervals are not allowed)
|
|
- mutually exclusive with `--duration`
|
|
- `--ci-project`
|
|
- optional string
|
|
- project path
|
|
- defaults to `$CI_PROJECT_PATH` if set
|
|
- defaults to `path.split.3` otherwise
|
|
- `--ci-ref`
|
|
- optional string
|
|
- branch or tag
|
|
- defaults to `$CI_COMMIT_REF_SLUG` if set
|
|
- defaults to `path.split.4` otherwise
|
|
- `--ci-commit`
|
|
- optional string
|
|
- SHA of ref
|
|
- defaults to `$CI_COMMIT_SHA` if set
|
|
- `--ci-pipeline`
|
|
- optional string
|
|
- pipeline ID
|
|
- defaults to `$CI_PIPELINE_ID` if set
|
|
- `--ci-job`
|
|
- optional string
|
|
- job ID
|
|
- defaults to `$CI_JOB_ID` if set
|
|
|
|
#### Storage Backend Options
|
|
|
|
- `--storage`
|
|
- string
|
|
- one of `dynamodb`, `memory`
|
|
- `--region`
|
|
- string, optional
|
|
- DynamoDB region name
|
|
- `--table`
|
|
- string
|
|
- DynamoDB table name
|
|
- `--endpoint`
|
|
- string, optional
|
|
- DynamoDB endpoint
|
|
- set to `http://localhost:8000` for testing with DynamoDB local
|
|
- `--fake`
|
|
- string, optional
|
|
- a fake lock that should be added to the in-memory data store
|
|
- the in-memory data store always starts empty, this is the only way to have an existing lock
|
|
|
|
#### Admission Controller Options
|
|
|
|
- `--admission-base`
|
|
- string
|
|
- base path for admission controller requests
|
|
- usually the path to the cluster within the deploy path, like `aws/staging/us-east-1/apps/a`
|
|
- `--admission-path`
|
|
- array, strings
|
|
- path keys to use when checking admission controller requests
|
|
- must be fields within the `AdmissionReview` kind
|
|
- usually something like `[namespace, name]` or `[namespace, requestResource.group, requestResource.resource, name]`
|
|
|
|
### REST API
|
|
|
|
#### Endpoints
|
|
|
|
- `/admission POST`
|
|
- Kubernetes admission controller webhook
|
|
- `/locks GET`
|
|
- equivalent to `deploy-lock list`
|
|
- `/locks POST`
|
|
- equivalent to `deploy-lock lock`
|
|
- path taken from request body
|
|
- `/locks DELETE`
|
|
- equivalent to `deploy-lock prune`
|
|
- `/locks/:path GET`
|
|
- equivalent to `deploy-lock check`
|
|
- `/locks/:path PUT`
|
|
- equivalent to `deploy-lock lock`
|
|
- path taken from URL path
|
|
- `/locks/:path DELETE`
|
|
- equivalent to `deploy-lock unlock`
|
|
- `/ok`
|
|
- health check
|
|
|
|
|