1
0
Fork 0
deploy-lock/README.md

6.5 KiB

Deploy Lock

This is a tool to lock a cluster or service, in order to prevent people from deploying changes during test automation or restarting pods during an infrastructure incident.

Features

  • in-memory data store, mostly for testing
  • DynamoDB data store
  • lock paths and recursive checking
  • infer lock data from CI variables

Contents

Concepts

readme banner top

Lock Path

Briefly describe paths.

More details here.

Lock Data

Each lock must contain the following fields:

interface Lock {
  type: 'automation' | 'deploy' | 'freeze' | 'incident' | 'maintenance';
  path: string;
  author: string;
  links: Map<string, string>;
  // often duplicates of path, but useful for cross-project locks
  source: string;

  // Timestamps, calculated from --duration and --until
  created_at: number;
  updated_at: number;
  expires_at: number;

  // CI fields, optional
  ci?: {
    project: string;
    ref: string;
    commit: string;
    pipeline: string;
    job: string;
  }
}

If $CI is not set, the ci sub-struct will not be present.

Usage

Command-line Interface

> deploy-lock check apps/staging/a/auth-app   # is equivalent to
> deploy-lock check apps && deploy-lock check apps/staging && deploy-lock check apps/staging/a && deploy-lock check apps/staging/a/auth-app
> deploy-lock check apps/staging/a/auth-app --recursive=false   # only checks the leaf node

> deploy-lock list
> deploy-lock list apps/staging    # list all locks within the apps/staging path

> deploy-lock lock apps/staging --type automation --duration 60m
> deploy-lock lock apps/staging/a/auth-app --type deploy --duration 5m
> deploy-lock lock apps/staging --until 2022-12-31T12:00   # local TZ, unless Z specified

> deploy-lock prune   # prune all expired locks
> deploy-lock prune apps/staging   # prune expired locks within the path
> deploy-lock prune apps/staging --now future-date   # prune locks that will expire by --now

> deploy-lock unlock apps/staging --type automation    # unlock type must match lock type

Basic Options

  • command
    • one of check, list, lock, prune, unlock
  • <path>
    • positional, required
    • string
    • lock path
    • always lowercase (forced in code)
    • /^[-a-z\/]+$/
  • --now
    • number, optional
    • defaults to current epoch time
  • --recursive
    • boolean
    • recursively check locks
    • defaults to true for check
    • defaults to false for lock, unlock
  • --type
    • string, enum
    • type of lock
    • one of automation, deploy, freeze, incident, or maintenance

Lock Data Options

  • --allow
    • array, strings
    • check types to allow while this lock is active
    • this does not allow creating other locks at the same path, it only changes whether the check returns allowed
  • --author
    • string
    • defaults to $GITLAB_USER_EMAIL if $GITLAB_CI is set
    • defaults to $USER otherwise
  • --duration
    • string
    • duration of lock, relative to now
    • may be given in epoch seconds (\d+), as an ISO-8601 date, or a human interval (30m)
    • mutually exclusive with --until
  • --link
    • array, strings
  • --source
    • string
    • each component:
      • first
        • defaults to $CLUSTER_NAME if set
        • defaults to path.split.0 otherwise
      • second
        • defaults to $DEPLOY_ENV if set
        • defaults to path.split.1 otherwise
      • third
        • defaults to $DEPLOY_TARGET if set
        • defaults to path.split.2 otherwise
      • fourth
        • defaults to --ci-project if set
        • defaults to $CI_PROJECT_PATH if set
        • defaults to path.split.3 otherwise
      • fifth
        • defaults to --ci-ref if set
        • defaults to $CI_COMMIT_REF_SLUG if set
        • defaults to path.split.4 otherwise
  • --until
    • string, timestamp
    • duration of lock, absolute
    • may be given in epoch seconds (\d+) or as an ISO-8601 date (intervals are not allowed)
    • mutually exclusive with --duration
  • --ci-project
    • optional string
    • project path
    • defaults to $CI_PROJECT_PATH if set
    • defaults to path.split.3 otherwise
  • --ci-ref
    • optional string
    • branch or tag
    • defaults to $CI_COMMIT_REF_SLUG if set
    • defaults to path.split.4 otherwise
  • --ci-commit
    • optional string
    • SHA of ref
    • defaults to $CI_COMMIT_SHA if set
  • --ci-pipeline
    • optional string
    • pipeline ID
    • defaults to $CI_PIPELINE_ID if set
  • --ci-job
    • optional string
    • job ID
    • defaults to $CI_JOB_ID if set

Storage Backend Options

  • --storage
    • string
    • one of dynamodb, memory
  • --region
    • string, optional
    • DynamoDB region name
  • --table
    • string
    • DynamoDB table name
  • --endpoint
    • string, optional
    • DynamoDB endpoint
    • set to http://localhost:8000 for testing with DynamoDB local
  • --fake
    • string, optional
    • a fake lock that should be added to the in-memory data store
    • the in-memory data store always starts empty, this is the only way to have an existing lock

Admission Controller Options

  • --admission-base
    • string
    • base path for admission controller requests
    • usually the path to the cluster within the deploy path, like aws/staging/us-east-1/apps/a
  • --admission-path
    • array, strings
    • path keys to use when checking admission controller requests
    • must be fields within the AdmissionReview kind
    • usually something like [namespace, name] or [namespace, requestResource.group, requestResource.resource, name]

REST API

Endpoints

  • /admission POST
    • Kubernetes admission controller webhook
  • /locks GET
    • equivalent to deploy-lock list
  • /locks POST
    • equivalent to deploy-lock lock
    • path taken from request body
  • /locks DELETE
    • equivalent to deploy-lock prune
  • /locks/:path GET
    • equivalent to deploy-lock check
  • /locks/:path PUT
    • equivalent to deploy-lock lock
    • path taken from URL path
  • /locks/:path DELETE
    • equivalent to deploy-lock unlock
  • /ok
    • health check

readme bottom banner