2023-02-01 14:53:20 +00:00
# Chain Pipelines
Chain pipelines are a new feature in v0.6 that allows you to run any combination of models on images
of almost any size, by automatically splitting them into smaller tiles as needed. Individual models
are run on each tile, then the results are recombined and passed on to the next stage.
## Contents
- [Chain Pipelines ](#chain-pipelines )
- [Contents ](#contents )
- [Overview ](#overview )
- [Format ](#format )
- [Stages ](#stages )
- [Blending Stages ](#blending-stages )
2023-12-16 23:03:25 +00:00
- [Blend: Denoise ](#blend-denoise )
- [Blend: Grid ](#blend-grid )
2023-02-01 14:53:20 +00:00
- [Blend: Img2img ](#blend-img2img )
2023-12-16 23:03:25 +00:00
- [Blend: Linear ](#blend-linear )
- [Blend: Mask ](#blend-mask )
2023-02-01 14:53:20 +00:00
- [Correction Stages ](#correction-stages )
2023-02-13 23:32:34 +00:00
- [Correct: CodeFormer ](#correct-codeformer )
2023-02-01 14:53:20 +00:00
- [Correct: GFPGAN ](#correct-gfpgan )
2023-12-16 23:03:25 +00:00
- [Compound Stages ](#compound-stages )
- [Highres Stage ](#highres-stage )
- [Upscale Stage ](#upscale-stage )
2023-02-01 14:53:20 +00:00
- [Persistence Stages ](#persistence-stages )
- [Persist: Disk ](#persist-disk )
- [Persist: S3 ](#persist-s3 )
- [Reduction Stages ](#reduction-stages )
- [Reduce: Crop ](#reduce-crop )
- [Reduce: Thumbnail ](#reduce-thumbnail )
- [Source Stages ](#source-stages )
- [Source: Noise ](#source-noise )
2023-12-16 23:03:25 +00:00
- [Source: S3 ](#source-s3 )
2023-02-01 14:53:20 +00:00
- [Source: Txt2img ](#source-txt2img )
2023-12-16 23:03:25 +00:00
- [Source: URL ](#source-url )
2023-02-01 14:53:20 +00:00
- [Upscaling Stages ](#upscaling-stages )
2023-12-16 23:03:25 +00:00
- [Upscale: BSRGAN ](#upscale-bsrgan )
- [Upscale: Highres ](#upscale-highres )
2023-02-01 14:53:20 +00:00
- [Upscale: Outpaint ](#upscale-outpaint )
- [Upscale: Real ESRGAN ](#upscale-real-esrgan )
2023-12-16 23:03:25 +00:00
- [Upscale: Simple ](#upscale-simple )
2023-02-01 14:53:20 +00:00
- [Upscale: Stable Diffusion ](#upscale-stable-diffusion )
2023-12-16 23:03:25 +00:00
- [Upscale: SwinIR ](#upscale-swinir )
2023-02-01 14:53:20 +00:00
## Overview
### Format
The `/api/chain` endpoint accepts a chain pipeline in JSON format and adds it to the queue of background jobs.
Pipelines are defined mostly through their `stages` , where each stage specifies a function to be run and the
parameters for that function, including the name of the model to be used.
The output of the pipeline _will not_ automatically be saved to disk, which is the case for the single-stage
endpoints. You must use at least one `persist-*` stage. Persist stages can be placed anywhere in the pipeline
and can also save intermediate output, such as the result of a `source-txt2img` stage before upscaling it.
```json
{
"stages": [
{
"name": "start",
"type": "source-txt2img",
"params": {
"prompt": "a magical wizard"
}
},
{
"name": "expand",
"type": "upscale-outpaint",
"params": {
"border": 256,
"prompt": "a magical wizard in a robe fighting a dragon"
}
},
{
"name": "save-local",
"type": "persist-disk",
"params": {
2023-09-13 22:28:38 +00:00
"tiles": "hd8k"
2023-02-01 14:53:20 +00:00
}
}
]
}
```
2023-12-17 22:53:18 +00:00
The complete schema can be found in [`api/schema.yaml` ](https://github.com/ssube/onnx-web/blob/main/api/schemas/chain.yaml ) and some example pipelines are available
in [`common/pipelines` ](https://github.com/ssube/onnx-web/tree/main/common/pipelines ).
2023-02-01 14:53:20 +00:00
## Stages
### Blending Stages
2023-12-16 23:03:25 +00:00
#### Blend: Denoise
2023-12-16 23:20:26 +00:00
Run [fast non-local means denoising ](https://docs.opencv.org/4.8.0/d5/d69/tutorial_py_non_local_means.html ) using `cv2` .
2023-12-16 23:03:25 +00:00
#### Blend: Grid
2023-12-16 23:20:26 +00:00
Combine the source images into a grid.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Blend: Img2img
2023-12-16 23:20:26 +00:00
Run an img2img pipeline.
2023-12-16 23:03:25 +00:00
#### Blend: Linear
2023-12-16 23:20:26 +00:00
Blend two images using linear interpolation (0.0 is the first image, 1.0 is the second).
2023-12-16 23:03:25 +00:00
#### Blend: Mask
2023-12-16 23:20:26 +00:00
Blend two images using a mask.
2023-02-01 14:53:20 +00:00
### Correction Stages
2023-02-13 23:32:34 +00:00
#### Correct: CodeFormer
2023-12-16 23:20:26 +00:00
Run correction using CodeFormer.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Correct: GFPGAN
2023-12-16 23:20:26 +00:00
Run correction using GFPGAN.
2023-12-16 23:03:25 +00:00
### Compound Stages
Not currently available through JSON API.
#### Highres Stage
2023-12-17 04:39:50 +00:00
Prep one or more highres iterations. Each iteration is an upscale stage followed by img2img.
2023-12-16 23:03:25 +00:00
#### Upscale Stage
2023-12-17 04:39:50 +00:00
Prep upscale and correction stages.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
### Persistence Stages
#### Persist: Disk
2023-12-16 23:20:26 +00:00
Save all of the sources to disk.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Persist: S3
2023-12-16 23:20:26 +00:00
Save all of the sources to an S3 bucket.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
### Reduction Stages
#### Reduce: Crop
2023-12-17 04:39:50 +00:00
Crop a section out of each source.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Reduce: Thumbnail
2023-12-17 04:39:50 +00:00
Downscale each image into a thumbnail of itself.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
### Source Stages
#### Source: Noise
2023-12-16 23:20:26 +00:00
Create a new source using a noise generator.
2023-12-16 23:03:25 +00:00
#### Source: S3
2023-12-16 23:20:26 +00:00
Load a new source from an S3 bucket.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Source: Txt2img
2023-12-16 23:20:26 +00:00
Run a txt2img pipeline.
2023-12-16 23:03:25 +00:00
#### Source: URL
2023-12-16 23:20:26 +00:00
Load a new source from a URL.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
### Upscaling Stages
2023-12-16 23:03:25 +00:00
#### Upscale: BSRGAN
2023-12-16 23:20:26 +00:00
Upscaling stage using BSRGAN.
2023-12-16 23:03:25 +00:00
#### Upscale: Highres
2023-12-16 23:20:26 +00:00
Upscaling stage using highres.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Upscale: Outpaint
Upscaling stage using outpainting. This adds empty borders to the source image, optionally fills them with noise, and
then runs inpainting on those areas.
#### Upscale: Real ESRGAN
Upscaling stage using the Real ESRGAN upscaling models, available in x2 and x4 versions:
- https://github.com/xinntao/Real-ESRGAN/releases
2023-12-16 23:03:25 +00:00
#### Upscale: Simple
2023-12-16 23:20:26 +00:00
Upscaling stage using bilinear or Lanczos upscaling.
2023-12-16 23:03:25 +00:00
2023-02-01 14:53:20 +00:00
#### Upscale: Stable Diffusion
Upscaling stage using the Stable Diffusion x4 upscaling model:
- https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler
- https://huggingface.co/ssube/stable-diffusion-x4-upscaler-onnx
2023-12-16 23:03:25 +00:00
#### Upscale: SwinIR
2023-12-16 23:20:26 +00:00
Upscaling stage using SwinIR.