295 lines
13 KiB
Markdown
295 lines
13 KiB
Markdown
# Setup Guide
|
|
|
|
This guide covers the setup process for onnx-web, including downloading the Windows bundle.
|
|
|
|
## Contents
|
|
|
|
- [Setup Guide](#setup-guide)
|
|
- [Contents](#contents)
|
|
- [Cross-platform method](#cross-platform-method)
|
|
- [Install Git and Python](#install-git-and-python)
|
|
- [Note about setup paths](#note-about-setup-paths)
|
|
- [Create a virtual environment](#create-a-virtual-environment)
|
|
- [Install pip packages](#install-pip-packages)
|
|
- [For AMD on Linux: PyTorch ROCm and ONNX runtime ROCm](#for-amd-on-linux-pytorch-rocm-and-onnx-runtime-rocm)
|
|
- [For AMD on Windows: PyTorch CPU and ONNX runtime DirectML](#for-amd-on-windows-pytorch-cpu-and-onnx-runtime-directml)
|
|
- [For CPU everywhere: PyTorch CPU and ONNX runtime CPU](#for-cpu-everywhere-pytorch-cpu-and-onnx-runtime-cpu)
|
|
- [For Nvidia everywhere: Install PyTorch GPU and ONNX GPU](#for-nvidia-everywhere-install-pytorch-gpu-and-onnx-gpu)
|
|
- [Test the models](#test-the-models)
|
|
- [Download the web UI bundle](#download-the-web-ui-bundle)
|
|
- [Windows-specific methods](#windows-specific-methods)
|
|
- [Windows all-in-one bundle](#windows-all-in-one-bundle)
|
|
- [Windows Python installer](#windows-python-installer)
|
|
|
|
## Cross-platform method
|
|
|
|
This works on both Linux and Windows, for both AMD and Nvidia, but requires some familiarity with the command line.
|
|
|
|
### Install Git and Python
|
|
|
|
Install Git and Python 3.10 for your environment:
|
|
|
|
- https://gitforwindows.org/
|
|
- https://www.python.org/downloads/
|
|
|
|
The latest version of git should be fine. Python should be 3.9 or 3.10, although 3.8 and 3.11 may work if the correct
|
|
packages are available for your platform. If you already have Python installed for another form of Stable Diffusion,
|
|
that should work, but make sure to verify the version in the next step.
|
|
|
|
Make sure you have Python 3.9 or 3.10:
|
|
|
|
```shell
|
|
> python --version
|
|
Python 3.10
|
|
```
|
|
|
|
If your system differentiates between Python 2 and 3 and uses the `python3` and `pip3` commands for the Python 3.x
|
|
tools, make sure to adjust the commands shown here. They should otherwise be the same: `python3 --version`.
|
|
|
|
Once you have those basic packages installed, clone this git repository:
|
|
|
|
```shell
|
|
> git clone https://github.com/ssube/onnx-web.git
|
|
```
|
|
|
|
### Note about setup paths
|
|
|
|
This project contains both Javascript and Python, for the client and server respectively. Make sure you are in the
|
|
correct directory when working with each part.
|
|
|
|
Most of these setup commands should be run in the Python environment and the `api/` directory:
|
|
|
|
```shell
|
|
> cd api
|
|
> pwd
|
|
/home/ssube/code/github/ssube/onnx-web/api
|
|
```
|
|
|
|
The Python virtual environment will be created within the `api/` directory.
|
|
|
|
The Javascript client can be built and run within the `gui/` directory.
|
|
|
|
### Create a virtual environment
|
|
|
|
Change into the `api/` directory, then create a virtual environment:
|
|
|
|
```shell
|
|
> pip install virtualenv
|
|
> python -m venv onnx_env
|
|
```
|
|
|
|
This will contain all of the pip libraries. If you update or reinstall Python, you will need to recreate the virtual
|
|
environment.
|
|
|
|
If you receive an error like `Error: name 'cmd' is not defined`, there may be [a bug in the `venv` module](https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1884072.html) on certain
|
|
Debian-based systems. You may need to install venv through apt instead:
|
|
|
|
```shell
|
|
> sudo apt install python3-venv # only if you get an error
|
|
```
|
|
|
|
Every time you start using onnx-web, activate the virtual environment:
|
|
|
|
```shell
|
|
# on linux:
|
|
> source ./onnx_env/bin/activate
|
|
|
|
# on windows:
|
|
> .\onnx_env\Scripts\Activate.bat
|
|
```
|
|
|
|
Update pip itself:
|
|
|
|
```shell
|
|
> python -m pip install --upgrade pip
|
|
```
|
|
|
|
### Install pip packages
|
|
|
|
You can install all of the necessary packages at once using [the `requirements/base.txt` file](./api/requirements/base.txt)
|
|
and the `requirements/` file for your platform. Install them in separate commands and make sure to install the
|
|
platform-specific packages first:
|
|
|
|
```shell
|
|
> pip install -r requirements/amd-linux.txt
|
|
> pip install -r requirements/base.txt
|
|
# or
|
|
> pip install -r requirements/amd-windows.txt
|
|
> pip install -r requirements/base.txt
|
|
# or
|
|
> pip install -r requirements/cpu.txt
|
|
> pip install -r requirements/base.txt
|
|
# or
|
|
> pip install -r requirements/nvidia.txt
|
|
> pip install -r requirements/base.txt
|
|
```
|
|
|
|
Only install one of the platform-specific requirements files, otherwise you may end up with the wrong version of
|
|
PyTorch or the ONNX runtime. The full list of available ONNX runtime packages [can be found here
|
|
](https://download.onnxruntime.ai/).
|
|
|
|
If you have successfully installed both of the requirements files for your platform, you do not need to install
|
|
any of the packages shown in the following sections and you should [skip directly to testing the models](#test-the-models).
|
|
|
|
The ONNX runtime nightly packages used by the `requirements/*-nightly.txt` files can be substantially faster than the
|
|
last release, but may not always be stable. Many of the nightly packages are specific to one version of Python and
|
|
some are only available for Python 3.8 and 3.9, so you may need to find the correct package for your environment. If
|
|
you are using Python 3.10, download the `cp310` package. For Python 3.9, download the `cp39` package, and so on.
|
|
Installing with pip will figure out the correct package for you.
|
|
|
|
#### For AMD on Linux: PyTorch ROCm and ONNX runtime ROCm
|
|
|
|
If you are running on Linux with an AMD GPU, install the ROCm versions of PyTorch and `onnxruntime`:
|
|
|
|
```shell
|
|
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/rocm5.2
|
|
# and one of
|
|
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.14.1%2Brocm54-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
|
|
# or
|
|
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.15.0.dev20230326001%2Brocm542-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
|
|
```
|
|
|
|
Make sure you have installed ROCm 5.x ([see their documentation
|
|
](https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.2.3/page/How_to_Install_ROCm.html#_How_to_Install) for more
|
|
details) and that the version of `onnxruntime` matches your ROCm drivers. The version of PyTorch does not need to match
|
|
exactly, and they only have limited versions available.
|
|
|
|
Ubuntu 20.04 supports ROCm 5.2 and Ubuntu 22.04 supports ROCm 5.4, unless you want to build custom packages. The ROCm
|
|
5.x series supports many discrete AMD cards since the Vega 20 architecture, with [a partial list of supported cards
|
|
shown here](https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4.3/page/Prerequisites.html#d5434e465).
|
|
|
|
#### For AMD on Windows: PyTorch CPU and ONNX runtime DirectML
|
|
|
|
If you are running on Windows with an AMD GPU, install the DirectML ONNX runtime as well:
|
|
|
|
```shell
|
|
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
|
|
# and one of
|
|
> pip install onnxruntime-directml
|
|
# or
|
|
> pip install ort-nightly-directml --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
|
|
```
|
|
|
|
If you DirectML package upgrades numpy to an incompatible version >= 1.24, downgrade it:
|
|
|
|
```shell
|
|
> pip install "numpy>=1.20,<1.24" --force-reinstall # the DirectML package will upgrade numpy to 1.24, which will not work
|
|
```
|
|
|
|
You can optionally install the latest DirectML ORT nightly package, which may provide a substantial performance
|
|
increase.
|
|
|
|
#### For CPU everywhere: PyTorch CPU and ONNX runtime CPU
|
|
|
|
If you are running with a CPU and no hardware acceleration, install `onnxruntime` and the CPU version of PyTorch:
|
|
|
|
```shell
|
|
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
|
|
# and
|
|
> pip install onnxruntime
|
|
# or
|
|
> pip install ort-nightly --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
|
|
```
|
|
|
|
#### For Nvidia everywhere: Install PyTorch GPU and ONNX GPU
|
|
|
|
If you are running with an Nvidia GPU on any operating system, install `onnxruntime-gpu` and the CUDA version of
|
|
PyTorch:
|
|
|
|
```shell
|
|
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cu117
|
|
# and
|
|
> pip install onnxruntime-gpu
|
|
# or
|
|
> pip install ort-nightly-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
|
|
```
|
|
|
|
Make sure you have installed CUDA 11.x and that the version of PyTorch matches the version of CUDA
|
|
([see their documentation](https://pytorch.org/get-started/locally/) for more details).
|
|
|
|
### Test the models
|
|
|
|
You should verify that all of the steps up to this point have worked correctly by attempting to run the
|
|
`api/scripts/test-diffusers.py` script, which is a slight variation on the original txt2img script.
|
|
|
|
If the script works, there will be an image of an astronaut in `outputs/test.png`.
|
|
|
|
If you get any errors, check [the known errors section of the user guide](user-guide.md#known-errors).
|
|
|
|
### Download the web UI bundle
|
|
|
|
Once the server environment is working, you will need the latest files for the web UI. This is a Javascript bundle and
|
|
you can download a pre-built copy from Github or compile your own.
|
|
|
|
From [the `gh-pages` branch](https://github.com/ssube/onnx-web/tree/gh-pages), select the version matching your server
|
|
and download all three files:
|
|
|
|
- `bundle/main.js`
|
|
- `config.json`
|
|
- `index.html`
|
|
|
|
Copy them into your local `api/gui` folder, making sure to keep the `main.js` bundle in the `bundle` subfolder.
|
|
|
|
For example, for a v0.11 server, copy the files from https://github.com/ssube/onnx-web/tree/gh-pages/v0.11.0 into your
|
|
local copy of https://github.com/ssube/onnx-web/tree/main/api/gui and
|
|
https://github.com/ssube/onnx-web/tree/main/api/gui/bundle.
|
|
|
|
## Windows-specific methods
|
|
|
|
These methods are specific to Windows, tested on Windows 10, and still experimental. They should provide an easier
|
|
setup experience.
|
|
|
|
### Windows all-in-one bundle
|
|
|
|
1. Install the latest Visual C++ 2019 redistributable
|
|
1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
|
|
2. https://aka.ms/vs/17/release/vc_redist.x64.exe
|
|
2. Download the latest ZIP file from [the apextoaster Nexus server](https://artifacts.apextoaster.com/#browse/browse:onnx-web-dist)
|
|
3. Find the ZIP file and `Extract All` to a memorable folder
|
|
4. Open the folder where you extracted the files
|
|
1. Your models will be converted into the `models` folder, and you can [add your own models](user-guide.md#adding-your-own-models)
|
|
2. Your images will be in the `outputs` folder, along with files containing the parameters used to generate them
|
|
5. Make sure the server is allowed to run
|
|
1. Open the `server` folder
|
|
2. Right-click the `onnx-web.exe` file and click `Properties`
|
|
3. On the `General` tab, click `Unblock` next to the message `This file came from another computer and might be
|
|
blocked to help protect this computer.`
|
|
4. Go back to the folder where you extracted the files
|
|
5. Repeat step 3 for the `onnx-web-*.bat` files
|
|
6. Run the local server using one of the `onnx-web-*.bat` scripts
|
|
1. Run `onnx-web-half.bat` if you are using a GPU and you have < 12GB of VRAM
|
|
- `-half` mode is compatible with both AMD and Nvidia GPUs
|
|
- `-half` mode is not compatible with CPU mode
|
|
2. Run `onnx-web-full.bat` if you are using CPU mode or if you have >= 16GB of VRAM
|
|
- Try the `onnx-web-half.bat` script if you encounter out-of-memory errors or generating images is very slow
|
|
7. Wait for the models to be downloaded and converted
|
|
1. Most models are distributed in PyTorch format and need to be converted into ONNX format
|
|
2. This only happens once for each model and takes a few minutes
|
|
8. Open one of the URLs shown in the logs in your browser
|
|
1. This will typically be http://127.0.0.1:5000?api=http://127.0.0.1:5000
|
|
2. If you running the server on a different PC and not accessing it from a browser on the same system, use that PC's
|
|
IP address instead of 127.0.0.1
|
|
3. Any modern browser should work, including Chrome, Edge, and Firefox
|
|
4. Mobile browsers also work, but have stricter mixed-content policies
|
|
|
|
### Windows Python installer
|
|
|
|
1. Install the latest Visual C++ 2019 redistributable
|
|
1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
|
|
2. https://aka.ms/vs/17/release/vc_redist.x64.exe
|
|
2. Install Git
|
|
- https://gitforwindows.org/
|
|
3. Install Python 3.10
|
|
- https://www.python.org/downloads/
|
|
4. Clone or download the onnx-web repository
|
|
- `git clone https://github.com/ssube/onnx-web.git`
|
|
- https://github.com/ssube/onnx-web/archive/refs/heads/main.zip
|
|
5. Open a command prompt window
|
|
6. Run one of the `setup-*.bat` scripts
|
|
1. Run `setup-amd.bat` if you are using an AMD GPU and DirectML
|
|
2. Run `setup-nvidia.bat` if you are using an Nvidia GPU and CUDA
|
|
3. Run `setup-cpu.bat` if you are planning on only using CPU mode
|
|
7. After the first run, you can run `launch.bat` instead of the setup script
|
|
1. You should only need to run the setup script once
|
|
2. If you encounter any errors with Python imports, run the setup script again
|