12 KiB
Setup Guide
This guide covers the setup process for onnx-web, including downloading the Windows bundle.
Contents
- Setup Guide
Cross-platform method
This works on both Linux and Windows, for both AMD and Nvidia, but requires some familiarity with the command line.
Install Git and Python
Install Git and Python 3.10 for your environment:
The latest version of git should be fine. Python should be 3.9 or 3.10, although 3.8 and 3.11 may work if the correct packages are available for your platform. If you already have Python installed for another form of Stable Diffusion, that should work, but make sure to verify the version in the next step.
Make sure you have Python 3.9 or 3.10:
> python --version
Python 3.10
If your system differentiates between Python 2 and 3 and uses the python3
and pip3
commands for the Python 3.x
tools, make sure to adjust the commands shown here. They should otherwise be the same: python3 --version
.
Once you have those basic packages installed, clone this git repository:
> git clone https://github.com/ssube/onnx-web.git
Note about setup paths
This project contains both Javascript and Python, for the client and server respectively. Make sure you are in the correct directory when working with each part.
Most of these setup commands should be run in the Python environment and the api/
directory:
> cd api
> pwd
/home/ssube/code/github/ssube/onnx-web/api
The Python virtual environment will be created within the api/
directory.
The Javascript client can be built and run within the gui/
directory.
Create a virtual environment
Change into the api/
directory, then create a virtual environment:
> pip install virtualenv
> python -m venv onnx_env
This will contain all of the pip libraries. If you update or reinstall Python, you will need to recreate the virtual environment.
If you receive an error like Error: name 'cmd' is not defined
, there may be a bug in the venv
module on certain
Debian-based systems. You may need to install venv through apt instead:
> sudo apt install python3-venv # only if you get an error
Every time you start using ONNX web, activate the virtual environment:
# on linux:
> source ./onnx_env/bin/activate
# on windows:
> .\onnx_env\Scripts\Activate.bat
Update pip itself:
> python -m pip install --upgrade pip
Install pip packages
You can install all of the necessary packages at once using the requirements/base.txt
file
and the requirements/
file for your platform. Install them in separate commands and make sure to install the
platform-specific packages first:
> pip install -r requirements/amd-linux.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/amd-windows.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/cpu.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/nvidia.txt
> pip install -r requirements/base.txt
Only install one of the platform-specific requirements files, otherwise you may end up with the wrong version of PyTorch or the ONNX runtime. The full list of available ONNX runtime packages can be found here .
If you have successfully installed both of the requirements files for your platform, you do not need to install any of the packages shown in the following sections and you should skip directly to testing the models.
The ONNX runtime nightly packages used by the requirements/*-nightly.txt
files can be substantially faster than the
last release, but may not always be stable. Many of the nightly packages are specific to one version of Python and
some are only available for Python 3.8 and 3.9, so you may need to find the correct package for your environment. If
you are using Python 3.10, download the cp310
package. For Python 3.9, download the cp39
package, and so on.
Installing with pip will figure out the correct package for you.
For AMD on Linux: PyTorch ROCm and ONNX runtime ROCm
If you are running on Linux with an AMD GPU, install the ROCm versions of PyTorch and onnxruntime
:
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/rocm5.2
# and one of
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.14.1%2Brocm54-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# or
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.15.0.dev20230326001%2Brocm542-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Make sure you have installed ROCm 5.x (see their documentation
for more
details) and that the version of onnxruntime
matches your ROCm drivers. The version of PyTorch does not need to match
exactly, and they only have limited versions available.
Ubuntu 20.04 supports ROCm 5.2 and Ubuntu 22.04 supports ROCm 5.4, unless you want to build custom packages. The ROCm 5.x series supports many discrete AMD cards since the Vega 20 architecture, with a partial list of supported cards shown here.
For AMD on Windows: PyTorch CPU and ONNX runtime DirectML
If you are running on Windows with an AMD GPU, install the DirectML ONNX runtime as well:
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and one of
> pip install onnxruntime-directml
# or
> pip install ort-nightly-directml --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
If you DirectML package upgrades numpy to an incompatible version >= 1.24, downgrade it:
> pip install "numpy>=1.20,<1.24" --force-reinstall # the DirectML package will upgrade numpy to 1.24, which will not work
You can optionally install the latest DirectML ORT nightly package, which may provide a substantial performance increase.
For CPU everywhere: PyTorch CPU and ONNX runtime CPU
If you are running with a CPU and no hardware acceleration, install onnxruntime
and the CPU version of PyTorch:
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and
> pip install onnxruntime
# or
> pip install ort-nightly --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
For Nvidia everywhere: Install PyTorch GPU and ONNX GPU
If you are running with an Nvidia GPU on any operating system, install onnxruntime-gpu
and the CUDA version of
PyTorch:
> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cu117
# and
> pip install onnxruntime-gpu
# or
> pip install ort-nightly-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall
Make sure you have installed CUDA 11.x and that the version of PyTorch matches the version of CUDA (see their documentation for more details).
Test the models
You should verify that all of the steps up to this point have worked correctly by attempting to run the
api/scripts/test-diffusers.py
script, which is a slight variation on the original txt2img script.
If the script works, there will be an image of an astronaut in outputs/test.png
.
If you get any errors, check the known errors section of the user guide.
Windows-specific methods
These methods are specific to Windows, tested on Windows 10, and still experimental. They should provide an easier setup experience.
Windows all-in-one bundle
- Install the latest Visual C++ 2019 redistributable
- Download the latest ZIP file from the apextoaster Nexus server
- Find the ZIP file and
Extract All
to a memorable folder - Open the folder where you extracted the files
- Your models will be converted into the
models
folder, and you can add your own models - Your images will be in the
outputs
folder, along with files containing the parameters used to generate them
- Your models will be converted into the
- Make sure the server is allowed to run
- Open the
server
folder - Right-click the
onnx-web.exe
file and clickProperties
- On the
General
tab, clickUnblock
next to the messageThis file came from another computer and might be blocked to help protect this computer.
- Go back to the folder where you extracted the files
- Repeat step 3 for the
onnx-web-*.bat
files
- Open the
- Run the local server using one of the
onnx-web-*.bat
scripts- Run
onnx-web-half.bat
if you are using a GPU and you have < 12GB of VRAM-half
mode is compatible with both AMD and Nvidia GPUs-half
mode is not compatible with CPU mode
- Run
onnx-web-full.bat
if you are using CPU mode or if you have >= 16GB of VRAM- Try the
onnx-web-half.bat
script if you encounter out-of-memory errors or generating images is very slow
- Try the
- Run
- Wait for the models to be downloaded and converted
- Most models are distributed in PyTorch format and need to be converted into ONNX format
- This only happens once for each model and takes a few minutes
- Open one of the URLs shown in the logs in your browser
- This will typically be http://127.0.0.1:5000?api=http://127.0.0.1:5000
- If you running the server on a different PC and not accessing it from a browser on the same system, use that PC's IP address instead of 127.0.0.1
- Any modern browser should work, including Chrome, Edge, and Firefox
- Mobile browsers also work, but have stricter mixed-content policies
Windows Python installer
- Install the latest Visual C++ 2019 redistributable
- Install Git
- Install Python 3.10
- Clone or download the onnx-web repository
git clone https://github.com/ssube/onnx-web.git
- https://github.com/ssube/onnx-web/archive/refs/heads/main.zip
- Open a command prompt window
- Run one of the
setup-*.bat
scripts- Run
setup-amd.bat
if you are using an AMD GPU and DirectML - Run
setup-nvidia.bat
if you are using an Nvidia GPU and CUDA - Run
setup-cpu.bat
if you are planning on only using CPU mode
- Run
- After the first run, you can run
launch.bat
instead of the setup script- You should only need to run the setup script once
- If you encounter any errors with Python imports, run the setup script again