apextoaster
/
onnx-web
1
0
Fork 0
Code Actions
onnx-web/docs/setup-guide.md

12 KiB
Raw Blame History

Setup Guide

This guide covers the setup process for onnx-web, including downloading the Windows bundle.

Contents

Cross-platform method

This works on both Linux and Windows, for both AMD and Nvidia, but requires some familiarity with the command line.

Install Git and Python

Install Git and Python 3.10 for your environment:

The latest version of git should be fine. Python should be 3.9 or 3.10, although 3.8 and 3.11 may work if the correct packages are available for your platform. If you already have Python installed for another form of Stable Diffusion, that should work, but make sure to verify the version in the next step.

Make sure you have Python 3.9 or 3.10:

> python --version
Python 3.10

If your system differentiates between Python 2 and 3 and uses the python3 and pip3 commands for the Python 3.x tools, make sure to adjust the commands shown here. They should otherwise be the same: python3 --version.

Once you have those basic packages installed, clone this git repository:

> git clone https://github.com/ssube/onnx-web.git

Note about setup paths

This project contains both Javascript and Python, for the client and server respectively. Make sure you are in the correct directory when working with each part.

Most of these setup commands should be run in the Python environment and the api/ directory:

> cd api
> pwd
/home/ssube/code/github/ssube/onnx-web/api

The Python virtual environment will be created within the api/ directory.

The Javascript client can be built and run within the gui/ directory.

Create a virtual environment

Change into the api/ directory, then create a virtual environment:

> pip install virtualenv
> python -m venv onnx_env

This will contain all of the pip libraries. If you update or reinstall Python, you will need to recreate the virtual environment.

If you receive an error like Error: name 'cmd' is not defined, there may be a bug in the venv module on certain Debian-based systems. You may need to install venv through apt instead:

> sudo apt install python3-venv   # only if you get an error

Every time you start using ONNX web, activate the virtual environment:

# on linux:
> source ./onnx_env/bin/activate

# on windows:
> .\onnx_env\Scripts\Activate.bat

Update pip itself:

> python -m pip install --upgrade pip

Install pip packages

You can install all of the necessary packages at once using the requirements/base.txt file and the requirements/ file for your platform. Install them in separate commands and make sure to install the platform-specific packages first:

> pip install -r requirements/amd-linux.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/amd-windows.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/cpu.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/nvidia.txt
> pip install -r requirements/base.txt

Only install one of the platform-specific requirements files, otherwise you may end up with the wrong version of PyTorch or the ONNX runtime. The full list of available ONNX runtime packages can be found here .

If you have successfully installed both of the requirements files for your platform, you do not need to install any of the packages shown in the following sections and you should skip directly to testing the models.

The ONNX runtime nightly packages used by the requirements/*-nightly.txt files can be substantially faster than the last release, but may not always be stable. Many of the nightly packages are specific to one version of Python and some are only available for Python 3.8 and 3.9, so you may need to find the correct package for your environment. If you are using Python 3.10, download the cp310 package. For Python 3.9, download the cp39 package, and so on. Installing with pip will figure out the correct package for you.

For AMD on Linux: PyTorch ROCm and ONNX runtime ROCm

If you are running on Linux with an AMD GPU, install the ROCm versions of PyTorch and onnxruntime:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/rocm5.2
# and one of
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.14.1%2Brocm54-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# or
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.15.0.dev20230326001%2Brocm542-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Make sure you have installed ROCm 5.x (see their documentation for more details) and that the version of onnxruntime matches your ROCm drivers. The version of PyTorch does not need to match exactly, and they only have limited versions available.

Ubuntu 20.04 supports ROCm 5.2 and Ubuntu 22.04 supports ROCm 5.4, unless you want to build custom packages. The ROCm 5.x series supports many discrete AMD cards since the Vega 20 architecture, with a partial list of supported cards shown here.

For AMD on Windows: PyTorch CPU and ONNX runtime DirectML

If you are running on Windows with an AMD GPU, install the DirectML ONNX runtime as well:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and one of
> pip install onnxruntime-directml
# or
> pip install ort-nightly-directml --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

If you DirectML package upgrades numpy to an incompatible version >= 1.24, downgrade it:

> pip install "numpy>=1.20,<1.24" --force-reinstall  # the DirectML package will upgrade numpy to 1.24, which will not work

You can optionally install the latest DirectML ORT nightly package, which may provide a substantial performance increase.

For CPU everywhere: PyTorch CPU and ONNX runtime CPU

If you are running with a CPU and no hardware acceleration, install onnxruntime and the CPU version of PyTorch:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and
> pip install onnxruntime
# or
> pip install ort-nightly --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

For Nvidia everywhere: Install PyTorch GPU and ONNX GPU

If you are running with an Nvidia GPU on any operating system, install onnxruntime-gpu and the CUDA version of PyTorch:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cu117
# and
> pip install onnxruntime-gpu
# or
> pip install ort-nightly-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

Make sure you have installed CUDA 11.x and that the version of PyTorch matches the version of CUDA (see their documentation for more details).

Test the models

You should verify that all of the steps up to this point have worked correctly by attempting to run the api/scripts/test-diffusers.py script, which is a slight variation on the original txt2img script.

If the script works, there will be an image of an astronaut in outputs/test.png.

If you get any errors, check the known errors section of the user guide.

Windows-specific methods

These methods are specific to Windows, tested on Windows 10, and still experimental. They should provide an easier setup experience.

Windows all-in-one bundle

  1. Install the latest Visual C++ 2019 redistributable
    1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
    2. https://aka.ms/vs/17/release/vc_redist.x64.exe
  2. Download the latest ZIP file from the apextoaster Nexus server
  3. Find the ZIP file and Extract All to a memorable folder
  4. Open the folder where you extracted the files
    1. Your models will be converted into the models folder, and you can add your own models
    2. Your images will be in the outputs folder, along with files containing the parameters used to generate them
  5. Make sure the server is allowed to run
    1. Open the server folder
    2. Right-click the onnx-web.exe file and click Properties
    3. On the General tab, click Unblock next to the message This file came from another computer and might be blocked to help protect this computer.
    4. Go back to the folder where you extracted the files
    5. Repeat step 3 for the onnx-web-*.bat files
  6. Run the local server using one of the onnx-web-*.bat scripts
    1. Run onnx-web-half.bat if you are using a GPU and you have < 12GB of VRAM
      • -half mode is compatible with both AMD and Nvidia GPUs
      • -half mode is not compatible with CPU mode
    2. Run onnx-web-full.bat if you are using CPU mode or if you have >= 16GB of VRAM
      • Try the onnx-web-half.bat script if you encounter out-of-memory errors or generating images is very slow
  7. Wait for the models to be downloaded and converted
    1. Most models are distributed in PyTorch format and need to be converted into ONNX format
    2. This only happens once for each model and takes a few minutes
  8. Open one of the URLs shown in the logs in your browser
    1. This will typically be http://127.0.0.1:5000?api=http://127.0.0.1:5000
    2. If you running the server on a different PC and not accessing it from a browser on the same system, use that PC's IP address instead of 127.0.0.1
    3. Any modern browser should work, including Chrome, Edge, and Firefox
    4. Mobile browsers also work, but have stricter mixed-content policies

Windows Python installer

  1. Install the latest Visual C++ 2019 redistributable
    1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
    2. https://aka.ms/vs/17/release/vc_redist.x64.exe
  2. Install Git
  3. Install Python 3.10
  4. Clone or download the onnx-web repository
  5. Open a command prompt window
  6. Run one of the setup-*.bat scripts
    1. Run setup-amd.bat if you are using an AMD GPU and DirectML
    2. Run setup-nvidia.bat if you are using an Nvidia GPU and CUDA
    3. Run setup-cpu.bat if you are planning on only using CPU mode
  7. After the first run, you can run launch.bat instead of the setup script
    1. You should only need to run the setup script once
    2. If you encounter any errors with Python imports, run the setup script again