From cebf5be1a42c3673d03e847e082bace2e03e0685 Mon Sep 17 00:00:00 2001 From: Sean Sube Date: Mon, 27 May 2024 12:03:50 -0500 Subject: [PATCH] start admin guide --- docs/guides/admin.md | 232 ++++++++++++++++++++++++++ docs/guides/{players.md => player.md} | 0 2 files changed, 232 insertions(+) create mode 100644 docs/guides/admin.md rename docs/guides/{players.md => player.md} (100%) diff --git a/docs/guides/admin.md b/docs/guides/admin.md new file mode 100644 index 0000000..7804793 --- /dev/null +++ b/docs/guides/admin.md @@ -0,0 +1,232 @@ +# Admin's Guide to TaleWeave AI + +## Contents + +- [Admin's Guide to TaleWeave AI](#admins-guide-to-taleweave-ai) + - [Contents](#contents) + - [Hardware](#hardware) + - [Disk space](#disk-space) + - [GPU and VRAM](#gpu-and-vram) + - [Networking and ports](#networking-and-ports) + - [Setup](#setup) + - [Clone the git repository](#clone-the-git-repository) + - [Set up a virtual environment](#set-up-a-virtual-environment) + - [Install the pip dependencies](#install-the-pip-dependencies) + - [Launch Ollama for text generation](#launch-ollama-for-text-generation) + - [Using vLLM](#using-vllm) + - [Using OpenAI](#using-openai) + - [Optional: Launch ComfyUI for image generation](#optional-launch-comfyui-for-image-generation) + - [Register](#register) + - [Register a Discord bot](#register-a-discord-bot) + - [Configure](#configure) + - [Configure the Discord bot](#configure-the-discord-bot) + - [Configure the server environment](#configure-the-server-environment) + - [Optional: Configure image generation](#optional-configure-image-generation) + - [Optional: Configure websocket server](#optional-configure-websocket-server) + - [Optional: Configure world size](#optional-configure-world-size) + - [Run](#run) + - [Choose a world prompt or template](#choose-a-world-prompt-or-template) + - [Launch the game](#launch-the-game) + - [Connect and play](#connect-and-play) + +## Hardware + +### Disk space + +The amount of disk space required depends primarily on the LLM and Stable Diffusion checkpoints being used. + +The game world is usually < 1MB and up to 2 copies of each world are kept on disk. The images rendered for various +objects and events in the game can grow over time, but each image should be about 1MB, depending on the image size +that you choose. + +Approximate disk space requirements: + +| LLM / SD | SD v1.5 (3-4GB) | SDXL (6-8GB) | +| ------------- | --------------- | ------------ | +| 7-8b (4-6GB) | 8-10GB | 10-12GB | +| 14b (8-10GB) | 15GB | 20GB | +| 30b (16-20GB) | 25GB | 30GB | +| 70b (30-40GB) | 35GB | 50GB | + +The exact size will vary by model, checkpoint, quantization, etc, and is likely to grow over time. + +### GPU and VRAM + +Technically, you do not need a GPU to run TaleWeave AI. However, performance will be much better with at least one +GPU. Running the LLM on CPU _is not recommended_ and will impact the game's performance, but it is possible. + +If you want to render images of the world while the characters are acting, you will need two GPUs or you will +need to run image generation on the CPU. + +The required amount of VRAM depends on which models you choose and their memory requirements, which is related to +parameter size and quantization. + +| LLM / SD | SD v1.5 | SDXL | +| -------- | --------------------- | ---------------------- | +| 7-8b | 1x 16GB | 1x 24GB or 8GB + 16GB | +| 14b | 1x 24GB or 8GB + 16GB | 2x 16-24GB | +| 30b | 1x 48GB or 8GB + 32GB | 1x 48GB or 16GB + 32GB | +| 70b* | 8GB + 48GB | 16GB + 48GB | + +Notes: + +1. 70b models need to be quantized to 4-bit or so to run on a 48GB GPU +2. While it may be possible to run both text and image generation on the same GPU, two GPUs is recommended + +### Networking and ports + +If you want to make the websocket server accessible over the internet, you will need at least one open port. + +If you are using a load balancer, make sure it is compatible with websockets. + +## Setup + +### Clone the git repository + +Clone the TaleWeave AI repository: + +```bash +git clone https://github.com/ssube/TaleWeaveAI.git +cd TaleWeaveAI +``` + +This will clone the `main` branch, which is currently the development branch and may not be totally stable. + +### Set up a virtual environment + +Set up a virtual environment to manage the dependencies: + +```bash +# Create a virtual environment +python3 -m venv venv + +# Activate the virtual environment +source venv/bin/activate +``` + +If you prefer using Conda or other tools, you can create an environment using their commands instead. Make sure they +are compatible with pip packages. + +### Install the pip dependencies + +Install the pip dependencies into your virtual environment: + +```bash +# Install dependencies +pip install -r requirements.txt +``` + +### Launch Ollama for text generation + +Since TaleWeave AI is a text adventure, some kind of text generator is required. By default, TaleWeave AI is designed +to work with locally-hosted LLM servers, like [Ollama](https://github.com/ollama/ollama). + +Ollama releases binaries for Linux, Mac, and Windows, along with a Docker container that works on RunPod and other GPU +container services. + +```shell +TODO +``` + +If you are using Ollama, you will need to set the following variables in your server environment: + +```shell +PACKIT_DRIVER=ollama +OLLAMA_API=http://127.0.0.1:11434 +``` + +If you are running Ollama with a GPU, you should also set the number of GPU layers and context length: + +```shell +OLLAMA_NUM_GPU=100 +OLLAMA_NUM_CTX=8192 +``` + +These environment variables need to be set in the terminal where TaleWeave AI will be running, they are _not_ used by +the Ollama server. + +Please see the Ollama docs for more details: + +- https://github.com/ollama/ollama?tab=readme-ov-file#ollama + +#### Using vLLM + +You can use vLLM for text generation instead of Ollama. This has not been thoroughly tested, but anything server that +is compatible with the OpenAI API should work, and vLLM seems to work in general. Some models are available for vLLM +on the HuggingFace Hub that are not available for Ollama, and performance differs on different hardware. Use the server +that works for you. + +If you are using vLLM, you will need to set the following variables in your server environment: + +```shell +PACKIT_DRIVER=openai +``` + +Please see the vLLM docs for more details: + +- https://docs.vllm.ai/en/latest/getting_started/installation.html + +#### Using OpenAI + +- https://platform.openai.com/docs/quickstart +- `PACKIT_DRIVER=openai` + +### Optional: Launch ComfyUI for image generation + +- https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file#installing + +## Register + +### Register a Discord bot + +## Configure + +### Configure the Discord bot + +### Configure the server environment + +### Optional: Configure image generation + +### Optional: Configure websocket server + +### Optional: Configure world size + +## Run + +### Choose a world prompt or template + +### Launch the game + +To start a game simulation using the "outback animals" example prompt and running both the Discord both and websocket +server: + +```bash +# Start the TaleWeave AI engine +python3 -m taleweave.main \ + --world worlds/outback-animals-1 \ + --world-template ./taleweave/prompts.yml:outback-animals \ + --discord \ + --server \ + --rooms 3 \ + --turns 30 \ + --optional-actions=true \ + --actions taleweave.systems.sim:init_actions \ + --systems taleweave.systems.sim:init_logic +``` + +This will generate a relatively small world with 3 rooms or areas, run for 30 steps, then save the game and shut down. + +The world will be saved to a file named `worlds/outback-animals-1.json` and the state will be saved after each step to +another file named `worlds/outback-animals-1.state.json`. The world can be stopped at any time by pressing Ctrl-C, +although the step in progress will be lost. The saved state can be resumed and played for any number of additional +steps by running the server again with the same arguments. + +> Note: `module.name:function_name` and `path/filename.yml:key` are patterns you will see repeated throughout TaleWeave AI. +> They indicate a Python module and function within it, or a data file and key within it, respectively. + +The `sim_systems` provide many mechanics from popular life simulations, including hunger, thirst, exhaustion, and mood. +Custom actions and systems can be used to provide any other mechanics that are desired for your setting. The logic +system uses a combination of Python and YAML to modify the prompts connected to rooms, characters, and items in the +world, influencing the behavior of the language models. + +### Connect and play diff --git a/docs/guides/players.md b/docs/guides/player.md similarity index 100% rename from docs/guides/players.md rename to docs/guides/player.md