start admin guide
This commit is contained in:
parent
16e7243da1
commit
cebf5be1a4
|
@ -0,0 +1,232 @@
|
||||||
|
# Admin's Guide to TaleWeave AI
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
- [Admin's Guide to TaleWeave AI](#admins-guide-to-taleweave-ai)
|
||||||
|
- [Contents](#contents)
|
||||||
|
- [Hardware](#hardware)
|
||||||
|
- [Disk space](#disk-space)
|
||||||
|
- [GPU and VRAM](#gpu-and-vram)
|
||||||
|
- [Networking and ports](#networking-and-ports)
|
||||||
|
- [Setup](#setup)
|
||||||
|
- [Clone the git repository](#clone-the-git-repository)
|
||||||
|
- [Set up a virtual environment](#set-up-a-virtual-environment)
|
||||||
|
- [Install the pip dependencies](#install-the-pip-dependencies)
|
||||||
|
- [Launch Ollama for text generation](#launch-ollama-for-text-generation)
|
||||||
|
- [Using vLLM](#using-vllm)
|
||||||
|
- [Using OpenAI](#using-openai)
|
||||||
|
- [Optional: Launch ComfyUI for image generation](#optional-launch-comfyui-for-image-generation)
|
||||||
|
- [Register](#register)
|
||||||
|
- [Register a Discord bot](#register-a-discord-bot)
|
||||||
|
- [Configure](#configure)
|
||||||
|
- [Configure the Discord bot](#configure-the-discord-bot)
|
||||||
|
- [Configure the server environment](#configure-the-server-environment)
|
||||||
|
- [Optional: Configure image generation](#optional-configure-image-generation)
|
||||||
|
- [Optional: Configure websocket server](#optional-configure-websocket-server)
|
||||||
|
- [Optional: Configure world size](#optional-configure-world-size)
|
||||||
|
- [Run](#run)
|
||||||
|
- [Choose a world prompt or template](#choose-a-world-prompt-or-template)
|
||||||
|
- [Launch the game](#launch-the-game)
|
||||||
|
- [Connect and play](#connect-and-play)
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
### Disk space
|
||||||
|
|
||||||
|
The amount of disk space required depends primarily on the LLM and Stable Diffusion checkpoints being used.
|
||||||
|
|
||||||
|
The game world is usually < 1MB and up to 2 copies of each world are kept on disk. The images rendered for various
|
||||||
|
objects and events in the game can grow over time, but each image should be about 1MB, depending on the image size
|
||||||
|
that you choose.
|
||||||
|
|
||||||
|
Approximate disk space requirements:
|
||||||
|
|
||||||
|
| LLM / SD | SD v1.5 (3-4GB) | SDXL (6-8GB) |
|
||||||
|
| ------------- | --------------- | ------------ |
|
||||||
|
| 7-8b (4-6GB) | 8-10GB | 10-12GB |
|
||||||
|
| 14b (8-10GB) | 15GB | 20GB |
|
||||||
|
| 30b (16-20GB) | 25GB | 30GB |
|
||||||
|
| 70b (30-40GB) | 35GB | 50GB |
|
||||||
|
|
||||||
|
The exact size will vary by model, checkpoint, quantization, etc, and is likely to grow over time.
|
||||||
|
|
||||||
|
### GPU and VRAM
|
||||||
|
|
||||||
|
Technically, you do not need a GPU to run TaleWeave AI. However, performance will be much better with at least one
|
||||||
|
GPU. Running the LLM on CPU _is not recommended_ and will impact the game's performance, but it is possible.
|
||||||
|
|
||||||
|
If you want to render images of the world while the characters are acting, you will need two GPUs or you will
|
||||||
|
need to run image generation on the CPU.
|
||||||
|
|
||||||
|
The required amount of VRAM depends on which models you choose and their memory requirements, which is related to
|
||||||
|
parameter size and quantization.
|
||||||
|
|
||||||
|
| LLM / SD | SD v1.5 | SDXL |
|
||||||
|
| -------- | --------------------- | ---------------------- |
|
||||||
|
| 7-8b | 1x 16GB | 1x 24GB or 8GB + 16GB |
|
||||||
|
| 14b | 1x 24GB or 8GB + 16GB | 2x 16-24GB |
|
||||||
|
| 30b | 1x 48GB or 8GB + 32GB | 1x 48GB or 16GB + 32GB |
|
||||||
|
| 70b* | 8GB + 48GB | 16GB + 48GB |
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
1. 70b models need to be quantized to 4-bit or so to run on a 48GB GPU
|
||||||
|
2. While it may be possible to run both text and image generation on the same GPU, two GPUs is recommended
|
||||||
|
|
||||||
|
### Networking and ports
|
||||||
|
|
||||||
|
If you want to make the websocket server accessible over the internet, you will need at least one open port.
|
||||||
|
|
||||||
|
If you are using a load balancer, make sure it is compatible with websockets.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
### Clone the git repository
|
||||||
|
|
||||||
|
Clone the TaleWeave AI repository:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ssube/TaleWeaveAI.git
|
||||||
|
cd TaleWeaveAI
|
||||||
|
```
|
||||||
|
|
||||||
|
This will clone the `main` branch, which is currently the development branch and may not be totally stable.
|
||||||
|
|
||||||
|
### Set up a virtual environment
|
||||||
|
|
||||||
|
Set up a virtual environment to manage the dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create a virtual environment
|
||||||
|
python3 -m venv venv
|
||||||
|
|
||||||
|
# Activate the virtual environment
|
||||||
|
source venv/bin/activate
|
||||||
|
```
|
||||||
|
|
||||||
|
If you prefer using Conda or other tools, you can create an environment using their commands instead. Make sure they
|
||||||
|
are compatible with pip packages.
|
||||||
|
|
||||||
|
### Install the pip dependencies
|
||||||
|
|
||||||
|
Install the pip dependencies into your virtual environment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install dependencies
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Launch Ollama for text generation
|
||||||
|
|
||||||
|
Since TaleWeave AI is a text adventure, some kind of text generator is required. By default, TaleWeave AI is designed
|
||||||
|
to work with locally-hosted LLM servers, like [Ollama](https://github.com/ollama/ollama).
|
||||||
|
|
||||||
|
Ollama releases binaries for Linux, Mac, and Windows, along with a Docker container that works on RunPod and other GPU
|
||||||
|
container services.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
TODO
|
||||||
|
```
|
||||||
|
|
||||||
|
If you are using Ollama, you will need to set the following variables in your server environment:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
PACKIT_DRIVER=ollama
|
||||||
|
OLLAMA_API=http://127.0.0.1:11434
|
||||||
|
```
|
||||||
|
|
||||||
|
If you are running Ollama with a GPU, you should also set the number of GPU layers and context length:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
OLLAMA_NUM_GPU=100
|
||||||
|
OLLAMA_NUM_CTX=8192
|
||||||
|
```
|
||||||
|
|
||||||
|
These environment variables need to be set in the terminal where TaleWeave AI will be running, they are _not_ used by
|
||||||
|
the Ollama server.
|
||||||
|
|
||||||
|
Please see the Ollama docs for more details:
|
||||||
|
|
||||||
|
- https://github.com/ollama/ollama?tab=readme-ov-file#ollama
|
||||||
|
|
||||||
|
#### Using vLLM
|
||||||
|
|
||||||
|
You can use vLLM for text generation instead of Ollama. This has not been thoroughly tested, but anything server that
|
||||||
|
is compatible with the OpenAI API should work, and vLLM seems to work in general. Some models are available for vLLM
|
||||||
|
on the HuggingFace Hub that are not available for Ollama, and performance differs on different hardware. Use the server
|
||||||
|
that works for you.
|
||||||
|
|
||||||
|
If you are using vLLM, you will need to set the following variables in your server environment:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
PACKIT_DRIVER=openai
|
||||||
|
```
|
||||||
|
|
||||||
|
Please see the vLLM docs for more details:
|
||||||
|
|
||||||
|
- https://docs.vllm.ai/en/latest/getting_started/installation.html
|
||||||
|
|
||||||
|
#### Using OpenAI
|
||||||
|
|
||||||
|
- https://platform.openai.com/docs/quickstart
|
||||||
|
- `PACKIT_DRIVER=openai`
|
||||||
|
|
||||||
|
### Optional: Launch ComfyUI for image generation
|
||||||
|
|
||||||
|
- https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file#installing
|
||||||
|
|
||||||
|
## Register
|
||||||
|
|
||||||
|
### Register a Discord bot
|
||||||
|
|
||||||
|
## Configure
|
||||||
|
|
||||||
|
### Configure the Discord bot
|
||||||
|
|
||||||
|
### Configure the server environment
|
||||||
|
|
||||||
|
### Optional: Configure image generation
|
||||||
|
|
||||||
|
### Optional: Configure websocket server
|
||||||
|
|
||||||
|
### Optional: Configure world size
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
### Choose a world prompt or template
|
||||||
|
|
||||||
|
### Launch the game
|
||||||
|
|
||||||
|
To start a game simulation using the "outback animals" example prompt and running both the Discord both and websocket
|
||||||
|
server:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Start the TaleWeave AI engine
|
||||||
|
python3 -m taleweave.main \
|
||||||
|
--world worlds/outback-animals-1 \
|
||||||
|
--world-template ./taleweave/prompts.yml:outback-animals \
|
||||||
|
--discord \
|
||||||
|
--server \
|
||||||
|
--rooms 3 \
|
||||||
|
--turns 30 \
|
||||||
|
--optional-actions=true \
|
||||||
|
--actions taleweave.systems.sim:init_actions \
|
||||||
|
--systems taleweave.systems.sim:init_logic
|
||||||
|
```
|
||||||
|
|
||||||
|
This will generate a relatively small world with 3 rooms or areas, run for 30 steps, then save the game and shut down.
|
||||||
|
|
||||||
|
The world will be saved to a file named `worlds/outback-animals-1.json` and the state will be saved after each step to
|
||||||
|
another file named `worlds/outback-animals-1.state.json`. The world can be stopped at any time by pressing Ctrl-C,
|
||||||
|
although the step in progress will be lost. The saved state can be resumed and played for any number of additional
|
||||||
|
steps by running the server again with the same arguments.
|
||||||
|
|
||||||
|
> Note: `module.name:function_name` and `path/filename.yml:key` are patterns you will see repeated throughout TaleWeave AI.
|
||||||
|
> They indicate a Python module and function within it, or a data file and key within it, respectively.
|
||||||
|
|
||||||
|
The `sim_systems` provide many mechanics from popular life simulations, including hunger, thirst, exhaustion, and mood.
|
||||||
|
Custom actions and systems can be used to provide any other mechanics that are desired for your setting. The logic
|
||||||
|
system uses a combination of Python and YAML to modify the prompts connected to rooms, characters, and items in the
|
||||||
|
world, influencing the behavior of the language models.
|
||||||
|
|
||||||
|
### Connect and play
|
Loading…
Reference in New Issue