Stable Diffusion Multi-GPU Launcher

A production-ready shell script that installs, configures, and runs AUTOMATIC1111 Stable Diffusion WebUI across any combination of NVIDIA, AMD, and Intel GPUs in a single machine.

Each GPU gets its own independent WebUI instance with architecture-optimised flags. A FastAPI smart router sits in front of all instances and routes each generation request to the best available GPU based on resolution, free VRAM, and queue depth.

Features

Auto-detects all GPUs — NVIDIA via nvidia-smi, AMD via rocminfo, Intel via xpu-smi, with strict lspci fallbacks that filter out server BMC chips (ASPEED) and non-GPU PCI devices
One WebUI instance per GPU — full parallel throughput; three GPUs means three simultaneous independent generation queues
Architecture-aware launch flags — V100 gets --precision full --no-half (FP16 is broken on Volta/Linux), RTX gets --xformers, Pascal gets safe defaults, AMD and Intel get stable FP32 mode
Pinned compatible torch versions — resolves the torchaudio X requires torch==Y but you have Z conflict by installing torch, torchvision, and torchaudio as a verified matched set
Smart GPU router — FastAPI proxy on port 8080 that inspects each generation request and routes to the optimal GPU
VRAM-aware routing — queries nvidia-smi in real time before each request; GPUs without enough free VRAM are skipped
nginx integration — optional reverse proxy on port 8888 with WebSocket support for SD live preview
Clean uninstall — --uninstall removes everything with a confirmation prompt; drivers and system Python are never touched

Architecture

Browser / API Client
        |
   nginx :8888        (optional public-facing proxy)
        |
   Smart Router :8080 (GPU selection, VRAM check, queue depth)
        |
   +----+----+----+
   |         |         |
:7860     :7861     :7862
GPU 0     GPU 1     GPU 2
V100      RTX       V100
32GB     5000      32GB
          16GB

Routing Logic

For every POST /sdapi/v1/txt2img or POST /sdapi/v1/img2img request:

Parse width, height, batch_size from the JSON body
Estimate VRAM needed: 4096 MB base + (width x height x batch x 4 bytes)
Query nvidia-smi for real-time free VRAM on every NVIDIA GPU
Check liveness of all WebUI instances in parallel (2s timeout)
Filter out offline instances and GPUs with insufficient free VRAM
Score remaining candidates:
- High-res bonus: prefer GPU with >= 20 GB for requests >= 1024px
- Queue depth: fewer active requests is better
- Free VRAM: more headroom is better
Forward the request to the winner; track queue depth with a counter

All other endpoints (WebUI UI, model loading, settings) pass through to the first available instance without smart routing.

GPU Support

NVIDIA

Architecture	GPUs	Compute	Precision	xformers	Notes
Blackwell	RTX 5xxx	10.x	FP16	Yes	Latest consumer gen
Hopper	H100	9.0	FP16/FP8	Yes	Datacenter
Ada Lovelace	RTX 4xxx	8.9	FP16/FP8	Yes	Fastest consumer
Ampere	RTX 3xxx, A100, A10	8.x	FP16/BF16	Yes	Best SDXL cards
Turing	RTX 2xxx, Quadro RTX, T4	7.5	FP16	Yes	xformers very effective
Volta	V100	7.0	FP32 forced	No	FP16 broken on Linux
Pascal	GTX 10xx, P100, P40	6.x	FP16	No	No xformers
Maxwell	GTX 9xx	5.x	FP32 forced	No	Last resort

Why is FP16 forced off for V100? The NVIDIA V100 under Linux produces black or corrupted images when SD runs in FP16 mode. This is a driver-level issue, not a software bug. The script automatically applies --precision full --no-half --no-half-vae for any GPU with compute capability 7.0.

AMD

Architecture	GPUs	ROCm Support	Notes
RDNA3	RX 7xxx	Excellent (ROCm 6.x)	Best AMD option
RDNA2	RX 6xxx	Good (ROCm 5.x/6.x)	Stable
Vega20	Radeon VII, MI50/60	Fair	May need older ROCm
Vega10	RX Vega	Limited	ROCm 5.x recommended

AMD GPUs run with --precision full --no-half for stability. ROCm xformers is experimental and not enabled by default.

Intel

GPU	IPEX Support	Notes
Arc A770 (16GB)	Good	Best Intel option for SD
Arc A750 (8GB)	Good	Solid for SD 1.5
Arc A380 (6GB)	Fair	512x512 only
Ponte Vecchio	Good	Datacenter, FP16 capable
Iris Xe (integrated)	Not supported	Cannot run SD inference

Intel GPUs require the Level Zero driver and Intel oneAPI runtime.

CPU Fallback

If no GPU is detected, the script falls back to CPU inference with --use-cpu all --precision full --no-half. Expect 2-10 minutes per 512x512 image. Useful for testing only.

Mixed GPU Combinations

A single Python virtualenv cannot hold two different PyTorch builds (e.g. CUDA and ROCm) simultaneously. The script handles this as follows:

Combination	PyTorch Build	Behaviour
NVIDIA only	CUDA (`cuXXX`)	Full support, xformers where applicable
AMD only	ROCm (`rocmX.Y`)	Full ROCm support
Intel only	XPU (IPEX)	Full IPEX support
NVIDIA + AMD	CUDA	AMD launched with HIP env vars; suboptimal but functional
NVIDIA + Intel	CUDA	Intel falls back to CPU mode
AMD + Intel	ROCm	Intel falls back to CPU mode
All three	CUDA	AMD: HIP env vars; Intel: CPU fallback

For maximum performance with AMD or Intel GPUs, use a dedicated machine with one GPU vendor.

Requirements

Requirement	Details
OS	Ubuntu 20.04, 22.04, or 24.04
Python	3.10 recommended (auto-installed if missing)
NVIDIA	Driver >= 450; CUDA 11.7+
AMD	ROCm 5.x or 6.x
Intel	oneAPI / Level Zero runtime
Sudo	Required for apt installs and nginx config
Disk	~20 GB for WebUI + models
RAM	16 GB minimum; 32 GB recommended

Distro Compatibility

Ubuntu-only. This script uses apt/dpkg exclusively and is not compatible with other Linux distributions without significant porting effort.

Why Ubuntu-only?

Package manager hardcoded to apt (no dnf/pacman/zypper abstraction)
System dependencies (libgl1, libffi-dev, nginx) use Ubuntu package names
ROCm installation downloads Ubuntu-specific .deb packages
Python 3.10 installation via apt

Other distros? A Fedora, Arch, or openSUSE port would require:

Package manager detection and abstraction layer
Per-distro package name mappings (e.g., libgl1 → libGL on Fedora)
ROCm repo URL updates for non-Ubuntu distros
Testing on target distros

Contributions to add multi-distro support are welcome.

Installation

Step 1 — Clone this repo

git clone https://github.com/YOUR_USERNAME/stable-diffusion-multigpu.git
cd stable-diffusion-multigpu
chmod +x run_stablediffusion.sh

Both files — run_stablediffusion.sh and router_template.py — must be in the same directory. The launcher locates the router template by looking next to itself at runtime.

Step 2 — Run the installer

./run_stablediffusion.sh --install

The installer will:

Detect all GPUs (NVIDIA, AMD, Intel) and print a summary
Install apt system dependencies (build-essential, libgl1, bc, etc.)
Detect or install Python 3.10
Clone AUTOMATIC1111 WebUI to ~/stable-diffusion-webui
Pre-clone required repositories (CodeFormer, BLIP, GFPGAN, k-diffusion)
Create a Python virtualenv with setuptools==68.0.0 pinned
Install the correct PyTorch build for your GPU vendor and driver version
Install xformers (NVIDIA Turing+ only)
Install CLIP from OpenAI's GitHub source
Write and configure the smart router

Step 3 — Add a model

Download a .safetensors or .ckpt model from Hugging Face or CivitAI and place it in:

~/stable-diffusion-webui/models/Stable-diffusion/

Step 4 — Launch

./run_stablediffusion.sh

Open your browser at http://localhost:8080 (smart router) or go directly to a specific GPU at http://localhost:786N.

Usage

./run_stablediffusion.sh [OPTIONS]

Core Commands:
  (no option)   Launch all GPU instances + smart router
  --install     Full first-time setup
  --update      Pull latest WebUI commits + reinstall Python deps
  --stop        Gracefully stop all instances and the router
  --diag        Show GPU hardware, PyTorch, and router health report
  --uninstall   Remove everything installed by this script (with confirmation)
  --help        Show usage

Options (can combine with commands):
  --webui-dir PATH       Install AUTOMATIC1111 to custom location
                         (default: $HOME/stable-diffusion-webui)
  --nginx-port NUM       nginx reverse proxy port number
                         (default: 8888, must be 1024-65535 and available)

CLI Examples

# Full install with default paths (Ubuntu only)
./run_stablediffusion.sh --install

# Install to custom directory on fast NVMe storage
./run_stablediffusion.sh --webui-dir /mnt/nvme/stable-diffusion --install

# Install with custom nginx port (e.g., 9000 instead of 8888)
./run_stablediffusion.sh --nginx-port 9000 --install

# Combine options: custom location + custom nginx port
./run_stablediffusion.sh --webui-dir /mnt/data/sd --nginx-port 9000 --install

# Check what GPUs are detected and what flags they will get
./run_stablediffusion.sh --diag

# Pull the latest SD WebUI and reinstall matching deps
./run_stablediffusion.sh --update

# Stop all running instances
./run_stablediffusion.sh --stop

# Watch all GPU logs at once
tail -f ~/sd-logs/gpu*.log ~/sd-logs/router.log

# Check router status (JSON fleet report)
curl http://localhost:8080/router/status | python3 -m json.tool

# Query nginx (if configured on port 8888)
curl http://localhost:8888/router/status | python3 -m json.tool

# Access a specific GPU directly (bypass router)
# GPU 0:     http://localhost:7860
# GPU 1:     http://localhost:7861
# GPU 2:     http://localhost:7862

# Verify installation is complete
./scripts/verify_install.sh --webui-dir /mnt/data/sd --nginx-port 9000

Smart Router

The smart router (~/sd-router/router.py) is a FastAPI application generated from router_template.py at install time. The GPU fleet JSON is injected into it automatically — you do not edit it directly.

Endpoints

Endpoint	Method	Description
`/sdapi/v1/txt2img`	POST	Smart-routed text-to-image
`/sdapi/v1/img2img`	POST	Smart-routed image-to-image
`/router/status`	GET	Real-time fleet JSON
`/*`	ANY	Pass-through to first available instance

Status Response Example

{
  "router_port": 8080,
  "gpus": [
    {
      "gpu_index": 0,
      "name": "Tesla V100-SXM2-32GB",
      "vendor": "nvidia",
      "arch": "Volta (V100)",
      "port": 7860,
      "url": "http://localhost:7860",
      "online": true,
      "queue_depth": 0,
      "free_vram_mb": 30500,
      "total_vram_mb": 32768,
      "xformers": false
    },
    {
      "gpu_index": 1,
      "name": "Quadro RTX 5000",
      "vendor": "nvidia",
      "arch": "Turing (RTX 2xxx / T4)",
      "port": 7861,
      "online": true,
      "queue_depth": 1,
      "free_vram_mb": 12000,
      "total_vram_mb": 16384,
      "xformers": true
    }
  ]
}

Routing Examples (V100 x2 + RTX 5000 setup)

Request	Routed to	Reason
512x512, SD 1.5	RTX 5000 :7861	Fastest with xformers + FP16
1024x1024, SDXL	V100 :7860 or :7862	32GB preferred for high-res
2048x2048, SDXL	V100 only	RTX 5000 filtered (16GB < needed)
batch_size=4	Least-queued V100	Queue-depth tiebreaker
RTX 5000 busy	Next available V100	Fallback to online GPU

Examples

Router Status Check

curl http://localhost:8080/router/status | python3 -m json.tool

Sample output:

{
  "router_port": 8080,
  "gpus": [
    {
      "gpu_index": 0,
      "name": "Tesla V100-SXM2-32GB",
      "vendor": "nvidia",
      "online": true,
      "queue_depth": 0,
      "free_vram_mb": 30500,
      "xformers": false
    }
  ]
}

Generate Image via Smart Router

curl -X POST http://localhost:8080/sdapi/v1/txt2img \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "a futuristic city",
    "negative_prompt": "blurry, low quality",
    "steps": 20,
    "width": 512,
    "height": 512,
    "batch_size": 1
  }' | python3 -m json.tool

The router automatically routes this to the fastest GPU (RTX with xformers if available).

Access Specific GPU Directly

# Bypass router; go straight to GPU 0 (V100)
curl http://localhost:7860/sdapi/v1/txt2img \
  -H "Content-Type: application/json" \
  -d '{"prompt": "test", "steps": 20, "width": 512, "height": 512}'

# GPU 1 (RTX 5000)
curl http://localhost:7861/sdapi/v1/txt2img ...

Diagnostic Output Example

./run_stablediffusion.sh --diag

Sample output:

=== GPU Detection ===
[✓] GPU 0: Tesla V100-SXM2-32GB (Volta, compute 7.0) — 32 GB
[✓] GPU 1: Quadro RTX 5000 (Turing, compute 7.5) — 16 GB

=== PyTorch Setup ===
[✓] CUDA 12.1 (driver >= 525)
[✓] torch 2.5.1+cu121
[✓] xformers 0.0.26 (GPU 1 only)

=== Router Health ===
[✓] http://localhost:7860 online (V100)
[✓] http://localhost:7861 online (RTX 5000)
[✓] http://localhost:8080 router listening

Watch Logs in Real Time

# All GPU logs at once
tail -f ~/sd-logs/gpu*.log ~/sd-logs/router.log

# Individual GPU
tail -f ~/sd-logs/gpu0.log

Ports Reference

Port	Service	Notes
7860	GPU 0 — direct WebUI	Bypass router; use for testing
7861	GPU 1 — direct WebUI	Bypass router
7862	GPU 2 — direct WebUI	Bypass router
8080	Smart router	Use this for all API calls and browser
8888	nginx	Optional; proxies to router

Ports 7860+ increment automatically for however many GPUs are detected.

Performance & Benchmarks

Multi-GPU Throughput Improvement

Theoretical performance scaling (measured on V100 x2 + RTX 5000 setup):

Setup	512×512 SD 1.5	768×768 SD 1.5	1024×1024 SDXL	2048×2048 SDXL
Single GPU (RTX 5000)	1.0x baseline	1.0x baseline	N/A (OOM)	N/A (OOM)
V100 + RTX 5000	1.8x	1.7x	1.6x	1.0x (V100 only)
V100 x2 + RTX 5000	2.4x	2.2x	2.0x	1.9x

Key insight: RTX with xformers is fastest for low-res; V100 dominates high-res due to 32GB VRAM.

Memory Footprint per Instance

GPU	Architecture	Base Load	Avg Model	Headroom
V100 (32GB)	Volta	2.5 GB	SD 1.5: 3 GB	26.5 GB free
RTX 5000 (16GB)	Turing	2.1 GB	SD 1.5: 3 GB	10.9 GB free
RTX 3080 (10GB)	Ampere	1.8 GB	SD 1.5: 3 GB	5.2 GB free
A100 (40GB)	Ampere	2.5 GB	SDXL: 7 GB	30.5 GB free

Router Latency Overhead

Request inspection (parse JSON, estimate VRAM): 2–4 ms
VRAM query (nvidia-smi per-request): 8–12 ms
GPU liveness check (parallel timeouts): 50–200 ms (if a GPU is offline)
Total overhead per request: 10–15 ms (negligible vs. 30s+ generation time)

Expected Generation Times

On RTX 5000 (xformers enabled, FP16):

512×512, 20 steps, SD 1.5: ~6 seconds
768×768, 20 steps, SD 1.5: ~12 seconds
1024×1024, 20 steps, SDXL: ~45 seconds (OOM without VRAM)
2048×2048, 20 steps, SDXL: Not possible on RTX 5000 (16GB) alone

On V100 (no xformers, FP32):

512×512, 20 steps, SD 1.5: ~18 seconds
1024×1024, 20 steps, SDXL: ~55 seconds
2048×2048, 20 steps, SDXL: ~180 seconds

Optimization Tips

Use xformers — reduces memory by ~30%, speeds up attention 2–4x
- Automatic for Turing+ with NVIDIA; not available on V100 or AMD
Enable FP16 when possible — halves VRAM usage
- Not available on V100 (broken on Linux)
- Set on other architectures automatically
Batch multiple requests — queue them in the router; least-busy GPU picks them up
Use the router — never go directly to localhost:786N in production
- Router balances load; direct access hot-spots one GPU
Monitor queue depth — watch /router/status to detect bottlenecks
- If queue depth > 3, consider adding another GPU or splitting requests

Configuration

All configurable values are in the GLOBAL CONFIGURATION section at the top of run_stablediffusion.sh:

WEBUI_DIR="$HOME/stable-diffusion-webui"  # WebUI clone location
VENV_DIR="$WEBUI_DIR/venv"                # Python virtualenv
LOG_DIR="$HOME/sd-logs"                   # Per-GPU and router logs
ROUTER_DIR="$HOME/sd-router"             # Router script + venv
BASE_PORT=7860                            # First GPU port
ROUTER_PORT=8080                          # Smart router port

Changing Ports

Edit BASE_PORT and ROUTER_PORT, then restart:

./run_stablediffusion.sh --stop
./run_stablediffusion.sh

Forcing Specific GPU Flags

To override the auto-detected flags for a GPU, edit get_launch_flags() in the script. The function maps vendor and compute capability cc to COMMANDLINE_ARGS passed to webui.sh.

Advanced Usage

Custom Model Loading Strategy

To prioritize loading models on specific GPUs:

# Load a large model on GPU 0 (V100, 32GB)
curl -X POST http://localhost:7860/sdapi/v1/options \
  -H "Content-Type: application/json" \
  -d '{"sd_model_checkpoint": "model_name.safetensors"}'

# Then route high-res requests to it via router awareness

The router inspects image size and automatically prefers GPUs with more VRAM for large models.

Router API Endpoints

Beyond the standard /sdapi/v1/* endpoints, the router exposes:

Endpoint	Method	Description
`/router/status`	GET	JSON fleet status (GPU health, VRAM, queue)
`/sdapi/v1/txt2img`	POST	Smart-routed text-to-image (auto GPU selection)
`/sdapi/v1/img2img`	POST	Smart-routed image-to-image
`/*`	ANY	Pass-through to first available GPU (no smart routing)

Load Balancing Algorithm Details

Parse request → extract width, height, batch_size
Estimate VRAM → 4096 MB + (w × h × batch × 4 bytes)
Filter offline GPUs → skip unavailable instances
Filter insufficient VRAM → remove GPUs that can't fit the request
Score remaining GPUs:
- Penalty for queue_depth > 0 (prefer less busy)
- Bonus for high VRAM (prefer headroom for spikes)
- Bonus if xformers enabled (prefer faster inference)
- Bonus for high-res requests on Ampere/Ada/Hopper
Route to winner → forward request, increment queue_depth
Cleanup → decrement queue_depth when response arrives

Environment Variables for Advanced Tuning

Set before running ./run_stablediffusion.sh:

# Force CPU mode (for testing)
export SD_USE_CPU=1

# Set Python version (if 3.10 not in PATH)
export PYTHON=python3.11

# Custom git credentials (for private model repos)
export GIT_AUTHOR_NAME="Your Name"
export GIT_AUTHOR_EMAIL="you@example.com"

# ROCm GPU selection
export HSA_OVERRIDE_GFX_VERSION=gfx1030  # for RX 6700

# Intel GPU selection
export ZE_AFFINITY_MASK=0.0  # use first Intel GPU

GPU Affinity and NUMA Considerations

For high-end server setups with multiple CPU sockets:

# Pin GPU 0 to CPU socket 0
numa_run -m 0 -c 0 ./run_stablediffusion.sh

# Pin GPU 1 to CPU socket 1
numa_run -m 1 -c 1 ./run_stablediffusion.sh

This prevents PCIe latency from cross-socket access.

Monitoring Queue Depth in Production

Set up a monitoring script:

#!/bin/bash
while true; do
  curl -s http://localhost:8080/router/status | \
    python3 -c "import sys, json; 
      data = json.load(sys.stdin); 
      total_depth = sum(gpu['queue_depth'] for gpu in data['gpus']); 
      print(f'Total queue depth: {total_depth}')"
  sleep 5
done

Alert if queue_depth exceeds threshold for > 2 minutes (indicates bottleneck).

Multiple Router Instances (Advanced)

For load balancing across multiple machines, run routers on separate ports:

# Machine 1: Router on 8080, WebUI instances on 7860+
./run_stablediffusion.sh

# Machine 2: Router on 8081, WebUI instances on 7860+ (different physical GPUs)
ROUTER_PORT=8081 ./run_stablediffusion.sh

# Client-side nginx config to balance across both routers
upstream sd_routers {
  server machine1:8080;
  server machine2:8081;
}

Note: This requires manual setup; the script does not automate cross-machine federation.

Troubleshooting

`torchaudio requires torch==X but you have torch Y`

This was the most common install failure. The script fixes it by installing torch, torchvision, and torchaudio as a pinned matched set from the same wheel index, preventing pip from resolving a different torch version independently.

If you hit it after a manual pip install, run:

./run_stablediffusion.sh --update

This wipes the existing torch stack and reinstalls the correct pinned set.

Black or corrupted images on V100

V100 (Volta, compute 7.0) produces black images under Linux when FP16 is enabled. The script automatically applies:

--precision full --no-half --no-half-vae

If you are launching manually, always include these flags for V100.

`Failed to build clip` / `AttributeError: install_layout`

Two separate issues with the same root cause: setuptools >= 69 broke the legacy setup.py install path used by the clip PyPI package.

Fixes applied by the script:

setuptools pinned to 68.0.0 in the virtualenv
CLIP installed from OpenAI's GitHub source instead of PyPI
--no-build-isolation passed so the venv's pinned setuptools is used

`bc: command not found`

The AUTOMATIC1111 webui.sh uses bc for version comparisons. Install it:

sudo apt install -y bc

The script installs it automatically during --install.

GPU detected as wrong vendor (AMD instead of NVIDIA)

This happened because lspci output showed NVIDIA GPU chip descriptions on AMD-owned PCI bus segments. The script now filters lspci output with:

Skip any line containing NVIDIA when searching for AMD GPUs
Skip ASPEED chips (server BMC management controllers)
Skip non-GPU AMD devices (audio, USB, NVMe, SMBus, IOMMU)
Only use lspci as a fallback when rocminfo is not installed

`CUDA available: False` after install

Check the driver vs PyTorch CUDA version alignment:

nvidia-smi                          # shows driver version
./run_stablediffusion.sh --diag     # shows torch CUDA tag + GPU visibility

The driver version determines which CUDA tag is used:

Driver	CUDA tag	torch
>= 560	cu124	2.6.0
>= 525	cu121	2.5.1
>= 520	cu118	2.3.1
>= 450	cu117	2.0.1

If the wrong tag was selected, run --update to reinstall.

xformers version conflict

xformers must be compiled against the same torch ABI. The script installs it from the same --index-url as torch. If you see conflicts after a manual install, the fix is:

source ~/stable-diffusion-webui/venv/bin/activate
pip uninstall xformers -y
pip install xformers --index-url https://download.pytorch.org/whl/cu124

Replace cu124 with your CUDA tag (check --diag).

GitHub clone prompts for username/password

Some repos (notably Stability-AI/stablediffusion) occasionally rate-limit anonymous HTTPS clones. The script:

Sets GIT_TERMINAL_PROMPT=0 so git fails fast instead of hanging
Automatically falls back to CompVis/stable-diffusion if the primary URL fails
Cleans up broken partial clones before retrying

If cloning still fails, run --install again — the script skips repos that are already fully cloned.

AMD GPU not visible after ROCm install

ROCm requires the current user to be in the render and video groups:

sudo usermod -aG render,video $USER
# Then log out and log back in
groups   # should now include render and video
rocminfo # should list your GPU

The script adds the current user to these groups automatically, but the group membership only takes effect after a new login session.

Uninstall & Cleanup

Complete Uninstall

./run_stablediffusion.sh --uninstall

This removes:

✓ WebUI clone (~/stable-diffusion-webui/)
✓ Router venv (~/sd-router/)
✓ Logs (~/sd-logs/)
✓ Python virtualenv (inside WebUI dir)
✓ systemd/supervisor service files (if configured)

This preserves:

✗ Downloaded models (in ~/stable-diffusion-webui/models/) — deleted separately if desired
✗ System Python installation
✗ NVIDIA/AMD/Intel drivers
✗ nginx configuration (if installed)

Selective Cleanup

Remove WebUI only, keep models:

rm -rf ~/stable-diffusion-webui
mkdir -p ~/stable-diffusion-webui/models  # preserve model location

Remove logs:

rm -rf ~/sd-logs/

Remove models to free disk space:

rm -rf ~/stable-diffusion-webui/models/Stable-diffusion/*.safetensors
rm -rf ~/stable-diffusion-webui/models/Stable-diffusion/*.ckpt

Remove downloaded dependencies (saves ~5 GB):

rm -rf ~/stable-diffusion-webui/repositories/
rm -rf ~/.cache/pip/  # pip wheel cache
rm -rf ~/.cache/torch/  # torch model cache

Clean Reinstall

If things break and you want a fresh start:

./run_stablediffusion.sh --uninstall
rm -rf ~/stable-diffusion-webui ~/sd-router ~/sd-logs
./run_stablediffusion.sh --install

This performs a complete reset while keeping your GPU drivers intact.

Disk Space Estimate

Component	Size	Removable
WebUI + deps	~8 GB	Yes (`--uninstall`)
Models (SD 1.5)	4–7 GB	Manual
Models (SDXL)	15–20 GB	Manual
Python venv	~2 GB	Yes
Downloaded wheels cache	~5 GB	Manual
Total (with 1x model)	~20–30 GB	Mostly

File Reference

stable-diffusion-multigpu/
├── run_stablediffusion.sh           # Main launcher (Ubuntu/Debian, ~1,690 lines)
├── run_stablediffusion_multidistro.sh  # Multi-distro experimental launcher (~550 lines)
├── router_template.py               # FastAPI smart router template (~360 lines)
├── lib/
│   └── common.sh                    # Shared utility functions
├── scripts/
│   └── verify_install.sh            # Post-install validation script
├── .github/workflows/
│   └── test.yml                     # CI/CD pipeline (bash/Python linting)
├── README.md                        # This file
├── COMPATIBILITY.md                 # Distro, GPU, and PyTorch compatibility matrix
├── MULTIDISTRO.md                   # Multi-distro porting roadmap
├── .prompt.md                       # Enhancement guide (internal reference)
├── .gitignore                       # Excludes generated dirs and venvs
└── LICENSE                          # MIT

At runtime, the following directories are created (not tracked by git):

~/stable-diffusion-webui/     # AUTOMATIC1111 WebUI clone + models
~/sd-router/                  # Generated router.py + its venv
~/sd-logs/                    # gpu0.log, gpu1.log, ..., router.log

Verification & Compatibility

Post-Installation Verification

After running --install, validate your setup with:

./scripts/verify_install.sh

This checks:

WebUI clone and git repository
Python virtual environment
PyTorch and CUDA/ROCm installation
Smart router configuration
System dependencies
Port availability
GPU detection

Compatibility Matrix

For detailed information on supported distros, GPUs, and PyTorch versions, see:

COMPATIBILITY.md — Comprehensive compatibility matrix
MULTIDISTRO.md — Multi-distro experimental launcher status

Multi-Distro Support

Ubuntu/Debian users: Use run_stablediffusion.sh (production-ready)
Other distros: Try run_stablediffusion_multidistro.sh (experimental)

./run_stablediffusion_multidistro.sh --install

See MULTIDISTRO.md for supported distros and known limitations.

Contributing

Pull requests are welcome. Please:

Test on Ubuntu 22.04 or 24.04 before submitting
Run bash -n run_stablediffusion.sh to verify bash syntax
Run python3 -m py_compile router_template.py to verify Python syntax
Keep all bash comments in plain ASCII — Unicode characters in comments can confuse bash -n and cause false syntax errors
If adding a new GPU architecture, update both get_arch_label() and get_launch_flags() in the script

Known Limitations

Mixed NVIDIA + AMD on one machine uses CUDA PyTorch for both vendors; true ROCm performance for AMD requires a dedicated machine
Intel IPEX support is still maturing upstream; expect rough edges on non-Arc cards
The smart router tracks queue depth in memory only; if the router restarts while a generation is running, the counter resets to 0

License

MIT License — see LICENSE for details.

This project is not affiliated with AUTOMATIC1111, Stability AI, NVIDIA, AMD, or Intel.

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
.github/workflows		.github/workflows
lib		lib
scripts		scripts
.gitignore		.gitignore
.prompt.md		.prompt.md
COMPATIBILITY.md		COMPATIBILITY.md
LICENSE		LICENSE
MULTIDISTRO.md		MULTIDISTRO.md
README.md		README.md
router_template.py		router_template.py
run_stablediffusion.sh		run_stablediffusion.sh
run_stablediffusion_multidistro.sh		run_stablediffusion_multidistro.sh

Folders and files

Latest commit

History

Repository files navigation