xformers for auto (#136)

Closes #128

.github/ISSUE_TEMPLATE/bug.md

@@ -7,30 +7,39 @@ assignees: ''
 
 ---
 
-**Has this issue been opened before? Check the [FAQ](https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/Main), the [issues](https://github.com/AbdBarho/stable-diffusion-webui-docker/issues?q=is%3Aissue)**
+<!--  PLEASE FILL THIS OUT, IT WILL MAKE BOTH OF OUR LIVES EASIER -->
 
+**Has this issue been opened before?**
+
+- [ ] It is not in the [FAQ](https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/FAQ), I checked.
+- [ ] It is not in the [issues](https://github.com/AbdBarho/stable-diffusion-webui-docker/issues?q=), I searched.
 
 
 **Describe the bug**
 
+<!--  tried to run the app, my cat exploded -->
+
 
 **Which UI**
 
 hlky or auto or auto-cpu or lstein?
 
+
+**Hardware / Software**
+ - OS: [e.g. Windows 10 / Ubuntu ]
+ - OS version: <!--  on windows, use the command `winver` to find out, on ubuntu `lsb_release -d`  -->
+ - WSL version (if applicable): <!-- get using `wsl -l -v` -->
+ - Docker Version: <!--  get using `docker version` -->
+ - Docker compose version: <!--  get using `docker compose version` -->
+ - Repo version: <!-- tag, commit sha, or "from master" -->
+ - RAM:
+ - GPU/VRAM:
+
 **Steps to Reproduce**
 1. Go to '...'
 2. Click on '....'
 3. Scroll down to '....'
 4. See error
 
-**Hardware / Software:**
- - OS: [e.g. Windows / Ubuntu and version]
- - RAM:
- - GPU: [Nvidia 1660 / No GPU]
- - VRAM:
- - Docker Version, Docker compose version
- - Release version [e.g. 1.0.1]
-
 **Additional context**
 Any other context about the problem here. If applicable, add screenshots to help explain your problem.

.github/pull_request_template.md

@@ -2,4 +2,4 @@
 
 - auto: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/
 - hlky: https://github.com/sd-webui/stable-diffusion-webui/commit/
-- lstein: https://github.com/lstein/stable-diffusion/commit/
+- lstein: https://github.com/invoke-ai/InvokeAI/commit/

.github/workflows/executable.yml1

@@ -1,22 +0,0 @@
-name: Check executable
-
-on: [push]
-
-jobs:
-  check:
-    runs-on: ubuntu-latest
-    name: Check all sh
-    steps:
-      - run: git config --global core.fileMode true
-      - uses: actions/checkout@v3
-      - shell: bash
-        run: |
-          shopt -s globstar;
-          FAIL=0
-          for file in **/*.sh; do
-              if [ -f "${file}" ] && [ -r "${file}" ] && [ ! -x "${file}" ]; then
-                  echo "$file" is not executable;
-                  FAIL=1
-              fi
-          done
-          exit ${FAIL}

.github/workflows/xformers.yml

@@ -0,0 +1,36 @@
+name: Build Xformers
+
+on:
+  workflow_dispatch: {}
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    timeout-minutes: 180
+    container:
+      image: python:3.10-slim
+      env:
+        DEBIAN_FRONTEND: noninteractive
+        XFORMERS_DISABLE_FLASH_ATTN: 1
+        FORCE_CUDA: 1
+        TORCH_CUDA_ARCH_LIST: "6.0;6.1;6.2;7.0;7.2;7.5;8.0;8.6"
+        NVCC_FLAGS: --use_fast_math -DXFORMERS_MEM_EFF_ATTENTION_DISABLE_BACKWARD
+        MAX_JOBS: 4
+    steps:
+      - run: |
+          apt-get update
+          apt-get install gpg wget git -y
+          wget https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/cuda-keyring_1.0-1_all.deb
+          dpkg -i cuda-keyring_1.0-1_all.deb
+          apt-get update
+          apt-get install cuda-nvcc-11-8 cuda-libraries-dev-11-8 -y
+
+          export PIP_CACHE_DIR=$(pwd)/cache
+          pip install ninja install torch --extra-index-url https://download.pytorch.org/whl/cu113
+
+          pip wheel --wheel-dir=data git+https://github.com/facebookresearch/xformers.git@3633e1afc7bffbe61957f04e7bb1a742ee910ace#egg=xformers
+      - name: Artifacts
+        uses: actions/upload-artifact@v3
+        with:
+          name: xformers
+          path: data/xformers-0.0.14.dev0-cp310-cp310-linux_x86_64.whl

README.md

@@ -40,15 +40,20 @@ Screenshots:
 | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
 | ![](https://user-images.githubusercontent.com/24505302/189541298-f902b021-a1eb-4e4b-b2eb-b6a696a8ec80.jpg) | ![](https://user-images.githubusercontent.com/24505302/189541295-7d7f2162-2189-4e0a-abbd-703f4779e1cd.jpg) | ![](https://user-images.githubusercontent.com/24505302/189541294-aa7f7735-a973-4e17-ada0-1fe3acbb1772.jpg) |
 
-<!--
+
+
 ### lstein
 
-[lstein's fork](https://github.com/lstein/stable-diffusion) is very mature when it comes to the cli, and the WebUI has potential.
+[lstein's fork](https://github.com/invoke-ai/InvokeAI) is one of the earliest with a wonderful WebUI.
+- Text to image, with many samplers
+- Image to image
+- 4GB GPU support
+- More coming!
+- [Full feature list here](https://github.com/invoke-ai/InvokeAI#features)
 
 | Text to image                                                                                              | Image to image                                                                                             | Extras                                                                                                     |
 | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
-| ![](https://user-images.githubusercontent.com/24505302/190662506-dabdc967-93af-4d78-8533-394604d29ba4.jpg) | ![](https://user-images.githubusercontent.com/24505302/190662557-7640d9f0-30d8-4527-97b0-07d3f48108d4.jpg) | ![](https://user-images.githubusercontent.com/24505302/190662588-37a01fad-f993-4674-9ae6-8714aa229f7b.jpg) |
+| ![](https://user-images.githubusercontent.com/24505302/195158552-39f58cb6-cfcc-4141-9995-a626e3760752.jpg) | ![](https://user-images.githubusercontent.com/24505302/195158553-152a0ab8-c0fd-4087-b121-4823bcd8d6b5.jpg) | ![](https://user-images.githubusercontent.com/24505302/195158548-e118206e-c519-4915-85d6-4c248eb10fc0.jpg) |
--->
 
 ## Setup & Usage
 

data/.gitignore

@@ -1,5 +1,7 @@
 # for all of the stuff downloaded by transformers, pytorch, and others
 /.cache
+# for UIs
+/config
 # for all stable diffusion models (main, waifu diffusion, etc..)
 /StableDiffusion
 # others
@@ -11,4 +13,5 @@
 /SwinIR
 /ScuNET
 /LDSR
+/Hypernetworks
 /embeddings

docker-compose.yml

@@ -29,17 +29,14 @@ services:
     build: ./services/hlky/
     environment:
       - CLI_ARGS=--optimized-turbo
+      - USE_STREAMLIT=0
 
   auto: &automatic
     <<: *base_service
     profiles: ["auto"]
     build: ./services/AUTOMATIC1111
-    volumes:
-      - *v1
-      - *v2
-      - ./services/AUTOMATIC1111/config.json:/stable-diffusion-webui/config.json
     environment:
-      - CLI_ARGS=--allow-code --medvram
+      - CLI_ARGS=--allow-code --medvram --xformers
 
   auto-cpu:
     <<: *automatic
@@ -53,5 +50,5 @@ services:
     profiles: ["lstein"]
     build: ./services/lstein/
     environment:
-      - PRELOAD=false
+      - PRELOAD=true
       - CLI_ARGS=

services/AUTOMATIC1111/Dockerfile

@@ -2,41 +2,45 @@
 
 FROM alpine/git:2.36.2 as download
 
+SHELL ["/bin/sh", "-ceuxo", "pipefail"]
 
 RUN git clone https://github.com/CompVis/stable-diffusion.git repositories/stable-diffusion && cd repositories/stable-diffusion && git reset --hard 69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc
 
 RUN git clone https://github.com/sczhou/CodeFormer.git repositories/CodeFormer && cd repositories/CodeFormer && git reset --hard c5b4593074ba6214284d6acd5f1719b6c5d739af
 RUN git clone https://github.com/salesforce/BLIP.git repositories/BLIP && cd repositories/BLIP && git reset --hard 48211a1594f1321b00f14c9f7a5b4813144b2fb9
-RUN git clone https://github.com/Hafiidz/latent-diffusion.git repositories/latent-diffusion && cd repositories/latent-diffusion && git reset --hard abf33e7002d59d9085081bce93ec798dcabd49af
 
 RUN <<EOF
 # because taming-transformers is huge
 git config --global http.postBuffer 1048576000
 git clone https://github.com/CompVis/taming-transformers.git repositories/taming-transformers
+cd repositories/taming-transformers
 git reset --hard 24268930bf1dce879235a7fddd0b2355b84d7ea6
-rm -rf repositories/taming-transformers/data repositories/taming-transformers/assets
+rm -rf data assets
 EOF
 
 RUN git clone https://github.com/crowsonkb/k-diffusion.git repositories/k-diffusion && cd repositories/k-diffusion && git reset --hard f4e99857772fc3a126ba886aadf795a332774878
 
+FROM python:3.10-slim as xformers
+SHELL ["/bin/bash", "-ceuxo", "pipefail"]
+RUN pip install gdown
+RUN gdown https://drive.google.com/uc?id=1SqwicrLx1TrG_sbbEoIF_3TUHd4EYSmw -O /wheel.whl
 
-FROM continuumio/miniconda3:4.12.0
+FROM python:3.10-slim
 
 SHELL ["/bin/bash", "-ceuxo", "pipefail"]
 
-ENV DEBIAN_FRONTEND=noninteractive
+ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1 PIP_NO_CACHE_DIR=1
 
-RUN conda install python=3.8.5 && conda clean -a -y
+RUN pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
-RUN conda install pytorch==1.11.0 torchvision==0.12.0 cudatoolkit=11.3 -c pytorch && conda clean -a -y
 
-RUN apt-get update && apt install fonts-dejavu-core rsync -y && apt-get clean
+RUN apt-get update && apt install fonts-dejavu-core rsync git -y && apt-get clean
 
 
 RUN <<EOF
 git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
 cd stable-diffusion-webui
 git reset --hard 1eb588cbf19924333b88beaa1ac0041904966640
-pip install --prefer-binary --no-cache-dir -r requirements_versions.txt
+pip install -r requirements_versions.txt
 EOF
 
 ENV ROOT=/stable-diffusion-webui \
@@ -46,32 +50,38 @@ ENV ROOT=/stable-diffusion-webui \
 COPY --from=download /git/ ${ROOT}
 RUN pip install --prefer-binary --no-cache-dir -r ${ROOT}/repositories/CodeFormer/requirements.txt
 
+# TODO: move to top
+RUN apt-get install jq moreutils -y
+
+
 # Note: don't update the sha of previous versions because the install will take forever
 # instead, update the repo state in a later step
 
-ARG SHA=2995107fa24cfd72b0a991e18271dcde148c2807
+ARG SHA=36a0ba357ab0742c3c4a28437b68fb29a235afbe
 RUN <<EOF
 cd stable-diffusion-webui
 git pull --rebase
 git reset --hard ${SHA}
 pip install --prefer-binary --no-cache-dir -r requirements_versions.txt
-pip install --prefer-binary --no-cache-dir -r requirements.txt
 EOF
 
 RUN pip install --prefer-binary --no-cache-dir opencv-python-headless \
   git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379 \
-  git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
+  git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1 \
-
+  pyngrok
 
+COPY --from=xformers /wheel.whl xformers-0.0.14.dev0-cp310-cp310-linux_x86_64.whl
+RUN pip install xformers-0.0.14.dev0-cp310-cp310-linux_x86_64.whl
 
 COPY . /docker
 RUN <<EOF
 chmod +x /docker/mount.sh && python3 /docker/info.py ${ROOT}/modules/ui.py
 EOF
 
+
 ENV CLI_ARGS=""
 WORKDIR ${WORKDIR}
 EXPOSE 7860
 # run, -u to not buffer stdout / stderr
 CMD /docker/mount.sh && \
-  python3 -u ../../webui.py --listen --port 7860 --hide-ui-dir-config --ckpt-dir ${ROOT}/models/Stable-diffusion ${CLI_ARGS}
+  python3 -u ../../webui.py --listen --port 7860 --ckpt-dir ${ROOT}/models/Stable-diffusion ${CLI_ARGS}

services/AUTOMATIC1111/config.json

@@ -6,74 +6,5 @@
   "outdir_txt2img_grids": "/output/txt2img-grids",
   "outdir_img2img_grids": "/output/img2img-grids",
   "outdir_save": "/output/saved",
-  "font": "DejaVuSans.ttf",
+  "font": "DejaVuSans.ttf"
-  "__WARNING__": "DON'T CHANGE ANYTHING BEFORE THIS",
-
-  "add_model_hash_to_info": false,
-  "code_former_weight": 0.5,
-  "directories_filename_pattern": "",
-  "directories_max_prompt_words": 8,
-  "enable_batch_seeds": true,
-  "enable_emphasis": true,
-  "enable_pnginfo": true,
-  "enable_quantization": false,
-  "ESRGAN_tile": 192,
-  "ESRGAN_tile_overlap": 8,
-  "export_for_4chan": true,
-  "face_restoration_model": null,
-  "face_restoration_unload": false,
-  "filter_nsfw": false,
-  "grid_extended_filename": false,
-  "grid_format": "png",
-  "grid_only_if_multiple": true,
-  "grid_save": true,
-  "grid_save_to_dirs": false,
-  "img2img_color_correction": false,
-  "img2img_fix_steps": false,
-  "interrogate_clip_dict_limit": 1500,
-  "interrogate_clip_max_length": 48,
-  "interrogate_clip_min_length": 24,
-  "interrogate_clip_num_beams": 1,
-  "interrogate_keep_models_in_memory": false,
-  "interrogate_use_builtin_artists": true,
-  "jpeg_quality": 80,
-  "js_modal_lightbox": true,
-  "js_modal_lightbox_initialy_zoomed": true,
-  "ldsr_post_down": 1,
-  "ldsr_pre_down": 1,
-  "ldsr_steps": 30,
-  "memmon_poll_rate": 8,
-  "multiple_tqdm": true,
-  "n_rows": -1,
-  "outdir_grids": "",
-  "random_artist_categories": [],
-  "realesrgan_enabled_models": [
-    "Real-ESRGAN 4x plus",
-    "Real-ESRGAN 4x plus anime 6B",
-    "Real-ESRGAN 2x plus",
-    "Real-ESRGAN AnimeVideo",
-    "Real-ESRGAN General WDN x4x3",
-    "Real-ESRGAN General x4x3"
-  ],
-  "return_grid": true,
-  "samples_filename_format": "",
-  "samples_filename_pattern": "",
-  "samples_format": "png",
-  "samples_log_stdout": false,
-  "samples_save": true,
-  "save_images_before_color_correction": false,
-  "save_images_before_face_restoration": false,
-  "save_selected_only": false,
-  "save_to_dirs": false,
-  "save_to_dirs_prompt_len": 10,
-  "save_txt": false,
-  "sd_model_checkpoint": null,
-  "show_progress_every_n_steps": 7,
-  "show_progressbar": true,
-  "SWIN_tile": 192,
-  "SWIN_tile_overlap": 8,
-  "upscale_at_full_resolution_padding": 16,
-  "upscaler_for_hires_fix": null,
-  "upscaler_for_img2img": null,
-  "use_original_name_batch": false
 }

services/AUTOMATIC1111/mount.sh

@@ -2,6 +2,10 @@
 
 set -Eeuo pipefail
 
+mkdir -p /data/config/auto/
+cp -n /docker/config.json /data/config/auto/config.json
+jq '. * input' /data/config/auto/config.json /docker/config.json | sponge /data/config/auto/config.json
+
 declare -A MOUNTS
 
 MOUNTS["/root/.cache"]="/data/.cache"
@@ -16,8 +20,10 @@ MOUNTS["${ROOT}/models/RealESRGAN"]="/data/RealESRGAN"
 MOUNTS["${ROOT}/models/SwinIR"]="/data/SwinIR"
 MOUNTS["${ROOT}/models/ScuNET"]="/data/ScuNET"
 MOUNTS["${ROOT}/models/LDSR"]="/data/LDSR"
+MOUNTS["${ROOT}/models/hypernetworks"]="/data/Hypernetworks"
 
 MOUNTS["${ROOT}/embeddings"]="/data/embeddings"
+MOUNTS["${ROOT}/config.json"]="/data/config/auto/config.json"
 
 # extra hacks
 MOUNTS["${ROOT}/repositories/CodeFormer/weights/facelib"]="/data/.cache"
@@ -26,8 +32,12 @@ for to_path in "${!MOUNTS[@]}"; do
   set -Eeuo pipefail
   from_path="${MOUNTS[${to_path}]}"
   rm -rf "${to_path}"
+  if [ ! -f "$from_path" ]; then
     mkdir -vp "$from_path"
+  fi
   mkdir -vp "$(dirname "${to_path}")"
   ln -sT "${from_path}" "${to_path}"
   echo Mounted $(basename "${from_path}")
 done
+
+mkdir -p /output/saved /output/txt2img-images/ /output/img2img-images /output/extras-images/ /output/grids/ /output/txt2img-grids/ /output/img2img-grids/

services/hlky/Dockerfile

@@ -22,10 +22,7 @@ conda clean -a -y
 EOF
 
 
-# Note: don't update the sha of previous versions because the install will take forever
+ARG BRANCH=dev SHA=bd57d22f2e0ec7eed084bdf2a4c0dcc0ed745daf
-# instead, update the repo state in a later step
-ARG BRANCH=master  SHA=1a9c053cb7b6832695771db2555c0adc9b41e95f
-# ARG BRANCH=dev SHA=1e7bdfe3f38a6dd37fc230f440ea1b0db0937240
 RUN <<EOF
 cd stable-diffusion
 git fetch
@@ -35,16 +32,17 @@ conda env update --file environment.yaml -n base
 conda clean -a -y
 EOF
 
-RUN pip install -U --no-cache-dir pyperclip
-
 # add info
 COPY . /docker/
-RUN python /docker/info.py /stable-diffusion/frontend/frontend.py && chmod +x /docker/mount.sh
+RUN <<EOF
+python /docker/info.py /stable-diffusion/frontend/frontend.py
+chmod +x /docker/mount.sh /docker/run.sh
+# streamlit
+sed -i -- 's/8501/7860/g' /stable-diffusion/.streamlit/config.toml
+EOF
 
 WORKDIR /stable-diffusion
-ENV PYTHONPATH="${PYTHONPATH}:${PWD}" CLI_ARGS=""
+ENV PYTHONPATH="${PYTHONPATH}:${PWD}" STREAMLIT_SERVER_HEADLESS=true USE_STREAMLIT=0 CLI_ARGS=""
 EXPOSE 7860
-# run, -u to not buffer stdout / stderr
+
-CMD /docker/mount.sh && \
+CMD /docker/mount.sh && /docker/run.sh
-  python3 -u scripts/webui.py --outdir /output --ckpt /data/StableDiffusion/model.ckpt ${CLI_ARGS}
-# sed -i -- 's/8501/7860/g' .streamlit/config.toml && STREAMLIT_SERVER_HEADLESS=true python -u -m streamlit run scripts/webui_streamlit.py  --theme.base dark

services/hlky/run.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+set -Eeuo pipefail
+
+echo "USE_STREAMLIT = ${USE_STREAMLIT}"
+if [ "${USE_STREAMLIT}" == "1" ]; then
+  python -u -m streamlit run scripts/webui_streamlit.py
+else
+  python3 -u scripts/webui.py --outdir /output --ckpt /data/StableDiffusion/model.ckpt ${CLI_ARGS}
+fi

services/lstein/Dockerfile

@@ -17,15 +17,14 @@ ENV PIP_EXISTS_ACTION=w
 RUN <<EOF
 git clone https://github.com/invoke-ai/InvokeAI.git stable-diffusion
 cd stable-diffusion
-git reset --hard 8a8be92eac17e0ef699528157596b2336bdee532
+git reset --hard 79e79b78aaeedb49afcc795e0e00eebfdbedee96
-sed -i -- 's/python=3.8.5/python=3.9/g' environment.yaml
 git config --global http.postBuffer 1048576000
-conda env update --file environment.yaml -n base
+conda env update --file environment.yml -n base
 conda clean -a -y
 EOF
 
 
-ARG BRANCH=development SHA=4f247a3672474bd9c46060bab6087dbf9e2531f3
+ARG BRANCH=development SHA=fe2a2cfc8b6156805c420c026c5ef8bbb256899e
 RUN <<EOF
 cd stable-diffusion
 git fetch
@@ -36,7 +35,7 @@ conda env update --file environment.yml -n base
 conda clean -a -y
 EOF
 
-RUN pip uninstall opencv-python -y && pip install --prefer-binary --force-reinstall --no-cache-dir opencv-python-headless transformers==4.19.2
+RUN pip uninstall opencv-python -y && pip install --prefer-binary --force-reinstall --no-cache-dir opencv-python-headless
 
 COPY . /docker/
 RUN <<EOF
@@ -50,8 +49,4 @@ WORKDIR /stable-diffusion
 EXPOSE 7860
 
 CMD /docker/mount.sh && \
-  # python3 -u backend/server.py --host 0.0.0.0 --port 7860 --cors http://localhost:7860
+  python3 -u scripts/invoke.py --outdir /output --web --host 0.0.0.0 --port 7860 ${CLI_ARGS}
-  python3 -u scripts/dream.py --outdir /output --web --host 0.0.0.0 --port 7860 ${CLI_ARGS}
-# echo The lstein webUI is currently deactivated due to implementation limitations: \
-# https://github.com/invoke-ai/InvokeAI/blob/8c9f2ae705cf723d4a8a73c416e8d8bf2d746977/backend/modules/create_cmd_parser.py#L26 \
-# Once the path the output is fixed, the UI will be activated again

services/lstein/mount.sh

@@ -9,10 +9,11 @@ MOUNTS["/root/.cache"]=/data/.cache
 # ui specific
 MOUNTS["${PWD}/models/ldm/stable-diffusion-v1/model.ckpt"]=/data/StableDiffusion/model.ckpt
 MOUNTS["${PWD}/src/gfpgan/experiments/pretrained_models/GFPGANv1.4.pth"]=/data/GFPGAN/GFPGANv1.4.pth
-MOUNTS["${PWD}/ldm/dream/restoration/codeformer/weights"]=/data/CodeFormer
+MOUNTS["${PWD}/ldm/invoke/restoration/codeformer/weights"]=/data/Codeformer
 # hacks
 MOUNTS["/opt/conda/lib/python3.9/site-packages/facexlib/weights"]=/data/.cache
 MOUNTS["/opt/conda/lib/python3.9/site-packages/realesrgan/weights"]=/data/RealESRGAN
+MOUNTS["${PWD}/gfpgan/weights"]=/data/.cache
 
 for to_path in "${!MOUNTS[@]}"; do
   set -Eeuo pipefail