v2 (#108 )

### Update versions - auto: 3f417566b0 ### Breaking changes: * renamed `automatic-1111` service to `auto` * the `cache` folder is now deprecated, replaced with `data` (see migration guide below) * `embeddings` folder has been moved to `data/embeddings` * use GFPGAN 1.4 ### Migration Guide Note: in theory, running the command ``` docker compose --profile download up --build ``` is all you need to use the new version, however, this means you will also have to download everything again. A new script is available under `scripts/migratev1tov2.sh` that will copy models to the new structure and should get you most of the way, run ```bash ./scripts/migratev1tov2.sh ``` or you can manually inspect the script and copy the files After that, run ``` docker compose --profile download up --build ``` to validate everything.
Update / Disable lstein Temporarily (#106 )
2025-10-27 16:24:26 -04:00 · 2022-10-01 12:57:53 +02:00 · 2022-09-30 09:37:27 +02:00 · 2022-09-29 12:04:40 +02:00 · 2022-09-28 10:18:07 +02:00 · 2022-09-28 08:36:53 +02:00
26 changed files with 341 additions and 296 deletions
--- a/.devscripts/chmod.sh
+++ b/.devscripts/chmod.sh
--- a/.github/pull_request_template.md
+++ b/.github/pull_request_template.md
@@ -0,0 +1,5 @@
+### Update versions
+
+- auto: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/
+- hlky: https://github.com/sd-webui/stable-diffusion-webui/commit/
+- lstein: https://github.com/lstein/stable-diffusion/commit/
--- a/.github/workflows/stale.yml
+++ b/.github/workflows/stale.yml
@@ -0,0 +1,20 @@
+name: 'Close stale issues and PRs'
+on:
+  schedule:
+    - cron: '30 1 * * *'
+
+jobs:
+  stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v5
+        with:
+          only-labels: awaiting-response
+          stale-issue-message: This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.
+          stale-pr-message: This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.
+          close-issue-message: This issue was closed because it has been stalled for 7 days with no activity.
+          close-pr-message: This PR was closed because it has been stalled for 7 days with no activity.
+          days-before-issue-stale: 14
+          days-before-pr-stale: 14
+          days-before-issue-close: 7
+          days-before-pr-close: 7
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,4 @@
 /dev
 /.devcontainer
+embeddings/*
+/docker-compose.override.yml
--- a/9
+++ b/9
@@ -86,4 +86,11 @@ administration of justice, law enforcement, immigration or asylum
 processes, such as predicting an individual will commit fraud/crime
 commitment (e.g. by text profiling, drawing causal relationships between
 assertions made in documents, indiscriminate and arbitrarily-targeted
-use).
+use).
+
+
+
+By using this software, you also agree to the following licenses:
+https://github.com/CompVis/stable-diffusion/blob/main/LICENSE
+https://github.com/TencentARC/GFPGAN/blob/master/LICENSE
+https://github.com/xinntao/Real-ESRGAN/blob/master/LICENSE
--- a/README.md
+++ b/README.md
@@ -16,6 +16,7 @@ This repository provides multiple UIs for you to play around with stable diffusi
 - Loopback, prompt weighting, prompt matrix, X/Y plot
 - Live preview of the generated images.
 - Highly optimized 4GB GPU support, or even CPU only!
+- Textual inversion allows you to use pretrained textual inversion embeddings
 - [Full feature list here](https://github.com/AUTOMATIC1111/stable-diffusion-webui-feature-showcase)

 | Text to image                                                                                              | Image to image                                                                                             | Extras                                                                                                     |
@@ -39,15 +40,22 @@ Screenshots:
 | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
 | ![](https://user-images.githubusercontent.com/24505302/189541298-f902b021-a1eb-4e4b-b2eb-b6a696a8ec80.jpg) | ![](https://user-images.githubusercontent.com/24505302/189541295-7d7f2162-2189-4e0a-abbd-703f4779e1cd.jpg) | ![](https://user-images.githubusercontent.com/24505302/189541294-aa7f7735-a973-4e17-ada0-1fe3acbb1772.jpg) |

+<!--
 ### lstein

-[lstein's fork](https://github.com/lstein/stable-diffusion) is very mature when it comes to the cli, but less so for the WebUI.
+[lstein's fork](https://github.com/lstein/stable-diffusion) is very mature when it comes to the cli, and the WebUI has potential.
+
+| Text to image                                                                                              | Image to image                                                                                             | Extras                                                                                                     |
+| ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
+| ![](https://user-images.githubusercontent.com/24505302/190662506-dabdc967-93af-4d78-8533-394604d29ba4.jpg) | ![](https://user-images.githubusercontent.com/24505302/190662557-7640d9f0-30d8-4527-97b0-07d3f48108d4.jpg) | ![](https://user-images.githubusercontent.com/24505302/190662588-37a01fad-f993-4674-9ae6-8714aa229f7b.jpg) |
+-->

 ## Setup & Usage

 Visit the wiki for [Setup](https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/Setup) and [Usage](https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/Usage) instructions, checkout the [FAQ](https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/FAQ) page if you face any problems, or create a new issue!

 ## Contributing
+
 Contributions are welcome! create an issue first of what you want to contribute (before you implement anything) so we can talk about it.

 ## Disclaimer
--- a/cache/.gitignore
+++ b/cache/.gitignore
@@ -1,4 +0,0 @@
-/torch
-/transformers
-/weights
-/models
--- a/data/.gitignore
+++ b/data/.gitignore
@@ -0,0 +1,13 @@
+# for all of the stuff downloaded by transformers, pytorch, and others
+/.cache
+# for all stable diffusion models (main, waifu diffusion, etc..)
+/StableDiffusion
+# others
+/Codeformer
+/GFPGAN
+/ESRGAN
+/BSRGAN
+/RealESRGAN
+/SwinIR
+/LDSR
+# TODO: add the embeddings folder here
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -4,7 +4,7 @@ x-base_service: &base_service
    ports:
      - "7860:7860"
    volumes:
-      - &v1 ./cache:/cache
+      - &v1 ./data:/data
      - &v2 ./output:/output
    deploy:
      resources:
@@ -30,7 +30,7 @@ services:
    environment:
      - CLI_ARGS=--optimized-turbo

-  automatic1111: &automatic
+  auto: &automatic
    <<: *base_service
    profiles: ["auto"]
    build: ./services/AUTOMATIC1111
@@ -39,9 +39,9 @@ services:
      - *v2
      - ./services/AUTOMATIC1111/config.json:/stable-diffusion-webui/config.json
    environment:
-      - CLI_ARGS=--medvram --opt-split-attention
+      - CLI_ARGS=--allow-code --medvram

-  automatic1111-cpu:
+  auto-cpu:
    <<: *automatic
    profiles: ["auto-cpu"]
    deploy: {}
@@ -52,3 +52,6 @@ services:
    <<: *base_service
    profiles: ["lstein"]
    build: ./services/lstein/
+    environment:
+      - PRELOAD=false
+      - CLI_ARGS=
--- a/scripts/migratev1tov2.sh
+++ b/scripts/migratev1tov2.sh
@@ -0,0 +1,30 @@
+mkdir -p data/.cache data/StableDiffusion data/Codeformer data/GFPGAN data/ESRGAN data/BSRGAN data/RealESRGAN data/SwinIR data/LDSR /data/embeddings
+
+cp -vf cache/models/model.ckpt data/StableDiffusion/model.ckpt
+
+cp -vf cache/models/LDSR.ckpt data/LDSR/model.ckpt
+cp -vf cache/models/LDSR.yaml data/LDSR/project.yaml
+
+cp -vf cache/models/RealESRGAN_x4plus.pth data/RealESRGAN/
+cp -vf cache/models/RealESRGAN_x4plus_anime_6B.pth data/RealESRGAN/
+
+cp -vrf cache/torch data/.cache/
+
+mkdir -p data/.cache/huggingface/transformers/
+cp -vrf cache/transformers/* data/.cache/huggingface/transformers/
+
+cp -v cache/custom-models/* data/StableDiffusion/
+
+mkdir -p data/.cache/clip/
+cp -vf cache/weights/ViT-L-14.pt data/.cache/clip/
+
+cp -vf cache/weights/codeformer.pth data/Codeformer/codeformer-v0.1.0.pth
+
+cp -vf cache/weights/detection_Resnet50_Final.pth data/.cache/
+cp -vf cache/weights/parsing_parsenet.pth data/.cache/
+
+cp -v embeddings/* data/embeddings/
+
+echo this script was created 10/2022
+echo Dont forget to run: docker compose --profile download up --build
+echo the cache and embeddings folders can be deleted, but its not necessary.
--- a/services/AUTOMATIC1111/Dockerfile
+++ b/services/AUTOMATIC1111/Dockerfile
@@ -1,16 +1,24 @@
 # syntax=docker/dockerfile:1

 FROM alpine/git:2.36.2 as download
+
+
+RUN git clone https://github.com/CompVis/stable-diffusion.git repositories/stable-diffusion && cd repositories/stable-diffusion && git reset --hard 69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc
+
+RUN git clone https://github.com/sczhou/CodeFormer.git repositories/CodeFormer && cd repositories/CodeFormer && git reset --hard c5b4593074ba6214284d6acd5f1719b6c5d739af
+RUN git clone https://github.com/salesforce/BLIP.git repositories/BLIP && cd repositories/BLIP && git reset --hard 48211a1594f1321b00f14c9f7a5b4813144b2fb9
+RUN git clone https://github.com/Hafiidz/latent-diffusion.git repositories/latent-diffusion && cd repositories/latent-diffusion && git reset --hard abf33e7002d59d9085081bce93ec798dcabd49af
+
 RUN <<EOF
 # because taming-transformers is huge
 git config --global http.postBuffer 1048576000
-git clone https://github.com/sczhou/CodeFormer.git repositories/CodeFormer
-git clone https://github.com/CompVis/stable-diffusion.git repositories/stable-diffusion
-git clone https://github.com/salesforce/BLIP.git repositories/BLIP
 git clone https://github.com/CompVis/taming-transformers.git repositories/taming-transformers
+git reset --hard 24268930bf1dce879235a7fddd0b2355b84d7ea6
 rm -rf repositories/taming-transformers/data repositories/taming-transformers/assets
 EOF

+RUN git clone https://github.com/crowsonkb/k-diffusion.git repositories/k-diffusion && cd repositories/k-diffusion && git reset --hard 9e3002b7cd64df7870e08527b7664eb2f2f5f3f5
+

 FROM continuumio/miniconda3:4.12.0

@@ -27,38 +35,41 @@ RUN apt-get update && apt install fonts-dejavu-core rsync -y && apt-get clean
 RUN <<EOF
 git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
 cd stable-diffusion-webui
-git reset --hard 13eec4f3d4081fdc43883c5ef02e471a2b6c7212
-conda env update --file environment-wsl2.yaml -n base
-conda clean -a -y
-pip install --prefer-binary --no-cache-dir -r requirements.txt
+git reset --hard 15f333a266c20319e2b95a47a8834adf7b914aec
+pip install --prefer-binary --no-cache-dir -r requirements_versions.txt
 EOF

 ENV ROOT=/stable-diffusion-webui \
  WORKDIR=/stable-diffusion-webui/repositories/stable-diffusion

+
 COPY --from=download /git/ ${ROOT}
 RUN pip install --prefer-binary --no-cache-dir -r ${ROOT}/repositories/CodeFormer/requirements.txt

 # Note: don't update the sha of previous versions because the install will take forever
 # instead, update the repo state in a later step

-ARG SHA=2ddaeb318a9626502ef4bf949a312253d8021ff0
+ARG SHA=3f417566b0bda8eab05d247567aebf001c1d1725
 RUN <<EOF
 cd stable-diffusion-webui
 git pull --rebase
 git reset --hard ${SHA}
+pip install --prefer-binary --no-cache-dir -r requirements_versions.txt
 pip install --prefer-binary --no-cache-dir -r requirements.txt
 EOF

 RUN pip install --prefer-binary -U --no-cache-dir opencv-python-headless

-ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch CLI_ARGS=""

 COPY . /docker
-RUN chmod +x /docker/mount.sh && python3 /docker/info.py ${ROOT}/modules/ui.py
-
+RUN <<EOF
+chmod +x /docker/mount.sh && python3 /docker/info.py ${ROOT}/modules/ui.py
+EOF

+ENV CLI_ARGS=""
 WORKDIR ${WORKDIR}
 EXPOSE 7860
 # run, -u to not buffer stdout / stderr
-CMD /docker/mount.sh && python3 -u ../../webui.py --listen --port 7860 --hide-ui-dir-config ${CLI_ARGS}
+CMD /docker/mount.sh && \
+  python3 -u ../../webui.py --listen --port 7860 --hide-ui-dir-config --ckpt-dir ${ROOT}/models/Stable-diffusion ${CLI_ARGS}
+#  --gfpgan-models-path /data/GFPGAN --codeformer-models-path /data/Codeformer --esrgan-models-path /data/ESRGAN --bsrgan-models-path /data/BSRGAN --realesrgan-models-path /data/RealESRGAN --swinir-models-path /data/SwinIR --ldsr-models-path /data/LDSR
--- a/services/AUTOMATIC1111/README.md
+++ b/services/AUTOMATIC1111/README.md
@@ -1,14 +0,0 @@
-# WebUI for AUTOMATIC1111
-
-The WebUI of [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui) as docker container!
-
-## Setup
-
-Clone this repo, download the `model.ckpt` and `GFPGANv1.3.pth` and put into the `models` folder as mentioned in [the main README](../README.md), then run
-
-```
-cd AUTOMATIC1111
-docker compose up --build
-```
-
-You can change the cli parameters in `AUTOMATIC1111/docker-compose.yml`. The full list of cil parameters can be found [here](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/shared.py)
--- a/services/AUTOMATIC1111/config.json
+++ b/services/AUTOMATIC1111/config.json
@@ -9,40 +9,71 @@
  "font": "DejaVuSans.ttf",
  "__WARNING__": "DON'T CHANGE ANYTHING BEFORE THIS",

-  "samples_filename_format": "",
-  "outdir_grids": "",
-  "save_to_dirs": false,
-  "grid_save_to_dirs": false,
-  "save_to_dirs_prompt_len": 10,
-  "samples_save": true,
-  "samples_format": "png",
-  "grid_save": true,
-  "return_grid": true,
-  "grid_format": "png",
-  "grid_extended_filename": false,
-  "grid_only_if_multiple": true,
-  "n_rows": -1,
-  "jpeg_quality": 80,
-  "export_for_4chan": true,
-  "enable_pnginfo": true,
  "add_model_hash_to_info": false,
+  "code_former_weight": 0.5,
+  "directories_filename_pattern": "",
+  "directories_max_prompt_words": 8,
+  "enable_batch_seeds": true,
  "enable_emphasis": true,
-  "save_txt": false,
+  "enable_pnginfo": true,
+  "enable_quantization": false,
  "ESRGAN_tile": 192,
  "ESRGAN_tile_overlap": 8,
-  "random_artist_categories": [],
-  "upscale_at_full_resolution_padding": 16,
-  "show_progressbar": true,
-  "show_progress_every_n_steps": 7,
-  "multiple_tqdm": true,
+  "export_for_4chan": true,
  "face_restoration_model": null,
-  "code_former_weight": 0.5,
-  "save_images_before_face_restoration": false,
  "face_restoration_unload": false,
+  "filter_nsfw": false,
+  "grid_extended_filename": false,
+  "grid_format": "png",
+  "grid_only_if_multiple": true,
+  "grid_save": true,
+  "grid_save_to_dirs": false,
+  "img2img_color_correction": false,
+  "img2img_fix_steps": false,
+  "interrogate_clip_dict_limit": 1500,
+  "interrogate_clip_max_length": 48,
+  "interrogate_clip_min_length": 24,
+  "interrogate_clip_num_beams": 1,
  "interrogate_keep_models_in_memory": false,
  "interrogate_use_builtin_artists": true,
-  "interrogate_clip_num_beams": 1,
-  "interrogate_clip_min_length": 24,
-  "interrogate_clip_max_length": 48,
-  "interrogate_clip_dict_limit": 1500.0
+  "jpeg_quality": 80,
+  "js_modal_lightbox": true,
+  "js_modal_lightbox_initialy_zoomed": true,
+  "ldsr_post_down": 1,
+  "ldsr_pre_down": 1,
+  "ldsr_steps": 30,
+  "memmon_poll_rate": 8,
+  "multiple_tqdm": true,
+  "n_rows": -1,
+  "outdir_grids": "",
+  "random_artist_categories": [],
+  "realesrgan_enabled_models": [
+    "Real-ESRGAN 4x plus",
+    "Real-ESRGAN 4x plus anime 6B",
+    "Real-ESRGAN 2x plus",
+    "Real-ESRGAN AnimeVideo",
+    "Real-ESRGAN General WDN x4x3",
+    "Real-ESRGAN General x4x3"
+  ],
+  "return_grid": true,
+  "samples_filename_format": "",
+  "samples_filename_pattern": "",
+  "samples_format": "png",
+  "samples_log_stdout": false,
+  "samples_save": true,
+  "save_images_before_color_correction": false,
+  "save_images_before_face_restoration": false,
+  "save_selected_only": false,
+  "save_to_dirs": false,
+  "save_to_dirs_prompt_len": 10,
+  "save_txt": false,
+  "sd_model_checkpoint": null,
+  "show_progress_every_n_steps": 7,
+  "show_progressbar": true,
+  "SWIN_tile": 192,
+  "SWIN_tile_overlap": 8,
+  "upscale_at_full_resolution_padding": 16,
+  "upscaler_for_hires_fix": null,
+  "upscaler_for_img2img": null,
+  "use_original_name_batch": false
 }
--- a/services/AUTOMATIC1111/mount.sh
+++ b/services/AUTOMATIC1111/mount.sh
@@ -1,36 +1,31 @@
 #!/bin/bash

-set -e
+set -Eeuo pipefail

-declare -A MODELS
+declare -A MOUNTS

-MODELS["${WORKDIR}/models/ldm/stable-diffusion-v1/model.ckpt"]=model.ckpt
-MODELS["${ROOT}/GFPGANv1.3.pth"]=GFPGANv1.3.pth
+MOUNTS["/root/.cache"]="/data/.cache"

-MODELS_DIR=/cache/models
+# main
+MOUNTS["${ROOT}/models/Stable-diffusion"]="/data/StableDiffusion"
+MOUNTS["${ROOT}/models/Codeformer"]="/data/Codeformer"
+MOUNTS["${ROOT}/models/GFPGAN"]="/data/GFPGAN"
+MOUNTS["${ROOT}/models/ESRGAN"]="/data/ESRGAN"
+MOUNTS["${ROOT}/models/BSRGAN"]="/data/BSRGAN"
+MOUNTS["${ROOT}/models/RealESRGAN"]="/data/RealESRGAN"
+MOUNTS["${ROOT}/models/SwinIR"]="/data/SwinIR"
+MOUNTS["${ROOT}/models/LDSR"]="/data/LDSR"

-for path in "${!MODELS[@]}"; do
-  name=${MODELS[$path]}
-  base=$(dirname "${path}")
-  from_path="${MODELS_DIR}/${name}"
-  if test -f "${from_path}"; then
-    mkdir -p "${base}" && ln -sf "${from_path}" "${path}" && echo "Mounted ${name}"
-  else
-    echo "Skipping ${name}"
-  fi
+MOUNTS["${ROOT}/embeddings"]="/data/embeddings"
+
+# extra hacks
+MOUNTS["${ROOT}/repositories/CodeFormer/weights/facelib"]="/data/.cache"
+
+for to_path in "${!MOUNTS[@]}"; do
+  set -Eeuo pipefail
+  from_path="${MOUNTS[${to_path}]}"
+  rm -rf "${to_path}"
+  mkdir -p "$(dirname "${to_path}")"
+  ln -sT "${from_path}" "${to_path}"
+  echo Mounted $(basename "${from_path}")
 done
-
-# force realesrgan cache
-rm -rf /opt/conda/lib/python3.8/site-packages/realesrgan/weights
-ln -s -T "${MODELS_DIR}" /opt/conda/lib/python3.8/site-packages/realesrgan/weights
-
-# force facexlib cache
-mkdir -p /cache/weights/ ${WORKDIR}/gfpgan/
-ln -sf /cache/weights/ ${WORKDIR}/gfpgan/
-# code former cache
-rm -rf ${ROOT}/repositories/CodeFormer/weights/CodeFormer ${ROOT}/repositories/CodeFormer/weights/facelib
-ln -sf -T /cache/weights ${ROOT}/repositories/CodeFormer/weights/CodeFormer
-ln -sf -T /cache/weights ${ROOT}/repositories/CodeFormer/weights/facelib
-
-# mount config
-# ln -sf /docker/config.json ${WORKDIR}/config.json
--- a/services/download/Dockerfile
+++ b/services/download/Dockerfile
@@ -1,6 +1,6 @@
 FROM bash:alpine3.15

-RUN apk add parallel
+RUN apk add parallel aria2
 COPY . /docker
 RUN chmod +x /docker/download.sh
 ENTRYPOINT ["/docker/download.sh"]
--- a/services/download/checksums.sha256
+++ b/services/download/checksums.sha256
@@ -1,6 +1,6 @@
-fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556  /cache/models/model.ckpt
-c953a88f2727c85c3d9ae72e2bd4846bbaf59fe6972ad94130e23e7017524a70  /cache/models/GFPGANv1.3.pth
-4fa0d38905f75ac06eb49a7951b426670021be3018265fd191d2125df9d682f1  /cache/models/RealESRGAN_x4plus.pth
-f872d837d3c90ed2e05227bed711af5671a6fd1c9f7d7e91c911a61f155e99da  /cache/models/RealESRGAN_x4plus_anime_6B.pth
-c209caecac2f97b4bb8f4d726b70ac2ac9b35904b7fc99801e1f5e61f9210c13  /cache/models/LDSR.ckpt
-9d6ad53c5dafeb07200fb712db14b813b527edd262bc80ea136777bdb41be2ba  /cache/models/LDSR.yaml
+fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556  /data/StableDiffusion/model.ckpt
+e2cd4703ab14f4d01fd1383a8a8b266f9a5833dacee8e6a79d3bf21a1b6be5ad  /data/GFPGAN/GFPGANv1.4.pth
+4fa0d38905f75ac06eb49a7951b426670021be3018265fd191d2125df9d682f1  /data/RealESRGAN/RealESRGAN_x4plus.pth
+f872d837d3c90ed2e05227bed711af5671a6fd1c9f7d7e91c911a61f155e99da  /data/RealESRGAN/RealESRGAN_x4plus_anime_6B.pth
+c209caecac2f97b4bb8f4d726b70ac2ac9b35904b7fc99801e1f5e61f9210c13  /data/LDSR/model.ckpt
+9d6ad53c5dafeb07200fb712db14b813b527edd262bc80ea136777bdb41be2ba  /data/LDSR/project.yaml
--- a/services/download/download.sh
+++ b/services/download/download.sh
@@ -2,32 +2,22 @@

 set -Eeuo pipefail

-# [[ "$(sha256sum -b $file | head -c 64)" == "$sha" ]]
+mkdir -p /data/.cache /data/StableDiffusion /data/Codeformer /data/GFPGAN /data/ESRGAN /data/BSRGAN /data/RealESRGAN /data/SwinIR /data/LDSR

-declare -A MODELS
+cat <<EOF
+By using this software, you agree to the following licenses:
+https://github.com/CompVis/stable-diffusion/blob/main/LICENSE
+https://github.com/TencentARC/GFPGAN/blob/master/LICENSE
+https://github.com/xinntao/Real-ESRGAN/blob/master/LICENSE
+EOF

-MODELS['model.ckpt']='https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media'
-MODELS['GFPGANv1.3.pth']='https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth'
-MODELS['RealESRGAN_x4plus.pth']='https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth'
-MODELS['RealESRGAN_x4plus_anime_6B.pth']='https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth'
-MODELS['LDSR.yaml']='https://heibox.uni-heidelberg.de/f/31a76b13ea27482981b4/?dl=1'
-MODELS['LDSR.ckpt']='https://heibox.uni-heidelberg.de/f/578df07c8fc04ffbadf3/?dl=1'
+echo "Downloading, this might take a while..."

-echo "Downloading..."
-
-for file in "${!MODELS[@]}"; do
-  url=${MODELS[$file]}
-  full_path="/cache/models/$file"
-
-  if [[ -f "$full_path" ]]; then
-    echo "- $file exists"
-    continue
-  fi
-
-  mkdir -p $(dirname $full_path)
-  wget --tries=10 -c -O $full_path $url
-done
+aria2c --input-file /docker/links.txt --dir /data --continue

 echo "Checking SHAs..."

-time parallel --will-cite -a /docker/checksums.sha256 "echo -n {} | sha256sum -c"
+parallel --will-cite -a /docker/checksums.sha256 "echo -n {} | sha256sum -c"
+
+# fix potential permissions
+chmod -R 777 /data /output
--- a/services/download/links.txt
+++ b/services/download/links.txt
@@ -0,0 +1,12 @@
+https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media
+  out=StableDiffusion/model.ckpt
+https://github.com/TencentARC/GFPGAN/releases/download/v1.3.4/GFPGANv1.4.pth
+  out=GFPGAN/GFPGANv1.4.pth
+https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth
+  out=RealESRGAN/RealESRGAN_x4plus.pth
+https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth
+  out=RealESRGAN/RealESRGAN_x4plus_anime_6B.pth
+https://heibox.uni-heidelberg.de/f/31a76b13ea27482981b4/?dl=1
+  out=LDSR/project.yaml
+https://heibox.uni-heidelberg.de/f/578df07c8fc04ffbadf3/?dl=1
+  out=LDSR/model.ckpt
--- a/services/hlky/Dockerfile
+++ b/services/hlky/Dockerfile
@@ -13,6 +13,7 @@ RUN apt-get update && apt install fonts-dejavu-core rsync gcc -y && apt-get clea


 RUN <<EOF
+git config --global http.postBuffer 1048576000
 git clone https://github.com/sd-webui/stable-diffusion-webui.git stable-diffusion
 cd stable-diffusion
 git reset --hard 7623a5734740025d79b710f3744bff9276e1467b
@@ -20,15 +21,11 @@ conda env update --file environment.yaml -n base
 conda clean -a -y
 EOF

-# new dependency, should be added to the environment.yaml
-RUN pip install -U --no-cache-dir pyperclip

 # Note: don't update the sha of previous versions because the install will take forever
 # instead, update the repo state in a later step
-ARG BRANCH=master
-ARG SHA=833a91047df999302f699637768741cecee9c37b
-# ARG BRANCH=dev
-# ARG SHA=b4de6caf697d311c1238c15a4c863fa529a35522
+# ARG BRANCH=master  SHA=d0bb60a139d60e6c2b9be4e18e0e29a86aa5af59
+ARG BRANCH=dev SHA=aaa3be16e0e66f07c63dbdb49b9b7cac8fef5cfb
 RUN <<EOF
 cd stable-diffusion
 git fetch
@@ -38,26 +35,16 @@ conda env update --file environment.yaml -n base
 conda clean -a -y
 EOF

-# Latent diffusion
-RUN <<EOF
-git clone https://github.com/Hafiidz/latent-diffusion.git
-cd latent-diffusion
-git reset --hard e1a84a89fcbb49881546cf2acf1e7e250923dba0
-# hacks all the way down
-mv ldm ldm_latent &&
-sed -i -- 's/from ldm/from ldm_latent/g' *.py
-# dont forget to update the yaml!!
-EOF
-
+RUN pip install -U --no-cache-dir pyperclip

 # add info
 COPY . /docker/
 RUN python /docker/info.py /stable-diffusion/frontend/frontend.py && chmod +x /docker/mount.sh

 WORKDIR /stable-diffusion
-ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch CLI_ARGS=""
+ENV PYTHONPATH="${PYTHONPATH}:${PWD}" CLI_ARGS=""
 EXPOSE 7860
 # run, -u to not buffer stdout / stderr
 CMD /docker/mount.sh && \
-  python3 -u scripts/webui.py --outdir /output --ckpt /cache/models/model.ckpt --ldsr-dir /latent-diffusion --inbrowser ${CLI_ARGS}
-  # STREAMLIT_SERVER_PORT=7860 python -m streamlit run scripts/webui_streamlit.py
+  python3 -u scripts/webui.py --outdir /output --ckpt /data/StableDiffusion/model.ckpt ${CLI_ARGS}
+# sed -i -- 's/8501/7860/g' .streamlit/config.toml && STREAMLIT_SERVER_HEADLESS=true python -u -m streamlit run scripts/webui_streamlit.py  --theme.base dark
--- a/services/hlky/mount.sh
+++ b/services/hlky/mount.sh
@@ -1,38 +1,28 @@
 #!/bin/bash

-set -e
+set -Eeuo pipefail

-declare -A MODELS
+declare -A MOUNTS

 ROOT=/stable-diffusion/src

-MODELS["${ROOT}/gfpgan/experiments/pretrained_models/GFPGANv1.3.pth"]=GFPGANv1.3.pth
-MODELS["${ROOT}/realesrgan/experiments/pretrained_models/RealESRGAN_x4plus.pth"]=RealESRGAN_x4plus.pth
-MODELS["${ROOT}/realesrgan/experiments/pretrained_models/RealESRGAN_x4plus_anime_6B.pth"]=RealESRGAN_x4plus_anime_6B.pth
-MODELS["/latent-diffusion/experiments/pretrained_models/model.ckpt"]=LDSR.ckpt
-# MODELS["/latent-diffusion/experiments/pretrained_models/project.yaml"]=LDSR.yaml
+# cache
+MOUNTS["/root/.cache"]=/data/.cache
+# ui specific
+MOUNTS["${ROOT}/gfpgan/experiments/pretrained_models/GFPGANv1.3.pth"]=/data/GFPGAN/GFPGANv1.4.pth
+MOUNTS["${ROOT}/realesrgan/experiments/pretrained_models"]=/data/RealESRGAN
+MOUNTS["${ROOT}/latent-diffusion/experiments/pretrained_models"]=/data/LDSR
+# hacks
+MOUNTS["/stable-diffusion/gfpgan/weights"]=/data/.cache

-MODELS_DIR=/cache/models
-
-for path in "${!MODELS[@]}"; do
-  name=${MODELS[$path]}
-  base=$(dirname "${path}")
-  from_path="${MODELS_DIR}/${name}"
-  if test -f "${from_path}"; then
-    mkdir -p "${base}" && ln -sf "${from_path}" "${path}" && echo "Mounted ${name}"
-  else
-    echo "Skipping ${name}"
-  fi
+for to_path in "${!MOUNTS[@]}"; do
+  set -Eeuo pipefail
+  from_path="${MOUNTS[${to_path}]}"
+  rm -rf "${to_path}"
+  mkdir -p "$(dirname "${to_path}")"
+  ln -sT "${from_path}" "${to_path}"
+  echo Mounted $(basename "${from_path}")
 done

-# hack for latent-diffusion
-if test -f "${MODELS_DIR}/LDSR.yaml"; then
-  sed 's/ldm\./ldm_latent\./g' "${MODELS_DIR}/LDSR.yaml" >/latent-diffusion/experiments/pretrained_models/project.yaml
-fi
-
-# force facexlib cache
-mkdir -p /cache/weights/ /stable-diffusion/gfpgan/
-ln -sf /cache/weights/ /stable-diffusion/gfpgan/
-
 # streamlit config
-ln -sf /docker/webui_streamlit.yaml /stable-diffusion/configs/webui/webui_streamlit.yaml
+ln -sf /docker/userconfig_streamlit.yaml /stable-diffusion/configs/webui/userconfig_streamlit.yaml
--- a/services/hlky/userconfig_streamlit.yaml
+++ b/services/hlky/userconfig_streamlit.yaml
@@ -0,0 +1,8 @@
+general:
+  outdir: /outputs
+  default_model: "Stable Diffusion v1.4"
+  default_model_path: /data/StableDiffusion/model.ckpt
+  outdir_txt2img: /outputs/txt2img-samples
+  outdir_img2img: /outputs/img2img-samples
+  optimized: True
+  optimized_turbo: True
--- a/services/hlky/webui_streamlit.yaml
+++ b/services/hlky/webui_streamlit.yaml
@@ -1,102 +0,0 @@
-# UI defaults configuration file. It is automatically loaded if located at configs/webui/webui_streamlit.yaml.
-general:
-  gpu: 0
-  outdir: /outputs
-  ckpt: "/cache/models/model.ckpt"
-  fp:
-    name: "embeddings/alex/embeddings_gs-11000.pt"
-  GFPGAN_dir: "./src/gfpgan"
-  RealESRGAN_dir: "./src/realesrgan"
-  RealESRGAN_model: "RealESRGAN_x4plus"
-  outdir_txt2img: /outputs/txt2img-samples
-  outdir_img2img: /outputs/img2img-samples
-  gfpgan_cpu: False
-  esrgan_cpu: False
-  extra_models_cpu: False
-  extra_models_gpu: False
-  save_metadata: True
-  skip_grid: False
-  skip_save: False
-  grid_format: "jpg:95"
-  save_format: "png"
-  n_rows: -1
-  no_verify_input: False
-  no_half: False
-  precision: "autocast"
-  optimized: False
-  optimized_turbo: False
-  update_preview: True
-  update_preview_frequency: 1
-
-txt2img:
-  prompt:
-  height: 512
-  width: 512
-  cfg_scale: 5.0
-  seed: ""
-  batch_count: 1
-  batch_size: 1
-  sampling_steps: 50
-  default_sampler: "k_lms"
-  separate_prompts: False
-  normalize_prompt_weights: True
-  save_individual_images: True
-  save_grid: True
-  group_by_prompt: True
-  save_as_jpg: False
-  use_GFPGAN: True
-  use_RealESRGAN: True
-  RealESRGAN_model: "RealESRGAN_x4plus"
-  variant_amount: 0.0
-  variant_seed: ""
-
-img2img:
-  prompt:
-  sampling_steps: 50
-  # Adding an int to toggles enables the corresponding feature.
-  # 0: Create prompt matrix (separate multiple prompts using |, and get all combinations of them)
-  # 1: Normalize Prompt Weights (ensure sum of weights add up to 1.0)
-  # 2: Loopback (use images from previous batch when creating next batch)
-  # 3: Random loopback seed
-  # 4: Save individual images
-  # 5: Save grid
-  # 6: Sort samples by prompt
-  # 7: Write sample info files
-  # 8: jpg samples
-  # 9: Fix faces using GFPGAN
-  # 10: Upscale images using Real-ESRGAN
-  sampler_name: k_lms
-  denoising_strength: 0.45
-  # 0: Keep masked area
-  # 1: Regenerate only masked area
-  mask_mode: 0
-  # 0: Just resize
-  # 1: Crop and resize
-  # 2: Resize and fill
-  resize_mode: 0
-  # Leave blank for random seed:
-  seed: ""
-  ddim_eta: 0.0
-  cfg_scale: 5.0
-  batch_count: 1
-  batch_size: 1
-  height: 512
-  width: 512
-  # Textual inversion embeddings file path:
-  fp: ""
-  loopback: True
-  random_seed_loopback: True
-  separate_prompts: False
-  normalize_prompt_weights: True
-  save_individual_images: True
-  save_grid: True
-  group_by_prompt: True
-  save_as_jpg: False
-  use_GFPGAN: True
-  use_RealESRGAN: True
-  RealESRGAN_model: "RealESRGAN_x4plus"
-  variant_amount: 0.0
-  variant_seed: ""
-
-gfpgan:
-  strength: 100
--- a/services/lstein/Dockerfile
+++ b/services/lstein/Dockerfile
@@ -6,26 +6,54 @@ SHELL ["/bin/bash", "-ceuxo", "pipefail"]

 ENV DEBIAN_FRONTEND=noninteractive

-RUN conda install python=3.8.5 && conda clean -a -y
+# now it requires python3.9
+RUN conda install python=3.9 && conda clean -a -y
 RUN conda install pytorch==1.11.0 torchvision==0.12.0 cudatoolkit=11.3 -c pytorch && conda clean -a -y

 RUN apt-get update && apt install fonts-dejavu-core rsync gcc -y && apt-get clean


 RUN <<EOF
-git clone https://github.com/lstein/stable-diffusion.git
+git clone https://github.com/invoke-ai/InvokeAI.git stable-diffusion
 cd stable-diffusion
-git reset --hard 751283a2de81bee4bb571fbabe4adb19f1d85b97
+git reset --hard a1739a73b48bfe98b6abcb67f5a0197a9ad270e0
+sed -i -- 's/python=3.8.5/python=3.9/g' environment.yaml
 conda env update --file environment.yaml -n base
 conda clean -a -y
 EOF

-ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch CLI_ARGS=""

+ARG BRANCH=development SHA=8c9f2ae705cf723d4a8a73c416e8d8bf2d746977
+# this breaks on generation:
+# there is a new UI anyway, but it is not by any means ready.
+# ARG BRANCH=development SHA=bdbc76fcd4bd3362312dc91b087d9af66de423b1
+RUN <<EOF
+cd stable-diffusion
+git fetch
+git reset --hard
+git checkout ${BRANCH}
+git reset --hard ${SHA}
+conda env update --file environment.yml -n base
+conda clean -a -y
+EOF
+
+RUN pip uninstall opencv-python -y && pip install --prefer-binary --force-reinstall --no-cache-dir opencv-python-headless transformers==4.19.2
+
+COPY . /docker/
+RUN <<EOF
+python3 /docker/info.py /stable-diffusion/static/dream_web/index.html
+chmod +x /docker/mount.sh
+sed -i -- 's/outputs\//\/output/g' /stable-diffusion/backend/server.py
+EOF
+
+
+ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch PRELOAD=false CLI_ARGS=""
 WORKDIR /stable-diffusion
-
 EXPOSE 7860
-# run, -u to not buffer stdout / stderr
-CMD mkdir -p /stable-diffusion/models/ldm/stable-diffusion-v1/ && \
-  ln -sf /cache/models/model.ckpt /stable-diffusion/models/ldm/stable-diffusion-v1/model.ckpt && \
-  python3 -u scripts/dream.py --outdir /output --web --host 0.0.0.0 --port 7860 ${CLI_ARGS}
+
+CMD /docker/mount.sh && \
+  # python3 -u backend/server.py --host 0.0.0.0 --port 7860 --cors http://localhost:7860
+  # python3 -u scripts/dream.py --outdir /output --web --host 0.0.0.0 --port 7860 ${CLI_ARGS}
+  echo The lstein webUI is currently deactivated due to implementation limitations: \
+  https://github.com/invoke-ai/InvokeAI/blob/8c9f2ae705cf723d4a8a73c416e8d8bf2d746977/backend/modules/create_cmd_parser.py#L26 \
+  Once the path the output is fixed, the UI will be activated again
--- a/services/lstein/README.md
+++ b/services/lstein/README.md
@@ -1,14 +0,0 @@
-# WebUI for lstein
-
-The WebUI of [lstein/stable-diffusion](https://github.com/lstein/stable-diffusion) as docker container!
-
-Although it is a simple UI, the project has a lot of potential.
-
-## Setup
-
-Clone this repo, download the `model.ckpt` and put into the `models` folder as mentioned in [the main README](../README.md), then run
-
-```
-cd lstein
-docker compose up --build
-```
--- a/services/lstein/info.py
+++ b/services/lstein/info.py
@@ -0,0 +1,10 @@
+import sys
+from pathlib import Path
+
+file = Path(sys.argv[1])
+file.write_text(
+  file.read_text()\
+  .replace('GitHub site</a>', """
+  GitHub site</a>, Deployed with <a href="https://github.com/AbdBarho/stable-diffusion-webui-docker/">stable-diffusion-webui-docker</a>
+""", 1)
+)
--- a/services/lstein/mount.sh
+++ b/services/lstein/mount.sh
@@ -0,0 +1,29 @@
+#!/bin/bash
+
+set -eu
+
+ROOT=/stable-diffusion
+
+mkdir -p "${ROOT}/models/ldm/stable-diffusion-v1/"
+ln -sf /cache/models/model.ckpt "${ROOT}/models/ldm/stable-diffusion-v1/model.ckpt"
+
+base="${ROOT}/src/gfpgan/experiments/pretrained_models/"
+mkdir -p "${base}"
+# TODO: "real" GFPGANv1.4.pth
+ln -sf /cache/models/GFPGANv1.3.pth "${base}/GFPGANv1.4.pth"
+echo "Mounted GFPGANv1.3.pth"
+
+# facexlib
+FACEX_WEIGHTS=/opt/conda/lib/python3.9/site-packages/facexlib/weights
+
+rm -rf "${FACEX_WEIGHTS}"
+mkdir -p /cache/weights
+ln -sf -T /cache/weights "${FACEX_WEIGHTS}"
+
+REALESRGAN_WEIGHTS=/opt/conda/lib/python3.9/site-packages/realesrgan/weights
+rm -rf "${REALESRGAN_WEIGHTS}"
+ln -sf -T /cache/weights "${REALESRGAN_WEIGHTS}"
+
+if "${PRELOAD}" == "true"; then
+  python3 -u scripts/preload_models.py
+fi
Author	SHA1	Message	Date
AbdBarho	14c4b36aff	v2 (#108 ) ### Update versions - auto: `3f417566b0` ### Breaking changes: * renamed `automatic-1111` service to `auto` * the `cache` folder is now deprecated, replaced with `data` (see migration guide below) * `embeddings` folder has been moved to `data/embeddings` * use GFPGAN 1.4 ### Migration Guide Note: in theory, running the command ``` docker compose --profile download up --build ``` is all you need to use the new version, however, this means you will also have to download everything again. A new script is available under `scripts/migratev1tov2.sh` that will copy models to the new structure and should get you most of the way, run ```bash ./scripts/migratev1tov2.sh ``` or you can manually inspect the script and copy the files After that, run ``` docker compose --profile download up --build ``` to validate everything.	2022-10-01 12:57:53 +02:00
AbdBarho	28f171e64d	Update / Disable lstein Temporarily (#106 ) - auto: `f80c3696f6` - model merger now works! the resulting model is saved in `cache/custom-models` - hlky: `aaa3be16e0` - lstein: `8c9f2ae705` - This UI has been temporarely disabled due to limitation in the output path: `8c9f2ae705/backend/modules/create_cmd_parser.py (L26)`	2022-09-30 09:37:27 +02:00
AbdBarho	9af4a23ec4	Stalebot: don't ignore updates	2022-09-29 12:04:40 +02:00
AbdBarho	24ecd676ab	Update versions (#104 ) - auto: `15f333a266` - Checkpoint merger NOT WORKING!!! - hlky: `7bd785d28f` - Streamlit UI still unstable and clunky	2022-09-28 10:18:07 +02:00
Sebastian Piechowiak	ef36c50cf9	Docker compose .gitignore update (#100 ) Docker compose allows override some settings in `docker-compose.yml` by using additional file: `docker-compose.override.yml`. This allows to hold own settings in override file which does not conflict with updates made by pulling newer version with "git pull" command. This feature requires three things: 1. Creating `docker-compose.override.yml-dist` file which is a distributed file inside repo. This file can be copied as `docker-compose.override.yml` and modified for own needs. 2. Change in `.gitignore` file so `docker-compose.override.yml` file is ignored, so git pull / commit will not complain about this file. 3. Modify wiki entry about setup to mention possibility to use this method. Closes #101	2022-09-28 08:36:53 +02:00
AbdBarho	43a5e5e85f	Update versions (#99 ) - auto: `ca3e5519e8` - hlky: `1fd28eed1e` - lstein: `b40bfb5116`	2022-09-26 08:31:47 +02:00
Rafael Goes	5bbc21ea3d	Adding embeddings volume for auto textual inversion (#98 ) Adding embeddings volume mapping for AUTOMATIC1111, enabling textual inversion feature. As discussed in #93	2022-09-25 18:56:38 +02:00
AbdBarho	09366ed955	Ignore Updates	2022-09-25 12:42:33 +02:00
AbdBarho	d4874e7c3a	Update versions (#96 ) - auto: `a2bea2f97a` - hlky: `f585ab1923` - New UI is still in works & extremely unstable - lstein: No new updates, especially not to the UI	2022-09-24 11:10:11 +02:00
AbdBarho	7638fb4e5e	Fix CLIP model caching #88 (#95 ) Refs #88 hacky solution but works for now	2022-09-24 09:57:57 +02:00
AbdBarho	15a61a99d6	Explicit path to GFPGAN model (#91 ) Refs #89	2022-09-23 16:38:50 +02:00
AbdBarho	556a50f49b	Pin transformers version (#90 ) Refs #88	2022-09-23 16:24:14 +02:00
AbdBarho	b899f4e516	Update (#87 ) ### Update versions - auto: `d6fd71f36f` - hlky: `2a911049aa`	2022-09-23 10:34:01 +02:00
AbdBarho	a8c85b4699	Update versions (#86 ) - auto: `5a1951f175` - Now with LDSR support - hlky: `fa6a31b23c` - lstein: prepare for new UI Closes #85	2022-09-21 19:10:27 +02:00
Abdullah Barhoum	a96285d10b	Update License	2022-09-20 19:35:10 +02:00
AbdBarho	83b78fe504	Update versions (#82 ) ### Update versions - auto: `dd911a47b3` - hlky: `17748cbc9c` - lstein: `50d607ffea`	2022-09-19 22:02:46 +02:00
AbdBarho	84f9cb84e7	Update versions (#77 ) AUTOMATIC1111/stable-diffusion-webui@9e892d9 lstein/stable-diffusion@9bcb0df transformers==4.22 for caching Refs #78	2022-09-18 13:49:06 +02:00
AbdBarho	6a66ff6abb	Update hlky to dev (#76 ) Update hlky to dev `abb0c1c377`	2022-09-17 16:09:54 +02:00
AbdBarho	59892da866	Custom Models Auto (#75 )	2022-09-17 13:44:00 +02:00
Abdullah Barhoum	fceb83c2b0	Dev hlky	2022-09-16 21:10:40 +02:00
AbdBarho	17b01a7627	Parallel Downloads (#74 )	2022-09-16 20:07:50 +02:00
Abdullah Barhoum	b96d7c30d0	make executable	2022-09-16 18:37:41 +02:00
AbdBarho	aae83bb8f2	Update lstein to dev branch (#73 )	2022-09-16 16:40:20 +02:00
AbdBarho	10763a8f61	Update Git Post Buffer	2022-09-16 06:51:14 +02:00
AbdBarho	64e8f093d2	Create stale.yml	2022-09-16 06:41:49 +02:00
AbdBarho	3e0a137c23	Remove outdated (#69 )	2022-09-15 22:48:38 +02:00