🖼️ Python Bindings for stable-diffusion.cpp

Simple Python bindings for @leejet's stable-diffusion.cpp library.

This package provides:

• Low-level access to C API via ctypes interface.
• High-level Python API for Stable Diffusion and FLUX image generation.

Installation

Requirements:

• Python 3.8+
• C compiler
  • Linux: gcc or clang
  • Windows: Visual Studio or MinGW
  • MacOS: Xcode

To install the package, run:

pip install stable-diffusion-cpp-python

This will also build stable-diffusion.cpp from source and install it alongside this python package.

If this fails, add --verbose to the pip install to see the full cmake build log.

Installation Configuration

stable-diffusion.cpp supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the stable-diffusion.cpp README for a full list.

All stable-diffusion.cpp cmake build options can be set via the CMAKE_ARGS environment variable or via the --config-settings / -C cli flag during installation.

# Linux and Mac
CMAKE_ARGS="-DSD_CUBLAS=ON" pip install stable-diffusion-cpp-python

# Windows
$env:CMAKE_ARGS="-DSD_CUBLAS=ON"
pip install stable-diffusion-cpp-python

pip install --upgrade pip # ensure pip is up to date
pip install stable-diffusion-cpp-python -C cmake.args="-DSD_CUBLAS=ON"

# requirements.txt

stable-diffusion-cpp-python -C cmake.args="-DSD_CUBLAS=ON"

Supported Backends

Below are some common backends, their build commands and any additional environment variables required.

CMAKE_ARGS="-DSD_CUBLAS=ON" pip install stable-diffusion-cpp-python

CMAKE_ARGS="-G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1101" pip install stable-diffusion-cpp-python

CMAKE_ARGS="-DSD_METAL=ON" pip install stable-diffusion-cpp-python

CMAKE_ARGS="-DSD_VULKAN=ON" pip install stable-diffusion-cpp-python

# Export relevant ENV variables
source /opt/intel/oneapi/setvars.sh

# Option 1: Use FP32 (recommended for better performance in most cases)
CMAKE_ARGS="-DSD_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx" pip install stable-diffusion-cpp-python

# Option 2: Use FP16
CMAKE_ARGS="-DSD_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL_F16=ON" pip install stable-diffusion-cpp-python

CMAKE_ARGS="-DSD_FLASH_ATTN=ON" pip install stable-diffusion-cpp-python

CMAKE_ARGS="-DGGML_OPENBLAS=ON" pip install stable-diffusion-cpp-python

Upgrading and Reinstalling

To upgrade and rebuild stable-diffusion-cpp-python add --upgrade --force-reinstall --no-cache-dir flags to the pip install command to ensure the package is rebuilt from source.

High-level API

The high-level API provides a simple managed interface through the StableDiffusion class.

Below is a short example demonstrating how to use the high-level API to generate a simple image:

Text to Image

from stable_diffusion_cpp import StableDiffusion

def callback(step: int, steps: int, time: float):
    print("Completed step: {} of {}".format(step, steps))

stable_diffusion = StableDiffusion(
      model_path="../models/v1-5-pruned-emaonly.safetensors",
      wtype="default", # Weight type (default: automatically determines weight type of model file)
)
output = stable_diffusion.txt_to_img(
      prompt="a lovely cat",
      width=512, # Must be a multiple of 64
      height=512, # Must be a multiple of 64
      progress_callback=callback,
      # seed=1337, # Uncomment to set a specific seed
)
output[0].save("output.png") # Image returned as list of PIL Images

With LoRA (Stable Diffusion)

You can specify the directory where the lora weights are stored via lora_model_dir. If not specified, the default is the current working directory.

• LoRA is specified via prompt, just like stable-diffusion-webui. (e.g. ``)
• LoRAs will not work when using quantized models. You must instead use a full precision .safetensors model.

Here's a simple example:

from stable_diffusion_cpp import StableDiffusion

stable_diffusion = StableDiffusion(
      model_path="../models/v1-5-pruned-emaonly.safetensors",
      lora_model_dir="../models/", # This should point to folder where LoRA weights are stored (not an individual file)
)
output = stable_diffusion.txt_to_img(
      prompt="a lovely cat",
)

• The lora_model_dir argument is used in the same way for FLUX image generation.

FLUX Image Generation

FLUX models should be run using the same implementation as the stable-diffusion.cpp FLUX documentation where the diffusion_model_path argument is used in place of the model_path. The clip_l_path, t5xxl_path, and vae_path arguments are also required for inference to function.

Download the weights from the links below:

• Preconverted gguf weights from FLUX.1-dev-gguf or FLUX.1-schnell, this way you don't have to do the conversion yourself.
• Download vae from https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/ae.safetensors
• Download clip_l from https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/clip_l.safetensors
• Download t5xxl from https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/t5xxl_fp16.safetensors

from stable_diffusion_cpp import StableDiffusion

stable_diffusion = StableDiffusion(
    diffusion_model_path="../models/flux1-schnell-q3_k.gguf", # In place of model_path
    clip_l_path="../models/clip_l.safetensors",
    t5xxl_path="../models/t5xxl_fp16.safetensors",
    vae_path="../models/ae.safetensors",
)
output = stable_diffusion.txt_to_img(
      prompt="a lovely cat holding a sign says 'flux.cpp'",
      sample_steps=4,
      cfg_scale=1.0, # a cfg_scale of 1 is recommended for FLUX
      sample_method="euler", # euler is recommended for FLUX
)

With LoRA (FLUX)

LoRAs can be used with FLUX models in the same way as Stable Diffusion models (as shown above).

Note that:

• It is recommended to use LoRA with naming formats compatible with ComfyUI.
• Only the Flux-dev q8_0 will work with LoRAs.
• You can download FLUX LoRA models from https://huggingface.co/XLabs-AI/flux-lora-collection/tree/main (you must use a comfy converted version!!!).

SD3.5 Image Generation

Download the weights from the links below:

• Download sd3.5_large from https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/sd3.5_large.safetensors
• Download clip_g from https://huggingface.co/Comfy-Org/stable-diffusion-3.5-fp8/blob/main/text_encoders/clip_g.safetensors
• Download clip_l from https://huggingface.co/Comfy-Org/stable-diffusion-3.5-fp8/blob/main/text_encoders/clip_l.safetensors
• Download t5xxl from https://huggingface.co/Comfy-Org/stable-diffusion-3.5-fp8/blob/main/text_encoders/t5xxl_fp16.safetensors

from stable_diffusion_cpp import StableDiffusion

stable_diffusion = StableDiffusion(
    model_path="../models/sd3.5_large.safetensors",
    clip_l_path="../models/clip_l.safetensors",
    clip_g_path="../models/clip_g.safetensors",
    t5xxl_path="../models/t5xxl_fp16.safetensors",
)
output = stable_diffusion.txt_to_img(
      prompt="a lovely cat holding a sign says 'Stable diffusion 3.5 Large'",
      height=1024,
      width=1024,
      cfg_scale=4.5,
      sample_method="euler",
)

Other High-level API Examples

Other examples for the high-level API (such as image to image, upscaling and model conversion) can be found in the tests directory.

Low-level API

The low-level API is a direct ctypes binding to the C API provided by stable-diffusion.cpp. The entire low-level API can be found in stable_diffusion_cpp/stable_diffusion_cpp.py and directly mirrors the C API in stable-diffusion.h.

Below is a short example demonstrating how to use the low-level API:

import stable_diffusion_cpp as sd_cpp
import ctypes
from PIL import Image

img = Image.open("path/to/image.png")
img_bytes = img.tobytes()

c_image = sd_cpp.sd_image_t(
      width=img.width,
      height=img.height,
      channel=channel,
      data=ctypes.cast(
            (ctypes.c_byte * len(img_bytes))(*img_bytes),
            ctypes.POINTER(ctypes.c_uint8),
      ),
) # Create a new C sd_image_t

img = sd_cpp.upscale(
      self.upscaler,
      image_bytes,
      upscale_factor,
) # Upscale the image

Development

To get started, clone the repository and install the package in editable / development mode.

git clone --recurse-submodules https://github.com/william-murray1204/stable-diffusion-cpp-python.git
cd stable-diffusion-cpp-python

# Upgrade pip (required for editable mode)
pip install --upgrade pip

# Install with pip
pip install -e .

Now you can make changes to the code within the stable_diffusion_cpp directory and test them in your python environment.

Cleanup

To clear the cache.

make clean

References

• stable-diffusion.cpp
• llama-cpp-python
• llama.cpp
• whisper-cpp-python
• Golang stable-diffusion
• StableDiffusion.NET

License

This project is licensed under the terms of the MIT license. See LICENSE for details.