Generating by Prompts using SDXL-lightning

Introduction

We create a module that generates images by user-defined (text) prompts. As a model, we utilize the pretrained version of Stable Diffusion XL-lightning provided by ByteDance at HuggingFace.

Setting Up the Environment

Using Conda (recommended)

1. Install Conda, if not already installed.
2. Clone the repository:

   git clone https://github.com/byrkbrk/generating-by-prompt-sdxl-lightning.git
   

3. Change the directory:

   cd generating-by-prompt-sdxl-lightning
   

4. Create the environment:

   conda env create -f generating-by-prompt-sdxl-lightning.yaml
   

5. Activate the environment:

   conda activate generating-by-prompt-sdxl-lightning
   

Using pip

1. Download & install Python (version==3.11)
2. Clone the repository:

   git clone https://github.com/byrkbrk/generating-by-prompt-sdxl-lightning.git
   

3. Change the directory:

   cd generating-by-prompt-sdxl-lightning
   

4. Install packages using pip:

   pip install -r requirements.txt
   

Generating Images

Check it out how to use

python3 generate.py --help

Output:

Generate images by prompts using SDXL-lightning

positional arguments:
  prompt                Text prompt that be used for generating

options:
  -h, --help            show this help message and exit
  --step_choice {1-step,2-step,4-step,8-step}
                        Step choice for inference. Default: '4-step'
  --scheduler_name SCHEDULER_NAME
                        Scheduler name for inference. Default: 'euler_discrete_scheduler'
  --device {cuda,mps,cpu}
                        GPU device that be used during inference. Default: None

Example usages

python3 generate.py\
 "an image of a turtle in Picasso style"\
 --step_choice 2-step

python3 generate.py\
 "an image of a turtle in Camille Pissarro style"\
 --step_choice 2-step

The output images seen below (left: Picasso style, right: Pissarro style) will be saved into ./generated-images folder.

Generating Images by Gradio

To run the gradio app on your local computer, execute:

python3 app.py

Then, visit the url http://127.0.0.1:7860 to open the interface.

Example usage

See the display below for an example usage of the module via Gradio for the prompt 'a painting titled "Lion" in Claude Monet style' with inference steps of 4.