In the realm of AI-generated imagery, Stable Diffusion 3 stands out as a powerful tool, enabling the creation of stunning visuals with remarkable detail. However, setting up Stable Diffusion 3 locally can be a daunting task if you’re a little new to setting up models locally. This blog aims to simplify the process, providing a straightforward pathway to get Stable Diffusion 3 up and running on your local machine.

A simple GitHub repository, Stable Diffusion 3 Notebook, has been designed to make this journey smoother. While many of the necessary resources can already be found on the Stable Diffusion Hugging Face repo by Stability AI, our project aggregates these essentials for seamless local setup. This setup is predominantly aimed at Linux users. However, with a little variation in the setup it should be doable on Windows WSL or Mac.

Find our simple notebook setup here

What is Stable Diffusion 3?

Stable Diffusion 3 is an advanced AI model designed for generating high-quality images from textual descriptions. It represents a significant leap forward from its predecessors, offering improved performance, better detail, and more realistic outputs.

Key Features

• Enhanced Image Quality: Generates high-resolution images with intricate details.
• Versatility: Adaptable to various use cases, from digital art to content creation.
• Efficiency: Optimised for faster processing times and reduced computational costs.

Use Cases

• Digital Art: Create unique, high-quality art pieces.
• Game Development: Generate textures, backgrounds, and concept character designs.
• Content Creation: Enhance blogs, articles, and social media posts with AI-generated imagery.
• Marketing and Advertising: Design eye-catching visuals for campaigns.

Why Set Up Stable Diffusion 3 Locally?

Running Stable Diffusion 3 on a local machine offers several advantages:

1. Privacy and Control: Keep your data and generated content private.
2. Performance: Leverage local hardware for intensive tasks.
3. Customisation: Integrate and tweak the model to fit your specific needs.

Overview of the Project Repository

Our GitHub repository, Stable Diffusion 3 Notebook, provides all the necessary resources to set up Stable Diffusion 3 locally. Key components include:

• Jupyter Notebook: For easy experimentation and interaction.
• Setup Scripts: Automates the environment setup process.
• requirements.txt: Manages dependencies.
• .env.hf: Manages secure environment variables.

Prerequisites and Initial Setup

Before diving into the setup, ensure you have the following prerequisites:

• Hardware: A machine with sufficient GPU memory (preferably 8GB or more).
• Software: Python 3.11.8 or later.
• Installing pyenv on a Linux Server: To manage multiple Python versions, we recommend using pyenv.
• Authozation to the Huggingface repo: Visit https://huggingface.co/stabilityai/stable-diffusion-3-medium and submit the form for access for your HF account.
• Huggingface token: You can obtain a token from your Hugging Face account settings. Read access is all you need. https://huggingface.co/settings/tokens

Install Dependencies

1. Install pyenv: Follow the official guide to install pyenv if you haven’t already.
2. Install a Specific Python Version:
   pyenv install 3.11.8
3. Set the local Python Version
   In the root of your project folder:
   pyenv local 3.11.8
4. Verify the Installation:
   python --version

Cloning and Setting Up the Repository

1. Clone the Repository:
   git clone https://github.com/pixelkey/stable-diffusion-3-notebook.git
cd stable-diffusion-3-notebook
2. Create a Virtual Environment:
   python -m venv .env source .env/bin/activate
3. Install Dependencies:
   pip install -r requirements.txt

Adding Your Hugging Face Token

Stable Diffusion 3 requires a Hugging Face token for authentication.

1. Create .env.hf file:
   touch .env.hf
2. Add Your Token to the file:
   Replace your_hugging_face_token_here with your actual token. You can obtain a token from your Hugging Face account settings.

Running the Notebook

• Launch Jupyter Notebook:
  jupyter notebook
• Open the Notebook:
  Navigate to sd3.ipynb and open it.
• Run the Cells:
  Follow the instructions within the notebook to set up the environment, generate images, and save them locally.

Troubleshooting and Tips

• Environment Issues:
  Ensure you are in the correct Python environment (which python).
• Dependency Issues:
  Verify all dependencies are installed (pip list).
• Performance Optimisation:
  Allocate sufficient GPU memory and tweak settings for optimal performance. You’ll notice we added the line “pipe.enable_model_cpu_offload()” to help with compatibility for smaller GPUs that only have 8GB. There are other tweaks available in Stability AI documentation. https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/stable_diffusion_3

Potential Use Cases

Stable Diffusion 3 offers numerous possibilities for developers:

1. Digital Art and Illustrations: Create unique, high-quality art pieces.
2. Game Development: Generate textures, backgrounds, and character designs.
3. Content Creation: Enhance blogs, articles, and social media posts with AI-generated imagery.
4. Marketing and Advertising: Design eye-catching visuals for campaigns.

Contributing to the Project

We welcome contributions! Here’s how you can get involved:

• Fork the Repository: Create your own fork on GitHub.
• Submit Pull Requests: Make improvements and submit PRs for review.
• Open Issues: Report bugs or suggest new features.

Setting up Stable Diffusion 3 locally doesn’t have to be a complicated process. Our project repository on GitHub provides everything you need to get started quickly and easily. By following this guide, you can leverage the power of Stable Diffusion 3 for your own projects, from digital art to game development and beyond. Visit our repository at Stable Diffusion 3 Notebook and start your journey today.