Stable Diffusion v1.5 — Pokemon LoRA
LoRA adaptation weights for Stable Diffusion v1.5, fine-tuned on the Pokemon BLIP Captions dataset (833 samples) to generate Pokemon-style images from text prompts.
Video walkthrough: Fine-tune Stable Diffusion with LoRA for as low as $1
Training Details
| Detail | Value |
|---|---|
| Method | 4-bit QLoRA following the official LoRA blog post |
| Hardware | Amazon EC2 g4dn.xlarge (1x NVIDIA T4 GPU) |
| Environment | AWS Deep Learning AMI for PyTorch |
| Training time | ~6 hours |
| Cost | ~$3 on-demand / ~$1 with EC2 Spot Instances |
Usage
from diffusers import StableDiffusionPipeline
import torch
pipe = StableDiffusionPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16
)
pipe.unet.load_attn_procs("juliensimon/stable-diffusion-v1-5-pokemon-lora")
pipe.to("cuda")
image = pipe("a pokemon with blue fire wings").images[0]
Resources
- Training script and training logs are included in the repository.
- Downloads last month
- 32
Model tree for juliensimon/stable-diffusion-v1-5-pokemon-lora
Base model
runwayml/stable-diffusion-v1-5