Illustrious Beginners Guide
Generating Images with Illustrious
(You Naughty Otaku)
Before we begin our journey into AI image generation, we need to understand what is actually happening behind the scenes.
What is image generation?
How does a diffusion model work?
What do all those confusing settings actually do?
Let's break it down.
What is a Diffusion Model?
A diffusion model is a type of generative AI model that creates new images by learning patterns from the images it was trained on.
The basic idea is surprisingly simple:
The AI is shown images, then those images are gradually destroyed by adding random noise. The model learns how to reverse that process.
After training, the AI can start with pure noise and slowly remove that noise until an image appears.
Think of it like developing a photograph from static.
How Image Generation Works
Step 1: The Canvas of Noise
When you click "Generate," the AI does not start with a blank white canvas.
Instead, it starts with random static, similar to an old television tuned to a dead channel.
At this point, there is no image. Just noise.
Step 2: Reading Your Instructions
The AI looks at several things:
Positive prompt:
What you want the image to contain.
Negative prompt:
Things you want to avoid.
CFG (Classifier-Free Guidance):
How strongly the AI should follow your instructions.
The model uses these as a guide to determine what shapes, colors, objects, and details should emerge from the noise.
Step 3: The Denoising Process
This is where the magic happens.
During generation, the AI performs multiple cleanup passes called steps.
For example, with 30 steps:
- Early steps create rough shapes, composition, and lighting.
- Middle steps define objects, faces, and major details.
- Final steps refine textures, edges, and small details.
The AI is not drawing the image from scratch. It is repeatedly correcting noise until the final image appears.
Step 4: The Final Image
Once the denoising process finishes, the remaining noise is converted into the final image you see.
The static has become art.
Samplers and Schedulers
The first settings we need to understand are:
- Sampler
- Scheduler
- CFG
- Steps
- VAE
These determine how the AI travels from random noise to a finished image.
What Are Samplers?
A sampler controls how the AI removes noise during generation.
Each step, the sampler:
- Predicts what parts of the image are noise
- Decides how much noise to remove
- Determines the direction of the next correction
Different samplers use different mathematical approaches, creating different balances between:
- Speed
- Quality
- Detail
- Randomness
- Consistency
Common Samplers
| Sampler | Speed | Quality | Best Use |
|---|---|---|---|
| Euler | Fast | Good | Quick generations |
| Euler a | Fast | Good | Creative exploration |
| DPM++ 2M | Medium | Excellent | General use |
| DPM++ 2M Karras | Medium | Excellent | High-quality images |
| DPM++ SDE | Slower | Excellent | Detailed images |
| DDIM | Fast | Good | Older workflows |
| UniPC | Fast | Very Good | Efficient generation |
Popular Samplers Explained
Euler
Euler is one of the classic samplers.
It is fast, reliable, and produces good results with fewer steps.
Great for:
- Quick generations
- Testing prompts
- Experimenting
Euler a (Euler Ancestral)
The "a" stands for ancestral.
Ancestral samplers introduce additional randomness during generation, which can create more variation.
Great for:
- Exploring ideas
- Finding unexpected results
- Creative experimentation
DPM++
DPM++ is a newer family of samplers designed for better accuracy and detail.
Compared to Euler, it generally provides:
- Cleaner details
- Better consistency
- Improved anatomy
- Better results at higher quality settings
DPM++ 2M
The "M" means multi-step.
It reuses previous calculations to improve efficiency while maintaining quality.
A very popular choice for production-quality images.
DPM++ SDE
A higher-quality variant that often produces excellent textures and details.
It is slower, but many users prefer the results.
What Does Karras Mean?
Karras is not a sampler. It is a scheduler/noise schedule.
The scheduler controls how noise is removed throughout the generation process.
Karras adjusts the noise curve so that:
- Early steps preserve composition
- Later steps focus more heavily on refinement
This often creates sharper details and better results with fewer steps.
What Does Ancestral Mean?
If you see an "A" after a sampler:
Example:
Euler a
It means the sampler adds some randomness back during the process.
This creates more variety but makes results less predictable.
Think:
Euler:
"Follow the road."
Euler a:
"Follow the road, but occasionally explore the side paths."
Common Illustrious Choices
Many users commonly use:
DPM++ 2M SDE Karras
- High-quality character art
- Detailed images
DPM++ 2M Karras
- Clean production images
- Reliable results
Euler a
- Exploring ideas
- Creative variations
My personal recommendations:
- Best quality character art: DPM++ 2M SDE Karras
- Clean production images: DPM++ 2M Karras
- Exploring concepts: Euler a
- Unexpected results: Euler a with lower CFG
However, Euler remains extremely popular because it is fast, versatile, and many models were trained using it.
Experiment and find what works for your workflow.
CFG - Classifier-Free Guidance
Think of CFG as the AI's "listen to my prompt!" volume knob.
It controls how strongly the AI follows your instructions.
A low CFG:
"I heard your suggestion, but I’m going to do my own thing."
A high CFG:
"I WILL FOLLOW EVERY WORD YOU SAID WITH MAXIMUM FORCE."
Too low:
- The image may ignore important prompt details.
- Characters may drift from what you wanted.
Too high:
- Images can become oversaturated.
- Details can look harsh or unnatural.
- The AI may overcook the image.
Since Illustrious is based on SDXL, a common sweet spot is:
CFG: 4-6
A CFG of 6 can work very well, but Illustrious generally does not like extremely high CFG values.
You can go higher, but expect the AI to start turning the image into a very enthusiastic, overly processed version of itself. Sometimes that works. Sometimes it looks like the AI drank five energy drinks and grabbed a paintbrush. 😂
Steps
Steps are the number of times the AI cleans up the image during generation.
Every step is a calculation loop:
- The AI looks at the current noisy image.
- It predicts what is noise and what belongs.
- It removes a small amount of noise.
- It repeats the process.
More steps generally mean more refinement.
However, more does not always mean better.
A common beginner mistake is thinking:
"If 30 steps looks good, 300 steps must look amazing!"
Usually, it just means you spent ten times longer waiting.
For Illustrious:
20-30 steps: Good starting point
30-40 steps: Fine detail work
50+ steps: Usually diminishing returns
A good starting recipe:
- Steps: 25-35
- CFG: 4-6
- DPM++ 2M SDE Karras
VAE (Otherwise Known As: "Why Do My Colors Look Amazing Now?")
A VAE (Variational Autoencoder) is the translator between the AI's internal image information and the final picture you see.
The diffusion model creates information in a compressed hidden space called the latent space.
The VAE converts that information into an actual image.
The VAE affects:
- Colors
- Contrast
- Lighting
- Shadows
- Sharpness
- Fine details
A simple way to remember:
🖌️ Model: What to paint.
🚶 Sampler: How to paint.
⏱️ Scheduler: When to make each correction.
🎨 VAE: How the painting is developed so you can actually see it.
VAE Precision: FP8, FP16, FP32
FP stands for floating point precision.
Basically, it describes how much numerical detail the computer keeps while processing the VAE.
FP8
Fastest option.
Pros:
- Uses less memory
- Faster processing
Cons:
- Lower precision
- May lose subtle color information
Good for:
- Limited hardware
- Speed-focused workflows
FP16
The standard choice.
Pros:
- Great balance of speed and quality
- Lower memory usage
- Usually looks excellent
For most users:
FP16 is perfectly fine.
FP32
Maximum precision.
Pros:
- More accurate calculations
- Can produce richer colors and smoother gradients
Cons:
- Uses more VRAM/RAM
- Slower
If your hardware can handle it, FP32 VAE can sometimes produce noticeably better colors, especially with certain Illustrious models.
NOW WE PROMPT!
We have our settings ready.
The AI is waiting.
Time to actually tell it what to create.
Prompting Illustrious
Illustrious is trained heavily on Danbooru-style tags.
This means it understands structured tags better than long natural language descriptions.
Example:
Better:
1girl, blue eyes, long hair, smile, school uniform
Less effective:
A beautiful girl with long blue hair smiling while wearing a school uniform.
Natural language can work, but Illustrious generally prefers tags.
Danbooru Tags
Danbooru is an anime-focused tagging system used to describe images.
Examples:
Character:
1girl
solo
blue eyes
long hair
Pose:
sitting
standing
looking at viewer
smile
crossed arms
Camera:
portrait
cowboy shot
full body
from above
Environment:
classroom
forest
city
bedroom
The more accurate the tag, the better the AI understands what you want.
Prompt Order
Illustrious generally responds well to this structure:
Quality
↓
Subject count
↓
Character
↓
Appearance
↓
Clothing
↓
Pose / Action
↓
Environment
↓
Lighting
↓
Camera
↓
Style
Example:
masterpiece, best quality, newest,
1girl, solo,
silver hair, blue eyes,
white dress,
standing, smiling,
garden,
sunset lighting,
cowboy shot
You do not have to follow this perfectly, but it is a great starting point.
Prompt Weighting
Weights allow you to tell the AI:
"This part matters more."
Parentheses
Small increase:
(detailed eyes)
More emphasis:
((detailed eyes))
Even stronger:
(((detailed eyes)))
Numeric Weights
More precise:
(blue eyes:1.3)
(long hair:1.2)
(smile:1.1)
General guideline:
1.1-1.3
- Gentle emphasis
1.4-1.6
- Strong emphasis
1.7+
- Use carefully
Too much weighting can overpower the rest of the prompt.
Reducing Importance
You can lower emphasis:
(background:0.8)
Some interfaces also support:
[background]
However, numeric weights are usually more consistent.
Prompt Length
Illustrious can handle fairly large prompts, but bigger is not always better.
A good target:
20-80 useful tags
You can technically use much longer prompts, but eventually:
- Earlier tags become more important.
- Later tags become weaker.
- Concepts compete with each other.
- The AI may blend conflicting ideas together.
The goal is not the longest prompt.
The goal is a prompt where every tag has a purpose.
LoRA: Small Add-ons With Big Impact
A LoRA (Low-Rank Adaptation) is a small file that teaches an existing model a specific concept.
Think of the main model as an artist.
A LoRA is a small lesson:
"Here is how to draw this character, style, outfit, or feature."
A LoRA does not replace the model.
It modifies the model temporarily while generating.
Common LoRA Uses
Character LoRA:
specific character
Style LoRA:
specific art style
Concept LoRA:
armor design
hairstyle
clothing
poses
LoRA Strength
Most LoRAs use a weight:
<lora:name:0.8>
Common ranges:
0.5-0.8
- Subtle influence
0.8-1.0
- Normal use
1.0+
- Strong influence
Too high:
- Features become exaggerated.
- The image may become distorted.
- The LoRA can overpower the base model.
A good rule:
Start low and increase.
Tools For Image Generation
ComfyUI
The powerhouse.
Pros:
- Maximum control
- Huge community
- Advanced workflows
- Excellent for SDXL, Flux, Illustrious
Cons:
- Steep learning curve
- Node graphs can become complicated
ComfyUI is a workshop full of tools. Powerful, but you have to learn where everything goes.
SwarmUI
The balance between power and simplicity.
Pros:
- Easier than ComfyUI
- Uses Comfy backend
- Great model management
- Great for batch generation
Cons:
- Less granular control than raw ComfyUI
A fantastic choice for people who want power without building a spaceship cockpit.
InvokeAI
Beautiful and artist-friendly.
Pros:
- Excellent canvas tools
- Great inpainting
- Polished interface
Cons:
- Some advanced workflows are less flexible
Great for artists who want AI integrated into a creative workflow.
AUTOMATIC1111
The classic.
Pros:
- Huge community
- Tons of extensions
- Endless tutorials
Cons:
- Older architecture
- Workflow management can become messy
Fooocus
Beginner friendly.
Pros:
- Very easy
- Great results quickly
Cons:
- Less control
Krita AI Diffusion
Best for artists who already draw.
Pros:
- AI inside a painting workflow
- Great control
Cons:
- Setup can require patience
Useful Resources
Models and Checkpoints
Hugging Face
A major repository for AI models, datasets, and research.
Civitai
A popular community site for checkpoints, LoRAs, embeddings, and workflows.
Tags
Danbooru
A reference for anime-style tags.
Workflows and Help
ComfyUI GitHub
Documentation and updates for ComfyUI.
Beginner Illustrious Recipe
Start here:
Model:
Illustrious XL
Resolution:
1024x1024
Sampler:
DPM++ 2M SDE Karras
Steps:
25-35
CFG:
4-6
VAE:
FP16 or FP32
Prompt:
Danbooru tags
Seed:
Random while exploring
Fixed when recreating results
Final Advice
Do not chase the "perfect" settings.
The best workflow is the one that lets you create.
Experiment.
Change one thing at a time.
Learn what each setting actually changes.
The AI is not a magic button.
It is a paintbrush with thousands of tiny knobs attached.
Now go make something awesome. 🎨