---

title: "ControlNet Field Guide: SwarmUI & InvokeAI" description: "A practical 2026 field guide to ControlNet concepts and workflows in SwarmUI and InvokeAI." date: 2026-08-23

PRACTICAL FIELD GUIDE

ControlNet in SwarmUI & InvokeAI

Structure, pose, depth, edges, and multi-control workflows

Verified scope UI terminology checked against current project documentation, current source/release materials, and current repositories on 23 August 2026. Controls vary by app version, base-model family, and installed adapter.

For artists and operators who need repeatable results—not a theory lecture.

How to read this guide

VERIFIED CURRENT UI Exact labels and workflows directly supported by current first-party SwarmUI documentation or current InvokeAI project materials.

GENERAL / VERSION-SENSITIVE Sound ControlNet practice, but the label or location may differ. If a control is absent, treat the section as conceptual guidance rather than an instruction to find a hidden switch.

The 60-second mental model

• A preprocessor converts an ordinary reference image into a control signal: edges, estimated depth, pose landmarks, linework, and so on.
• A matching ControlNet (or related control adapter) tells the diffusion model how strongly to respect that signal.
• Weight/strength controls influence; start/end or control range controls when during denoising the influence is active.
• The ControlNet model must match both the signal type and the base-model family. An SD 1.5 ControlNet does not belong on an SDXL checkpoint.

Start here: a safe baseline

Shared ControlNet concepts

Preprocessor vs. control model

• Preprocessor: produces the map. Canny produces hard edges; Depth estimates relative distance; Pose/OpenPose extracts a skeleton; Lineart simplifies drawn contours.
• Control model/adapter: interprets that map during generation. A depth model expects a depth-style map; a pose model expects pose landmarks.
• Already have a correct map? Skip preprocessing. In SwarmUI this is explicitly documented as Preprocessor = None; the same principle applies elsewhere when the UI supports direct control images.

Weight / strength

Rule of thumb If the result ignores the map, raise weight in small steps. If it looks over-constrained, reduce weight before changing the prompt.

Start/end or control range

• Early denoising establishes global composition. Starting at 0 applies control while the scene is being formed.
• Late denoising refines texture and small details. Ending early can preserve the layout while allowing natural detail to emerge.
• SwarmUI: Display Advanced exposes ControlNet Start and End. Current InvokeAI control-layer builds expose comparable adapter settings when supported, but naming and availability vary by adapter/model family.

Side-by-side workflow

SwarmUI: practical procedure

1. Match the architecture — Select the base checkpoint first; then choose a ControlNet trained for the same family (SD 1.5, SDXL, Flux/other only where supported).

2. Select the control model — At the bottom of Generate, select from the ControlNets model list. If newly installed, use the list’s refresh icon—not browser refresh.

3. Open the parameter group — Open ControlNet. Drop a reference into Choose File, or deliberately use Init Image fallback.

4. Set or confirm the preprocessor — Autodetection is documented for some Canny and Depth models. For other controls, select manually. Click Preview and judge the map, not just the source photo.

5. Establish a baseline — Start near 0.7–0.8 strength, Start 0.0, End 0.8–1.0. Lock the seed and generate.

6. Add controls carefully — Display Advanced reveals ControlNet Two and Three. Add one at a time and reduce competing weights.

Verified SwarmUI detail A preprocessed OpenPose image can be used with Preprocessor = None and an OpenPose ControlNet. Preview errors point toward the preprocessor/input; generation-only errors point more toward model compatibility.

InvokeAI: practical procedure

1. Work in Canvas — Set the generation model and bounding box/output area. Current InvokeAI uses Canvas Control Layers for ControlNet-style conditioning.

2. Add a Control Layer — Use the layer controls (located at the bottom of the layers list in current releases) and create a Control Layer. Import, paste, or draw the source content.

3. Choose a compatible adapter — Select the layer’s ControlNet/control-adapter model. Do not assume every model family supports every adapter; externally hosted models generally do not support ControlNets.

4. Prepare the signal — Select a processor if the build offers one, or supply an already processed map. For drawn Scribble/Lineart-style control, make the signal clean and high contrast.

5. Tune layer settings — Start with moderate weight. If begin/end or equivalent range controls are shown, use 0.0 to 0.8 as a flexible baseline; exact labels can vary by adapter and version.

6. Stack as layers — Create additional Control Layers for pose + depth, canny + tile, etc. Disable layers to run controlled comparisons.

InvokeAI version note InvokeAI’s project materials confirm Control Layers and multiple current model-family adapters, but its documentation is being actively reorganized. Treat screenshots or older tutorials showing a standalone ControlNet panel as version-specific.

Preprocessors and model pairings

Sizing, cropping, and framing

• Match the reference and output aspect ratio whenever structure matters. A 16:9 source forced into 1:1 either crops important structure or distorts it.
• Prefer crop-to-fit over stretch. If the UI offers fit/fill behavior, inspect the resulting control map and confirm the subject remains where expected.
• Size the generation for the base family’s normal operating range. Larger is not automatically better; preprocessing detail that exceeds model needs can add noise.
• Pose and line controls are sensitive to thin features. Avoid tiny figures, clipped limbs, or lines that vanish after resizing.
• On InvokeAI Canvas, align the control content with the generation bounding box. Off-box content should not be assumed to influence the generated region.

Fast check Overlay the imagined output crop on the control image. If the subject’s face, hands, horizon, or product edges land outside the crop, fix framing before tuning weight.

Using multiple ControlNets

Reliable combinations

Avoid the “two bosses” problem

• Two high-weight controls that disagree will produce artifacts, ignored controls, or a compromised composition.
• Name a primary control. Give it the higher weight and/or longer range. Make every other control secondary.
• Stage your test: primary only → add secondary at 0.25–0.40 → raise slowly. Keep seed, prompt, sampler, and dimensions fixed.
• SwarmUI documents up to three standard UI groups under Display Advanced. InvokeAI stacks multiple Control Layers; practical limits depend on VRAM, graph/model support, and adapter rules.

Troubleshooting decision guide

Quick reference

• Tile behavior and range are especially model/workflow-dependent.

Before you blame the prompt

• Correct base-model family?
• Correct map/model pairing?
• Preview map clean and correctly cropped?
• Only one variable changed?
• Seed locked?
• Secondary controls below the primary weight?

Verified sources and version notes

• SwarmUI, “ControlNets in SwarmUI” (current master documentation, accessed 23 Aug 2026): https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Features/ControlNet.md
• SwarmUI, “Model Support” (current master documentation, accessed 23 Aug 2026): https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md
• InvokeAI repository and current documentation tree (accessed 23 Aug 2026): https://github.com/invoke-ai/InvokeAI
• InvokeAI current releases (accessed 23 Aug 2026): https://github.com/invoke-ai/InvokeAI/releases
• Original ControlNet paper (general concepts): Zhang, Rao & Agrawala, “Adding Conditional Control to Text-to-Image Diffusion Models,” 2023: https://arxiv.org/abs/2302.05543

Verification boundary SwarmUI’s exact labels in this guide are directly documented. InvokeAI’s Control Layer terminology and current model-family support are verified from current project materials; individual adapter-setting labels are marked version-sensitive because the UI and documentation are evolving.