Generative AI for Architecture

05 — Generative AI

Generative AI for Architecture

Reinterpreting icons with ControlNet and LoRA

Tools
ComfyUI · ControlNet · LoRA
Studies
Dancing House · Fallingwater
Method
Parameter sweeps

Using ComfyUI to reinterpret two icons of twentieth-century architecture - the Dancing House and Fallingwater - while controlling how far each design departs from the original.

Each study holds on to the building through a ControlNet - line art for the Dancing House, a depth map for Fallingwater - while LoRA models steer material and form. Lowering the ControlNet strength, or ending its guidance earlier in the generation (End %), gradually frees the model from the original geometry.

Dancing House

ControlNet LineArt, with the Curved and Concrete LoRAs.

Input - the Dancing House
Input - the Dancing House
ControlNet LineArt
ControlNet LineArt

ControlNet strength

Strength 0.8
Strength 0.8
Strength 0.6
Strength 0.6
Strength 0.4
Strength 0.4
Strength 0.2
Strength 0.2
Strength 0
Strength 0

Start 0 · End 1 · Curved LoRA 1 · Concrete LoRA 1.

ControlNet end %

End 0.8
End 0.8
End 0.6
End 0.6
End 0.4
End 0.4
End 0.2
End 0.2
End 0.01
End 0.01

Strength 1 · Start 0 · Curved LoRA 1 · Concrete LoRA 1.

Fallingwater

ControlNet Depth, with the Details LoRA alongside the Origami or Concrete LoRA.

Input - Fallingwater
Input - Fallingwater
ControlNet Depth
ControlNet Depth

ControlNet strength

Strength 0.8
Strength 0.8
Strength 0.6
Strength 0.6
Strength 0.4
Strength 0.4
Strength 0.2
Strength 0.2
Strength 0
Strength 0

Start 0 · End 1 · Details LoRA 1 · Origami LoRA 1.

ControlNet end %

End 0.8
End 0.8
End 0.6
End 0.6
End 0.4
End 0.4
End 0.2
End 0.2
End 0.01
End 0.01

Strength 1 · Start 0 · Details LoRA 1 · Concrete LoRA 1.