Landbook Rendering with Diffuser

April 28, 2024

Objective

The purpose of this project is to develop a PoC by integrating the Landbook architectural engine with Hugging Face's Diffusers for generating architectural renderings. This PoC aims to: 1) Evaluate the feasibility of using Diffusers for rendering architectural outputs generated by the Landbook engine. 2) Experiment with prompt engineering to create workflows for producing consistent visualizations. 3) Explore the variability in results based on seed adjustments, negative prompts, and prompt refinements. This PoC explores the foundation for advancing AI-powered rendering solutions in architectural design.

Tests

• Landbook architecture engine result rendering test with Diffusers of Hugging Face
• Rendering test for the building created by Landbook achitecture engine
• Explore prompt to fix for applying it to rendering module
• Diffuser returns different styles by seed even if uses the same prompt
• Image to test:

• Cherry-picked results
• simplicity prompt produces results with simpler, solid-color background.
• The keyword for brick facade was used, but the diversity of colors was reduced by keyword for simplicity
• This way may lead to multi-layer-based rendering as Photoshop
• Prmopt:

        {
            "original_image_path": "spacediffusion/landbook-premium-clean-2.png",
            "prompt": [
                "simplicity",
                "architecture",
                "orthographic camera view",
                "white background",
                "brick facade",
                "a small building with multiple stories on the white flat ground",
                "multiplex housing",
                "windows",
                "with shadows",
                "photorealistic",
                "canon EOS 5d",
                "geometric shaped outline",
                "ultra realistic",
                "many details",
                "dramatic lighting",
                "lighting effects"
            ],
            "negative_prompt": [
                "complexity",
                "intricacy",
                "background-color like red / gray / brown",
                "colorful ground",
                "mountain",
                "snow",
                "background",
                "landscape",
                "cityscape",
                "many buildings",
                "ugly",
                "weird",
                "facade",
                "grayscale",
                "low brightness",
                "sketch"
            ],
        }
    

• Cherry-picked intermediate results 1
• Meaningless surrounding contexts are generated
• Prompt:

        {
            "original_image_path": "spacediffusion/landbook-premium-clean-2.png",
            "prompt": [
                "architecture",
                "orthographic camera view",
                "white solid color background",
                "brick facade",
                "a small building with multiple stories on the white flat ground",
                "multiplex housing",
                "windows",
                "with shadows",
                "photorealistic",
                "canon EOS 5d",
                "geometric shaped outline",
                "ultra realistic",
                "many details",
                "dramatic lighting",
                "lighting effects"
            ],
            "negative_prompt": [
                "ugly",
                "weird",
                "facade",
                "background",
                "grayscale",
                "cityscape",
                "low brightness",
                "sketch"
            ],
        }
    

• Cherry-picked intermediate results 2
• Meaningless surrounding contexts are still generated but the ratio was reduced
• Prompt:

        {
            "original_image_path": "spacediffusion/landbook-premium-clean-2.png",
            "prompt": [
                "architecture",
                "orthographic camera view",
                "white background",
                "brick facade",
                "a small building with multiple stories on the white flat ground",
                "multiplex housing",
                "windows",
                "with shadows",
                "photorealistic",
                "canon EOS 5d",
                "geometric shaped outline",
                "ultra realistic",
                "many details",
                "dramatic lighting",
                "lighting effects"
            ],
            "negative_prompt": [
                "colorful ground",
                "background",
                "landscape",
                "cityscape",
                "ugly",
                "weird",
                "facade",
                "grayscale",
                "low brightness",
                "sketch"
            ],
        }
    

• Cherry-picked intermediate results 3
• After adding red for the negative prompt, green or blue color background was generated
• There was no change in the ratio for the solid-color background results
• Prompt:

        {
            "original_image_path": "spacediffusion/landbook-premium-clean-2.png",
            "prompt": [
                "architecture",
                "orthographic camera view",
                "white background",
                "brick facade",
                "a small building with multiple stories on the white flat ground",
                "multiplex housing",
                "windows",
                "with shadows",
                "photorealistic",
                "canon EOS 5d",
                "geometric shaped outline",
                "ultra realistic",
                "many details",
                "dramatic lighting",
                "lighting effects"
            ],
            "negative_prompt": [
                "red",
                "colorful ground",
                "background",
                "landscape",
                "cityscape",
                "ugly",
                "weird",
                "facade",
                "grayscale",
                "low brightness",
                "sketch"
            ],
        }
    

---