Projects, PoCs, Prototypes, Studies, Tests, ...
Image to 3D Scene Pipeline
Introduction
This project aims to develop an interior scene generation system, funded by KOCCA, that converts single
interior images into structured 3D scenes.
By combining models for 2D object detection, automatic mask
and semantic segmentation, and single-view image-to-3D reconstruction, the pipeline detects and segments
interior elements from a single input photo, reconstructs each object as an aligned and textured 3D mesh,
and then merges them into a single reconstructed scene.
Applications include virtual walkthroughs, scene editing, AR/VR staging, and interior design automation.
SAM 3D
SAM 3D Objects represents a new approach
to 3D reconstruction and object pose estimation
from a single natural image, reconstructing detailed 3D shapes, textures, and layouts of objects from everyday images.
.
.
#generative-design #sam #sam-3d-objects
Wave Function Collapse
What is Wave Function Collapse?
Wave function collapse is a quantum mechanics concept
where a system in a superposition of states
instantly reduces to a single, definite state upon measurement or observation.
In computer science, the Wave Function Collapse (WFC) algorithm is a technique for generating procedural,
constraint-based content, such as tilemaps or textures, similar to solving a 3D Sudoku puzzle.
Each cell in a grid is initially treated as a superposition of all possible states.
Rather than sampling all cells simultaneously, WFC proceeds by selecting a single cell,
collapsing its state space to one concrete value, and then propagating the resulting constraints to neighboring cells.
Intuition of how the WFC algorithm works
Consider first the 3x3 block in the right-center of the leftmost image.
.
.
#generative-design #procedural generation
Latent Shapes
Objectives
Latent vectors are typically initialized from a normal distribution and then updated during training to minimize the loss function.
Although linear interpolation can be used between latent vectors,
and similar vectors can be placed in spatially close locations, the latent vectors themselves do not have a specific geometric shape or structure.
This project takes inspiration from this perspective and explores two main ideas:
1) Latent vectors with geometric shape ─ exploring what happens if latent vectors are given a geometric structure, and how this can help us better understand, interpret, or control the results;
2) Interactive manipulation using the latent vector ─ making it possible for users to modify latent vectors easily, so they can immediately see the effects and create new shapes or designs in an interactive way.
Latent Vector Manipulation in real-time with Mouse Dragging
Data Preprocessing
This project uses the chair dataset (03001627) from ShapeNetCore.
The original data is available here.
.
.
#deep-sdf #generative-design #interpolation #latent-vector
Office Layout Generation Agents
PlanNext
PlanNext is a cloud-based API service that uses Large Language Models to generate optimized office layouts from natural language input.
By interpreting text-based requirements, it automatically generates layouts divided into essential work zones: Focus, Collaboration, Social, Functional, Service, and Private.
Visualized results from PlanNext API
This flow chart shows the complete pipeline of an LLM-based office layout generation system.
When a user provides layout requirements in natural language, the OrchestrationAgent analyzes the request and selects the required specialized agents.
.
.
#agent #generative-design #llm
Voxels to Graph
Introduction 🔗
This project explores the key contributions of Building-GAN by AutodeskAILab:
1) a novel 3D representation called voxel graph that can encode irregular voxel grids with non-uniform space partitioning, overcoming limitations of traditional regular voxel grids, and
2) a graph-conditioned generative adversarial network (GAN) leveraging graph neural networks (GNNs).
The implementation uses their dataset to study these approaches.
The structure of the raw data is examined first in the following section.
Generated Buildings
Raw Data
In the Data Collection part of the paper, it says that the dataset was created as follows:
Since there is no publicly available dataset for volumetric designs from real buildings, we create a synthetic
dataset with 120,000 volumetric designs for commercial buildings using parametric models.
The heuristics behind the parametric models are based on the rules and knowledge provided by professional architects.
.
.
#building-gan #gnn #graph #voxel
Landbook Diffusion Pipeline
Landbook 🏠
Landbook is a service that supports the entire process of new construction development for small and medium-sized land investors.
Landbook's AI architect service provides building owners with various architectural design proposals
by considering different plot sizes, zoning areas, and building regulations for each region.
Landbook's AI architect service
This project develops a pipeline that renders the final results of the Landbook AI architect service using a generative image model such as Diffusers.
By taking 3D modeling data as input and generating realistic images that resemble actual buildings,
it allows building owners to visualize and review how their design proposals would look after construction.
Unlike conventional architectural 3D rendering tools such as V-Ray and Enscape, this pipeline aims to provide high-quality visualization that considers both building textures
and the surrounding context by using AI-based generative models.
.
.
#diffusers #huggingface #landbook
Floor Plan Generation with Voronoi Diagram
Introduction 🔹
This project reviews
and implements the paper Free-form Floor Plan Design using Differentiable Voronoi Diagram.
In deep learning and gradient-based optimization methods, gradients are usually computed using tensor operations, but this approach is not intuitive for geometric operations.
Therefore, this project integrates tensor operations and geometric operations using Pytorch and Shapely.
The main difference between the paper and this project is the use of autograd.
In the paper, the Differentiable Voronoi Diagram is used to propagate gradients,
while this project uses Numerical Differentiation to approximate the gradients directly.
Floor plan generation with voronoi diagram
The following section describes numerical differentiation.
Numerical Differentiation
Numerical differentiation
is a method used to approximate derivatives using finite perturbation differences.
.
.
#numerical-differentiation #voronoi-diagram
Polygon segmentations
Objective ✂️
In the early stages of architectural design, the axis defines the direction in which a building is to be placed.
This axis plays an important role in determining the optimal layout with respect to the functionality, aesthetics, and environmental conditions of the building.
The goal of this project is to develop a segmenter that determines how many axes a given 2D polygon should be segmented into and how those segmentations should be formed.
Segmented lands by a human architect
The figures above show imaginary dividing lines set arbitrarily by the architect.
The apartments in the figures are placed based on the segmented polygons.
Human architects intuitively know how many axes a given 2D polygon should be segmented into and how to create these segmentations, but expressing this intuition to a computer is difficult.
To achieve this, a combination of deep learning and graph theory is used.
In the graph, each point of the polygon is treated as a node, and the connections between points are treated as edges.
Based on this concept, a GNN-based model is implemented to learn how to optimally segment given polygons.
.
.
#deep-learning #geometry #graph-neural-network #link-prediction
Landbook Rendering with Diffuser
Objective
The purpose of this project is to develop a proof of concept
by integrating the Landbook architectural engine with Hugging Face's Diffusers
for generating architectural renderings.
This proof of concept aims to:
1) Evaluate the feasibility of using Diffusers to render architectural outputs generated by the Landbook engine.
2) Experiment with prompt engineering to establish workflows that produce consistent visualizations.
3) Explore the variability of results based on seed adjustments, negative prompts, and prompt refinements.
This work establishes the foundation for advancing AI-powered rendering solutions in architectural design.
Tests
Rendering test of the Landbook architecture engine result with Hugging Face Diffusers
The building generated by the Landbook architecture engine is rendered as a test case.
A prompt is explored and fixed before it is applied to the rendering module.
Diffusers returns different styles depending on the seed, even when the same prompt is used.
Image to test:
Cherry-picked results
The simplicity prompt produces results with a simpler, solid-color background.
The keyword brick facade was used, but the diversity of colors was reduced by the keyword simplicity.
This approach may lead to multi-layer-based rendering, similar to Photoshop.
.
.
#diffusers #huggingface
Infinite synthesis
Introduction 🏢
Deep Generative AI, a field of artificial intelligence concerned with generating new data resembling its training data, is exerting an influence not only on text and image generation but also on 3d model generation within the design industry.
In the domain of architectural design, and particularly in the phase of initial design, generative AI can serve as a useful instrument for examining a large number of design options.
Physical model examination by humans
Shou Sugi Ban / BYTR Architects
By leveraging the Deep Signed Distance Functions model (DeepSDF) together with latent vectors, this project aims to build the algorithm that can synthesize infinite number of skyscrapers similar to the trained data.
These vectors, mapped within a high-dimensional latent space, serve as the DNA for synthesizing potential skyscrapers.
Through a system that manipulates the latent vectors that express the shape of the buildings, architectural designers can rapidly generate and examine a diverse array of design options.
By manipulating two or more latent vectors through interpolation and arithmetic operations, the model provides virtually infinite design options.
This method not only introduces a new design process but also enables the exploration of novel architectural forms that were previously unattainable through conventional design methodologies.
.
.
#deep-learning #deep-sdf #generative-design #geometry
Seeing is solving
Difficulty of the geometric imagination
"I have no geometric imagination.
.
.
"
I often feel this way, especially when facing complex geometry problems or writing code that deals with shapes and spaces.
Picturing complex 3D shapes in the mind is not easy, at least for me.
This is a significant challenge, and it is the reason visualization is so useful.
Visualizing geometry is a key to solving such problems.
Being able to see the geometry under construction makes a substantial difference.
General debugging tools such as print are of limited help here, because they provide numbers and text but do not show the geometry that needs to be observed.
.
.
#debugger #geometry #visualization
Shape-conditional GANs
What is Conditional Generative Adversarial Networks ❓
Conditional Generative Adversarial Networks (cGANs) are an extension of the original Generative Adversarial Networks.
cGANs extend conventional GANs by feeding additional information into both the generator and the discriminator, which enables the generation of data that is more specific and controlled.
This information acts as a directive or constraint that specifies the type of data the generator is expected to produce.
The paper for Conditional Generative Adversarial Nets says:
" Generative adversarial nets can be extended to a conditional model if both the generator and discriminator are conditioned on some extra information y.
y could be any kind of auxiliary information, such as class labels or data from other modalities.
We can perform the conditioning by feeding y into the both the discriminator and generator as additional input layer.
"
.
.
#conditional-generative-adversairal-networks #deep-learning #geometry
Local coordinate system
Types of coordinate systems: Global vs.
Local
In engineering, computer graphics, and physics, understanding the relationship between different coordinate systems is essential for accurate modeling, simulation, and analysis.
The attached figure describes the concept of a global coordinate system (GCS) and a local coordinate system (LCS),
both of which are commonly used to define the position and orientation of objects in 3D space.
The GCS is represented by the axes x , y , and z .
This system provides a fixed reference for describing the location of all objects within the space.
In contrast, the LCS, denoted by the axes x' , y' , and z' , is tied to a specific object within the GCS.
It moves and rotates with the object, providing a reference frame that is local to the object itself.
.
.
#algorithm #coordinate-system #geometry
Mass GANs
Objectives
Generative Adversarial Networks (GANs) have enabled significant advances across numerous fields, ranging from art creation to deepfake video generation.
The potential of GANs, however, is not restricted to 2D space.
The development and application of 3D GANs have opened new possibilities, particularly in the field of design.
This project examines the possibilities of 3D GANs in the design field with the following objectives:
Grasp the fundamental concepts behind GANs and their 3D extension.
Appreciate the power and nuances of 3D GANs through hands-on experiments.
Examine how 3D GANs can be harnessed for product design, architectural modeling, and virtual environment creation.
Visualize and manipulate the latent space to generate novel and innovative designs.
Understand the limitations of current 3D GAN models and the potential areas of improvement.
.
.
#deep-learning #gan #generative-adversarial-networks #machine-learning
The selfish gene
What is natural selection? 🦕
Natural selection is a mechanism of evolution.
Organisms that are more adapted to their environment are more likely to survive and pass on the genes that aided their success.
This process causes species to change and diverge over time.
Natural selection is one of the ways to account for the millions of species that have lived on Earth.
For example, the evolution of long necks has enabled giraffes to feed on leaves that others cannot reach, giving them a competitive advantage.
Owing to this better food source, individuals with longer necks were more likely to survive, while those with shorter necks and access to less food were less likely to survive.
.
.
#evolution #natural-selection #simulation
Parallel execution within GhPython
A simple understanding of multiprocessing
Multiprocessing refers to the ability of a computer system to execute multiple processes or tasks concurrently.
It involves the use of multiple processors or processor cores to perform multiple tasks simultaneously, thereby improving overall system performance and efficiency.
In a multiprocessing system, each process is an independent unit of execution with its own program counter, stack, and data section.
These processes can run concurrently, enabling parallelism and the efficient utilization of system resources.
Processes: A process is an instance of a running program.
Each process has its own memory space, file descriptors, and system resources.
Processes are created and managed by the operating system.
They can communicate with each other using inter-process communication mechanisms.
Multiprocessing can be understood easily through a very simple example, as follows:
.
.
#algorithm #multiprocessing #python
Drafting with AI
Stable diffusion 🤖
Stable Diffusion is a deep learning, text-to-image model released in 2022.
It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt.
An image generated by Stable Diffusion based on the text prompt: "a photograph of an astronaut riding a horse"
Using this model to create images makes it possible to obtain plausible rendered images quickly and easily from only a few lines of text.
This raises the question of how such a capability can be utilized in the field of architecture.
.
.
#controlnet #drawing #rendering #stable-diffusion
Analysis of Apt state-spaces
Objectives
This analysis aims to develop a prototype for deriving the apartment layout starting point for the street housing maintenance project (가로주택정비사업) solution called LBDeveloper (at Spacewalk).
The plots under analysis and the corresponding layout analysis are visualized in the images below.
A further objective is to define the parameters and the range of each parameter used to optimize the apartment complex layout.
Plots to analyze and visualized layout analysis
Axes rules
Because the orientation of street houses must account for the south-facing of all units, the orientation and the placement axis can be treated separately.
The array axes are therefore divided broadly into the following:
Longest segment axis of the plot geometry
Boundary axis
South-facing axis
The rules governing the array axes, derived from the layout analysis, are as follows:
One array axis has a minimum of 1 and a maximum of 3 origins.
The array axes have a minimum of 1 and a maximum of 2 axes when the Boundary axis is not selected.
If the selected array-axis parameter is the Boundary axis, N line segments are offset into the site.
A corner-type building is placed at the intersection of the lines offset from the site boundary axis.
.
.
#algorithm #automation #lbdeveloper #visualization
Plan to Model
Edge detection
An edge is a boundary line or an outline.
In an image, an edge refers to a region located where the brightness changes from a low value to a high value, or vice versa.
Ultimately, an edge indicates the boundary of an object in an image and contains various information, such as cues for shape and direction detection.
Edge detection is the process of finding the pixels that correspond to an edge.
It includes a variety of mathematical methods that aim to identify edges, that is, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities.
Lenna
Lenna (or Lena) is a standard test image used in the field of digital image processing
.
.
#algorithm #automation #open-cv
Klemens Torggler's door
Evolution door
The Evolution Door is a 2012 development and Klemens Torggler's best-known work.
Unlike his previous kinetic works, this object remains on a single level and unfolds into a third dimension as it moves.
Klemens Torggler's The Evolution Door
The Evolution Door is connected to the door frame by two pivots — at the top and bottom — and consists of four triangular “flip-panels.
” When movement is initiated, the four sections collapse in on themselves and rotate 90 degrees before straightening back into a rectangle.
Door opening process
Implementation
The door consists of two square sections, each divided in half along the diagonal.
The diagonals are where two triangular panels meet, connected by a mechanism resembling a piano hinge so that each square can fold as the door moves.
Each part of the rectangle of the door must be square; in other words, the door's width must be half of the door's height.
.
.
#algorithm #formfollowsfunction #python
K-Rooms clusters
K-Means clustering 🕃
The K-Means clustering algorithm partitions a given set of data
into K clusters.
It operates by minimizing the distance differences
within each cluster.
The algorithm is a form of Unsupervised-Learning, and it serves to label unlabeled input data.
Among clustering methods, the K-Means algorithm belongs to the partitioning method.
A partitioning method divides a given set of data into multiple partitions.
For example, assume that n data objects are provided as input.
The partitioning method then divides the given data into K groups, where K is less than N, and each group forms a cluster.
In other words, a body of data is divided into one or more data objects.
K-means clustering, divided into 10
.
.
#machine-learning #unsupervised-learning
Loading the geometries I have
three.
js features
Three.
js includes the following features:
Lights: ambient, direction, point, and spot lights; shadows: cast and receive
Export and import: utilities to create Three.
js-compatible JSON files from within: Blender, openCTM, FBX, Max, and OBJ
OBJLoader
The OBJ file format is a simple data-format that represents 3D geometry alone — namely,
the position of each vertex, the UV position of each texture coordinate vertex,
vertex normals, and the faces that make each polygon defined as a list of vertices, and texture vertices.
.
.
#WebGL #ifc.js #javascript #three.js
ghpythonutils
GhPython Utilities 🔧
ghpythonutils serves as a utility library for GhPython users who develop components, algorithms, automation tools, and related functionality.
For reference, this project is always a work in progress.
Any ideas, questions, or suggestions are welcome and may be submitted as new issues.
.
.
#algorithm #geometry #python
three.js geoms
About the three.
js
Three.
js is a cross-browser JavaScript library
and application programming interface used to create and display animated 3D computer graphics in a web browser using WebGL.
The source code is hosted in a repository on GitHub.
Three.
js allows the creation of graphical processing unit (GPU)-accelerated 3D animations using the JavaScript language as part of a website without relying on proprietary browser plugins.
This is possible due to the advent of WebGL, a low-level graphics API created specifically for the web.
.
.
#WebGL #javascript #three.js
Windows placement helper
Boring, boring, boring 😑
Architects often download open CAD sources and use them to draw plans.
In doing so, they repeatedly perform the task of moving design elements such as windows and doors to the desired location and drilling holes in the wall at that location.
.
.
#automation #drawing #ezdxf #shapely
Voxelate
What is a Voxel ? 🧊
In computer graphics, a voxel, also known as a 3D pixel, is a 3D cube located on a three-dimensional grid and used to construct 3D models.
Similar to a pixel in a 2D image, a voxel occupies a specific location inside a 3D grid and has a color value assigned to it.
All individual voxels are the same size; however, voxels can be combined within the same grid at different positions and with different colors to create what is known as a voxel model.
What is a voxel ? From the left, Original shape · Voxelated shape
In addition, a voxel is used as the smallest unit composing an architectural space, because voxels have no deadspace and can be applied to architectural purposes in a reasonable and consistent manner.
For this reason, voxels are used in many architectural offices.
The architectural process below illustrates how voxelization is applied to a building.
This article covers the automation of voxelization, how a given shape is transformed into a voxelated shape, and how information can be embedded within each voxel.
.
.
#formfollowsfunction #information #visualization
Plot shape estimator
How many shapes does the plot have? 🏷️
The shapes of plots in Seoul are highly diverse.
They include flags, squares, long-squares, trapezoids, triangles and other forms that are difficult to define, and a building exists within each plot.
Since architects design within a given plot, the direction of the design may change depending on the shape of the site.
An architect empirically judges the shape of a plot and develops the design accordingly.
For an excessively long-square shape, the architect scales the building down along the long-side axis.
For a triangular plot, the architect designs the building to eliminate acute angles.
In this way, a person can subjectively judge the type and define the shape.
.
.
#machine-learning #python
Space syntax
Depth of space
The depth of a space can serve as a criterion for evaluating the accessibility of that space.
Consider the image below.
It shows two spaces whose layouts are identical.
However, the accessibility of a space changes depending on how the spaces are connected, that is, on the depth.
.
.
#informatin #visualization
Mass generator
Automating building generation with GeoJSON
1.
After reading the shapefile in QGIS, the necessary parts are extracted and converted to the GeoJSON file format.
Getting GeoJSON
2.
The GeoJSON file is read and the geometry is parsed.
The GeoJSON file contains information about the coordinates of the geometry, as shown below.
.
.
#automation #information #visualization
Traveling salesman problem
What is the Genetic Algorithm 🧬
Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems, drawing inspiration from biology through operations such as mutation, crossover, and selection.
A genetic algorithm is a metaheuristic inspired by the biological process of natural selection and belongs to the larger class of evolutionary algorithms.
This algorithm has the following characteristics:
.
.
#genetic-algorithm #optimization
Secure room in seoul
Recommend a room with a high security score
This project maps rooms with high security scores by overlapping the number of crimes with location data from 2014 to 2019.
The open API of the "Getting a good room with Peter Pan" property sales site is used to scrape room information data in Seoul.
The scraped data is stored as csv, inserted into Elasticsearch using Logstash and Beats from the Elastic Stack, and visualized with Kibana.
.
.
#information #visualization #web-scrapping