Projects, PoCs, Prototypes, Studies, Tests, ...
Image to 3D Scene Pipeline
Introduction
This project aims to develop an interior scene generation system, funded by KOCCA, that converts single interior images into structured 3D scenes.
By combining state-of-the-art models for 2D object detection, automatic mask and semantic segmentation,
and single-view image-to-3D reconstruction, the pipeline automatically identifies
and segments all interior elements from a single input photo,
reconstructs each detected object as an aligned, textured 3D mesh, and assembles them coherently into a recreated scene.
Applications range from virtual walkthroughs and editing to AR/VR staging and interior design automation.
SAM 3D
SAM 3D Objects represents a new approach
to tackling robust, visually grounded 3D reconstruction and object pose estimation
from a single natural image, reconstructing detailed 3D shapes, textures, and layouts of objects from everyday images.
.
.
#generative-design #sam #sam-3d-objects
Wave Function Collapse
What is Wave Function Collapse?
Wave function collapse is a quantum mechanics concept
where a system in a superposition of states
instantly reduces to a single, definite state upon measurement or observation.
In computer science, the Wave Function Collapse (WFC) algorithm is a technique for generating procedural,
constraint-based content, such as tilemaps or textures, similar to solving a 3D Sudoku puzzle.
Each cell in a grid is initially treated as a superposition of all possible states.
Rather than sampling all cells simultaneously, WFC proceeds by selecting a single cell,
collapsing its state space to one concrete value, and then propagating the resulting constraints to neighboring cells.
Sudoku puzzle can easily explain how WFC works
The above figure shows intuition of how the WFC algorithm works.
.
.
#generative-design #procedural generation
Latent Shapes
Objectives
In the best of my knowledge, latent vectors are initialized from a normal distribution and then updated during training to minimize the loss.
Although linear interpolation can be used in the manipulations between latent vectors,
and similar vectors can be placed in spatially close locations, the latent vectors themselves do not have a specific shape.
This project takes inspiration from this perspective and explores two main conceptions:
1) Latent vectors with geometric shape ─ exploring what happens if we give the latent vectors a geometric structure, and how that might help us better understand or control the results;
2) Interactive manipulation using the latent vector ─ making it possible for people to easily change and play with the latent vectors, so they can see and understand the effects right away and create new shapes or designs interactively.
Latent Vector Manipulation in real-time with Mouse Dragging
Data Preprocessing
This project is based on the chair dataset (03001627) from ShapeNetCore.
You can access the original data here.
.
.
#deep-sdf #generative-design #interpolation #latent-vector
Office Layout Generation Agents
PlanNext
PlanNext is a cloud-based API service that uses Large Language Models to design optimized office layouts from natural language input.
By interpreting your text-based requirements, it automatically generates layouts divided into essential work zones: Focus, Collaboration, Social, Functional, Service, and Private.
Visualized results from PlanNext API
This flow chart shows the complete pipeline of an LLM-based office layout generation system.
When a user requests layout requirements in natural language, OrchestrationAgent analyzes this and selects the necessary specialized agents.
.
.
#agent #generative-design #llm
Voxels to Graph
Introduction 🔗
This project explores the key contributions of Building-GAN by AutodeskAILab:
1) a novel 3D representation called voxel graph that can encode irregular voxel grids with non-uniform space partitioning, overcoming limitations of traditional regular voxel grids, and
2) a graph-conditioned generative adversarial network (GAN) leveraging graph neural networks (GNNs).
The implementation uses their dataset to study these approaches.
First, let's examine the structure of the raw data in the next section.
Generated Buildings
Raw Data
In the Data Collection part of the paper, it says that the dataset was created as follows:
Since there is no publicly available dataset for volumetric designs from real buildings, we create a synthetic
dataset with 120,000 volumetric designs for commercial buildings using parametric models.
The heuristics behind the parametric models are based on the rules and knowledge provided by professional architects.
.
.
#building-gan #gnn #graph #voxel
Landbook Diffusion Pipeline
Landbook 🏠
Landbook is a service that supports all the steps of new construction development for small and medium-sized land investors.
Landbook's AI architect service provides building owners with various architectural design proposals
by considering different plot sizes, zoning areas, and building regulations for each region.
Landbook's AI architect service
This project is to develop a pipeline that renders the final result of the Landbook AI architect service by using a generative image model such as diffusers.
By taking 3D modeling data as input and generating realistic images that closely resemble actual buildings,
it allows building owners to visualize and review what their design proposals would look like when actually constructed.
Unlike conventional 3D rendering, this aims to provide high-quality visualization that considers both the texture of actual buildings
and their harmony with the surrounding environment by utilizing AI-based generative models.
.
.
#diffusers #huggingface #landbook
Floor Plan Generation with Voronoi Diagram
Introduction 🔹
This project is to review
and implement the paper for Free-form Floor Plan Design using Differentiable Voronoi Diagram.
In Deep learning or any gradient-based optimization approach, it uses only tensors to compute gradients, but I think it is not intuitive in geometries.
Therefore, I aim to integrate the tensor operations and the geometric operations using Pytorch, Shapely.
The biggest difference between the paper and this project is whether using autograd.
In the paper, they used the Differentiable Voronoi Diagram for chaining gradient flow,
but, I used Numerical Differentiation to approximate the gradients directly.
Floor plan generation with voronoi diagram
So, what is the numerical differentiation?
Numerical Differentiation
Numerical differentiation
is a method used to approximate a derivative using finite perturbation differences.
.
.
#numerical-differentiation #voronoi-diagram
Polygon segmentations
Objective ✂️
In the early stages of architectural design, there is a concept about the axis in which direction the building will be placed.
This plays an important role in determining the optimal layout considering the functionality, aesthetics, and environmental conditions of the building.
The goal of this project is to develop a segmenter that can determine how many axes a given 2D polygon should be segmented into and how to make those segmentations.
Segmented lands by a human architect
The figures above are imaginary dividing lines arbitrarily set by the architect.
The apartments in the figures are placed based on segmented polygons.
Human architects intuitively know how many axes a given 2D polygon should be segmented into and how to create these segmentations, but explaining this intuition to a computer is difficult.
To achieve this, I will use a combination of deep learning and graph theory.
In the graph, each point of the polygon will be a node and the connections between points will be edges.
Based on this concept, I will implement a GNN-based model, which will learn how to optimally segment given polygons.
.
.
#deep-learning #geometry #graph-neural-network #link-prediction
Landbook Rendering with Diffuser
Objective
The purpose of this project is to develop a PoC
by integrating the Landbook architectural engine with Hugging Face's Diffusers
for generating architectural renderings.
This PoC aims to:
1) Evaluate the feasibility of using Diffusers for rendering architectural outputs generated by the Landbook engine.
2) Experiment with prompt engineering to create workflows for producing consistent visualizations.
3) Explore the variability in results based on seed adjustments, negative prompts, and prompt refinements.
This PoC explores the foundation for advancing AI-powered rendering solutions in architectural design.
Tests
Landbook architecture engine result rendering test with Diffusers of Hugging Face
Rendering test for the building created by Landbook achitecture engine
Explore prompt to fix for applying it to rendering module
Diffuser returns different styles by seed even if uses the same prompt
Image to test:
Cherry-picked results
simplicity prompt produces results with simpler, solid-color background.
The keyword for brick facade was used, but the diversity of colors was reduced by keyword for simplicity
This way may lead to multi-layer-based rendering as Photoshop
.
.
#diffusers #huggingface
Infinite synthesis
Introduction 🏢
Deep Generative AI, a field of artificial intelligence that focuses on generating new data similar to training data, is having an impact not only in text generation and image generation but also 3d model generation in the design industry.
In the realm of architectural design, especially in the phase of initial design, generative AI can serve as a useful tool for examining many design options.
Physical model examination by humans
Shou Sugi Ban / BYTR Architects
Leveraging the Deep Signed Distance Functions model (DeepSDF) with latent vectors, this project aims to build the algorithm that can synthesize infinite number of skyscrapers simliar to trained data.
These vectors, mapped within a high-dimensional latent space, serve as the DNA for synthesizing potential skyscrapers.
So, through the system that manipulates the latent vectors for expressing the shape of the buildings, architectural designers can rapidly generate and examine a diverse array of design options.
By manipulating(interpolation, and arithmetic operations) between two or more latent vectors, the model gives us infinite design options virtually.
This method not only provides a brand-new design process but also will lead us to explore the novel architectural forms previously unattainable through conventional design methodologies.
.
.
#deep-learning #deep-sdf #generative-design #geometry
Seeing is solving
Difficulty of the geometric imagination
"I have no geometric imagination.
.
.
"
I often feel this way, especially when facing complex geometry problems or writing code that deals with shapes and spaces.
At least for me, it is not easy to picture complex 3D shapes in my mind.
This is a big challenge and why visualization is so useful.
I think visualizing geometry is key to solving problems.
Seeing what I am doing makes a huge difference.
General debugging tools like print don't help much here.
They give me numbers and text but don't show me the geometry I need to see.
.
.
#debugger #geometry #visualization
Shape-conditional GANs
What is Conditional Generative Adversarial Networks ❓
Conditional Generative Adversarial Networks (cGANs) is an extension of the original Generative Adversarial Networks.
cGANs take the concept of conventional GANs further by feeding additional information into both the generator and discriminator, allowing the generation of data that is more specific and controlled.
This information acts as a directive or constraint for the generator on what type of data to produce.
The paper for Conditional Generative Adversarial Nets says:
" Generative adversarial nets can be extended to a conditional model if both the generator and discriminator are conditioned on some extra information y.
y could be any kind of auxiliary information, such as class labels or data from other modalities.
We can perform the conditioning by feeding y into the both the discriminator and generator as additional input layer.
"
.
.
#conditional-generative-adversairal-networks #deep-learning #geometry
Local coordinate system
Types of coordinate systems: Global vs.
Local
In the field of engineering, computer graphics, and physics, understanding the relationship between different coordinate systems is crucial for accurate modeling, simulation, and analysis.
The figure attached describes the concept of a global coordinate system (GCS) and a local coordinate system (LCS),
both of which are commonly used to define the position and orientation of objects in 3D space.
The GCS is represented by the axes x , y , and z .
This system provides a fixed reference for describing the location of all objects within the space.
In contrast, the LCS, denoted by the axes x' , y' , and z' , is tied to a specific object within the GCS.
It moves and rotates with the object, providing a reference frame that is local to the object itself.
.
.
#algorithm #coordinate-system #geometry
Mass GANs
Objectives
Generative Adversarial Networks (GANs) have paved the way for unprecedented advancements in numerous areas, from art creation to deepfake video generation.
However, the potential of GANs isn't restricted to 2D space.
The development and application of 3D GANs have opened new possibilities, especially in the realm of design.
This project delves deep into the possibilities of 3D GANs in the design field with the following objectives:
Grasp the fundamental concepts behind GANs and their 3D extension
Appreciate the power and nuances of 3D GANs through hands-on experiments
Examine how 3D GANs can be harnessed for product design, architectural modeling, and virtual environment creation
visualize and manipulate the latent space to generate novel and innovative designs
Understand the limitations of current 3D GAN models and the potential areas of improvement
.
.
#deep-learning #gan #generative-adversarial-networks #machine-learning
The selfish gene
What is natural selection 🦕
Natural selection is a mechanism of evolution.
Organisms that are more adapted to their environment are more likely to survive and pass on the genes that aided their success.
This process causes species to change and diverge over time.
Natural selection is one of the ways to account for the millions of species that have lived on Earth.
For example, evolving long necks have enabled giraffes to feed on leaves that others can't reach, giving them a competitive advantage.
Thanks to a better food source, those with longer necks were able to survive.
Those with shorter necks and access to less food would be less likely to survive.
.
.
#evolution #natural-selection #simulation
Parallel execution within GhPython
A simple understanding of multiprocessing
Multiprocessing refers to the ability of a computer system to execute multiple processes or tasks concurrently.
It involves the use of multiple processors or processor cores to perform multiple tasks simultaneously, thereby improving overall system performance and efficiency.
In a multiprocessing system, each process is an independent unit of execution with its own program counter, stack, and data section.
These processes can run concurrently, allowing for parallelism and efficient utilization of system resources.
Processes: A process is an instance of a running program.
Each process has its own memory space, file descriptors, and system resources.
Processes are created and managed by the operating system.
They can communicate with each other using inter-process communication mechanisms.
You can understand multiprocessing with a very simple example easily.
That is the following:
.
.
#algorithm #multiprocessing #python
Drafting with AI
Stable diffusion 🤖
Stable Diffusion is a deep learning, text-to-image model released in 2022.
It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt.
An image generated by Stable Diffusion based on the text prompt: "a photograph of an astronaut riding a horse"
If you use this AI to create images, you can get plausible rendered images easily and shortly with a few lines of text.
Let's think about how this can be utilized in the field of architecture.
.
.
#controlnet #drawing #rendering #stable-diffusion
Analysis of Apt state-spaces
Objectives
This analysis aims to write a prototype for deriving the apartment layout starting point for the street housing maintenance project (가로주택정비사업) solution called LBDeveloper (at Spacewalk).
The plots to analyze and layout analysis visualized are the following images.
Also it aims to set the parameters and the range of parameters to optimize apartment complex layout.
Plots to analyze and visualized layout analysis
Axes rules
Since the orientation of street houses takes into account the south-facing of all units, the orientation and placement axis can be separated.
Therefore, the array axes is largely divided into the following:
Longest segment axis of the plot geometry
Boundary axis
South-facing axis
The rules of array axes by layout analysis:
One array axis has a minimum of 1 and a maximum of 3 origins
The array axes have a minimum of 1 and a maximum of 2 axes when not Boundary axis
If the selected array axes parameter is the Boundary axis, offset N line segments into the site
Place a corner-type building at the intersection of lines that offset the site boundary axis
.
.
#algorithm #automation #lbdeveloper #visualization
Plan to Model
Edge detection
Edge means a boundary line or an outline.
An edge in an image refers to a portion existing at a point where the brightness of an image changes from a low value to a high value or vice versa.
Ultimately, an edge indicates the boundary of an object in an image and contains various information such as shape and direction detection.
Edge detection is the process of finding a pixel corresponding to an edge.
And it includes a variety of mathematical methods that aim at identifying edges, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities.
Lenna
Lenna (or Lena) is a standard test image used in the field of digital image processing
.
.
#algorithm #automation #open-cv
Klemens Torggler's door
Evolution door
Evolution Door is a 2012 development and Klemens Torggler's best known work.
Unlike previous kinetic works, this object is only on one level and unfolds into a third dimension as it moves.
Klemens Torggler's The Evolution Door
The Evolution Door is connected to the door frame with two pivots — at the top and bottom — and consists of four triangular “flip-panels.
” When movement is instigated, the four sections collapse in on themselves and rotate 90 degrees before straightening back up into a rectangle.
Door opening process
Implementation
The door consists of two square sections, each divided in half along the diagonal.
The diagonals are where two triangular panels meet, connected with something like a piano hinge so that each square can fold as the door moves.
And each part of the rectangle of the door must be square.
In other words, the door's width must be half of the door's height.
.
.
#algorithm #formfollowsfunction #python
K-Rooms clusters
K-Means clustering 🕃
The K-Means clustering algorithm is the algorithm
that clusters to a K number when given data.
And it operates as a way to minimize
between each cluster about distance difference.
This algorithm is a type of the Unsupervised-Learning and serves to label unlabeled input data.
The K-Means algorithm belongs to a partitioning method among clustring method.
A partitioning method is a way of splitting that divides multiple partitions when given data.
For example, let's assume that n data objects are input.
That's when partitioning method divides the given data into K groups less than N, at this time, each group forms a cluster.
That is, dividing a piece of data into one or more data objects.
K-means clustering, divided by 10
.
.
#machine-learning #unsupervised-learning
Loading the geometries I have
three.
js features
Three.
js includes the following features:
Lights: ambient, direction, point, and spot lights; shadows: cast and receive
Export and import: utilities to create Three.
js-compatible JSON files from within: Blender, openCTM, FBX, Max, and OBJ
OBJLoader
The OBJ file format is a simple data-format that represents 3D geometry alone — namely,
the position of each vertex, the UV position of each texture coordinate vertex,
vertex normals, and the faces that make each polygon defined as a list of vertices, and texture vertices.
.
.
#WebGL #ifc.js #javascript #three.js
ghpythonutils
GhPython Utilities 🔧
ghpythonutils serves as a utility for GhPython users engaged in the development of components, algorithms, automation tools, and other functionalities.
For reference, this project is always a work in progress.
Therefore, if you have any ideas, questions, or suggestions, please feel free to create new issues.
.
.
#algorithm #geometry #python
three.js geoms
About the three.
js
Three.
js is a cross-browser JavaScript library
and application programming interface used to create and display animated 3D computer graphics in a web browser using WebGL.
The source code is hosted in a repository on GitHub.
Three.
js allows the creation of graphical processing unit (GPU)-accelerated 3D animations using the JavaScript language as part of a website without relying on proprietary browser plugins.
This is possible due to the advent of WebGL, a low-level graphics API created specifically for the web.
.
.
#WebGL #javascript #three.js
Windows placement helper
Boring, boring, boring 😑
If you are an architect, you may have downloaded open cad sources and used them to draw plans.
And you may also have experienced the repetitive task of moving design elements such as windows and doors to the desired location and drilling holes in the wall at that location.
.
.
#automation #drawing #ezdxf #shapely
Voxelate
What is a Voxel ? 🧊
In computer graphics, a voxel, also known as a 3D pixel, is a 3D cube located on a three-dimensional grid used to create 3D models.
Similar to a pixel in a 2D image, a voxel can contain a specific location inside a 3D grid and has a color value assigned to it.
All individual voxels are the same size, however voxels can then be combined in the same grid at different positions with different colors to create what is known as a voxel model.
What is a voxel ? From the left, Original shape · Voxelated shape
In addition, a voxel is also used as the smallest unit composing an architectural space.
This is because voxels have no deadspace and can be used for all architectural purposes and reasonable.
Therefore It's being used in many architectural offices.
The process of architecture below shows how to apply voxelization to the building.
This article covers automation of voxelization and how change to the voxelated shape when given specific shape.
And let's put some information in the voxel.
.
.
#formfollowsfunction #information #visualization
Plot shape estimator
How many shapes does the plot have ? 🏷️
The shape of the plot in Seoul is diverse.
There are flags, squares, long-squares, trapezoids, triangles and other shapes that are hard to define.
building will exists within the plot.
Since architects design within a given plot, the direction of design may change depending on the shape of the site.
The architect empirically judges the shape of plot and develops the design.
In the case of too long-square shapes, the architect will scale the building down in the direction of the long side axis.
If given a triangle shapes of land, architects design it to eliminate acute angles.
In this way, people can subjectively judge the type and define the shape.
.
.
#machine-learning #python
Space syntax
Depth of space
The depth of spcae can be a criterion for evaluating the accessibility of the space.
Let's take a look at the image below.
There are two spaces, and the way they are all laid out is the same.
However, you can see that the accessibility to the space changes depending on the way the spaces are connected, that is, the depth.
.
.
#informatin #visualization
Mass generator
Automating buildings generation with GeoJSON
1.
after reading the shapefile from QGIS, cut out the necessary parts and convert it to the GeoJSON file format.
Getting GeoJSON
2.
Read the GeoJSON file and parse the geometry.
The GeoJSON file contains information about the coordinates of the geometry like the one below.
.
.
#automation #information #visualization
Travelling salesman problem
What is the Genetic Algorithm 🧬
Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems relying on inspiration from biology such as mutation, crossover, and selection.
It is a metaheuristic inspired by the process of natural selection of biology that belongs to the larger class of evolutionary algorithms.
And this algorithm has characteristics following below:
.
.
#genetic-algorithm #optimization
Secure room in seoul
Recommend a room with a high security score
This project maps rooms with high security scores by overlapping the number of crimes and locations data from 2014 to 2019.
I used the open api on the "Getting a good room with Peter Pan" property sales site to scrap room information data in Seoul.
The scrapped data is stored in the form of csv, inserted into the Elastic Search using the Logstash and Beats of the Elastic Stack, and visualized by the Kibana
.
.
#information #visualization #web-scrapping