Skip Connection

03/21/2024

• Skip Connection
• Concepts
• Skip Connection is to bypass a particular layer and combine its input directly to the output of the next few layers
• Prevent loss of low level information and vanishing gradient
• Block with skip-connection is called Residual (잔차) Block
• Methods to combine
• Summation
• ResNet
• Transformer

Fig.1: Summation method


• Concatenation
• UNet
• Denoising Diffusion

Fig.2: Concatenation method


• Simple implementation with MLP