首页 > 人工智能 > 机器学习 > 基于PyTorch的深度残差收缩网络
深度残差收缩网络(Deep Residual Shrinkage Network,DRSN)是一种面向含噪一维信号的特征学习网络,通过在残差模块中嵌入软阈值化操作,在网络前向传播过程中自适应地抑制与噪声相关的冗余特征。该方法发表于《IEEE Transactions on Industrial Informatics》期刊,论文题为《Deep Residual Shrinkage Networks for Fault Diagnosis》。本文仅围绕该论文提出的网络结构及其PyTorch实现进行介绍。
深度残差收缩网络以经典残差网络(ResNet)为基础框架,将原有的残差基本单元替换为残差收缩基本单元(RSBU)。根据阈值设定方式的不同,论文提出了两种网络变体:通道共享阈值的深度残差收缩网络(DRSN-CS)和通道独立阈值的深度残差收缩网络(DRSN-CW),其整体架构与核心模块如图4所示。

图4 原论文中的RSBU-CS、DRSN-CS、RSBU-CW及DRSN-CW结构示意图
在整体结构上(见图4(b)与图4(d)),网络输入为一维振动信号,首先经过一个步长为2的卷积层进行初步特征提取和降采样,随后堆叠多层残差收缩基本单元以逐级提取判别性特征并抑制噪声,最后经由批归一化、激活函数、全局平均池化和全连接层输出分类结果。按照原论文实验部分的配置,网络共包含三个特征尺度阶段,每个阶段依次堆叠四个基本单元(其中第1个单元采用步长为2进行降采样,并在后两个阶段将输出通道数分别翻倍),通道数依次由4增加至8和16,最终通过全局平均池化和全连接层映射为8类输出。
在核心模块设计上(见图4(a)与图4(c)),残差收缩基本单元在主干路径中依次包含两组批归一化、线性整流函数及卷积层,并在第二个卷积层之后接入一个阈值估计子网络与软阈值化层,再与恒等映射分支(当通道数或特征宽度变化时采用步长为2的一维卷积对齐维度)相加。其中,阈值估计子网络先对特征图取绝对值并执行全局平均池化以压缩为一维向量,再送入包含批归一化和激活函数的两层全连接网络,并经Sigmoid函数映射至零到一之间得到缩放系数。对于通道共享单元(RSBU-CS),全连接网络最终仅输出单个缩放系数,并与所有通道绝对值的全局均值相乘,从而为整个特征图生成统一的阈值;对于通道独立单元(RSBU-CW),全连接网络输出与通道数相等的缩放系数,并与对应通道绝对值的均值相乘,从而为每个通道分配独立的阈值。最终,软阈值化层利用估计出的阈值将幅值较小的特征置零,同时保留幅值较大的正负有效特征。
下面给出PyTorch代码:
import torch
import torch.nn as nn
class RSBU(nn.Module):
"""残差收缩基本单元(支持通道共享CS与通道独立CW两种模式)"""
def __init__(self, in_channels: int, out_channels: int, stride: int = 1, channel_wise: bool = True):
super().__init__()
self.channel_wise = channel_wise
self.residual = nn.Sequential(
nn.BatchNorm1d(in_channels),
nn.ReLU(inplace=True),
nn.Conv1d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False),
nn.BatchNorm1d(out_channels),
nn.ReLU(inplace=True),
nn.Conv1d(out_channels, out_channels, kernel_size=3, stride=1, padding=1, bias=False),
)
fc_out_dim = out_channels if channel_wise else 1
self.threshold_net = nn.Sequential(
nn.Linear(out_channels, out_channels),
nn.BatchNorm1d(out_channels),
nn.ReLU(inplace=True),
nn.Linear(out_channels, fc_out_dim),
nn.Sigmoid(),
)
if stride != 1 or in_channels != out_channels:
self.shortcut = nn.Conv1d(in_channels, out_channels, kernel_size=1, stride=stride, bias=False)
else:
self.shortcut = nn.Identity()
def forward(self, x: torch.Tensor) -> torch.Tensor:
identity = self.shortcut(x)
out = self.residual(x)
out_abs = torch.abs(out)
gap = torch.mean(out_abs, dim=-1, keepdim=True)
scales = self.threshold_net(gap.view(gap.size(0), -1)).unsqueeze(-1)
if self.channel_wise:
threshold = scales * gap
else:
avg_abs = torch.mean(gap, dim=1, keepdim=True)
threshold = scales * avg_abs
# 软阈值化:将接近零的特征置零,保留大于阈值的正负特征
out = torch.sign(out) * torch.clamp(out_abs - threshold, min=0.0)
return out + identity
class DRSN(nn.Module):
"""深度残差收缩网络(依据原论文表II及图4构建)"""
def __init__(self, in_channels: int = 1, num_classes: int = 8, channel_wise: bool = True):
super().__init__()
self.stem = nn.Conv1d(in_channels, 4, kernel_size=3, stride=2, padding=1, bias=False)
self.stage1 = self._make_stage(4, 4, num_blocks=4, stride=2, channel_wise=channel_wise)
self.stage2 = self._make_stage(4, 8, num_blocks=4, stride=2, channel_wise=channel_wise)
self.stage3 = self._make_stage(8, 16, num_blocks=4, stride=2, channel_wise=channel_wise)
self.head = nn.Sequential(
nn.BatchNorm1d(16),
nn.ReLU(inplace=True),
nn.AdaptiveAvgPool1d(1),
nn.Flatten(),
nn.Linear(16, num_classes),
)
@staticmethod
def _make_stage(
in_channels: int,
out_channels: int,
num_blocks: int,
stride: int,
channel_wise: bool,
) -> nn.Sequential:
layers = [RSBU(in_channels, out_channels, stride=stride, channel_wise=channel_wise)]
for _ in range(1, num_blocks):
layers.append(RSBU(out_channels, out_channels, stride=1, channel_wise=channel_wise))
return nn.Sequential(*layers)
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.stem(x)
x = self.stage1(x)
x = self.stage2(x)
x = self.stage3(x)
return self.head(x)
if __name__ == "__main__":
sample_input = torch.randn(16, 1, 2048)
drsn_cs = DRSN(in_channels=1, num_classes=8, channel_wise=False)
drsn_cw = DRSN(in_channels=1, num_classes=8, channel_wise=True)
print("DRSN-CS 输出维度:", drsn_cs(sample_input).shape)
print("DRSN-CW 输出维度:", drsn_cw(sample_input).shape)
上述代码通过参数channel_wise切换DRSN-CS与DRSN-CW两种结构,由于内部全连接子网络使用了批归一化层,训练时需保证输入批量大小大于1。
Deep Residual Shrinkage Networks for Fault Diagnosis[J]. IEEE Transactions on Industrial Informatics, 2020.
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/31557240/viewspace-3189466/ ,如需转载,请注明出处,否则将追究法律责任。