Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack

Introduction

Deep Neural Networks (DNNs) have achieved great success in a variety of applications, including but not limited to image classification , speech recognition , machine translation , and autonomous driving . Despite the remarkable accuracy imrovement , recent studies have shown that DNNs are vulnerable to adversarial examples. In image classification task, an adversarial example is a natural image intentionally perturbed by visually imperceptible variation, but can cause drastic classification accuracy degradation. Fig. 1 provides an illustration of adversarial example and its original counterpart. In addition to image classification, attacks to other DNN-powered tasks have also been actively investigated, such as visual question answering , image captioning , semantic segmentation and etc .

There has been a cohort of works on generating adversarial attacks and developing corresponding defense methods. The adversarial attacks can be categorized as white-box attack and black-box attack based on the attacker’s knowledge to the target model. For white-box attack , the adversary has full access to the network architecture and parameters. Whereas, only the input and output to the network can be externally accessed by the black-box attacks . White-box attack can often achieve high success rates for various applications .

Recently, different works have viewed the problem of adversarial examples from an unified perspective of model robustness and regularization. Conventional regularization mainly serves the purpose of reducing the generation error, thus preventing model from overfiting the training set. Traditional regularization methods have been effective in neural network training. For example, dropout , Batch Normalization (BN) and quantization all serve the purpose of model regularization. However, BN is specifically effective in convolution networks and dropout is applicable for fully connected network. Hinton discusses that adding Gaussian noise into the model (input, weight and activation) during training performs as a regularizer in his lecture notehttps://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec9.pdf and dropout work .

It is evident that a more general model regularization method specifically directed for improving neural network robustness can serve the purpose of defending adversarial example more effectively. Recently, different works have implemented noise regularization method both during training and inference phases . In this work, we propose to improve neural network robustness against adversarial attack by regularizing the training process through adding noise during the training phase. We believe adversarial defense methodologies that focus on defending the network at the inference only will eventually fall short in the advent of new attack methods. Thus a more general regularized training method can generate robust DNN to defend against a wide range of attacks.

Overview of our approach: In this work, we propose a novel noise injection method called Parametric Noise Injection (PNI) to improve neural network robustness against adversarial attack. It has the flexibility to inject trainable noise at the input (to whole network), activation and weights during both training and inference. The proposed PNI is embedded with well-known adversarial training, where Gaussian Noise with trainable parameters could adjust injected noise level at each neural network layer. In this work, we conduct a wide range of white-box and black-box adversarial attack experiments to demonstrate the effectiveness of our proposed PNI method accross different popular DNN architectures. Our simulation shows accuracy improvement for both clean data accuracy and under attack accuracy. PNI achieves a 1.1 % improvement on the clean test data on Resnet-20 compared to Vanilla Resnet-20 with adversarial training. Along with the improvement on clean test data, our defense shows 6.8% improvement on the test accuracy under PGD wite box attack. Additionally, Our result shows improved robustness under FGSM, C & W attack and various black-box attack.

Related works

Recently, various powerful adversarial attack methods have been proposed to totally fool a trained deep neural network through introducing barely visible perturbation upon input data. Several state-of-the-art white-box (i.e., PGD , FGSM and C&W ) and black-box (i.e., Substitute and ZOO ) adversarial attack method are briefly introduced as follows.

Fast Gradient Sign Method (FGSM) is a single-step efficient adversarial attack method, which alters each element $x$ of nature sample $\bm{x}$ along the direction of its gradient w.r.t the loss function $\partial\mathcal{L}/\partial x$. The generation of adversarial example $\bm{\hat{x}}$ can be described as:

where the attack is followed by a clipping operation to ensure the $\hat{x}\in$. The attack strength is determined by the perturbation constraint $\epsilon$.

Projected Gradient Descent (PGD) is the multi-step variant of FGSM, which is one of the strongest $L^{\infty}$ adversarial example generation algorithm. With $\bm{\hat{x}}^{1}=\bm{x}$ as the initialization, the iterative update of perturbed data $\bm{\hat{x}}$ can be expressed as:

where $P_{\epsilon}(\bm{x})$ is the projection space which is bounded by $\bm{x}\pm\epsilon$, $t$ is the step index up to $N_{\textup{step}}$, and $a$ is the step size. Madry et al. proposed that PGD is a universal adversary among all the first-order adversaries (i.e., attacks only rely on first-order information).

In C & W attack method, Carlini and Wagner consider the generation of adversarial example as an optimization problem, which optimize the $L^{p}$-norm of distance metric $\delta$ w.r.t the given input data $\bm{x}$, which can be described as:

where $\bm{\delta}$ is taken as the perturbation added upon the input data, and a proper loss function $\hat{\mathcal{L}}$ is chosen in to to solve the optimization problem via gradient descent method. $c$ is a constant set by attacker. In this work, we use $L^{2}$-norm based C&W attack and take $||\bm{\delta}||_{p=2}$ as the evaluation metric to measure the network’s robustness, where a higher value of $||\bm{\delta}||_{p=2}$ indicates a more robust network or potential failure of the attack.

The most popular black-box attack is conducted using a substitute model , where the attacker trains a substitute model to mimic the functionality of target model, then use the adversarial example generated from the substitute model to attack target model. In this work, we specifically investigate the transferable adversarial attack , which is a variant of substitute model attack. In transferable adversarial attack, the adversarial example is generated from one source model to attack another target model. The source model and target can own the absolutely different structure but trained on the identical dataset. Moreover, Zero-th Order Optimization (ZOO) attack is also considered. Rather than training a substitute model, it directly approximates the gradient of target model just based on the input data and output scores using stochastic gradient coordinate.

2 Adversarial Defenses:

Improving network robustness by training the model with adversarial examples is the most popular defense approach now-a-days. Most of later works have followed this path to supplement their defense with adversarial training . The first step in adversarial training is to choose an attack model to generate adversarial examples. Adopting Projected Gradient Descent (PGD) based attack model to adversarial training is becoming popular since it can generate universal adversarial examples among the first order approaches . Additionally, among many recent defense methods, only PGD based adversarial training can sustain state-of-the-art accuracy under attacks . The reported DNN accuracy in CIFAR10 dataset remains a major success to defend very strong adversarial attacks .

Recent works have merged the concept of improving model robustness through regularization to defend adversarial examples. Among them, an unified perspective of regularization and robustness was presented by . Again, randomly pruning some activation during the inference or randomizing the input layer serve the purpose of injecting randomness to somehow prevent the attacker from accessing the gradient. However, these approaches achieve good success against gradient based attacks at the cost of obfuscated gradient .

In order to make the model more robust to adversarial attack, several works have adopted the concept of adding a noise layer just before convolution layer during both training and inference phases . Even though we agree with the core idea of these works as they certainly makes the model more robust, but there are some fundamental advantages of our work compared to theirs. PNI improves the model robustness by regularizing the model while training more effectively. As classical machine learning demonstrated weight noise performs the regularization even better . We also show experimentally that particularly adding noise to the weights improves the robustness even more. While these works have chosen level of noise to be injected manually, we propose to inject different level of noise at different layers using trainable parameters. As choosing the level of noise manually for different layers even by validation set is not practically feasible.

Approach

In this section, we first introduce the proposed Parametric Noise Injection (PNI) function and will investigate the impact of noise injection on input (to the whole DNN), weight and activation.

The method that we propose to inject noise to different components or locations within DNN can be described as:

where $v_{i}$ is the element of noise-free tensor $\bm{v}$, and such $\bm{v}$ can be input/weight/inter-layer tensor in this work. $\eta$ is the additive noise term which follows the Gaussian distribution with zero mean and standard deviation $\sigma$, and $\alpha_{i}$ is the coefficient scales the magnitude of injected noise $\eta$. We adopt the scheme that $\eta$ shares the identical standard deviation of $\bm{v}$ as in Eq. 6, thus the injected additive noise is correlated to the distribution of $\bm{v}$ and $\alpha$ simultaneously. Moreover, rather than manually configuring $\alpha_{i}$ to restrict the noise level, we set $\alpha_{i}$ as learnable parameter which can be optimized for network robustness improvement. We name such method as Parametric Noise Injection (PNI). Considering the over-parameterization and the convergence of training $\alpha_{i}$, we make the element-wise noise term ($\alpha_{i}\cdot\eta$) shares the same scaling coefficient across the entire tensor. Assuming we performs the proposed PNI on the weight tensors of convolution/fully-connected layers throughout entire DNN, for each parametric layer there is only one layer-wise noise scaling coefficient to be optimized. We takes such layer-wise configuration as default in this work.

In this work, we treat the noise scaling coefficient as a model parameter which can be optimized through back-propagation training process. For $f_{\textup{PNI}}(\cdot)$ configuration which shares the noise scaling coefficient layer-wise, the gradient computation can be described as:

where the $\sum_{i}$ takes the summation over the entire tensor $\bm{v}$, and $\partial\mathcal{L}/\partial f_{\textup{PNI}}(v_{i})$ is the gradient back-propagated from the followed layers. The gradient calculation of the PNI function is:

It is noteworthy that even though $\eta$ is a Gaussian random variable, each sample of $\eta$ is taken as a constant during the back-propagation. Using the gradient descent optimizer with momentum, the optimization of $\alpha$ at step $j$ can be written as:

where $m$ is the momentum, $\epsilon$ is the learning rate, and $V$ is the updating velocity. Moreover, since weight decay tends to make the learned noise scaling coefficient converge to zero, there is no weight decay term on the $\alpha$ during the parameter updating in this work. We set $\alpha=0.25$ as default initialization.

We expect to utilize the aforementioned PNI technique to improve the network robustness. However, directly optimizing the noise scaling coefficient normally leads $\alpha$ to converge at a small close-to-zero value, owing to the model optimization tends to over-fit the training dataset (referring to Table 1).

In order to succeed in adversarial defense, we jointly use the PNI method with robust optimization (a.k.a. Adversarial Training) which can boost the inference accuracy for the perturbed data under attack. Given inputs- $\bm{x}$ and target labels- $t$, the adversarial training is to obtain the optimal solution of network parameter $\bm{\theta}$ for the following min-max problem:

where the inner maximization tends to acquire the perturbed data $\hat{\bm{x}}$, and $P_{\epsilon}(\bm{x})$ is the input data perturb set constrained by $\epsilon$. While the outer minimization is optimized through gradient descent method as regular network training. $L^{\infty}$ PGD attack is adopted as the default inner maximization solver (i.e., generating $\hat{\bm{x}}$). Note that, in order to prevent the label leaking during adversarial training, the perturbed data $\hat{\bm{x}}$ is generated through taking the predicted result of $\bm{x}$ as the label (i.e. $t$ in Eq. 2).

Moreover, in order to balance the clean data accuracy and perturbed data accuracy for practical application, rather than performing the outer minimization solely on the loss of perturbed data as in Eq. 10, we minimize the ensemble loss $\mathcal{L}^{\prime}$ which is the weighted sum of losses for clean- and perturbed-data. The ensemble loss is described as:

where $w_{c}$ and $w_{a}$ are the weights for clean data loss and adversarial data loss. $w_{c}=w_{a}=0.5$ is the default configuration in this work. Optimizing the ensemble loss $\mathcal{L}^{\prime}$ with gradient decent method leads to successful training of $f_{\textup{PNI}}(\bm{\theta})$ for both the model’s inherent parameter (e.g. weight, bias) and the add-on noise scaling coefficient $\alpha$ from PNI.

Experiments

The CIFAR-10 dataset is composed of 50K training samples and 10K test samples of 32$\times$32 color image. For CIFAR-10, the classical Residual Networks (ResNet-20/32/44/56) architecture are used, and ResNet-20 is taken as the baseline for most of the comparative experiments and ablation studies. A redundant network ResNet-18 is also used to report the performance for CIFAR-10, since large network capacity is helpful for adversarial defense. Moreover, rather than including the input normalization within the data augmentation, we place a non-trainable data normalization layer in front of the DNN to perform the identical function, thus attacker can directly add the perturbation on the nature image. Note that, since both PNI and PGD attack include randomness, we report the accuracy in the format of mean$\pm$std% with 5 trials to alleviate error.

To evaluate the performance of our proposed PNI technique, we employ multiple powerful white-box and black-box attacks as introduced in Section 2.1. For PGD attack on MNIST and CIFAR-10, $\epsilon$ is set to 0.3/1 and 8/255, and $N_{\textup{step}}$ is set to 40 and 7 respectively. FGSM attack adopt the same $\epsilon$ setup as PGD. The attack configurations of PGD and FGSM are identical as the setup in . For C&W attack, we set the constant $c$ as 0.01. ADAM is used to optimize the Eq. 4 with learning rate as $5e^{-4}$. We choose 0 for the confidence coefficient $k$, which is defined in $\hat{\mathcal{L}}$ used by C&W $L^{2}$ attack in . The binary search steps for the attack is 9, while number of iteration to perform the gradient descent is 10. Moreover, We also conduct the PNI defense against several state-of-the-art black-box attacks (i.e. substitute , ZOO and transferable attack) in a Section 4.2.2 to examine the robustness improvement resulted from the proposed PNI technique.

As far as we know, the adversarial training with PGD is the only unbroken defense method , which is labeled as vanilla adversarial training and taken as the baseline in this work. Beyond that, several recent works also utilize similar concept as ours in their defense method are discussed as well, including certified robustness and random self-ensemble .

2 PNI for adversarial attacks

As the aforementioned discussion in Section 3.1, the noise scaling coefficient will not be properly trained without utilizing the adversarial training (i.e., solving the min-max problem). We conduct the experiments for training the layer-wise PNI on weight (PNI-W) of ResNet-20, to compare the convergence of trained noise. As tabulated in Table 1, simply performing the vanilla training using momentum SGD optimizer totally fails the adversarial defense, where the noise scaling coefficients $\alpha$ are converged to the negligible values. On the contrary, with the aid of adversary training (i.e., optimization of Eq. 11), convolution layers in the network’s front-end has obtained relatively large $\alpha$ which are the bold values in Table 1, and the corresponding evolution curve are shown in Fig. 2.

Since the PGD attack is taken as the inner maximization solver, the generation of adversarial example $\hat{\bm{x}}$ in Eq. 2 is reformatted as:

where the difference between Eq. 2 and Eq. 12 is with/without PNI within $\hat{\bm{x}}$ generation. It is noteworthy that, keeping the noise term in the model for both adversarial example generation (Eq. 12) and model parameter update is the critical factor for the PNI optimization with adversarial training, since the optimization of $\alpha$ is also a min-max game. Increasing noise level enhances the defense strength, but hampers the network inference accuracy for natural clean image. Lowering $\alpha$, however, makes the network vulnerable to adversarial attack. As listed in Table 1, without PNI-W in $\hat{\bm{x}}$ generation indeed leads to the failure of PNI optimization, and the large value ($\alpha=5.856$ in Table 1) is not converged due to the probable gradient explosion.

In this work, even though the scheme of injecting noise on the weight (PNI-W) is taken as the default PNI setup, more results about PNI on activation (PNI-A-a/b), input (PNI-I) and hybrid-mode (e.g. PNI-W+A) are provided in Table 2 for a comprehensive study. PNI-A-a/PNI-A-b denotes injecting noise on the output/input tensor of the convolution/fully-connected layer respectively. Moreover, PNI-A-b scheme intrinsically includes the PNI-I, since PNI-I is applying the noise on the input tensor of first layer. Note that, all models with PNI variants are jointly trained with PGD-based adversarial training as discussed above. Then, with the same trained model, we report the accuracy with/without the trained noise term (left/right in Table 2) during the test phase. As shown in Table 2, with the noise term enabled during test phase, PNI-W on ResNet-20 gives the best performance to defend PGD and FGSM attack, in comparison to PNI on other locations. Although it is elusive to fully understand the mechanism that PNI-W outperforms other counterparts, the intuition is that PNI-W is the generalization of PNI-A in each connection instead of each output unit, similar as relation between the regularization technique DropConnect and Dropout .

Furthermore, we also observe that disabling PNI during test phase leads to significant accuracy drop for defending PGD and FGSM attack, while the clean-data accuracy maintains the same level as PNI enabled. Such observation raises two concerns about our PNI techniques: 1) Does the improvement of clean-/perturbed-data accuracy with PNI mainly comes from the attack strength reduction caused by the randomness (potential gradient obfuscation )? 2) Is PNI just an negligible trick or it performs the model regularization to construct a more robust model? Our answers to both questions are negative, where the explanations are elaborated under Section 5.

In order to investigate the relation between network capacity (i.e., number of trainable parameters) and robustness improvement by PNI, we examine various network architectures in terms of both depth and width. For different network depths, experiments on ResNet 20/32/44/56 are conducted under vanilla adversarial training and our proposed PNI robust optimization method. For different network widths, we adopt the original ResNet-20 as baseline and expand its input&output channel of each layer by 1.5$\times$/2$\times$/4$\times$ respectively. Same as Table 2, we report clean- and perturbed-data accuracy with/without PNI term during the test phase. The results in Table 3 indicates that increasing the model’s capacity indeed improves network robustness against white-box adversarial attacks, and our proposed PNI outperforms vanilla adversary training in terms of both clean-data accuracy and perturbed data accuracy for PGD and FGSM attack. Such observation demonstrates that the perturbed-data accuracy improvement does not come from trading off clean-data accuracy as reported in . Through increasing the network capacity, the drop perturbed-data accuracy, when disabling the PNI noise term during test phase, also becomes less significant. Although both adversarial training and PNI techniques perform regularization, the network structure still needs careful construction to prevent the over-fitting resulted from over-parameterization.

Improved robustness does not necessarily mean improving the test data accuracy against any particular attack method. Typically $L_{2}$ norm based C & W attack should reach 100 % success rate against any defense. Thus average $L_{2}$ norm required to fool the network gives more insight about a network’s robustness in general . The result presented in Table 4 represents the overall performance of our model against C & W attack. Our method of training the noise parameter becomes more effective for more redundant network. We demonstrate this phenomena by performing comparison study between Resnet-20 and Resnet-18 architecture. Clearly Resnet-18 shows the improvement in robustness from Vanilla adv. training much more than Resnet-20 against C & W attack.

2.2 PNI against black-box attack

In this section, we test our proposed PNI technique against transferable adversarial attack and ZOO attack. Following the transferable adversarial attack , two trained neural network are taken as the source model ($S$) and target model ($T$). The adversarial examples $\hat{\bm{x}}_{s}$ is generated from the source model then attack the target model using $\hat{\bm{x}}_{s}$, which is denoted as $S\Rightarrow T$. We take ResNet-18 on CIFAR-10 as an example. We train two ResNet-18 model (model-A and B) on CIFAR-10 dataset to attack each other, where model-A is optimized through vanilla adversarial training, while model-B is trained using our proposed PNI variants (i.e., PNI-W/A-a/W+A-a) robust optimization method. Table 5 shows almost equal perturbed-data accuracy for A $\Rightarrow$ B and B $\Rightarrow$ A under various PNI scenarios, which indicates that our PNI technique does not reduce the attack strength.

For ZOO attack, we test our defense on 200 randomly selected test samples for un-targeted attack. The Attack success rate denotes the percentage of test sample change their classification to a wrong class after attack. ZOO attack success rate for vanilla Resnet-18 with adversarial training is close to 80 %. The robustness of PNI is more evident from Table 5 as the attack success rate drops significantly for PNI-W+A-a and PNI-W. However, PNI-A-a fails to resist ZOO attack even though it still maintains a lower success rate than baseline. The failure of PNI-A-a shows that just adding noise in-front of the activation does not necessarily achieves the desired robustness as claimed by some of the previous defenses .

2.3 Comparison to competing methods

As discussed in Section 2.2, a large number of adversarial defense works have been proposed recently, however most of them are already broken by stronger attacks proposed in . As a result, in this work we choose to compare with the most effective one till date - PGD based adversarial training . Additionally, we compare with other randomness-based works in Table 6 for examining the effectiveness of PNI.

Previous defense works have shown a trade-off between clean-data accuracy and perturbed-data accuracy, where the perturbed-data accuracy improvement normally at the cost of lowering the clean-data accuracy. It is worthy to highlight that our proposed PNI improves both clean- and perturbed data accuracy under white-box attack, in comparison to PGD-based adversarial training . Differential Privacy (DP) is a similar method of utilizing noise injection at various locations within the network. Although their defense guarantees a certified defense it does not perform well against $L^{\infty}$-norm based attack (e.g., PGD and FGSM). In order to achieve a higher level of certified defense, DP significantly sacrifices the clean-data accuracy as well. Another randomness-based approach is Random Self-ensemble (RSE) , which inserts noise-layer before all the convolution layer. Even though their defense performs well against C & W attack but poor against strong PGD attack. In our black-box attack simulation Table 5 we demonstrate that adding activation noise may not be as effective as weight noise. Beyond that, both DP and RSE manually configure the noise level which is extremely difficult to find the optimal setup. Whereas, in our proposed PNI method, the noise level is determined by a trainable layer-wise noise scaling coefficient and distribution of noise injected location.

Discussion

The defense performance improvement led by our proposed PNI does not come from the stochastic gradients. The stochastic gradient is considered to incorrectly approximate the true gradient based on a single sample. We try to show that PNI is not relying on the gradient obfuscation from two perspectives: 1) Our proposed PNI method passes each inspection item proposed by to identify gradient obfuscation. 2) Under PGD attack, through increasing the attack steps, our PNI robust optimization method still outperforms vanilla adversarial training (certified as non-obfuscated gradients in ).

The famous gradient obfuscation work enumerates several characteristic behaviors as listed in Table 7 which can be observed when the defense method owns gradient obfuscation. Our experiments show that PNI passes each inspection item in Table 7.

For item.1, all the experiments in Table 2 and Table 3 report that FGSM attack (one-step) performs worse than PGD attack (iterative). For item.2, our black-box attack experiment in Table 5 shows that the black-box attack strength is worse than white-box attack. For items.3, as plotted in Fig. 3, we run experiments through increasing the distortion bound-$\epsilon$. The result shows that the unbounded attacks do lead to 0% accuracy under attack. For item.4, the prerequisite is the gradient-based attack (e.g., PGD and FGSM) cannot find the adversarial examples, however the experiments in Fig. 3 reveals that our method still can be broken when increasing the distortion bound. It just increases the resistance against the adversarial attacks, in comparison to the vanilla adversarial training. For item.5, again as shown in Fig. 3, increasing the distortion bound increase the attack success rate.

As shown in Fig. 3, gradually increasing the PGD attack steps $N_{step}$ raises the attack strength , thus leading to perturbed-data accuracy degradation for both vanilla adversary training and our PNI technique. However, for both cases the perturbed-data accuracy start saturating and do not degrade any further when $N_{step}=40$. If our PNI’s success comes from the stochastic gradient which gives incorrect gradient owing to the single sample, increasing the attack steps suppose to eventually break the PNI defense which is not observed here. Our PNI method still outperforms vanilla adversarial training even when $N_{step}$ is increased up to 100. Therefore, we can draw the conclusion that, even if PNI does include gradient obfuscation, the stochastic gradient is not the dominant role in PNI for the robustness improvement.

Conclusion

In this paper, we present a parametric noise injection technique where the noise intensity can be trained through solving the min-max optimization problem during adversarial training. Through extensive experiments, the proposed PNI method can outperforms the state-of-the-art defense method in terms of both clean-data accuracy and perturbed-data accuracy.

References