Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models

Introduction

Image classification is a fundamental problem in computer vision and serves as the foundation of multiple tasks such as object detection, image segmentation, object tracking, action recognition, and autonomous driving. Since the breakthrough achieved by AlexNet in ImageNet Challenge (ILSVRC) 2012 , deep neural networks (DNNs) have become the dominant force in this domain. From then on, DNN models with increasing depth and more complex building blocks have been proposed. While these models continue to achieve steadily increasing accuracies, their robustness has not been thoroughly studied, thus little is known if the high accuracies come at the price of reduced robustness.

A common approach to evaluate the robustness of DNNs is via adversarial attacks , where imperceptible adversarial examples are crafted to mislead DNNs. Generally speaking, the easier an adversarial example can be generated, the less robust the DNN is. Adversarial examples may lead to significant property damage or loss of life. For example, has shown that a subtly-modified physical Stop sign can be misidentified by a real-time object recognition system as a Speed Limit sign. In addition to adversarial attacks, neural network robustness can also be estimated in an attack-agnostic manner. For example, and theoretically analyzed the robustness of some simple neural networks by estimating their global and local Lipschitz constants, respectively. proposes to use extreme value theory to estimate a lower bound of the minimum adversarial distortion, and can be efficiently applied to any neural network classifier. proposes a robustness lower bound based on linear approximations of ReLU activations. In this work, we evaluate DNN robustness by using specific attacks as well as attack-agnostic approaches. We also note that the adversarial robustness studied in this paper is different from , where “robustness” is studied in the context of label semantics and accuracy.

Since the last ImageNet challenge has ended in 2017, we are now at the beginning of post-ImageNet era. In this work, we revisit 18 DNN models submitted to the ImageNet Challenge or achieved state-of-the-art performance. These models have different sizes, classification performance, and belong to multiple architecture families such as AlexNet , VGG Nets , Inception Nets , ResNets , DenseNets , MobileNets , and NASNets . Therefore, they are suitable to analyze how different factors influence the model robustness. Specifically, we aim to examine the following questions in this study:

Has robustness been sacrificed for the increased classification performance?

Which factors influence the robustness of DNNs?

In the course of evaluation, we have gained a number of insights and we summarize our contributions as follows:

The adversarial examples generated by the VGG family can transfer very well to all the other 17 models, while most adversarial examples of other models can only transfer within the same model family. Interestingly, this finding provides us an opportunity to reverse-engineer the architecture of black-box models.

We present the first comprehensive study that compares the robustness of 18 popular and state-of-the-art ImageNet models, offering a complete picture of the accuracy v.s. robustness trade-off. In terms of transferability of adversarial examples, we conduct thorough experiments on each pair of the 18 ImageNet networks (306 pairs in total), which is the largest scale to date.

Background and Experimental Setup

In this section, we introduce the background knowledge and how we set up experiments. We study both untargeted attack and targeted attack in this paper. Let $\mathbf{x}_{0}$ denote the original image and $\mathbf{x}$ denote the adversarial image of $\mathbf{x}_{0}$. The DNN model $F(\cdot)$ outputs a class label (or a probability distribution of class labels) as the prediction. Without loss of generality, we assume that $F(\mathbf{x}_{0})=y_{0}$, which is the ground truth label of $\mathbf{x}_{0}$, to avoid trivial solution. For untargeted attack, the adversarial image $\mathbf{x}$ is crafted in a way that $\mathbf{x}$ is close to $\mathbf{x}_{0}$ but $F(\mathbf{x})\neq y_{0}$. For targeted attack, a target class $t$ ($t\neq y_{0}$) is provided and the adversarial image $\mathbf{x}$ should satisfy that (i) $\mathbf{x}$ is close to $\mathbf{x}_{0}$, and (ii) $F(\mathbf{x})=t$.

In this work, we study the robustness of 18 deep image classification models belonging to 7 architecture families, as summarized below. Their basic properties of these models are given in Table 1.

AlexNet AlexNet is one of the pioneering and most well-known deep convolutional neural networks. Compared to many recent architectures, AlexNet has a relatively simple layout that is composed of $5$ convolutional layers followed by two fully connected layers and a softmax output layer.

VGG Nets The overall architecture of VGG nets are similar to AlexNet, but they are much deeper with more convolutional layers. Another main difference between VGG nets and AlexNet is that all the convolutional layers of VGG nets use a small (3$\times$3) kernel while the first two layers of AlexNet use 11$\times$11 and 5$\times$5 kernels, respectively. In our paper, we study VGG networks with $16$ and $19$ layers, with 138 million and 144 million parameters, respectively.

Inception Nets The family of Inception nets utilizes the inception modules that act as multi-level feature extractors. Specifically, each inception module consists of multiple branches of $1\times 1$, $3\times 3$, and $5\times 5$ filters, whose outputs will stack along the channel dimension and be fed into the next layer in the network. In this paper, we study the performance of all popular networks in this family, including Inception-v1 (GoogLeNet) , Inception-v2 , Inception-v3 , Inception-v4, and Inception-ResNet . All these models are much deeper than AlexNet/VGG but have significantly fewer parameters.

ResNets To solve the vanishing gradient problem for training very deep neural networks, the authors of proposes ResNets, where each layer learns the residual functions with reference to the input by adding skip-layer paths, or “identity shortcut connections”. This architecture enables practitioners to train very deep neural networks to outperform shallow models. In our study, we evaluate 3 ResNets with different depths.

DenseNets To further exploit the “identity shortcut connections” techniques from ResNets, proposes DenseNets that connect all layers with each other within a dense block. Besides tackling gradient vanishing problem, the authors also claimed other advantages such as encouraging feature reuse and reducing the number of parameters in the model. We study 3 DenseNets with different depths and widths.

MobileNets MobileNets are a family of light weight and efficient neural networks designed for mobile and embedded systems with restricted computational resources. The core components of MobileNets are depthwise separable filters with factorized convolutions. Separable filters can factorize a standard convolution into two parts, a depthwise convolution and a $1\times 1$ pointwise convolution, which can reduce computation and model size dramatically. In this study, we include 3 MobileNets with different depths and width multipliers.

NASNets NASNets are a family of networks automatically generated by reinforcement learning using a policy gradient algorithm to optimize architectures . Building blocks of the model are first searched on a smaller dataset and then transfered to a larger dataset.

2 Robustness Evaluation Approaches

We use both adversarial attacks and attack-agnostic approaches to evaluate network robustness. We first generate adversarial examples of each network using multiple state-of-the-art attack algorithms, and then analyze the attack success rates and the distortions of adversarial images. In this experiment, we assume to have full access to the targeted DNNs, known as the white-box attack. To further study the transferability of the adversarial images generated by each network, we consider all the 306 network pairs and for each pair, we conduct transfer attack that uses one model’s adversarial examples to attack the other model. Since transfer attack is widely used in the black-box setting , where an adversary has no access to the explicit knowledge of the target models, this experiment can provide some evidence on networks’ black-box robustness. Finally, we compute CLEVER score, a state-of-the-art attack-agnostic network robustness metric, to estimate each network’s intrinsic robustness. Below, we briefly introduce all the evaluation approaches used in our study.

We evaluate the robustness of DNNs using the following adversarial attacks:

Fast Gradient Sign Method (FGSM) FGSM is one of the pioneering and most efficient attacking algorithms. It only needs to compute the gradient once to generate an adversarial example $\mathbf{x}$:

C&W attack formulates the problem of generating adversarial examples $\mathbf{x}$ as the following optimization problem

where $f(\mathbf{x},t)$ is a loss function to measure the distance between the prediction of $\mathbf{x}$ and the target label $t$. In this work, we choose

where $f(\mathbf{x},t)$ is the same as used in the C&W attack. show that EAD-L1 attack is highly transferable and can bypass many defenses and analysis.

We also evaluate network robustness using an attack-agnostic approach:

3 Dataset

In this work, we use the ImageNet as the benchmark dataset, due to the following reasons: (i) ImageNet dataset can take full advantage of the studied DNN models since all of them were designed for ImageNet challenges; (ii) comparing to the widely-used small-scale datasets such as MNIST, CIFAR-10 , and GTSRB , ImageNet has significantly more images and classes and is more challenging; and (iii) it has been shown by that ImageNet images are easier to attack but harder to defend than the images from MNIST and CIFAR datasets. Given all these observations, ImageNet is an ideal candidate to study the robustness of state-of-the-art deep image classification models.

A set of randomly selected 1,000 images from the ImageNet validation set is used to generate adversarial examples from each model. For each image, we conduct targeted attacks with a random target and a least likely target as well as an untargeted attack. Misclassified images are excluded. We follow the setting in to compute CLEVER scores for 100 out of the all 1,000 images, as CLEVER is relatively more computational expensive. Additionally, we conducted another experiment by taking the subset of images (327 images in total) that are correctly classified by all of 18 examined ImageNet models. The results are consistent with our main results and are given in supplementary material.

4 Evaluation Metrics

In our study, the robustness of the DNN models is evaluated using the following four metrics:

Attack success rate For non-targeted attack, success rate indicates the percentage of the adversarial examples whose predicted labels are different from their ground truth labels. For targeted attack, success rate indicates the percentage of the adversarial examples that are classified as the target class. For both attacks, a higher success rate suggests that the model is easier to attack and hence less robust. When generating adversarial examples, we only consider original images that are correctly classified to avoid trial attacks.

Transferability We follow to define targeted and non-targeted transferability. For non-targeted attack, transferability is defined as the percentage of the adversarial examples generated for one model (source model) that are also misclassified by another model (target model). We refer to this percentage as error rate, and a higher error rate means better non-targeted transferability. For targeted attack, transferability is defined as matching rate, i.e., the percentage of the adversarial examples generated for source model that are misclassified as the target label (or within top-k labels) by the target model. A higher matching rate indicates better targeted transferability.

Experiments

After examining all the 18 DNN models, we have learned insights about the relationships between model architectures and robustness, as discussed below.

We have carefully conducted a controlled experiment by pulling images from a common set of 1000 test images when evaluating the robustness of different models. For assessing the robustness of each model, the originally misclassified images are excluded. We compare the success rates of targeted attack with a random target of FGSM, I-FGSM, C&W and EAD-L1 with different parameters for all 18 models. The success rate of FGSM targeted attack is low so we also show its untargeted attack success rate in Figure 1(b).

For targeted attack, the success rate of FGSM is very low (below 3% for all settings), and unlike in the untargeted setting, increasing $\epsilon$ in fact decreases attack success rate. This observation further confirms that FGSM is a weak attack, and targeted attack is more difficult and needs iterative attacking methods. Figure 1(c) shows that, with only 10 iterations, I-FGSM can achieve a very good targeted attack success rate on all models. C&W and EAD-L1 can also achieve almost 100% success rate on almost all of the models when $\kappa=0$.

We defer the results for targeted attack with the least likely target label to the Supplementary section because the conclusions made are similar.

2 Linear Scaling Law in Robustness v.s. Accuracy

3 Robustness of Different Model Sizes and Architectures

We find that model architecture is a more important factor to model robustness than the model size. Each family of networks exhibits a similar level of robustness, despite different depths and model sizes. For example, AlexNet has about 60 million parameters but its robustness is the best; on the other hand, Mobilenet-0.50-160 has only 1.5 million parameters but is more vulnerable to adversarial attacks in all metrics.

4 Transferability of Adversarial Examples

Figures 3, 4 and 5 show the transferability heatmaps of FGSM, I-FGSM and EAD-L1 over all 18 models (306 pairs in total). The value in the $i$-th row and $j$-th column of each heatmap matrix is the proportion of the adversarial examples successfully transferred to target model $j$ out of all adversarial examples generated by source model $i$ (including both successful and failed attacks on the source model). Specifically, the values on the diagonal of the heatmap are the attack success rate of the corresponding model. For each model, we generate adversarial images using the aforementioned attacks and pass them to the target model to perform black-box untargeted and targeted transfer attacks. To evaluate each model, we use the success rate for evaluating the untargeted transfer attacks and the top-5 matching rate for evaluating targeted transfer attacks.

Note that not all models have the same input image dimension. We also find that simply resizing the adversarial examples can significantly decrease the transfer attack success rate . To alleviate the disruptive effect of image resizing on adversarial perturbations, when transferring an adversarial image from a network with larger input dimension to a smaller dimension, we crop the image from the center; conversely, we add a white boarder to the image when the source network’s input dimension is smaller.

Generally, the transferability of untargeted attacks is significantly higher than that of targeted attacks, as indicated in Figure 3, 4 and 5. We highlighted some interesting findings in our experimental results:

In the untargeted transfer attack setting, FGSM and I-FGSM have much higher transfer success rates than those in EAD-L1 (despiting using a large $\kappa$). Similar to the results in , we find that the transferability of C&W is even worse than that of EAD-L1 and we defer the results to the supplement. The ranking of attacks on transferability in untargeted setting is given by

Again in the untargeted transfer attack setting, for FGSM, a larger $\epsilon$ yields better transferability, while for I-FGSM, less iterations yield better transferability. For untargeted EAD-L1 transfer attacks, a higher $\kappa$ value (confidence parameter) leads to better transferability, but it is still far behind I-FGSM.

Transferability of adversarial examples is sometimes asymmetric; for example, in Figure 4, adversarial examples of VGG 16 are highly transferable to Inception-v2, but adversarial examples of Inception-v2 do not transfer very well to VGG.

We find that VGG 16 and VGG 19 models achieve significantly better transferability than other models, in both targeted and untargeted setting, for all attacking methods, leading to the “stripe patterns”. This means that adversarial examples generated from VGG models are empirically more transferable to other models. This observation might be explained by the simple convolutional nature of VGG networks, which is the stem of all other networks. VGG models are thus a good starting point for mounting black-box transfer attacks. We also observe that the most transferable model family may vary with different attacks.

Most recent networks have some unique features that might restrict adversarial examples’ transferability to only within the same family. For example, as shown in Figure 4, when using I-FGSM in the untargeted transfer attack setting, for DenseNets, ResNets and VGG, transferability between different depths of the same architecture is close to 100%, but their transfer rates to other architectures can be much worse. This provides us an opportunity to reserve-engineer the internal architecture of a black-box model, by feeding it with adversarial examples crafted for a certain architecture and measure the attack success rates.

Conclusions

In this work, we focus on image classification. To the best of our knowledge, the scale and profound analysis on 18 ImageNet models have not been studied thoroughly in the previous literature. We believe our findings could also provide insights to robustness and adversarial examples in other computer vision tasks such as object detection and image captioning , since these tasks often use the same pre-trained image classifiers studied in this paper for feature extraction.

Supplementary Materials

To further validate our robustness analysis, we conducted another experiment by taking the subset of images (327 images in total) that are correctly classified by all of 18 examined ImageNet models and show their accuracy-vs-robustness figures on C&W and I-FGSM targeted attacks in Figure 6. The trends and conclusions are consistent with our reported main results.

2 Robustness vs. Accuracy of Least-Likely Attacks

In this section, we summarize the results of using the least-likely label (the class with the smallest probability of the original image) as the target class. Figure 7 (a) and (b) show the distortions of adversarial examples found by I-FGSM and C&W attacks, respectively. Although the least-likely label attack is even more challenging, both I-FGSM and C&W algorithms can still achieve a close to 100% success rate. Similar to Figure 2 of the main text, Figure 7 clearly shows an accuracy v.s. robustness trade-off for models on the Pareto frontier, e.g., AlexNet is the most robust network while the model with the highest accuracy (NASNet) is most prone to adversarial attacks. Likewise, we fit the Pareto frontier and still observe a similar log-linear scaling law.

3 The Transferability of C&W Attack

4 More Experiments on the Transferability of I-FGSM Attack

In this section, we show more experimental results on I-FGSM attack with different $\epsilon$ values. Figures 9 and 10 demonstrate the transferability heatmaps of I-FGSM with $\epsilon=0.1$ and $\epsilon=0.2.$ Comparing these two heatmaps with Figure 4 in the main text (transferability of I-FGSM with $\epsilon=0.3$), we observe that: (i) I-FGSM’s transferability improves when $\epsilon$ increases; (ii) less iterations usually yield better transferability; (iii) transferability of untargeted attacks is significantly higher than that of targeted attacks; (iv) adversarial examples of VGG networks consistently transfer very well; and (v) adversarial examples are easier to be transfered between the models sharing a same architecture (e.g., ResNets and DenseNets) but different depths.

5 Additional Remarks

Images in ImageNet are organized according to the WordNet hierarchy. To justify that least likely labels used in our experiments are indeed irrelevant to the original labels, we show their corresponding synsets’ shortest path distances in the WordNet hierarchy in Figure 11. We use Inception-v1 as the model in the experiment. Two labels of shortest path distances greater than 5 are considered irrelevant. In our case, this applies to 96.6% of our least likely attacks and hence the vulnerability is not from the label sensitivity effect as studied in .

In light of , our findings on accuracy-robustness trade-off could be explained by the increasing sensitivity in more accurate models – these two robustness conclusions actually complement each other, rather than being exclusive or contradictory. Specifically, increasing sensitivity aids in improved accuracy but might also make the model more vulnerable. For example, increasing the sensitivity in classifying different dog species can improve the model accuracy, but may at the same time contribute to smaller adversarial perturbations.

References