Towards Accurate One-Stage Object Detection with AP-Loss

Introduction

Object detection needs to localize and recognize the objects simultaneously from the large backgrounds, which remains challenging due to the imbalance between foreground and background. Deep learning based detection solutions usually adopt a multi-task architecture, which handles classification task and localization task with different loss functions. The classification task aims to recognize the object in a given box, while the localization task aims to predict the precise bounding box of the object. Two-stage detectors first generates a limited number of object box proposals, so that the detection problem can be solved by adopting classification task on those proposals. However, the circumstance is different for one-stage detectors, which need to predict the object class directly from the densely pre-designed candidate boxes. The large number of boxes yield the imbalance between foreground and background which makes the optimization of classification task easily biased and thus impacts the detection performance. It is observed that the classification metric could be very high for a trivial solution which predicts negative label for almost all candidate boxes, while the detection performance is poor. 1(a) illustrates one such example.

To tackle this issue in one-stage object detectors, some works introduce new classification losses such as balanced loss , Focal Loss , as well as tailored training method such as Online Hard Example Mining (OHEM) . These losses model each sample (anchor box) independently, and attempt to re-weight the foreground and background samples in classification losses to cater for the imbalance condition; this is done without considering the relationship among different samples. The designed balance weights are hand-crafted hyper-parameters, which do not generalize well across datasets. We argue that the gap between classification task and detection task hinder the performance of one-stage detectors. In this paper, instead of modifying the classification loss, we propose to replace classification task with ranking task in one-stage detectors, where the associated ranking loss explicitly models sample relationship, and is invariant to the ratio of positive and negative samples. As shown in 1(b), we adopt Average Precision (AP) as our target loss which is inherently more consistent with the evaluation metric for object detection.

However, it is non-trivial to directly optimize the AP-loss due to the non-differentiability and non-decomposability, so that standard gradient descent methods are not amenable for this case. There are three aspects of studies for this issue. First, AP based loss is studied within structured SVM models , which restricts in linear SVM model so that the performance is limited. Second, a structured hinge loss is proposed to optimize the upper bound of AP-loss instead of the loss itself. Third, approximate gradient methods are proposed for optimizing the AP-loss, which are less efficient and easy to fall into local optimum even for the case of linear models due to the non-convexity and non-quasiconvexity of the AP-loss. Therefore, it is still an open problem for the optimization of the AP-loss.

In this paper, we address this challenge by replacing the classification task in one-stage detectors with a ranking task, so that we handle the class imbalance problem with a ranking based loss named AP-loss. Furthermore, we propose a novel error-driven learning algorithm to effectively optimize the non-differentiable AP based objective function. More specifically, some extra transformations are added to the score output of one-stage detector to obtain the AP-loss, which includes a linear transformation that transforms the scores to pairwise differences, and a non-linear and non-differentiable “activation function” that transform the pairwise differences to the primary terms of the AP-loss. Then the AP-loss can be obtained by the dot product between the primary terms and the label vector. It is worth noting that the difficulty for using gradient method on the AP-loss lies in passing gradients through the non-differentiable activation function. Inspired by the perceptron learning algorithm , we adopt an error-driven learning scheme to directly pass the update signal through the non-differentiable activation function. Different from gradient method, our learning scheme gives each variable an update signal proportional to the error it makes. Then, we adopt the backpropagation algorithm to transfer the update signal to the weights of neural network. We theoretically and experimentally prove that the proposed optimization algorithm does not suffer from the non-differentiability and non-convexity of the objective function. The main contributions of this paper are summarized as below:

We propose a novel framework in one-stage object detectors which adopts the ranking loss to handle the class imbalance issue.

We propose an error-driven learning algorithm that can efficiently optimize the non-differentiable and non-convex AP-based objective function with both theoretical and experimental verifications.

We show notable performance improvement with the proposed method on state-of-the-art one-stage detectors over different kinds of classification-losses without changing the model architecture.

Related Work

One-stage detectors: In object detection, the one-stage approaches have relatively simpler architecture and higher efficiency than two-stage approaches. OverFeat is one of the first CNN-based one-stage detectors. Thereafter, different designs of one-stage detectors are proposed, including SSD , YOLO , DSSD and DSOD . These methods demonstrate good processing efficiency as one-stage detectors, but generally yield lower accuracy than two-stage detectors. Recently, RetinaNet and RefineDet narrow down the performance gap (especially on the challenging COCO benchmark ) between one-stage approaches and two-stage approaches with some innovative designs. As commonly known, the performance of one-stage detectors benefits much from densely designed anchors, which introduce extreme imbalance between foreground and background samples. To address this challenge, methods like OHEM and Focal Loss have been proposed to reduce the loss weight for easy samples. However, there are two hurdles that are still open to discussion. Firstly, hand-crafted hyper-parameters for weight balance do not generalize well across datasets. Secondly, the relationship among sample anchors is far from well modeled.

AP as a loss for Object Detection: Average Precision (AP) is widely used as the evaluation metric in many tasks such as object detection and information retrieval . However, AP is far from a good and common choice as an optimization goal in object detection due to its non-differentiability and non-convexity. Some methods have been proposed to optimize the AP-loss in object detection, such as AP-loss in the linear structured SVM model , structured hinge loss as upper bound of the AP-loss , approximate gradient methods , reinforcement learning to fine-tune a pre-trained object detector with AP based metric . Although these methods give valuable results in optimizing the AP-loss, their performances are still limited due to the intrinsic limitations. In details, the proposed approach differs from them in 4 aspects. (1) Our approach can be used for any differentiable linear or non-linear models such as neural networks, while only work for linear SVM model. (2) Our approach directly optimizes the AP-loss, while introduces notable loss gap after relaxation. (3) Our approach dose not approximate the gradient and dose not suffer from the non-convexity of objective function as in . (4) Our approach can train the detectors in an end-to-end way, while cannot.

Perceptron Learning Algorithm: The core of our optimization algorithm is the “error-driven update” which is generalized from the perceptron learning algorithm , and helps overcome the difficulty of the non-differentiable objective functions. The perceptron is a simple artificial neuron using the Heaviside step function as the activation function. The learning algorithm was first invented by Frank Rosenblatt . As the Heaviside step function in perceptron is non-differentiable, it is not amenable for gradient method. Instead of using a surrogate loss like cross-entropy, the perceptron learning algorithm employs an error-driven update scheme directly on the weights of neurons. This algorithm is guaranteed to converge in finite steps if the training data is linearly separable. Further works like have studied and improved the stability and robustness of the perceptron learning algorithm.

Method

We aim to replace the classification task with AP-loss based ranking task in one-stage detectors such as RetinaNet . Figure 2 shows the two key components of our approach, i.e., the ranking procedure and the error-driven optimization algorithm. Below, we will first present how AP-loss is derived from traditional score output. Then, we will introduce the error-driven optimization algorithm. Finally, we also present the theoretical analyses of the proposed optimization algorithm and outline the training details. Note that all changes are made on the loss part of the classification branch without changing the backbone model and localization branch.

In traditional one-stage detectors, given input image $I$, suppose the pre-defined boxes (also called anchors) set is $B$, each box $b_{i}\in B$ will be assigned a label $t_{i}\in\{-1,0,1,\ldots,K\}$ based on ground truth and the IoU strategy , where label $1\sim K$ means the object class ID, label “0” means background and label “$-1$” means ignored boxes. During training and testing phase, the detector outputs a score-vector $(s_{i}^{0},\cdots,s_{i}^{K})$ for each box $b_{i}$.

In our framework, instead of one box with $K+1$ dimensional score predictions, we replicate each box $b_{i}$ for $K$ times to obtain $b_{ik}$ where $k=1,\cdots,K$, and the $k$-th box is responsible for the $k$-th class. Each box $b_{ik}$ will be assigned a label $t_{ik}\in\{-1,0,1\}$ through the same IoU strategy (label $-1$ for not counted into the ranking loss). Therefore, in the training and testing phase, the detector will predict only one scalar score $s_{ik}$ for each box $b_{ik}$. Figure 3 illustrates our label formulation and the difference to traditional case.

The ranking task dictates that every positive boxes should be ranked higher than all the negative boxes w.r.t their scores. Note that AP of our ranking result is computed over the scores from all classes. This is slightly different from the evaluation metric meanAP for object detection systems, which computes AP for each class and obtains the average value. We compute AP this way because the score distribution should be unified for all classes while ranking each class separately cannot achieve this goal.

1.2 AP-Loss

where $s(b_{i};\bm{\theta})$ is a CNN based score function with weights $\bm{\theta}$ for box $b_{i}$. The ranking label transformation transfers labels $t_{i}$ to the corresponding pairwise ordering form

where $H(\cdot)$ is the Heaviside step function:

A ranking is denoted as proper ranking when there are no two samples scored equally (i.e., $\forall i\neq j,\,\,s_{i}\neq s_{j}$). Without loss of generality, we will treat all rankings as a proper ranking by breaking ties arbitrarily. Now, we can formulate the AP-loss $\mathcal{L}_{AP}$ as

Finally, the optimization problem can be written as:

where $\bm{\theta}$ denotes the weights of detector model. As the activation function $\bm{L}(\cdot)$ is non-differentiable, a novel optimization/learning scheme is required instead of the standard gradient descent method.

Besides the AP metric, other ranking based metric can also be used to design the ranking loss for our framework. One example is the AUC-loss which measures the area under ROC curve for ranking purpose, and has a slightly different “activation function” as

As AP is consistent with the evaluation metric of the object detection task, we argue that AP-loss is intuitively more suitable than AUC-loss for this task, and will provide empirical study in our experiments.

2 Optimization Algorithm

Recalling the perceptron learning algorithm, the update for input variable is “error-driven”, which means the update is directly derived from the difference between desired output and current output. We adopt this idea and further generalize it to accommodate the case of activation function with vector-valued input and output. Suppose $x_{ij}$ is the input and $L_{ij}$ is the current output, the update for $x_{ij}$ is thus

where $L^{*}_{ij}$ is the desired output. Note that the AP-loss achieves its minimum possible value when each term $L_{ij}\cdot y_{ij}=0$. There are two cases. If $y_{ij}=1$, we should set the desired output $L^{*}_{ij}=0$. If $y_{ij}=0$, we do not care the update and set it to , since it does not contribute to the AP-loss. Consequently, the update can be simplified as

2.2 Backpropagation

We now have the desired vector-form update $\Delta\bm{x}$, and then will find an update for model weights $\Delta\bm{\theta}$ which will produce most appropriate movement for $\bm{x}$. We use dot-product to measure the similarity of successive movements, and regularize the change of weights (i.e. $\Delta\bm{\theta}$) with $L_{2}$-norm based penalty term. The optimization problem can be written as:

where $\bm{\theta}^{(n)}$ denotes the model weights at the $n$-th step. With that, the first-order expansion of $\boldsymbol{x}(\boldsymbol{\theta})$ is given by:

where $\partial\bm{x}(\bm{\theta}^{(n)})/\partial\bm{\theta}$ is the Jacobian matrix of vector-valued function $\bm{x}(\bm{\theta})$ at $\bm{\theta}^{(n)}$. Ignoring the high-order infinitesimal, we obtain the step-wise minimization process:

The optimal solution can be obtained by finding the stationary point. Then, the form of optimal $\Delta\bm{\theta}$ is consistent with the chain rule of derivative, which means, it can be directly implemented by setting the gradient of $x_{ij}$ to $-\Delta x_{ij}$ (c.f. Equation 9) and proceeding with backpropagation. Hence the gradient for score $s_{i}$ can be obtained by backward propagating the gradient through the difference transformation:

3 Analyses

Convergence: To better understand the characteristics of the AP-loss, we first provide a theoretical analysis on the convergence of the optimization algorithm, which is generalized from the convergence property of the original perceptron learning algorithm.

The AP-loss optimizing algorithm is guaranteed to converge in finite steps if below conditions hold: (1) the learning model is linear;

(2) the training data is linearly separable.

The proof of this proposition is provided in Appendix-1 of supplementary. Although convergence is somewhat weak due to the need of strong conditions, it is non-trivial since the AP-loss function is not convex or quasiconvex even for the case of linear model and linearly separable data, so that gradient descent based algorithm may still fail to converge on a smoothed AP-loss function even under such strong conditions. One such example is presented in Appendix-2 of supplementary. It means that, under such conditions, our algorithm still optimizes better than the approximate gradient descent algorithm for AP-loss. Furthermore, with some mild modifications, even though the training data is not separable, the accumulated AP-loss can also be bounded proportionally by the best performance of the learning model. More details are presented in Appendix-3 of supplementary.

Consistency: Besides convergence, We observed that the proposed optimization algorithm is inherently consistent with widely used classification-loss functions.

When the activation function $L(\cdot)$ takes the form of softmax function and loss-augmented step function, our optimization algorithm can be expressed as the gradient descent algorithm on cross-entropy loss and hinge loss respectively.

The detailed analysis of this observation is presented in Appendix-4 of supplementary. We argue that the observed consistency is on the basis of the “error-driven” property. As is known, the gradients of those widely used loss functions are proportional to their prediction errors, where the prediction here refers to the output of activation function. In other words, their activation functions have a nice property: the vector field of prediction errors is conservative, allowing it being the gradient of some surrogate loss function. However, our activation function does not have this property, which makes our optimization not able to express as gradient descent with any surrogate loss function.

4 Details of Training Algorithm

Minibatch Training The minibatch training strategy is widely used in deep learning frameworks as it accounts for more stability than the case with batch size equal to 1. The mini-batch training helps our optimization algorithm quite a lot for escaping the so-called “score-shift” scenario. The AP-loss can be computed both from a batch of images and from a single image with multiple anchor boxes. Consider an extreme case: our detector can predict perfect ranking in both image $I_{1}$ and image $I_{2}$, but the lowest score in image $I_{1}$ is even greater than the highest score in image $I_{2}$. There are “score-shift” between two images so that the detection performance is poor when computing AP-loss per-image. Aggregating scores over images in a mini-batch can avoid such problem, so that the minibatch training is crucial for good convergence and good performance.

Piecewise Step function During early stage of training, scores $s_{i}$ are very close to each other (i.e. almost all inputs to Heaviside step function $H(x)$ are near zero), so that a small change of input will cause a big output difference, which destabilizes the updating process. To tackle this issue, we replace $H(x)$ with a piecewise step function:

The piecewise step functions with different $\delta$ are shown in Figure 4. When $\delta$ approaches $+0$, the piecewise step function approaches the original step function. Note that $f(\cdot)$ is only different from $H(\cdot)$ near zero point. We argue that the precise form of the piecewise step function is not crucial. Other monotonic and symmetric smooth functions that only differs from $H(\cdot)$ near zero point could be equally effective. The choice of $\delta$ relates closely to the weight decay hyper-parameter in CNN optimization. Intuitively, parameter $\delta$ controls the width of decision boundary between positive and negative samples. Smaller $\delta$ enforces a narrower decision boundary, which causes the weights to shrink correspondingly (similar effect to that caused by the weight decay). Further details are presented in the experiments.

Interpolated AP The interpolated AP is widely adopted by many object detection benchmarks like PASCAL VOC and MS COCO . The common justification for interpolating the precision-recall curve is “to reduce the impact of ’wiggles’ in the precision-recall curve, caused by small variations in the ranking of examples”. Under the same consideration, we adopt the interpolated AP instead of the original version. Specifically, the interpolation is applied on $L_{ij}$ to make the precision at the $k$-th smallest positive sample monotonically increasing with $k$ where the precision is $(1-\sum_{j\in\mathcal{N}}L_{ij})$ in which $i$ is the index of the $k$-th smallest positive sample. It is worth noting that the interpolated AP is a smooth approximation of the actual AP so that it is a practical choice to help to stabilize the gradient and to reduce the impact of ’wiggles’ in the update signals. The details of the interpolated AP based algorithm is summarized in Algorithm 1.

Experiments

We evaluate the proposed method on the state-of-the-art one-stage detector RetinaNet . The implementation details are the same as in unless explicitly stated. Our experiments are performed on two benchmark datasets: PASCAL VOC and MS COCO . The PASCAL VOC dataset has 20 classes, with VOC2007 containing 9,963 images for train/val/test and VOC2012 containing 11,530 for train/val. The MS COCO dataset has 80 classes, containing 123,287 images for train/val. We implement our codes with the MXNET framework, and conduct experiments on a workstation with two NVidia TitanX GPUs.

PASCAL VOC: When evaluated on the VOC2007 test set, models are trained on the VOC2007 and VOC2012 trainval sets. When evaluated on the VOC2012 test set, models are trained on the VOC2007 and VOC2012 trainval sets plus the VOC2007 test set. Similar to the evaluation metrics used in the MS COCO benchmark, we also report the AP averaged over multiple IoU thresholds of $0.50:0.05:0.95$. We set $\delta=1$ in Equation 14. We use ResNet as the backbone model which is pre-trained on the ImageNet-1k classification dataset . At each level of FPN , the anchors have 2 sub-octave scales ($2^{k/2}$, for $k\leq 1$) and 3 aspect ratios [0.5, 1, 2]. We fix the batch normalization layers to be frozen in training phase. We adopt the minibatch training on 2 GPUs with 8 images per GPU. All evaluated models are trained for 160 epochs with an initial learning rate of 0.001 which is then divided by 10 at 110 epochs and again at 140 epochs. Weight decay of 0.0001 and momentum of 0.9 are used. We adopt the same data augmentation strategies as , while do not use any data augmentation during testing phase. In training phase, the input image is fixed to 512$\times$512, while in testing phase, we maintain the original aspect ratio and resize the image to ensure the shorter side with 600 pixels. We apply the non-maximum suppression with IoU of 0.5 for each class.

MS COCO: All models are trained on the widely used trainval35k set (80k train images and 35k subset of val images), and tested on minival set (5k subset of val images) or test-dev set. We train the networks for 100 epochs with an initial learning rate of 0.001 which is then divided by 10 at 60 epochs and again at 80 epochs. Other details are similar to that for PASCAL VOC.

2 Ablation Study

We first investigate the impact of our design settings of the proposed framework. We fix the ResNet-50 as backbone and conduct several controlled experiments on PASCAL VOC2007 test set (and COCO minival if stated) for this ablation study.

Here we study the impact of the practical modifications introduced in Section 3.4. All results are shown in Table 1.

Minibatch Training: First, we study the mini-batch training, and report detector results at different batch-size in 1(a). It shows that larger batch-size (i.e. 8) outperforms all the other smaller batch-size. This verifies our previous hypothesis that large minibatch training helps to eliminate the “score-shift” from different images, and thus stabilizes the AP-loss through robust gradient calculation. Hence, $\text{batch-size}=8$ is used in our further studies.

Piecewise Step Function: Second, we study the piecewise step function, and report detector performance on the piecewise step function with different $\delta$ in 1(b). As mentioned before, we argue that the choice of $\delta$ is trivial and is dependent on other network hyper-parameters such as weight decay. Smaller $\delta$ makes the function sharper, which yields unstable training at initial phase. Larger $\delta$ makes the function deviate from the properties of the original AP-loss, which also worsens the performance. $\delta=1$ is a good choice we used in our further studies.

Interpolated AP: Third, we study the impact of interpolated AP in our optimization algorithm, and list the results in 1(c). Marginal benefits are observed for interpolated AP over standard AP, so we use interpolated AP in all the following studies.

2.2 Comparison on Different Losses

We evaluate with different losses on RetinaNet . Results are shown in Table 2. We compare traditional classification based losses like focal loss and cross entropy loss (CE-loss) with OHEM to the ranking based losses like AUC-loss and AP-loss. Although focal loss is significantly better than CE-loss with OHEM on COCO dataset, it is interesting that focal-loss does not perform better than CE-loss at AP50 on PASCAL VOC. This is likely because the hyper-parameters of focal loss are designed to suit the imbalance condition on COCO dataset which is not suitable for PASCAL VOC, so that focal loss cannot generalize well to PASCAL VOC without tuning its hyper-parameters. The proposed AP-loss performs much better than all the other losses on both two datasets, which demonstrates its effectiveness and stronger generalization ability on handling the imbalance issue. It is worth noting that AUC-loss performs much worse than AP-loss, which may be due to the fact that AUC has equal penalty for each misordered pair while AP imposes greater penalty for the misordering at higher positions in the predicted ranking. It is obvious that object detection evaluation concerns more on objects with higher confidence, which is why AP provides a better loss measure. Furthermore, an assessment of the detection performance at different training iterations, as shown in 5(a), outlines the superiority of the AP-loss for snapshot time points.

2.3 Comparison on Different Optimization Methods

We also compare our optimization method with the approximate gradient method and structured hinge loss method . Both approximate the AP-loss with a smooth expectation and envelope function, respectively. Following their guidance, we replace the step function in AP-loss with a sigmoid function to constrain the gradient to neither zero nor undefined, while still keep the shape similar to the original function. Same as , we adopt the log space objective function, i.e. $log(\text{AP}+\epsilon)$, to allow the model to quickly escape from the initial state. We train the detector on VOC2007 trainval set and turn off the bounding box regression task. The convergence curves shown in 5(b) reveal some essential observations. It can be seen that AP-loss optimized by approximate gradient method does not even converge, likely because its non-convexity and non-quasiconvexity fail on a direct gradient descent method. Meanwhile, AP-loss optimized by the structured hinge loss method converges slowly and stabilizes near 0.8, which is significantly worse than the asymptotic limit of AP-loss optimized by our error-driven update scheme. We believe that this method does not optimize the AP-loss directly but rather an upper bound of it, which is controlled by a discriminant function . In ranking task, this discriminant function is hand-picked and has an AUC-like form, which may cause variability in optimization.

3 Benchmark Results

Conclusion

In this paper, we address the class imbalance issue in one-stage object detectors by replacing the classification sub-task with a ranking sub-task, and proposing to solve the ranking task with AP-Loss. Due to non-differentiability and non-convexity of the AP-loss, we propose a novel algorithm to optimize it based on error-driven update scheme from perceptron learning. We provide a grounded theoretical analysis of the proposed optimization algorithm. Experimental results show that our approach can significantly improve the state-of-the-art one-stage detectors.

Acknowledgements. This paper is supported in part by: National Natural Science Foundation of China (61471235), Shanghai ’The Belt and Road’ Young Scholar Exchange Grant (17510740100), CREST Malaysia (No. T03C1-17), and the PKU-NTU Joint Research Institute (JRI) sponsored by a donation from the Ng Teng Fong Charitable Foundation. We gratefully acknowledge the support from Tencent YouTu Lab.

References

A1. Convergence

We provide proof for the proposition mentioned in Section 3.3.1 of the paper. The proof is generalized from the original convergence proof for perceptron learning algorithm.

The AP-loss optimizing algorithm is guaranteed to converge in finite steps if below conditions hold: (1) the learning model is linear;

(2) the training data is linearly separable.

Proof. Let $\bm{\theta}$ denote the weights of the linear model. Let $\bm{f}_{k}^{(n)}$ denote the feature vector of $k$-th box in $n$-th training sample. Assume the number of training samples is finite and each training sample contains at most $M$ boxes. Hence the score of $k$-th box is $s_{k}^{(n)}=\langle\bm{f}_{k}^{(n)},\bm{\theta}\rangle$. Define $x_{ij}^{(n)}=-(s_{i}^{(n)}-s_{j}^{(n)})$. Note that the training data is separable, which means there are $\epsilon>0$ and $\bm{\theta}^{*}$ that satisfy:

In the $t$-th step, a training sample which makes an error (if there is no such training sample, the model is already optimal and algorithm will stop) is randomly chosen. Then the update of $\bm{\theta}$ is:

For convenience, let $\bm{\theta}^{(0)}=0$ (if $\bm{\theta}^{(0)}\neq 0$, we can still find a $c>0$ that satisfies (20) for sufficiently large $t$), we have

Here, $C$ is a positive constant. Let $\bm{\theta}^{(0)}=0$ (again, if $\bm{\theta}^{(0)}\neq 0$, we can still find a $C>0$ that satisfies (22) for sufficiently large $t$), we arrive at:

It shows that the algorithm will stop at most after $C/c^{2}$ steps, which means that the training model will achieve the optimal solution in finite steps.

A2. An Example of Gradient Descent Failing on Smoothed AP-loss

We approximate the step function in AP-loss by sigmoid function to make it amenable to gradient descent. Specifically, the smoothed AP-loss function is given by:

Consider a linear model $s=f_{1}\theta_{1}+f_{2}\theta_{2}$ and three training samples $(0,0),(1,0),(-3,1)$ (the first one is negative sample, others are positive samples). Then we have

Note that the training data is separable since we have $s^{(2)}>s^{(1)}$ and $s^{(3)}>s^{(1)}$ when $0<\theta_{1}<\frac{1}{3}\cdot\theta_{2}$.

Under this setting, the smoothed AP-loss become

If $\theta_{1}$ is sufficiently large and $\theta_{1}>\theta_{2}>0$, then the partial derivatives satisfy the following condition:

which means $\theta_{1}$ and $\theta_{2}$ will keep increasing with the inequality $\theta_{1}>\theta_{2}$ according to the gradient descent algorithm. Hence the objective function $F$ will approach $1/6$ here. However, the objective function $F$ approaches the global minimum value if and only if $\theta_{1}\rightarrow+\infty$ and $\theta_{2}-3\theta_{1}\rightarrow+\infty$. This shows that the gradient descent fails to converge to global minimum in this case.

A3. Inseparable Case

In this section, we will provide analysis for our algorithm with inseparable training data. We demonstrate that the bound of accumulated AP-loss depends on the best performance of learning model. The analysis is based on online learning bounds .

To handle the inseparable case, a mild modification on the proposed algorithm is needed, i.e. in the error-driven update scheme, $L_{ij}$ is modified to

where $\widetilde{H}(\cdot)$ is defined in Section 3.4.2 (Piecewise Step Function) of the paper. The purpose is to introduce a non-zero decision margin for the pairwise score $x_{ij}$ which makes the algorithm more robust in the inseparable case. In contrast to the case in Section 3.4.2, here we only change $H(\cdot)$ to $\widetilde{H}(\cdot)$ in the numerator for the convenience of theoretical anaysis. However, such algorithm still suffers from the discontinuity of $H(\cdot)$ in the denominator. Hence the strategy in Section 3.4.2 is also practical consideration, necessary for good performance. Then, consider the AP-loss:

where $Q(x)=\int_{-\infty}^{x}{\widetilde{H}(\upsilon)d\upsilon}$. Note that the AP-loss is upper bounded by the surrogate loss:

The learning model can be written as $\bm{x}=\bm{X}_{d}(\bm{\theta})$, where $d\in\mathcal{D}$ denotes the training data for one iteration and $D$ is the whole training set. Then, the modified error-driven algorithm is equivalent to gradient descent on surrogate loss $l(\bm{X}_{d^{(t)}}(\bm{\theta}),\bm{X}_{d^{(t)}}(\bm{\theta}^{(t)});\mathcal{P}_{d^{(t)}},\mathcal{N}_{d^{(t)}})$ at each step $t$. We further suppose below conditions are satisfied:

(1) For all $\hat{\bm{\theta}}$ and $d\in\mathcal{D}$, $l(\bm{X}_{d}(\bm{\theta}),\bm{X}_{d}(\hat{\bm{\theta}});\mathcal{P}_{d},\mathcal{N}_{d})$ is convex w.r.t $\bm{\theta}$.

(2) For all $d\in\mathcal{D}$, $\|\partial\bm{X}_{d}(\bm{\theta})/\partial\bm{\theta}\|$ is upper bounded by a constant $R$. Here $\|\cdot\|$ is the matrix norm induced by the 2-norm for vectors.

Remark 1. Note that these two conditions are satisfied if the learning model is linear.

A3.2. Bound of Accumulated Loss

where $l^{(t)}(\bm{\theta})$ denotes $l(\bm{X}_{d^{(t)}}(\bm{\theta}),\bm{X}_{d^{(t)}}(\bm{\theta}^{(t)});\mathcal{P}_{d^{(t)}},\mathcal{N}_{d^{(t)}})$ and $\bm{u}$ can be any vector of model weights. Then, let $\bm{\theta}=\bm{\theta}^{(t)}$ and compute the sum over $t=1\sim T$, we have:

where $\eta$ is the step size of gradient descent. Note that

Let $\eta=\delta/R^{2}$, rearrange and get the expression:

This entails the bound of surrogate loss $l$:

which implies the bound of AP-loss $\mathcal{L}_{AP}$:

As a special case, if there exists a $\bm{u}$ such that $l^{(t)}(\bm{u})=0$ for all $t$, then the accumulated AP-loss is bounded by a constant, which implies that convergence can be achieved with finite steps (similar to that of the separable case). Otherwise, with sufficiently large $T$, the average AP-loss mainly depends on $\frac{1}{T}\frac{8}{\delta}\sum_{t=1}^{T}l^{(t)}(\bm{u})$. This implies that the bound is meaningful if there still exists a sufficiently good solution $\bm{u}$ in such inseparable case.

A3.3. Offline Setting

The second last inequality is based on the fact that $(1+\sum_{j\in\mathcal{P}\cup\mathcal{N},j\neq i}H(X_{ij}(\bm{\theta}^{(t)})))$ are picked from $1\sim(|\mathcal{P}|+|\mathcal{N}|)$ without replacement (assume no ties; if ties exist, this inequality still holds). Combining the results from Equation 42 and Equation 41, we have:

Combining the results from Equation 44 and Equation 41, we have:

If $Z(\bm{u})$ is small, the bound in Equation 43 is active, otherwise the bound in Equation 45 is active. Consequently, we have:

where $\overline{\mathcal{L}_{AP}}$ denotes the average AP-loss, $\epsilon\rightarrow 0$ as $T$ increases.

A4. Consistency

When the activation function $L(\cdot)$ takes the form of softmax function and loss-augmented step function, our optimization algorithm can be expressed as the gradient descent algorithm on cross-entropy loss and hinge loss respectively.

Cross Entropy Loss: Consider the multi-class classification task. The outputs of neural network are $(x_{1},\ldots,x_{K})$ where $K$ is the number of classes, and the ground truth label is $y\in\{1,\ldots,K\}$. Using softmax as the activation function, we have:

Note that $g_{i}$ is “error-driven” with the desired output $\mathbf{1}_{y=i}$ and current output $L_{i}$. This form is consistent with our error-driven update scheme (c.f. Section 3.2.1 of the paper).

Hinge Loss: Consider the binary classification task. The output of neural network is $x$, and the ground truth label is $y\in\{1,2\}$. Define $(x_{1},x_{2})=(-x,x)$. Using loss-augmented step function as the activation function, we have:

where $H(\cdot)$ is the Heaviside step function. The hinge loss is:

There are two cases. If $y=i$, the gradient $g_{i}$ is “error-driven” with the desired output $1$ and current output $L_{i}$. If $y\neq i$, the gradient $g_{i}$ equals zero, since $x_{i}$ does not contribute to the loss. This form is consistent with our error-driven update scheme (c.f. Section 3.2.1 of the paper).

A5. Additional Experiments on SSD

We also evaluate the proposed AP-loss on another one-stage detector SSD . The models are trained on VOC2007 and VOC2012 trainval sets, and tested on VOC2007 test set. We use VGG-16 as the backbone model which is pre-trained on the ImageNet-1k classification dataset . We use conv4_3, conv7, conv8_2, conv9_2, conv10_2, conv11_2, conv12_2 to predict both location and their corresponding confidences. An additional convolution layer is added after conv4_3 to scale the feature. The associated anchors are the same as that designed in . In testing phase, the input image is fixed to 512$\times$512. For focal loss with SSD, we observe that the hyper-parameters $\gamma=1,\alpha=0.25$ lead to a much better performance than the original settings in which are $\gamma=2,\alpha=0.25$. Hence we evaluate the focal loss with new $\gamma$ and $\alpha$ in our experiments on SSD. Other details are similar to that in Section 4.1 of the paper.

A5.2. Results

The results are shown in Table 4 and Figure 7. Note that the AP-loss outperforms all the other losses at both the final state and various snapshot time points. Together with the results on RetinaNet in Section 4.2.2 of the paper, we observe the robustness of the proposed AP-loss, which performs much better than the other competing losses on different datasets (i.e. PASCAL VOC , MS COCO ) and different detectors (i.e. RetinaNet , SSD ). This demonstrates the effectiveness and strong generalization ability of our proposed approach.