Convolutional Neural Networks Applied to House Numbers Digit Classification

. Introduction

Character recognition in documents can be considered a solved task for computer vision, whether handwritten or typed. It is however a harder problem in the context of complex natural scenes like photographs where the best current methods lag behind human performance, mainly due to non-contrasting backgrounds, low resolution, de-focused and motion-blurred images and large illumination differences (Figure 1).

recently introduced a new digit classification dataset of house numbers extracted from street level images. It is similar in format to the popular MNIST dataset (10 digits, 32x32 inputs), but an order of magnitude bigger (600,000 labeled digits), contains color information and various natural backgrounds.

Previous approaches in classifying characters and digits from natural images used multiple hand-crafted features and template-matching . In contrast, ConvNets learn features all the way from pixels to the classifier. demonstrated the superiority of learned features over hand-designed ones. Such superiority was also previously shown among others in a traffic sign classification challenge where two independent teams obtained the best performance against various other approaches using ConvNets . also show superior results with unsupervised learning, we however only report results with fully-supervised training. We obtain a $4.25$ points improvement in accuracy (with $94.85\%$ accuracy) over the previous state-of-the-art of $90.6\%$. We use the traditional ConvNet architecture augmented with different pooling methods and with multi-stage features . This work was implemented with the EBLearn http://eblearn.sf.net C++ open-source framework .

Architecture

The ConvNet architecture is composed of repeatedly stacked feature stages. Each stage contains a convolution module, followed by a pooling/subsampling module and a normalization module. While traditional pooling modules in ConvNet are either average or max poolings, we use an Lp pooling here. The normalization module is subtractive only as opposed to subtractive and divisive, i.e. the mean value of each neighborhood is subtracted to the output of each stage (but not divided by the standard deviation as it decreases performance with this dataset). Finally, multi-stage features are also used as opposed to single-stage features.

Lp pooling is a biologically inspired pooling layer modelled on complex cells who’s operation can be summarized in equation (1), where $G$ is a Gaussian kernel, $I$ is the input feature map and $O$ is the output feature map. It can be imagined as giving an increased weight to stronger features and suppressing weaker features. Two special cases of Lp pooling are notable. $P=1$ corresponds to a simple Gaussian averaging, whereas $P=\infty$ corresponds to max-pooling (i.e only the strongest signal is activated). Lp-pooling has been used previously in and a theoretical analysis of this method is described in .

Figure 2 demonstrates a simple example of L2-pooling.

2 Multi-Stage Features

Multi-Stage features (MS) are obtained by branching out outputs of all stages into the classifier (Figure 3). They provide richer representations compared to Single-Stage features (SS) by adding complementary information such as local textures and fine details lost by higher levels. MS features have consistently improved performance in other work and in this work as well (Figure 4). However we observe minimal gains on this dataset compared to other types of objects such as pedestrians and traffic signs (Table 1). The likely explanation for this observation is that gains are correlated to the amount of texture and multi-scale characteristics of the objects of interest.

. Experiments

The SVHN classification dataset contains 32x32 images with 3 color channels. The dataset is divided into three subsets: train set, extra set and test set. The extra set is a large set of easy samples and train set is a smaller set of more difficult samples. Since we are given no information about how the sampling of these images was done, we assume a random order to construct our validation set. We compose our validation set with $2/3$ from training samples (400 per class) and $1/3$ from extra samples (200 per class), yielding a total of 6000 samples. This distribution allows to measure success on easy samples but puts more emphasis on difficult ones.

Samples are pre-processed with a local contrast normalization (with a 7x7 kernel) on the Y channel of the YUV space followed by a global contrast normalization over each channel. No sample distortions were used to improve invariance.

2 Architecture Details

The ConvNet has 2 stages of feature extraction and a two-layer non-linear classifier. The first convolution layer produces 16 features with 5x5 convolution filters while the second convolution layer outputs 512 features with 7x7 filters. The output to the classifier also includes inputs from the first layer, which provides local features/motifs to reinforce the global features. The classifier is a 2-layer non-linear classifier with 20 hidden units. Hyper-parameters such as learning rate, regularization constant and learning rate decay were tuned on the validation set. We use stochastic gradient descent as our optimization method and shuffle our dataset after each training iteration.

For the pooling layers, we compare Lp-pooling for the value $p=1,2,4,8,12,16,32,\infty$ on the validation set and use the best performing pooling on the final testing. The performance of different pooling methods on the validation set can be seen in Figure 5. Insights from tell us that the optimal value of $p$ varies for different input spaces and there is no single globally optimal value for $p$. For our validation data, we observe that $p=2,4,12$ give the best performance ($5.62\%,5.64\%$ and $5.61\%$ respectively). Max-pooling, which corresponds to $p=\infty$ yielded a validation error rate of $7.57\%$.

Results & Future Work

Our experiments demonstrate a clear advantage of Lp pooling with $1<p<\infty$ on this dataset in validation (Figure 5) and test (Average pooling is 3.58 points inferior to L2 pooling in Table 2). With L4 pooling, we obtain a state-of-the-art performance on the test set with an accuracy of 94.85% compared to the previous best of 90.6% (Table 2). We also show that using multi-stage features gives only a slight increase in performance, compared to the performance increase seen in other vision applications.

Additionally, it is important to note that our approach is trained fully supervised only, whereas the best previous methods are unsupervised learning methods (k-means, auto-encoders). We shall, in the future, run experiments with unsupervised learning, to compare the accuracy improvement that can be attributed to supervision. Figure 6 shows the validation samples with highest energy. Many of these seem to exhibit large scale variations, future work could address this problem by introducing artificial scale deformations during training.

References