Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams

Introduction

The prevalence of data streams in contemporary applications urges systems to learn in a continual fashion. Autonomous vehicles, sensory robot data, and video streaming yield never-ending streams of data, with abrupt changes in the observed environment behind every vehicle turn, robot entering a new room, or camera cut to a subsequent scene. Alas, learning from streaming data is far from trivial due to these changes, as learning schemes for neural networks have been developed assuming identically and independently distributed (iid) data. When this assumption is violated, catastrophic forgetting of previously acquired knowledge occurs , emanating a trade-off between neural stability to retain the current state of knowledge and neural plasticity to swiftly adopt the new knowledge . The search for this delicate balance is considered the main hurdle for continual learning systems.

Although a lot of progress has been established in the literature, often strong assumptions apply , impeding applicability for real-world systems. Static training and testing paradigms prevail, whereas a true continual learner should enable both training and testing simultaneously and independently. Therefore, we propose the two-agent learner-evaluator framework, redefining perspective on existing paradigms in the field. Within this framework, we introduce data incremental learning, enabling completely task-free learning and evaluation.

Furthermore, we introduce Continual Prototype Evolution (CoPE), a new online data incremental learner with prototypes perpetually representing the most salient features of the class population, shifting the catastrophic forgetting problem from the full network parameter space to the lower-dimensional latent space. As a first, our prototypes evolve continually with the data stream, enabling learning and evaluation at any point in time. Similar to representativeness heuristics in human cognition , the class prototypes are the cornerstone for nearest neighbor classification. Additionally, our system is robust to highly imbalanced data streams by the combination of replay with a balancing memory population scheme. We find batch information in the latent space to be crucial in the challenging non-stationary and online processing regime, which we incorporate in our novel pseudo-prototypical proxy loss.

The learner-evaluator framework

To date, the paradigms of task, class, and domain incremental learning dominate the continual learning literature. However, strong and differing assumptions often lead to confusion and overlap between implementations of these definitions. Furthermore, the concept of a static training and testing phase is still ubiquitous, whereas continual learning systems should enable both aspects continually and independently. Therefore, we propose a generalizing framework which disentangles the continual learning system into two agents: the learner and the evaluator. Figure 1 presents an overview of the framework.

The learning agent learns predicting function ${f_{\theta}:\mathcal{X}\rightarrow\mathcal{Y}}$ parameterized by $\theta$, mapping the input space $\mathcal{X}$ to the target output space $\mathcal{Y}$. The learner receives data samples $\left({\bf x}_{i},{\bf y}_{i}\right)$ from stream $S$ and has simultaneous access to the horizon $\mathcal{D}$, i.e. the observable subset of stream $S$ which can be processed for multiple iterations. Data sample $i$ is constituted by input feature ${\bf x}_{i}\in\mathcal{X}$ and corresponding (self-)supervision signal ${\bf y}_{i}$ for which the output space for classification is defined as a discrete set of observed classes $\mathcal{Y}_{i}\leftarrow\mathcal{Y}_{i-1}\cup\left\{{\bf y}_{i}\right\}$. To manage memory usage and to enable multiple updates and stochasticity in the optimization process, updates for $\theta$ are typically performed based on a small-scale processing batch $B\subseteq\mathcal{D}$. The data and size of the horizon $\mathcal{D}$ are determined by the specific setup or application, ranging from standard offline learning with $\mathcal{D}=S$ to online continual learning with $\mathcal{D}=B$. Furthermore, the learner might need additional resources after observing data from $B\subseteq\mathcal{D}$, such as stored samples or model copies, confined by the operational memory $\mathcal{M}$.

The evaluating agent acts independently from the learner by evaluating $f_{\theta}$ with horizon $\mathcal{D}_{eval}$ from the evaluation stream $S_{eval}$, with small-scale processing batches $B_{eval}\subseteq\mathcal{D}_{eval}$. This stream can contain yet unobserved concepts by the learner in $S$ to measure zero-shot performance. The framework provides leeway for the concept distributions in $S_{eval}$ being either static or dynamically evolving, determining how performance of the learner is measured. On the one hand, static concept distributions can measure the degree to which the knowledge of learned concepts is preserved, as commonly used in continual learning. On the other hand, evolving concept distributions measure performance for the current distribution in horizon $\mathcal{D}_{eval}$ only, where concepts might drift from their original representation, also known as concept drift . Evaluation can occur asynchronously on-demand or periodically with periodicity $\rho$ determining the resolution of the evaluation samples.

Task, class, and domain incremental learning are based on the composition of the learner’s observable stream subset in horizon $\mathcal{D}_{t}$. In all three scenarios, the learner receives data of the form $\left({\bf x}_{i},{\bf y}_{i},t_{i}\right)$ with $t_{i}$ the indicator for horizon $\mathcal{D}_{t=t_{i}}$ of the current task, set of classes, or domain. In task incremental learning the learner’s horizon spans all data of a given task, i.e. $\mathcal{D}_{t}=\left\{\left({\bf x}_{i},{\bf y}_{i},t_{i}\right)\in S\ |\ t=t_{i}\right\}$ . Moreover, this setup assumes the evaluator to receive data in the same form as the learner, hence having explicit access to $t_{i}$. This confines prediction to a task-specific subset of output nodes, also referred to as a head. In class incremental learning, the horizon is determined based on a subset of classes $\mathcal{Y}_{t}$, i.e. $\mathcal{D}_{t}=\left\{\left({\bf x}_{i},{\bf y}_{i},t_{i}\right)\in S\ |\ {\bf y}_{i}\in\mathcal{Y}_{t=t_{i}}\right\}$, for which the learner requires $t_{i}$ to identify the transition between subsequent class subsets $\mathcal{Y}_{t_{i}}$ in $S$ . However, the evaluator considers the entire output space without the need for identifier $t$. Domain incremental learning holds the same assumptions as class incremental learning, although with concepts drifting from one domain to the other for a typically fixed output space, exemplified by the widely used permuted-MNIST setup . To measure to what extend knowledge is preserved, continual learning assumes in general that for new $t_{i}$’s a fixed set of evaluation samples is added to the evaluator’s horizon $\mathcal{D}_{eval,t=t_{i}}$, drawn mutually exclusively from the same distribution as the learner’s horizon $\mathcal{D}_{t=t_{i}}$.

Data incremental learning is a more general paradigm we introduce to facilitate learning from any data stream, with no assumption on the horizon or the order of observing the data. In contrast to existing paradigms, when the learner observes horizon $\mathcal{D}$ of data stream $S$, data incremental learning does not disclose an identifier $t$. Consequently, there is no explicit indication to which subset of the stream is being observed in the horizon $\mathcal{D}$. Therefore, the learner either processes observed data directly in an online fashion with processing batch $B=\mathcal{D}$, or infers an implicit identifier $t$ from statistics in stream $S$. Similar to class and domain incremental learning, the evaluator operates without $t$ on the full output space. The data incremental learning paradigm endows continual learning systems with increased practical use, as real-world streaming applications typically lack supervision signal $t$. Moreover, making this identifier available may depend on arbitrary choices, such as how to delineate tasks, introducing unwanted bias.

Data incremental learning resembles online learning in its independence of identifier $t$. However, online learning makes the rigorous assumption of learning from an iid data stream, whereas data incremental learning disregards this assumption, and additionally generalizes beyond the online learning horizon ($\mathcal{D}=B$), by allowing to store larger subsets of the data stream ($\mathcal{D}\subset S$). The data incremental learning paradigm is also referred to as task-free or task-agnostic learning, but is here defined based on transition $t$ of the horizon, which is more general than the task-based definition generating possible ambiguity of how a task is specifically defined. Note that online continual learning mostly refers to the four continual learning paradigms with horizon $\mathcal{D}=B$, which can also be viewed as learning for a single epoch for each task, class subset, or domain. Table 1 compares data incremental learning with online learning and the three main continual learning paradigms.

Prior work

Continual learning systems are able to learn with limited resources from data streams prone to severe distribution shifts. The main body of works presumes the presence of tasks, which divide the data streams into large discrete subsets, and are indicated to the learner with a task identifier . Replay methods retain representative data for observed data distributions, currently unavailable in the learner’s horizon $\mathcal{D}$. The replay data is either obtained directly from operational memory $\mathcal{M}$ with stored samples or generated using generative models . GEM uses replay in a constraint optimization perspective to project gradients towards a local joint task optimum. iCaRL employs exemplars to distill knowledge to the learner from a previous model version, with new class exemplars stored in a queue to optimally represent the class mean in feature space. The prototypes are then used for nearest neighbor prediction by the evaluator, in the same vein as . Nonetheless, all three works strongly rely on task identifier $t$ for the learner, mostly unavailable for real-world data streams. Moreover, in both prototypical approaches the prototypes remain static between the given task transitions and become outdated. Consequently, before using the evaluator they exhaustively recalculate the prototypes based on all exemplars in memory. In contrast, our prototypes evolve in an online fashion with the data stream and remain representative for the continual learner and evaluator at all times.

Recent works focus on online data incremental learning (Section 2) in which the learner operates completely task-free. Reservoir is a replay baseline with strong potential to outperform continual learning methods . Samples are stored in replay memory $\mathcal{M}_{r}$ with probability $|\mathcal{M}_{r}|/n$, with $n$ the number of observed samples and fixed replay buffer size $|\mathcal{M}_{r}|$. MIR extends Reservoir sampling with a loss-based retrieval strategy, with the cost of additional forward passes and a model copy to attain the losses for a subset of samples. The Reservoir buffer population approximately follows the data stream distribution, severely deteriorating the performance of underrepresented tasks in imbalanced data streams, as shown in Section 6.2. An alternative memory population scheme is used in GSS by extending the GEM constraint optimization perspective to an instance-based level. GSS adds samples to the buffer based on their gradients, whereas GEM requires the number of tasks and the task transitions to divide memory equally over all tasks a priori. Another memory population is used in iCaRL, incrementally subdividing over all classes after learning a task by iteratively adding observed samples from the horizon $\mathcal{D}_{t}$ to optimally approximate the class mean in feature space. As this is computationally expensive, recent works explore other balancing schemes , where we propose a simple but effective class-based Reservoir scheme with uniform retrieval.

Another branch of works are parameter isolation methods , allocating parameters to subsets of the data. Several task incremental works assign parameters based on the task identifier . A new line of work instead focuses on data incremental model expansion. CURL enables task-free and unsupervised adaptation using a multi-component variational auto-encoder, with generative replay from a model copy avoiding forgetting in the current model. CN-DPM allocates data subsets to expert networks following a Dirichlet process mixture. In contrast to these capacity expansion based methods, CoPE evades unbound allocation of resources, as the memory and network capacity are fixed with the replay memory dynamically subdivided over categories occurring in the data stream. Note that new categories require an additional prototype, but these are only $d$-dimensional and therefore insignificant in size, and the set of categories is typically limited as well.

Besides the focus on continual learning in this work, our learner-evaluator framework generalizes to concept drift as well , for which we refer to an overview in . Further, in deep embedding learning most commonly pairs and triplets of samples are considered in contrastive losses, whereas other works use batch information in lifted structure embeddings or instance-wise softmax embeddings . These approaches fully depend on the batch size, whereas our pseudo-prototypical proxy loss aggregates both decoupled prototypes and the additional batch pseudo-prototypes to defy class interference in the latent space. Learning prototypical representations also shows promising results in few-shot learning .

Continual Prototype Evolution

Each observed class $c\in\mathcal{Y}$ is represented by a slowly progressing prototype $\bf p^{c}$ in operational memory $\mathcal{M}_{p}$. The nearest neighbor classifier finds the most similar prototype for the given query ${\bf x}_{i}$, predicting $c^{*}=\operatorname*{arg\,max}_{c\in\mathcal{Y}}\bf f_{i}^{T}p^{c}$. Similar to , the class-prototype approximates the center of mass in the latent space, which we formally justify in Section 4.4. The main crux with storing representations is to prevent them from becoming obsolete as the embedding network evolves. This is further complicated by the shifting data distributions in the non-stationary regime, incurring catastrophic forgetting. Experience replay from a buffer $\mathcal{M}_{r}$ is a well known approach to address this forgetting. In our setup the replayed exemplars provide additional information about the current state of the embedding space, enabling rehearsal to rectify approximation ${\bf p}^{c}$ to the true center of mass. Concretely, the sampled batch $B_{n}$ equals the horizon $\mathcal{D}$ from data stream $S$ and joins batch $B_{\mathcal{M}}$ of equal size from memory $\mathcal{M}_{r}$, constituting $B$ as $B_{n}\cup B_{\mathcal{M}}$. However, updating the prototypes by fully relying on features extracted from $B$ incurs an unstable optimization process as the representative prototypes depend on stochastic sampling of the class distributions. Therefore, we design the prototypes to evolve continually with a high momentum based update for each observed batch, aiming to stabilize the impetuous changes in the data stream:

with momentum parameter $\alpha\in$, the batch subset $B^{c}=\{({\bf x}_{i},y_{i}=c)\in B\}$ of class $c$, and ${\bf\bar{p}}^{c}$ the corresponding center of mass in latent space for the current batch. Due to triangle inequality ${\bf p}^{c}$ is no longer unit length and requires to be L2-normalized after the update in Eq.(1). We empirically validate the effectiveness of high momentum with $\alpha\approx 1$ in the ablation study in Appendix D.

2 Balanced replay

Similar to , the total buffer size $|\mathcal{M}_{r}|$ is equally divided over the number of observed classes $|\mathcal{Y}|$ in a dynamic fashion. This scheme ensures consistent buffer capacity for all classes, making memory allocation independent of the data stream characteristics. As $S$ is typically highly imbalanced in real-world scenarios, this memory scheme prevents classes to be eradicated from the buffer and assumes equal importance to represent each class at all times. Consequently, random retrieval from the buffer resembles sampling an iid replay batch. Furthermore, each class-specific replay memory $\mathcal{M}^{c}_{r}$ can simply capture a random subset of its parent class distribution to approximate its center of mass. This avoids computationally expensive herding techniques as in iCaRL , which would require recalculation of the feature means for each change of the memory size or network parameters.

3 Pseudo-Prototypical Proxy loss

with the assumption of independence between ${\bf x}_{i}^{c}$ and ${\bf x}_{j}^{k}$ being recognized as $c$. We define the expected posterior probabilities for the attractor and repellor sets of instance ${\bf x}_{i}^{c}$ respectively as

where temperature $\tau$ controls the concentration level of the distribution , assuming a cosine similarity metric ${\bf f}_{i}^{T}{\bf f}_{j}$ with vectors normalized to unit length. We reformulate the objective in Eq.(2) as loss function $\mathcal{L}$ by taking the negative log-likelihood and summing over all the instances in $B$, which approximates the true joint probability with assumed independent pairs in the batch:

4 Optimal prototypes

We update prototypes to approximate the mean of the parent distribution in Eq.(1). This assumption is optimal for Bregman divergences for which the cluster mean is shown to have minimal distance to its population . This Bregman divergence is defined for a differentiable, strictly convex function $\varphi$ as

for which the squared Euclidean distance with $\varphi({\bf f})=||{\bf f}||^{2}$ is a canonical example. The squared Euclidean distance is proportional to the cosine distance with vectors normalized to unit length: ${\frac{1}{2}||{\bf f}_{i}-{\bf f}_{j}||^{2}=1-\cos\angle({\bf f}_{i},{\bf f}_{j})}$. As the PPP-loss for Eq.(5) requires a similarity measure instead of a distance measure, we employ the complementary normalized cosine similarity $\cos\angle({\bf f}_{i},{\bf f}_{j})={\bf f}_{i}^{T}{\bf f}_{j}$ with $||{\bf f}_{i}||=||{\bf f}_{j}||=1$. Besides the desirable cluster-mean property of its complement, this metric is also efficient for calculating the full batch similarity matrix using matrix multiplication libraries.

Experiments

This work examines five balanced data streams and 15 highly imbalanced variants based on Split-MNIST, Split-CIFAR10 and Split-CIFAR100, from which two low-capacity balanced setups are discussed in Appendix E. The learner is presented a data stream $S$, constituted by a sequence of tasks, each delineated by a subset of classes from the original dataset. Although the learner in CoPE is completely ignorant to the notion of task, this setup enables comparing to methods requiring task boundaries such as GEM and iCaRL. The evaluator uses a held-out dataset of static concepts in $S_{eval}$, evaluating with the subset of seen concepts $\mathcal{Y}$ in $\mathcal{D}_{eval}$ using the accuracy metric. The CoPE learner processes data online with $B_{n}=\mathcal{D}$ in the data incremental setup. To enable fair comparison, this allows per-task processing of 1 epoch for methods requiring task boundaries with $B\subset\mathcal{D}_{t}$. We use vanilla stochastic gradient descent with a limited processing batch size $|B_{n}|$ of 10 as in . All results are averaged over 5 different network initializations. Appendix details the full setup with additional experiments.

Balanced data streams contain a similar amount of data per task. We consider three benchmarks. First, Split-MNIST constitutes the MNIST handwritten digit recognition dataset with 60k training samples, split into 5 tasks according to pairs of incrementing digits. Second, Split-CIFAR10 considers the CIFAR10 dataset, subdivided into 5 tasks with 2 labels each, where each task entails 10k training samples. Third, Split-CIFAR100 is a variant of the CIFAR dataset with 100 different classes. The 50k training samples are subdivided in 20 tasks of 2.5k samples as in . For all datasets the evaluator considers the entire original test subset for $S_{eval}$.

Imbalanced data streams introduce a more realistic scenario without equality assumptions on the task durations in $S$. This addresses a common weakness in the literature mostly balancing the data streams artificially. Besides the imbalanced Split-MNIST setup , we introduce two novel and more challenging benchmarks based on Split-CIFAR10 and Split-CIFAR100, where data stream $S$ comprises significantly more data in task $T_{i}$, denoted by $S(T_{i})$. Split-MNIST and Split-CIFAR10 have respectively 2k and 4k samples in $T_{i}$, whereas tasks $T_{j}$ for $j\neq i$ contain factor $10$ less data for five variants $S(T_{i}),\ \forall i\in\{1,...,5\}$. Split-CIFAR100 defines $T_{i}$ with 2.5k samples and 1k for the remaining tasks, with variants $i\in\{1,5,10,15,20\}$.

Architectures. MNIST setups use an MLP with 2 hidden layers of 400 units with $2$k memories for the balanced setup as in , and 100 units with $|\mathcal{M}_{r}|=0.3$k for the imbalanced setup as in . CIFAR setups use a slim version of Resnet18 with a $1$k memory size for CIFAR10 , and 5k for CIFAR100 .

Methods compared to CoPE entail 11 baselines, with details on prior work discussed in Section 3. The upper reference point for performance when relaxing the challenging non-iid property of continual learning is set by iid-online & iid-offline. The learner shuffles the full data stream $S$ to ensure the iid property, for which iid-online trains a single epoch and iid-offline multiple epochs. In contrast, the finetune learner considers non-iid data stream $S$ sequentially, but optimizes solely for the new batch which typically results in worst-case catastrophic forgetting. CoPE-CE is a reference point for the merits of a prototypical approach by solely using the CoPE memory and sampling scheme, but with a typical cross-entropy loss and softmax classifier. GEM and iCaRL are standard replay methods considered in a class incremental setup, with the learner requiring knowledge about task boundaries. For online data incremental learning, we consider the reservoir, MIR and greedy GSS replay baselines, with CURL and CN-DPM instead relying on model expansion.

Results and discussion

The results for the three balanced data streams in Table 2 consistently report state-of-the-art results for CoPE. The difficulty for learning online is reflected in the discrepancy of performance between iid-offline and iid-online, indicating increasing difficulty for a minimal $2\%$ discrepancy for Split-MNIST, raised to $20\%$ for Split-CIFAR10, and culminating to $30\%$ in Split-CIFAR100. For Split-MNIST the gap with iid-online performance is closed by $0.7\%$ compared to main competitors GEM and DN-CPM. Furthermore, in the more challenging Split-CIFAR10 setup CoPE significantly increases the gained margin to $3.7\%$. In the most challenging Split-CIFAR100, CN-DPM, Reservoir and MIR are able to perform on par with the iid-online baseline, however, CoPE establishes an improvement of at least $1.5\%$ over all four baselines.

Compared to balanced replay with standard cross-entropy (CoPE-CE), the prototypical approach (CoPE) proves effective with significant gains of $2.2\%$, $9.2\%$ and $3.3\%$ respectively over the three benchmarks. Except for GEM in Split-MNIST, class incremental learning methods GEM and iCaRL are not competitive in the online setting and additionally require from the setup to reveal an identifier $t$ to the learner. From the expansion-based methods DN-CPM is competitive, whereas CURL is more suited for unsupervised learning and lags behind. Although Reservoir and extension MIR perform on par with iid-online for Split-CIFAR100, the imbalanced experiments in Section 6.2 show that full reservoir-based population of the buffer strongly relies on the assumption of equally sized tasks, which is unlikely to occur in real-world data streams.

Buffer size ablation study in Figure 3 shows CoPE to prevail over all sizes of replay buffer $\mathcal{M}_{r}$ compared to other replay methods, extending robustness to low capacity regimes. Although iCaRL shows competitive results for low capacity, CoPE scales with growing capacity leading to significantly outperforming iCaRL with $11\%$ in Split-MNIST ($2k$) and Split-CIFAR100 ($5k$), and $17\%$ in Split-CIFAR10 ($2k$).

2 Imbalanced data streams

Results for the highly imbalanced data stream benchmarks are reported in Figure 4. CoPE significantly outperforms all baselines in the three scenarios, with low standard deviation for the 15 variants indicating robustness over a wide spectrum of imbalanced sequences. GSS outperforms Reservoir and MIR for Split-MNIST, in correspondence with results in , whereas loss-based retrieval in MIR has significant gains for the challenging Split-CIFAR100 setting. However, CoPE surpasses both GSS and MIR for all three benchmarks, and on top of that operates profusely more resource efficient as discussed in Appendix C. The balancing memory scheme in CoPE-CE highly improves Reservoir over imbalanced Split-MNIST and Split-CIFAR10 variants with $10.8\%$ and $3.4\%$ respectively, and performs on par for Split-CIFAR100 where balancing over 100 classes with limited batch size proves more difficult. Although CoPE and CoPE-CE share memory and retrieval schemes, the prototypical CoPE surpasses the cross-entropy based CoPE-CE with $4.0\%$, $2.9\%$ and $6.7\%$ respectively on the three benchmarks, indicating the merits of the PPP-loss and continually evolving prototypes. Figure 5 compares the CoPE and CoPE-CE confusion matrices at the end of learning, with CoPE-CE exhibiting high plasticity for the later observed classes, whereas CoPE better preserves the recall over early learned classes, hence effectively alleviating catastrophic forgetting.

3 PPP-loss analysis

In the challenging setting for online processing of non-iid data streams, the PPP-loss exploits information in the small processing batch $B$, introducing pseudo-prototypes $\hat{\bf p}$ on top of the prototypes. This leads to questioning to what extent the pseudo-prototypes actually contribute to the quality of the embedding, and how this relates to the batch size. We examine both inquiries in Table 3 for the three balanced data streams by comparing inclusion and exclusion of the pseudo-prototypes $\hat{\bf p}$ in the PPP-loss, and extending the batch size $|B_{n}|$. First, including the pseudo-prototypes significantly improves overall performance, especially for the harder CIFAR-based data streams. Although both setups use batch information to update the prototypes following Eq.(1), it seems crucial to use additional pseudo-prototypes in the PPP-loss to improve latent space quality. Second, results for smaller batch sizes of 10 and 20 are very similar, and deteriorate towards increasing sizes. The PPP-loss implements the expectation over the prototype and the pseudo-prototypes, assuming uniform distribution in Eq.(3) and Eq.(4). Although this assumption impedes significance of the prototype for increasingly higher batch sizes, it results in ideal robustness for small online processing batches. Small batches maintain the additional benefit of more frequent prototype and network parameter updates for the same amount of processed data.

Conclusion

The novel two-agent learner-evaluator framework introduced a new perspective on current paradigms in continual learning. To overcome the standard paradigm of static training and testing phases, we explicitly model continual optimization and evaluation in the learner and evaluator agents respectively. We formalized the required resources as the horizon $\mathcal{D}$, containing the simultaneously available data of the data stream, and the operational memory $\mathcal{M}$ for operation of the learning algorithm. Transitions in the horizon $\mathcal{D}_{t}\rightarrow\mathcal{D}_{t+1}$ enable a uniform differentiation between existing incremental learning paradigms and the horizon size encloses the range from online ($\mathcal{D}=B$) to offline ($\mathcal{D}=S$) learning. Using the framework, we defined the task-free data incremental learning paradigm, requiring no additional information on the horizon’s identifier $t$ for both the learner and evaluator. In this challenging setup, we proposed Continual Prototype Evolution (CoPE) as a prototypical solution to learn online from non-stationary data streams. As a first, CoPE prevents the prototypes becoming obsolete in an ever evolving representation space, while using the prototypes to combat catastrophic forgetting. The three main components, continually evolving prototypes, a novel pseudo-prototypical proxy loss, and an efficient balancing replay scheme are proven remarkably effective over 11 baselines in both balanced and highly imbalanced benchmarks. We hope to encourage further research in online data incremental learning.

References