Evolutionary Generative Adversarial Networks

Introduction

Generative adversarial networks (GAN) are one of the main groups of methods used to learn generative models from complicated real-world data. As well as using a generator to synthesize semantically meaningful data from standard signal distributions, GANs (GAN and its variants) train a discriminator to distinguish real samples in the training dataset from fake samples synthesized by the generator. As the confronter, the generator aims to deceive the discriminator by producing ever more realistic samples. The training procedure continues until the generator wins the adversarial game; that is, the discriminator cannot make a better decision than randomly guessing whether a particular sample is fake or real. GANs have recently been successfully applied to image generation , image editing , video prediction, and many other tasks .

Although GANs already produce visually appealing samples in various applications, they are often difficult to train. If the data distribution and the generated distribution do not substantially overlap (usually at the beginning of training), the generator gradients can point to more or less random directions or even result in the vanishing gradient issue. GANs also suffer from mode collapse, i.e., the generator assigns all its probability mass to a small region in the space . In addition, appropriate hyper-parameters (e.g., learning rate and updating steps) and network architectures are critical configurations in GANs. Unsuitable settings reduce GAN’s performance or even fail to produce any reasonable results.

Many recent efforts on GANs have focused on overcoming these training difficulties by developing various adversarial training objectives. Typically, assuming the optimal discriminator for the given generator is learned, different objective functions of the generator aim to measure the distance between the data distribution and the generated distribution under different metrics. The original GAN uses Jensen-Shannon divergence as the metric. A number of metrics have been introduced to improve GAN’s performance, such as least-squares , absolute deviation , Kullback-Leibler divergence , and Wasserstein distance . However, according to both theoretical analyses and experimental results, minimizing each distance has its own pros and cons. For example, although measuring Kullback-Leibler divergence largely eliminates the vanishing gradient issue, it easily results in mode collapse . Likewise, Wasserstein distance greatly improves training stability but can have non-convergent limit cycles near equilibrium .

To exploit the advantages and suppress the weaknesses of different metrics (i.e., GAN objectives), we devise a framework that utilizes different metrics to jointly optimize the generator. In doing so, we improve both the training stability and generative performance. We build an evolutionary generative adversarial network (E-GAN), which treats the adversarial training procedure as an evolutionary problem. Specifically, a discriminator acts as the environment (i.e., provides adaptive loss functions) and a population of generators evolve in response to the environment. During each adversarial (or evolutionary) iteration, the discriminator is still trained to recognize real and fake samples. However, in our method, acting as parents, generators undergo different mutations to produce offspring to adapt to the environment. Different adversarial objective functions aim to minimize different distances between the generated distribution and the data distribution, leading to different mutations. Meanwhile, given the current optimal discriminator, we measure the quality and diversity of samples generated by the updated offspring. Finally, according to the principle of “survival of the fittest”, poorly-performing offspring are removed and the remaining well-performing offspring (i.e., generators) are preserved and used for further training.

Based on the evolutionary paradigm to optimize GANs, the proposed E-GAN overcomes the inherent limitations in the individual adversarial training objectives and always preserves the best offspring produced by different training objectives (i.e., mutations). In this way, we contribute to progress in and the success of GANs. Experiments on several datasets demonstrate the advantages of integrating different adversarial training objectives and E-GAN’s convincing performance for image generation.

Related Works

In this section, we first review some previous GANs devoted to reducing training instability and improving the generative performance. We then briefly summarize some evolutionary algorithms on deep neural networks.

Generative adversarial networks (GAN) provides an excellent framework for learning deep generative models, which aim to capture probability distributions over the given data. Compared to other generative models, GAN is easily trained by alternately updating a generator and a discriminator using the back-propagation algorithm. In many generative tasks, GANs (GAN and its variants) produce better samples than other generative models .

However, some problems still exist in the GANs training process. In the original GAN, training the generator was equal to minimizing the Jensen-Shannon divergence between the data distribution and the generated distribution, which easily resulted in the vanishing gradient problem. To solve this issue, a non-saturating heuristic objective (i.e., ‘$-\log D$ trick’) replaced the minimax objective function to penalize the generator . Then, and designed specified network architectures (DCGAN) and proposed several heuristic tricks (e.g., feature matching, one-side label smoothing, virtual batch normalization) to improve training stability. Meanwhile, energy-based GAN and least-squares GAN improved training stability by employing different training objectives. Although these methods partly enhanced training stability, in practice, the network architectures and training procedure still required careful design to maintain the discriminator-generator balance. More recently, Wasserstein GAN (WGAN) and its variant WGAN-GP were proposed to minimize the Wasserstein-1 distance between the generated and data distributions. Since the Wasserstein-1 distance is continuous everywhere and differentiable almost everywhere under only minimal assumptions , these two methods convincingly reduce training instability. However, to measure the Wasserstein-1 distance between the generated distribution and the data distribution, they are asked to enforce the Lipschitz constraint on the discriminator (aka critic), which may restrict critic capability and result in some optimization difficulties .

2 Evolutionary Algorithms

Over the last twenty years, evolutionary algorithms have achieved considerable success across a wide range of computational tasks including modeling, optimization and design . Inspired by natural evolution, the essence of an evolutionary algorithm is to equate possible solutions to individuals in a population, produce offspring through variations, and select appropriate solutions according to fitness .

Recently, evolutionary algorithms have been introduced to solve deep learning problems. To minimize human participation in designing deep algorithms and automatically discover such configurations, there have been many attempts to optimize deep learning hyper-parameters and design deep network architectures through an evolutionary search . Evolutionary algorithms have also demonstrated their capacity to optimize deep neural networks . Moreover, proposed a novel evolutionary strategy as an alternative to the popular MDP-based reinforcement learning (RL) techniques, achieving strong performance on RL benchmarks. Last but not least, an evolutionary algorithm was proposed to compress deep learning models by automatically eliminating redundant convolution filters .

Method

In this section, we first review the GAN formulation. Then, we introduce the proposed E-GAN algorithm. By illustrating E-GAN’s mutations and evaluation mechanism, we further discuss the advantage of the proposed framework. Finally, we conclude with the entire E-GAN training process.

GAN, first proposed in , studies a two-player minimax game between a discriminative network $D$ and a generative network $G$. Taking noisy sample $z\sim p(z)$ (sampled from a uniform or normal distribution) as the input, the generative network $G$ outputs new data $G(z)$, whose distribution $p_{g}$ is supposed to be close to that of the data distribution $p_{\text{data}}$. Meanwhile, the discriminative network $D$ is employed to distinguish the true data sample $x\sim p_{\text{data}}(x)$ and the generated sample $G(z)\sim p_{g}(G(z))$. In the original GAN, this adversarial training process was formulated as:

The adversarial procedure is illustrated in Fig. 1 (a). Most existing GANs perform a similar adversarial procedure in different adversarial objective functions.

2 Evolutionary Algorithm

In contrast to conventional GANs, which alternately update a generator and a discriminator, we devise an evolutionary algorithm that evolves a population of generator(s) $\{G\}$ in a given environment (i.e., the discriminator $D$). In this population, each individual represents a possible solution in the parameter space of the generative network $G$. During the evolutionary process, we expect that the population gradually adapts to its environment, which means that the evolved generator(s) can generate ever more realistic samples and eventually learn the real-world data distribution. As shown in Fig. 1 (b), during evolution, each step consists of three sub-stages:

Variation: Given an individual $G_{\theta}$ in the population, we utilize the variation operators to produce its offspring $\{G_{\theta_{1}},G_{\theta_{2}},\cdots\}$. Specifically, several copies of each individual—or parent—are created, each of which are modified by different mutations. Then, each modified copy is regarded as one child.

Evaluation: For each child, its performance—or individual’s quality—is evaluated by a fitness function $\mathcal{F}(\cdot)$ that depends on the current environment (i.e., discriminator $D$).

Selection: All children will be selected according to their fitness value, and the worst part is removed—that is, they are killed. The rest remain alive (i.e., free to act as parents), and evolve to the next iteration.

After each evolutionary step, the discriminative network $D$ (i.e., the environment) is updated to further distinguish real samples $x$ and fake samples $y$ generated by the evolved generator(s), i.e.,

Thus, the discriminative network $D$ (i.e., the environment) can continually provide the adaptive losses to drive the population of generator(s) evolving to produce better solutions. Next, we illustrate and discuss the proposed variation (or mutation) and evaluation operators in detail.

3 Mutations

We employ asexual reproduction with different mutations to produce the next generation’s individuals (i.e., children). Specifically, these mutation operators correspond to different training objectives, which attempt to narrow the distances between the generated distribution and the data distribution from different perspectives. In this section, we introduce the mutations used in this workMore mutation operations were tested, but the mutation approaches described already delivered a convincing performance.. To analyze the corresponding properties of these mutations, we suppose that, for each evolutionary step, the optimal discriminator $D^{*}(x)=\frac{p_{data}(x)}{p_{data}(x)+p_{g}(x)}$, according to Eq. (2), has already been learned .

The minimax mutation corresponds to the minimax objective function in the original GAN:

According to the theoretical analysis in , given the optimal discriminator $D^{*}$, the minimax mutation aims to minimize the Jensen-Shannon divergence (JSD) between the data distribution and the generated distribution. Although the minimax game is easy to explain and theoretically analyze, its performance in practice is disappointing, a primary problem being the generator’s vanishing gradient. If the support of two distributions lies in two manifolds, the JSD will be a constant, leading to the vanishing gradient . This problem is also illustrated in Fig. 2. When the discriminator rejects generated samples with high confidence (i.e., $D(G(z))\to 0$), the gradient tends to vanishing. However, if the generated distribution overlaps with the data distribution, meaning that the discriminator cannot completely distinguish real from fake samples, the minimax mutation provides effective gradients and continually narrows the gap between the data distribution and the generated distribution.

3.2 Heuristic mutation

Unlike the minimax mutation, which minimizes the log probability of the discriminator being correct, the heuristic mutation aims to maximize the log probability of the discriminator being mistaken, i.e.,

Compared to the minimax mutation, the heuristic mutation will not saturate when the discriminator rejects the generated samples. Thus, the heuristic mutation avoids vanishing gradient and provides useful generator updates (Fig. 2). However, according to , given the optimal discriminator $D^{*}$, minimizing the heuristic mutation is equal to minimizing $[KL(p_{g}||p_{\text{data}})-2JSD(p_{g}||p_{\text{data}})]$, i.e., inverted KL minus two JSDs. Intuitively, the JSD sign is negative, which means pushing these two distributions away from each other. In practice, this may lead to training instability and generative quality fluctuations .

3.3 Least-squares mutation

The least-squares mutation is inspired by LSGAN , where the least-squares objectives are utilized to penalize its generator to deceive the discriminator. In this work, we formulate the least-squares mutation as:

As shown in Fig. 2, the least-squares mutation is non-saturating when the discriminator can recognize the generated sample (i.e., $D(G(z))\to 0$). When the discriminator output grows, the least-squares mutation saturates, eventually approaching zero. Therefore, similar to the heuristic mutation, the least-squares mutation can avoid vanishing gradient when the discriminator has a significant advantage over the generator. Meanwhile, compared to the heuristic mutation, although the least-squares mutation will not assign an extremely high cost to generate fake samples, it will also not assign an extremely low cost to mode dropping demonstrated that the heuristic objective suffers from mode collapse since $KL(p_{g}||p_{\text{data}})$ assigns a high cost to generating fake samples but an extremely low cost to mode dropping., which partly avoids mode collapse .

Note that, different from GAN-minimax and GAN-heuristic, LSGAN employs a different loss (‘least-squares’) from ours (Eq. (2)) to optimize the discriminator. Yet, as shown in the Supplementary Material, the optimal discriminator of LSGAN is equivalent to ours. Therefore, although we employ only one discriminator as the environment to distinguish real and generated samples, it is sufficient to provide adaptive losses for mutations described above.

4 Evaluation

In an evolutionary algorithm, evaluation is the operation of measuring the quality of individuals. To determine the evolutionary direction (i.e., individuals’ selection), we devise an evaluation (or fitness) function to measure the performance of evolved individuals (i.e., children). Typically, we focus on two generator properties: 1) the quality and 2) the diversity of generated samples. First, we simply feed generator produced images into the discriminator $D$ and observe the average value of the output, which we name the quality fitness score:

Note that discriminator $D$ is constantly upgraded to be optimal during the training process, reflecting the quality of generators at each evolutionary (or adversarial) step. If a generator obtains a relatively high quality score, its generated samples can deceive the discriminator and the generated distribution is further approximate to the data distribution.

Besides generative quality, we also pay attention to the diversity of generated samples and attempt to overcome the mode collapse issue in GAN optimization. Recently, proposed a gradient-based regularization term to stabilize the GAN optimization and suppress mode collapse. Through their observation, when the generator collapses to a small region, the discriminator will subsequently label collapsed points as fake with obvious countermeasure (i.e., big gradients).

We employ a similar principle to evaluate generator optimization stability and generative diversity. Formally, the diversity fitness score is defined as:

The log gradient value of updating $D$ is utilized to measure the diversity of generated samples. If the updated generator obtains a relatively high diversity score, which corresponds to small discriminator gradients, its generated samples tend to spread out enough, to avoid the discriminator has obvious countermeasures. Thus, the mode collapse issue can be suppressed and the discriminator will change smoothly, which helps to improve the training stability.

Based on the aforementioned two fitness scores, we can finally give the evaluation (or fitness) function of the proposed evolutionary algorithm:

where $\gamma\geq 0$ balances two measurements: generative quality and diversity. Overall, a relatively high fitness score $\mathcal{F}$, leads to higher training efficiency and better generative performance.

5 E-GAN

Having introduced the proposed evolutionary algorithm and corresponding mutations and evaluation criteria, the complete E-GAN training process is concluded in Algorithm 1. Overall, in E-GAN, generators $\{G\}$ are regarded as an evolutionary population and discriminator $D$ acts as an environment. For each evolutionary step, generators are updated with different objectives (or mutations) to accommodate the current environment. According to the principle of “survival of the fittest”, only well-performing children will survive and participate in future adversarial training. Unlike the two-player game with a fixed and static adversarial training objective in conventional GANs, E-GAN allows the algorithm to integrate the merits of different adversarial objectives and generate the most competitive solution. Thus, during training, the evolutionary algorithm not only largely suppresses the limitations (vanishing gradient, mode collapse, etc.) of individual adversarial objectives, but it also harnesses their advantages to search for a better solution.

Experiments

To evaluate the proposed E-GAN, in this section, we run and analyze experiments on several generation tasks.

We evaluate E-GAN on two synthetic datasets and three image datasets: CIFAR-10 , LSUN bedroom , and CelebA . For all of these tasks, the network architectures are based on DCGAN and are briefly introduced here, more details can be found in the Supplementary Material. We use the default hyper-parameter values listed in Algorithm 1 for all experiments. Note that the number of parents $n_{p}$ is set as 1, which means only one (i.e., the best) child is retained in each evolutionary step. On the one hand, this reduces E-GAN’s computational cost, thereby accelerating training. On the other, our experiments show that E-GAN already achieves impressive performance and stability even with only one survivor at each step. Furthermore, all experiments were trained on Nvidia GTX 1080Ti GPUs. To train a model for $64\times 64$ images using the DCGAN architecture cost around 30 hours on a single GPU.

2 Synthetic Datasets and Mode Collapse

In the first experiment, we adopt the experimental design proposed in , which trains GANs on 2D Gaussian mixture distributions. The mode collapse issue can be accurately measured on these synthetic datasets, since we can clearly observe the data distribution and the generated distribution. As shown in Fig. 3, we employ two challenging distributions to evaluate E-GAN, a mixture of 8 Gaussians arranged in a circle and a mixture of 25 Gaussians arranged in a grid.We obtain both 2D distributions and network architectures from the code provided in .

We first compare the proposed evolutionary adversarial training framework with one using an individual adversarial objective (i.e., conventional GANs). We train each method 50K iterations and report the KDE plots in Fig. 3. The results show that all of the individual adversarial objectives suffer from mode collapse to a greater or lesser degree. However, by combining different objectives in our evolution framework, model performance is largely improved and can accurately fit the target distributions. This further demonstrates, during the evolutionary procedure, the proposed evaluation mechanism can recognize well-performing updatings (i.e., offspring), and promote the population to a better evolutionary direction.

3 CIFAR-10 and Inception Score

When evaluating a GAN model, sample quality and convergence speed are two important criteria. We train different GANs on CIFAR-10 and plot inception scores over the course of training (Fig. 4-left, middle). The same network architecture based on DCGAN is used in all methods.

As shown in Fig. 4-left, E-GAN can get higher inception score with less training steps. Meanwhile, E-GAN also shows comparable stability when it goes to convergence. By comparison, conventional GANs expose their different limitations, such as instability at convergence (GAN-Heuristic), slow convergence (GAN-Least square) and invalid (GAN-minimax). As mentioned above, different objectives aim to measure the distance between the generated and data distributions under different metrics which have different pros and cons. Here, utilizing the evolutionary framework, E-GAN not only overcomes the limitations of these individual adversarial objectives, but it also outperforms other GANs (the WGAN and its improved variation WGAN-GP). Furthermore, although E-GAN takes more time for each iteration, it achieves comparable convergence speed in terms of wall-clock time (Fig. 4-middle).

During training E-GAN, we recorded the selected objective in each evolutionary step (Fig. 4-right). At the beginning of training, the heuristic mutation and the least-square mutation are selected more frequently than the minimax mutation. It may due to the fact that the minimax mutation is hard to provide effective gradients (i.e., vanishing gradient) when the discriminator can easily recognize generated samples. Along with the generator approaching convergence (after 20K steps), ever more minimax mutations are employed, yet the number of selected heuristic mutations is falling. As aforementioned, the minus JSDs of the heuristic mutation may tend to push the generated distribution away from data distribution and lead to training instability. However, in E-GAN, beyond the heuristic mutation, we have other options of mutation, which improves the stability at convergence.

4 LSUN and Architecture Robustness

The architecture robustness is another advantage of E-GAN. To demonstrate the training stability of our method, we train different network architectures on the LSUN bedroom dataset and compare with several existing works. In addition to the baseline DCGAN architecture, we choose three additional architectures corresponding to different training challenges: (1) limiting the recognition capability of the discriminator $D$, i.e., 2-Conv-1-FC LeakyReLU discriminator; (2) limiting the expression capability of the generator $G$, i.e., no batchnorm and a constant number of filters in the generator; (3) reducing the network capability of the generator and discriminator together, i.e., remove the BN in both the generator $G$ and discriminator $D$. For each architecture, we test five different methods: DCGAN, LSGAN, standard WGAN (with weight clipping), WGAN-GP (with gradient penalty) ,and our E-GAN. For each method, we used the default configurations recommended in the respective studies (these methods are summarized in ) and train each model for 200K iterations. As shown in Fig. 6, E-GAN generates reasonable results even when other methods are failed. Furthermore, based on the DCGAN architecture, we train E-GAN to generate $128\times 128$ bedroom imagesWe remove batchnorm layers in the generator. The detailed architecture and more generated images are reported in the Supplementary Material. (Fig. 5). Observing generated images, we demonstrate that E-GAN can be trained to generate diversity and high-quality images from the target data distribution.

5 CelebA and Space Continuity

Since humans excel at identifying facial flaws, generating high-quality human face images is challenging. Similar to generating bedrooms, we employ the same architectures to generate $128\times 128$ RGB human face images (Fig. 7). In addition, given a well-trained generator, we evaluate the performance of the embedding in the latent space of noisy vectors $z$. In Fig. 8, we first select pairs of generated faces and record their corresponding latent vectors $z_{1}$ and $z_{2}$. The two images in one pair have different attributes, such as gender, expression, hairstyle, and age. Then, we generate novel samples by linear interpolating between these pairs (i.e., corresponding noisy vectors). We find that these generated samples can seamlessly change between these semantically meaningful face attributes. This experiment demonstrates that generator training does not merely memorize training samples but learns a meaningful projection from latent noisy space to face images. Meanwhile, it also shows that the generator trained by E-GAN does not suffer from mode collapse, and shows great space continuity.

Conclusion

In this paper, we present an evolutionary GAN framework (E-GAN) for training deep generative models. To reduce training difficulties and improve generative performance, we devise an evolutionary algorithm to evolve a population of generators to adapt to the dynamic environment (i.e., the discriminator $D$). In contrast to conventional GANs, the evolutionary paradigm allows the proposed E-GAN to overcome the limitations of individual adversarial objectives and preserve the best offspring after each iteration. Experiments show that E-GAN improves the training stability of GAN models and achieves convincing performance in several image generation tasks. Future works will focus on further exploring the relationship between the environment (i.e., discriminator) and evolutionary population (i.e., generators) and further improving generative performance.

References