BRIO: Bringing Order to Abstractive Summarization

Introduction

Neural methods for abstractive summarization Rush et al. (2015); Nallapati et al. (2016); Chopra et al. (2016); Lewis et al. (2020); Zhang et al. (2020) formulate summarization as a sequence-to-sequence (Seq2Seq) problem (Sutskever et al., 2014), learning to generate the summary in an autoregressive manner. Such models are commonly trained with maximum likelihood estimation (MLE), maximizing predictive probability of the reference output given the gold sub-sequence before it. However, during inference the model must also generate the output based on possibly erroneous previous steps. This can hurt model performance, a phenomenon often called exposure bias (Bengio et al., 2015; Ranzato et al., 2016). To maintain reasonable performance even in the case of a sub-sequence with errors, we argue that the model must accurately estimate relative quality of different generated outputs, since effective inference requires comparison among these candidates.

To understand whether existing models can accurately perform such relative comparisons, we conducted a preliminary study on pre-trained BART (Lewis et al., 2020), first generating two candidate summaries from the model and observing whether a higher probability is assigned to the candidate with a higher ROUGE (Lin, 2004) score. As Tab. 1 shows, the accuracy is far from ideal. This is likely due to the fact that MLE training only encourages the model to assign high probability to the reference summary, and is agnostic about any relative comparison between non-reference summaries. However, we argue that it is also important for the order of model scores to be coordinated with the actual quality metrics by which the summaries will be evaluated – higher model scores should indicate better quality summaries. In the following we will refer to models that have such scores as “coordinated” for conciseness.

We introduce a training paradigm which requires the abstractive model to be able to be accurate with respect to predicting the tokens in the reference summaries and coordinated with respect to the candidate summaries. In other words, we give the abstractive model a dual role: as a generation model, it generates the output summaries in an autoregressive way; as an evaluation model, it can be used to score the quality of candidate summaries by estimating a probability distribution over candidate outputs. The generation model is trained using the standard MLE loss, but to train the evaluation model we introduce a contrastive loss (Hadsell et al., 2006) defined over different candidate summaries generated by pre-trained abstractive models (Fig. 1), following previous work on ranking-based or contrastive learning (Hopkins and May, 2011; Zhong et al., 2020; Liu et al., 2021b).

Our main contribution is to change the target distribution of abstractive models from a one-point deterministic distribution assumed by MLE training to a non-deterministic distribution in which candidate summaries are also assigned probability mass according to their quality. The new SOTA performance on CNN/DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018) datasets demonstrated the effectiveness of our method. Our in-depth analysis also found that the abstractive models trained using our method can estimate the candidate summary quality more accurately, in concert with the the objective of our training paradigm.

Neural Abstractive Summarization

The goal of abstractive summarization is to create a function $g$ that takes a source document $D$ and generates an appropriate summary $S$

Neural abstractive summarization models aim to learn a neural model $g$ that results in good summaries. Maximum likelihood estimation (MLE) is the standard training algorithm. It aims to maximize the likelihood of the reference summary $S^{*}$, i.e.,

where $\theta$ denotes the parameters of $g$ and $p_{g_{\theta}}$ denotes the probability distribution entailed by these parameters. The summation is over the training set and $\{D^{(i)},{S^{*}}^{(i)}\}$ is the $i$-th training sample.

For a specific sample $\{D^{(i)},{S^{*}}^{(i)}\}$, Eq. 2 is equivalent to minimizing the sum of negative log-likelihoods of the tokens $\{s_{1}^{*},\cdots,s_{j}^{*},\cdots,s_{l}^{*}\}$ in the reference summary $S^{*}$ whose length is $l$, which is the cross-entropy loss:

In practice, label smoothing (Szegedy et al., 2016) is a widely used and effective technique that modifies the target distribution in Eq. 4 to a "soft" label by assigning probability mass $\beta$ to other tokens:

Inference and Exposure Bias

During inference, the abstractive model $g$ is used to generate the candidate summary in an autoregressive manner. It is intractable to enumerate all the possible candidate outputs, so in practice methods such as beam search are used to reduce the search space.

One important step in search is estimating the probability of the next word $s_{t}$ given the previous predicted sequence $S_{<t}$:

Comparing Eq. 6 with Eq. 3, the major difference is that during inference the model makes new predictions based on its own previous predictions $S_{<t}$ instead of the reference $S^{*}_{<t}$. As a result, even if the generation model $g$ achieves very high accuracy w.r.t. Eq. 3, once $S_{<t}$ starts to deviate from $S^{*}$, there is the risk that the performance of $g$ will significantly degrade. This problem has been identified as the exposure bias (Bengio et al., 2015).

Coordinating Abstractive Models

Eq. 6 implies that the abstractive model $g$ should be able to assign higher estimated probability to the better candidate summary during inference. However, this intuition is not directly captured in the standard MLE objective used in training – a model obtaining zero MLE loss would assign zero probability to any candidate summary different from the reference. This is obviously improper for any task where multiple reasonable generations may exist (Khayrallah et al., 2020), and also does not say anything about the ordering of two imperfect references. We therefore advocate for making the alternative assumption that the probability of one candidate should be well-correlated with its quality as evaluated by an automatic metric $M$. Since it is intractable to enumerate all the possible candidate outputs, we only require our model to be able to accurately predict the ranking order of a set of the most probable candidate summaries $\hat{\mathcal{S}}$, which are its own beam search results. In order to achieve this objective, we slightly modify the conditions of Eq. 5, maintaining the general functional form, but instead specifying the marginal probability of the non-reference candidates $\mathcal{S}$ to be $\beta$, and encouraging coordination of probabilities and qualities among non-reference candidates as follows:

We next describe precisely how we encourage coordination through contrastive learning.

The candidate quality measure $M$ can be defined in many ways. In this work we define it as the ROUGE (Lin, 2004) score of a candidate summary $S_{i}$ given the reference summary $S^{*}$. To coordinate a pre-trained abstractive model, we 1) use it to generate different candidate summaries with various levels of quality,This is achieved by using diverse beam search (Vijayakumar et al., 2018). then 2) encourage the model to assign higher estimated probabilities to better candidates by fine-tuning the model with a contrastive loss, following the previous work (Hopkins and May, 2011; Zhong et al., 2020):

where $S_{i}$ and $S_{j}$ are two different candidate summaries and $\textrm{ROUGE}(S_{i},S^{*})>\textrm{ROUGE}(S_{j},S^{*})$, $\forall i,j,i<j$. $\lambda_{ij}$ is the margin multiplied by the difference in rank between the candidates, i.e., $\lambda_{ij}=(j-i)*\lambda$. $f(S_{i})$ is the length-normalized estimated log-probabilityWe length-normalize as it is standard in comparing hypotheses in neural sequence generation (Cho et al., 2014).

where $\alpha$ is the length penalty hyperparameter.

This loss gives the abstractive model a dual purpose, first as a reference-free evaluation model, which can be used in a two-stage summarization pipeline, where it is used to score the candidates generated by a pre-trained generation model and select the final output from them. However, since the autoregressive generation depends on both the token-level prediction accuracy and sequence-level coordination, the model fine-tuned with the contrastive loss alone can no longer be used as a generation model.

Multi-task Fine-tuning

Following Edunov et al. (2018), we combine the contrastive (Eq. 8) and cross-entropy (Eq. 3) losses to preserve the generation ability of the pre-trained abstractive model:

where $\gamma$ is the weight of the contrastive loss. We note that the contrastive and the cross-entropy loss can effectively complement each other – since the contrastive loss is defined on the sequence level, the token-level cross-entropy loss serves as a normalization to ensure that the model could assign balanced probability mass across the whole sequence.

Related Work

In order to align the training objective and evaluation metric, structured losses have been used for the Seq2Seq model training. Among them, margin-based losses (Herbrich et al., 1999; Taskar et al., 2004; Gimpel and Smith, 2010), which require the model to assign higher probability to the better output, are a major category. Many margin-based losses used in modern seq2seq models (Wiseman and Rush, 2016; Edunov et al., 2018) assume a deterministic (one-point) distribution: a model can achieve zero loss if it can assign a much higher probability to the (pseudo)-reference, regardless of relative comparisons of other candidate summaries. By contrast, our method has a non-deterministic assumption (Eq. 7), which focuses on the pair-wise ranking of a set of candidate summaries.

One main challenge of directly optimizing a Seq2Seq model with quality scores of the output is that the discrete sampling process makes the loss non-differentiable. To circumvent this problem, reinforcement learning has been used to reformulate the conditional text generation tasks (Ranzato et al., 2016; Bahdanau et al., 2016; Li et al., 2016; Paulus et al., 2018; Li et al., 2019). Compared to this school of methods, our method is based on supervised learning, and it is more stable and less sensitive to the design choices (e.g. reward shaping), which are well-known challenges of reinforcement learning methods. Minimum risk training (Shen et al., 2016; Wieting et al., 2019) and other online sampling based methods (Bengio et al., 2015; Norouzi et al., 2016; Zhang et al., 2019) belong to another school of methods used to circumvent the problem of non-differentiability. However, they also exhibit similar problems of stability as reinforcement learning.

Contrastive Learning

Recently, contrastive learning (Hadsell et al., 2006) has been introduced into several conditional text generation tasks, such as machine translation (Yang et al., 2019; Pan et al., 2021), text summarization (Cao and Wang, 2021; Xu et al., 2021; Sun and Li, 2021), and other tasks (Uehara et al., 2020; Cho et al., 2021; Lee et al., 2021b). Among these application scenarios, most work deployed contrastive learning in the latent representation space, following the framework proposed in Chen et al. (2020). However, in this work we adopt contrastive learning over the discrete space of the generated texts.

Besides, instead of constructing the contrastive learning examples by rule-based methods (e.g. perturbing the reference output), we use the generation models to construct the examples, which makes the contrastive learning task closer to the generation task. Sun and Li (2021) also adopted contrastive learning on the generated texts. However, their formulation belongs to the margin-based losses. We have discussed the difference between our method and the margin-based losses in the previous paragraphs.

Discriminative Reranking Discriminative reranking has been widely studied for conditional generation tasks (Shen et al., 2004; Och et al., 2004; Wan et al., 2015; Mizumoto and Matsumoto, 2016). Some recent works (Liu and Liu, 2021; Lee et al., 2021a) have also explored discriminative reranking of candidates from neural natural language generation models, which adopt large pre-trained language models (e.g. BERT (Devlin et al., 2019)) as the reranker. In this work, we factorize the Seq2Seq model (e.g., BART) trained on the same dataset as the reranking model, which maximizes the parameter sharing across two stages. Besides, our approach contributes an instance of leveraging large pre-trained Seq2Seq models as a quality estimation model (Yuan et al., 2021).

Experiments

We mainly use three datasets in our experiments (statistics in Appendix A).

CNNDMhttps://cs.nyu.edu/~kcho/DMQA/ (Hermann et al., 2015) is a large scale news dataset. Following Nallapati et al. (2016), we treat the news articles as the source documents and the associated highlights as the summaries.

XSumhttps://github.com/EdinburghNLP/XSum (Narayan et al., 2018) is a highly abstractive dataset of articles from the British Broadcasting Corporation (BBC).

NYThttps://catalog.ldc.upenn.edu/LDC2008T19 (Sandhaus, 2008) contains articles from the New York Times and the associated summaries. We follow Kedzie et al. (2018) for data preprocessing and splitting, and use the associated archival abstracts as the summaries.

Baselines

We choose a variety of related models with strong performance as baselines. BART (Lewis et al., 2020) and PEGASUS (Zhang et al., 2020) are both large pre-trained Seq2Seq LMs standard in the literature. GSum (Dou et al., 2021) is built on BART, and improves performance by using additional guidance from an extractive summarizer. SimCLS (Liu and Liu, 2021) introduces a two-stage framework where the pre-trained BART model is used to generate candidates and a pre-trained RoBERTa (Liu et al., 2019) model is fine-tuned as an evaluation model to score the candidate summaries and select from them. It achieves state-of-the-art performance on both CNNDM and XSum. GOLD (Pang and He, 2021) uses offline reinforcement learning to train the BART model by treating the reference summaries as the demonstrations, a different formulation that can also improve the performance of the original BART. SeqCo (Xu et al., 2021) and ConSum (Sun and Li, 2021) are two recent methods that aim to leverage contrastive learning to improve the performance of the abstractive summarization model (BART).

Implementation Details

In the following experiments, we use either BART or PEGASUS as a backbone. We label our proposed methods BRIO, with two variants: (1) BRIO-Ctr is fine-tuned with the contrastive loss (Eq. 8) only; (2) BRIO-Mul is fine-tuned with the multi-task loss (Eq. 10). We use BRIO-Ctr as an evaluation model that scores different candidate summaries generated by a Seq2Seq abstractive model and selects the final output from them, and BRIO-Mul as a standard Seq2Seq model that takes the source documents as input and generates the output in an autoregressive manner. Further details are in Appendix B.

2 Results

The results are shown in Tab 2. For CNNDM and NYT we use BART as the backbone model while for XSum we use the pre-trained PEGASUS model as our base model since it achieves better performance than BART. We have the following observations:

(1) BRIO-Ctr outperforms SimCLS, its counterpart as an evaluation model in a two-stage summarization framework. Specifically, both BRIO-Ctr and SimCLS are used to score the candidate summaries generated by a Seq2Seq abstractive model (BART). The final outputs are selected based on those scores. We attribute BRIO-Ctr’s superior performance to its use of the same model architecture (BART) for both candidate generation and scoring, while SimCLS uses RoBERTa as the evaluation model. As a result, BRIO-Ctr maximizes the parameter sharing between the two stages, and preserves the power of the Seq2Seq model pre-trained on the same dataset.

(2) BRIO-Mul is able to establish the new stare-of-the-art performance on CNNDM. Notably, the previous state-of-the-art model, GSum, takes additional guidance as input and needs a separate encoder to encode the guidance information, while BRIO-Mul uses the same parameterization of BART. Compared to other methods (ConSum, SeqCo, GOLD) that aim to improve upon BART, BRIO-Mul performs much better, showing the effectiveness of our training method.

(3) Since on XSum we use PEGASUS instead of BART as the base model, the result shows that our method is not restricted to the specific choice of the base model.

3 Analysis

We further perform some in-depth analyses from diverse perspectives on the CNNDM dataset to gain more insights into our proposed method.

The multi-task loss (Eq. 10) used to train our model contains two parts: the cross-entropy loss and the contastive loss. As shown in Tab. 3, as the weight of the contrastive loss ($\gamma$) increases, the model’s performance improves. However, the cross-entropy loss is still necessary to preserve the model’s ability as a generation model. We argue that this is because the token level accuracy is still important during the auto-regressive generation process, where the individual tokens are predicted sequentially. In addition, we also found that the model tends to achieve the best performance (w.r.t the ROUGE scores on the development set) faster with a higher $\gamma$. Specifically, it requires less than one entire epoch to achieve the best performance on CNNDM, making our approach an efficient fine-tuning method.

Generation-Finetuning as a Loop

Since the fine-tuned model (BRIO-Mul) is still able to generate, we can use it to generate a new set of candidates in the same way as we used the pre-trained BART model, and continue fine-tuning it on this newly created set of candidates (Och, 2003). Fig. 2 illustrates this iterative process. The results shown in Tab. 4 illustrate that this new model (BRIO-Loop) outperforms BRIO-Mul. Besides, the model reached the best performance very quickly, showing the potential of adopting our method in an online framework where the new candidates are dynamically generated from the current model. We leave this direction for future work.

Increasing the Beam Width

While theoretically a larger beam width (i.e. the number of candidates maintained during beam search) would allow more candidates to be considered and therefore increase the upper bound of the performance, in practice model performance may be lower if the beam width is too large. The reason for this phenomenon is closely related to the low sequence-level coordination of the generator. Specifically, increasing the beam width may introduce candidates with lower quality (Stahlberg and Byrne, 2019), and the generator may not be able to differentiate them from high-quality candidates.

In Tab. 5, we compare the performance of the pre-trained BART and our model (BRIO-Mul) with different beam widths used during inference. We observe that the performance of BART goes down as the beam width increases. On the other hand, our model is able to achieve better performance with a larger number of beams, demonstrating that our training method can improve the coordination of the model by encouraging the model to assign estimated probabilities to candidate summaries well-correlated with their quality.

Training with Different Evaluation Metrics

In the previous experiments, we used ROUGE as the evaluation metric to define the target ordering of the candidate summaries (Eq.7). To evaluate our method’s performance beyond ROUGE, we use a model-based semantic similarity metric, BERTScore (Zhang* et al., 2020),https://github.com/Tiiiger/bert_score. We use its default version for English texts. as the evaluation metric $M$ in Eq.7 to compare the performance of different candidate summaries. Then, we trained another version of BRIO-Mul based on the order of candidate summaries calculated by BERTScore.

The results in Tab. 6 show that (1) Our model can significantly improve the model performance when either ROUGE or BERTScore is used as the target evaluation metric for ordering candidate summaries. This suggests that it is possible to use our method to optimize any specific target metric, making our method an alternative to reinforcement learning or minimum risk training. (2) Our model that is trained on one evaluation metric (e.g. BERTScore) also achieves improvement on another metric (e.g. ROUGE) compared with the baseline model, which indicates that the improvement made by our model is not from exploiting the potential weaknesses of individual metrics. Besides, this result also demonstrates a non-trivial degree of agreement between ROUGE and BERTScore.

Novel n𝑛n-grams

We compare the ratio of novel $n$-grams in reference, BRIO-Mul’s, and BART’s summaries. As Tab. 7 shows, our model is more “abstractive” compared to BART, although reference summaries still contain more novel $n$-grams. This is likely due to the fact that our model is optimized at the sequence-level, allowing more freedom for paraphrasing and compression.

We further investigate the relation of the “abstractiveness" and model performance by comparing our model (BRIO-Mul) with the baseline model (BART) on different buckets of test examples grouped by the “novelty" of the reference summaries,The calculation is performed using ExplainaBoard (Liu et al., 2021a). https://github.com/neulab/ExplainaBoard. i.e.,

Rank Correlation

We computed the rank correlation between the estimated probabilities of the candidate summaries calculated by the generators and the quality scores of the candidate summaries. We use Eq. 9 to calculate the estimated probabilitiesWe found the value of the length penalty factor $\alpha$ in Eq. 9 by maximizing the rank correlation on the validation set. and we use ROUGE-1 as the quality score metric of the candidate summaries. We calculate Spearman’s rank correlation for each sample, and use the average score as the overall correlation,

We investigated two specific settings: 1) ranking candidate summaries generated by a different model (PEGASUS); 2) ranking candidate summaries generated by themselves (BART & BRIO-Mul). We use 16 candidates in total for calculation. As Tab. 8 shows, our model achieves better rank correlation on the candidate summaries generated by both itself and the independent model. This suggests that our model can better estimate the quality of candidate summaries.

4 Token-level Calibration

Calibration requires that a model’s confidence on its predictions is equal to the accuracy of these predictions (Guo et al., 2017). Previous work (Müller et al., 2019; Kumar and Sarawagi, 2019; Wang et al., 2020) has found that a more calibrated text generation model tends to have better performance, and techniques like label smoothing can improve both the token-level calibration and sequence-level accuracy (i.e. the ability of generating better results). One intuitive explanation of this phenomenon is to interpret the model’s estimated probability of a generated summary as the product of the model’s confidences on a series of token-level predictions. Then, since a more calibrated model’s confidence estimates better the accuracy of its predictions, the model’s estimated probability of one sequence should be more indicative of the quality of this sequence, which is essential for the beam search during inference. However, the relation of token-level calibration and sequence-level performance remains inconclusive (Müller et al., 2019).In general, better token-level calibration doesn’t guarantee better sequence-level performance. For example, a generator that always predicts a uniform distribution over all tokens would be perfectly calibrated, however, such a model would not generate high-quality outputs.

We investigate this relation from the opposite direction by evaluating whether our model (BRIO-Mul), which is trained to have better sequence-level performance, would also be more calibrated at the token-level compared with the baseline models that are trained using MLE and label smoothing. We follow previous work by using the Expected Calibration Error (Naeini et al., 2015) (ECE) as the evaluation metric of calibration:

where the samples are grouped into $M$ equal-width buckets by confidence (conf), $B_{m}$ denotes the $m$-th bucket, and $n$ is the total number of samples. Following Wang et al. (2020), we evaluate model calibration on the system-generated summaries during inference and use the tercom toolkithttp://cs.umd.edu/~snover/tercom/ to assign labels (correct/incorrect) to the system-generated summaries based on the reference summaries.

The results in Tab. 9 show that BRIO-Mul is better calibrated compared to BART, suggesting that our method helps to improve the token-level calibration by explicitly encouraging the model to have more accurate sequence-level probability estimations. The reliability graph is shown in Fig. 4. We found that (1) abstractive models are generally over-confident on their own predictions, (2) models are generally more calibrated on XSum than CNNDM. This is likely due to the fact that XSum has shorter summaries therefore it is less likely to be affected by the exposure bias.

5 Few-shot Fine-tuning

The training paradigm proposed in this paper may be extended to any Seq2Seq model. However, it can be a non-trivial overhead to generate the candidate summaries using large neural models on the entire training set. On the other hand, recent work (Raffel et al., 2020; Zhang et al., 2020; Schick and Schütze, 2021; Fabbri et al., 2021) has shown that few-shot learning can be an effective fine-tuning method of pre-trained models for text generation tasks.

Therefore, we investigate our model’s performance in a few-shot setting. Specifically, we randomly sample 100/1000 examples from the training set of CNNDM/XSum, and fine-tune the models that are pre-trained using MLE loss on those examples. More training details can be found in Appendix C. The results are shown in Tab. 11. All experiments are repeated three times, and the reported results are the average performance. The results indicate that our model can achieve improvement over the baseline model under the few-shot learning setting with a small computational overhead.

6 Case Study on CNNDM

Tab. 10 presents an interesting pattern we observed when comparing the results of BRIO-Mul and BART, which demonstrates that our method helps the abstractive model to filter out noise patterns in the original data. Specifically, some of the reference summaries (331/11490) in CNNDM contains the phrase “click here”, pointing to a hyperlink, and 103 source documents also contain this phrase. BART picked up this pattern, and generates this phrase in 96 output summaries. On the contrary, our model learns to ignore this noise pattern and never generated it across the whole test set, likely because it identified that generated candidates with this pattern rarely achieve a high ROUGE score, and downweighted the probability accordingly.

Conclusion and Future Work

In this work, we presented a new training paradigm that assigns candidate outputs probability mass according to their quality using contrastive learning. While our method has achieved significant improvement on abstractive summarization, we note several directions for the future work to explore. First, since our method makes no assumptions specifically about the summarization task, it can be extended to other conditional text generation tasks such as machine translation. Second, it is possible to apply our method in a reinforcement learning setting, where the candidate summaries are dynamically generated. Finally, in experiments we only used diverse beam search to generate the candidate summaries, but it is likely that other candidate generation methods could yield further improvements.

Acknowledgements

We thank the anonymous reviewers for valuable feedback and helpful suggestions.

References

Appendix A Datasets Statistics

Appendix B Implementation Details

We use diverse beam search (Vijayakumar et al., 2018) to generate 16 candidates for each data sample. On CNNDM and XSum, we use the pre-trained BARTThe checkpoint is “facebook/bart-large-cnn”, containing around 400M parameters. and PEGASUSThe checkpoint is “google/pegasus-xsum”” containing around 568M parameters. models from the Transformers (Wolf et al., 2020) library as the base abstractive models for candidate summary generation and model finetuning respectively. On NYT, we first fine-tuned a BART modelThe checkpoint is “facebook/bart-large”. with MLE training as the base abstractive model, since our data pre-processing is sightly different from the previous work and there are no available pre-trained checkpoints. We use 4 NVIDIA RTX 3090 GPUs for the model training, and the average running time for one epoch is around 20 hours. We use the Adam optimizer (Kingma and Ba, 2015) with learning rate scheduling for the model training:

We set the length penalty factor $\alpha$ in the scoring function (Eq. 9) to the same value as used in the original beam search. We search the value of the margin $\lambda$ in the contrastive loss (Eq. 8) within the range $[$1\text{\times}{10}^{-5}$,1]$, and decide the value based on the model performance on the validation set. We also performed extensive search for the coefficient $\gamma$ in Eq. 10. The specific hyper-parameter setting is reported in Tab. 13.

We use the standard ROUGE (Lin, 2004) Perl packagehttps://github.com/summanlp/evaluation/tree/master/ROUGE-RELEASE-1.5.5 for evaluation. The command line parameters are ‘-c 95 -r 1000 -n 2 -m’. Before the ROUGE evaluation, the reference summaries and system outputs are lower-cased and tokenized.PTB tokenizer is used for tokenization. https://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/process/PTBTokenizer.html

Appendix C Details of Few-shot Fine-tuning

On CNNDM, we randomly select 100 examples from the training set for fine-tuning. On XSum, we found that at least 1000 examples are needed for the model to achieve better performance compared to the baseline model. All experiments are repeated three times. We randomly select 1000 examples from the original validation set for hyper-parameter selection. We use the Adam optimizer with the learning rate set to $1\text{\times}{10}^{-6}$. The model is trained for 15 epochs on CNNDM and 10 epochs on XSum.