Neural Response Generation with Dynamic Vocabularies

Introduction

Together with the rapid growth of social conversation data on Internet, there has been a surge of interest on building chatbots for open domain conversation with data driven approaches. Existing methods are either retrieval based (?; ?; ?) or generation based (?; ?; ?). Recently, generation based approaches are becoming popular in both academia and industry, and a common practice is to learn a response generation model within an encoder-decoder framework (a.k.a., a sequence-to-sequence model) from the large scale conversation data. The mainstream of implementation of the encoder-decoder framework is using neural networks, because they are powerful on capturing complicated semantic and syntactic relations between messages and responses and are end-to-end learnable. On top of the architecture, various models have been proposed to tackle the notorious “safe reply” problem (?; ?; ?); to take conversation history into consideration (?; ?; ?; ?); and to bias responses to some specific persona or emotions (?; ?).

Although existing work has made great progress on generating proper responses, they all assume a static vocabulary in decoding, that is they use the same large set of words to generate responses regardless of inputs. The assumption, however, is a simplification of the real scenario, as proper responses to a specific input (either a message or a conversation context) could only relate to a small specific set of words, and the sets of words could be different from input to input. As a result, the assumption may cause some problems in practice: (1) words that are semantically far from the current conversation also take part in decoding. These words may bias the process of generation and increase the probability of irrelevant responses and generic responses when some of them appear very frequently in the entire data set; (2) the decoding process becomes unnecessarily slow, because one has to estimate a probability distribution for the entire static vocabulary in decoding of each word of a response. More seriously, to suppress the irrelevant responses and the generic responses, state-of-the-art methods have to either complicate their decoders (?; ?) or append a heavy post-processing procedure after decoding (?), which further deteriorates efficiency. These problems widely exist in the existing methods, but have not drawn enough attention yet.

In this paper, we aim to achieve high quality response generation and fast decoding at the same time. Our idea is that we dynamically allocate a vocabulary for each input at the decoding stage. The vocabulary is small as it only covers words that are useful in forming relevant and informative responses for the input and filters most irrelevant words out. Because response decoding of each input only focuses on their own relevant words, the process can be conducted efficiently without loss of response quality. We formulate the idea as a dynamic vocabulary sequence-to-sequence (DVS2S) model. The model defines a dynamic vocabulary in decoding through a multivariate Bernoulli distribution (?) on the entire vocabulary and factorizes the generation probability as the product of a vocabulary generation probability conditioned on the input and a response generation probability conditioned on both the input and the vocabulary. DVS2S follows the encoder-decoder framework. In encoding, an input is transformed to a sequence of hidden vectors. In decoding, the model first estimates the multivariate Bernoulli distribution using the hidden vectors given by the encoder, and then selects words to form a vocabulary for the decoder according to the distribution. Responses are generated only using the selected words. Vocabulary construction and response generation are jointly learned from training data, and thus in parameter learning errors in response prediction can be backpropagated to vocabulary formation and used to calibrate word selection. In training, as target vocabularies can only be partially observed from data, we treat them as a latent variable, and optimize a lower bound of the true objective through a Monte Carlo sampling method.

We conduct an empirical study using the data in (?), and compare DVS2S with state-of-the-art generation methods using extensive automatic evaluation metrics and human judgment. In terms of automatic evaluation, DVS2S achieves $3\%$ gain on BLEU-1 and $5\%$ gain on Embedding Average (?) over the best performing baseline. On human evaluation, DVS2S significantly outperforms the baseline methods, which is consistent with the automatic evaluation results. Moreover, the model also achieves 6$\%$ gain on the metric of distinct-1 over the best baseline model, indicating that it can generate more informative and diverse responses. Upon the significant improvement on response quality, DVS2S can save 40$\%$ decoding time compared to the most efficient baseline in the same running environment.

Our contributions in the paper are three-folds: (1) proposal of changing the static vocabulary mechanism to a dynamic vocabulary mechanism in the response generation for chatbots; (2) proposal of a dynamic vocabulary sequence-to-sequence model and derivation of a learning approach that can jointly optimize word selection and response generation; (3) empirical verification of the effectiveness and efficiency of the proposed model on large scale conversation data.

Related Work

Recent years have witnessed remarkable success on open domain response generation for chatbots. In a single-turn scenario, Ritter (?) formulated response generation as a machine translation problem by regarding messages and responses as a source language and a target language respectively. Due to the success on machine translation, sequence-to-sequence (S2S) models (?) have been widely used in response generation recently. For instance, Vinyals et al. (?) and Shang et al. (?) applied S2S with attention on this task. To address the “general response” issue of the standard S2S, Li et al. (?) presented a maximum mutual information objective function, and Mou et al. (?) and Xing et al. (?) incorporated external knowledge into the S2S model. Shao et al. (?) proposed a target attention neural conversation model to generate long and diverse responses. Reinforcement learning (?) and adversarial learning (?; ?) techniques have also been exploited to enhance the existing models. Apart from the effort on improving response quality, researchers also considered varing persona and emotions of generated responses (?; ?). In a multi-turn scenario, Sordoni et al. (?) compressed context information into a vector and injected the vector into response generation. Serban et al. (?) adopted a hierarchical recurrent structure to model multi-turn conversations. As an extension of the model, latent variables were introduced to model the “one-to-many” relation in conversation (?; ?).

In this work, we focus on an important but less explored problem: vocabulary selection in decoding. We propose changing the widely used static vocabulary decoder in both single-turn generation and multi-turn generation to a dynamic vocabulary decoder, and derive an approach to jointly learn vocabulary construction and response generation from data. The proposed method can improve response quality and at the same time speed up decoding process.

Before us, some work in machine translation has already exploited dynamic vocabularies (?; ?; ?). These work often treats vocabulary construction and translation as two separate steps. The same practice, however, cannot be easily transplanted to conversation, as there are no clear “one-to-one” translation relations in responding. To maintain response quality while improve efficiency in conversation, we propose joint learning of vocabulary construction and response generation in order to let them supervise each other. As far as we know, we are the first who explore the application of dynamic vocabularies in response generation for open domain conversation.

Approach

Suppose that we have a data set $\mathcal{D}=\{(X_{i},Y_{i})\}^{N}_{i=1}$, where $Y_{i}$ is a response of an input $X_{i}$. Here $X_{i}$ can be either a message or a message with several previous turns as a context. As the first step, we assume $X_{i}$ a message in this work, and leave the verification of the same technology to context-based response generation as future work. $\forall i$, $X_{i}$ corresponds to a target vocabulary (i.e., vocabulary in decoding) $T_{i}=(t_{i,1},\ldots,t_{i,|V|})$ sampled from a multivariate Bernoulli distribution $(\beta_{i,1},\ldots\beta_{i,|V|})$ where $|V|$ is the size of the entire vocabulary $V$ and $t_{i,j}\in\{0,1\},\forall 1\leqslant j\leqslant|V|$. $t_{i,j}=1$ means that the $j$-th word $w_{j}$ in $V$ is selected for generating responses for $X_{i}$, otherwise the word will not be used in generation. $\beta_{i,j}=p(t_{i,j}=1)$ is the probability of the $j$-th word being selected which is parameterized by a function $f(X_{i})$. Generation probability of $Y_{i}$ given $X_{i}$ is formulated as $p(Y_{i}|X_{i})=p(Y_{i}|T_{i},X_{i})p(T_{i}|X_{i})$.

Our goal is to learn a word selection model $f(X)$ (corresponds to $p(T|X)$) and a response generation model $g(X,T)$ (corresponds to $p(Y|T,X)$) by maximizing log-likelihood $\sum_{i=1}^{N}\text{log}[p(Y_{i}|X_{i})]$ of $\mathcal{D}$. Thus given a new message $X^{\prime}$, we can estimate its target vocabulary $T^{\prime}$ with $f(X^{\prime})$ and generate a response $Y^{\prime}$ using $g(X^{\prime},T^{\prime})$. In the following sections, we first introduce our DVS2S model (i.e. $g(X,T)$) by assuming that $T$ is obtained. Then we present how to sample $T$ with the use of $f(X)$. Finally, we show how to jointly learn $f(X)$ and $g(X,T)$ from $\mathcal{D}$.

Dynamic Vocabulary Sequence-to-Sequence Model

Figure 1 illustrates the architecture of our dynamic vocabulary sequence-to-sequence (DVS2S) model. DVS2S is built in an encoder-decoder framework (?) with an attention mechanism (?). For each input, it equips the decoder with a specific vocabulary that consists of useful words sampled from the entire vocabulary according to a distribution and performs response generation with the vocabulary. Specifically, given a message $X=(x_{1},x_{2},\ldots,x_{t})$ where $x_{i}$ is the embedding of the $i$-th word, the encoder exploits a bidirectional recurrent neural network with gated recurrent units (biGRU) (?) to transform $X$ into hidden vectors $h=(h_{1},h_{2},\ldots,h_{t})$. A biGRU comprises a forward GRU that reads a sentence in its order and a backward GRU that reads the sentence in its reverse order. The forward GRU encodes the sentence into hidden vectors $(\overrightarrow{h}_{1},\ldots,\overrightarrow{h}_{t})$ by

where $z_{i}$ and $r_{i}$ are an update gate and a reset gate respectively, $\overrightarrow{h}_{0}=0$, and $W_{z}$, $W_{h}$, $W_{r}$, $U_{z}$, $U_{r}$,$U_{h}$ are parameters. The backward hidden state $\overleftarrow{h}_{i}$ is obtained similarly. Then $\forall i\in[1,t]$, $h_{i}$ is the concatenation of $\overrightarrow{h}_{i}$ and $\overleftarrow{h}_{i}$.

The decoder takes $h=(h_{1},h_{2},\ldots,h_{t})$ as an input and generates a response by a language model with an attention mechanism. When generating the $i$-th word $y_{i}$, the decoder estimates a word distribution $\hat{y_{i}}$ by

where $c_{i}$ is a context vector formed by the attention mechanism, $h^{\prime}_{i}$ is the $i$-th hidden state of the decoder, and $y_{i-1}$ is the $(i-1)$-th word of the response. Specifically, the decoder also exploits a GRU to encode $y_{i-1}$ into $h^{\prime}_{i}$ whose initial state is the last hidden vector of the encoder. $c_{i}$ is a linear combination of $\{h_{1},\ldots,h_{t}\}$ which is formulated as

$W_{\alpha}$ and $v$ are parameters, and $[\cdot;\cdot]$ means concatenation of the two arguments. $l(y_{i-1},c_{i},h^{\prime}_{i},T)$ is a $|T|$-dimensional probability distribution where $|T|=\sum_{k=1}^{|V|}t_{k}$. $\forall t_{k}\in T$, if $t_{k}=1$, then the corresponding element in $l(y_{i-1},c_{i},h^{\prime}_{i},T)$ is defined by

$W_{w_{k}}$ and $b_{w_{k}}$ are two parameters. Equation (6) and (7) are called projection operation.

Time complexity of decoding of DVS2S is $\mathcal{O}(len_{r}\cdot m\cdot p+len_{r}\cdot len_{m}\cdot m^{2}+len_{r}\cdot(m+p)\cdot|T|+m\cdot|V|)$ (GRU+attention+projection+vocabulary construction), while time complexity of decoding of the existing methods is at least $\mathcal{O}(len_{r}\cdot m\cdot p+len_{r}\cdot len_{m}\cdot m^{2}+len_{r}\cdot(m+p)\cdot|V|)$ (GRU+attention+projection), where $len_{r}$ is the length of the generated response, $len_{m}$ is the length of the message, $m$ is the hidden state size of the decoder, and $p$ is the embedding size of target words. In practice, $|V|$ is much larger than other parameters, so the cost of decoding in existing methods is dominated by $len_{r}\cdot(m+p)\cdot|V|$ (i.e., time complexity of projection). DVS2S reduces it to $len_{r}\cdot(m+p)\cdot|T|$ in Equation (6) and (7). Since $len_{r}$ is usually much larger than $1$, $len_{r}\cdot(m+p)\cdot|T|+m\cdot|V|$ is much smaller than $len_{r}\cdot(m+p)\cdot|V|$. Therefore, DVS2S could enjoy a faster decoding process than the existing methods (the conclusion is also verified in experiments).

Dynamic Vocabulary Construction

In this section, we elaborate dynamic vocabulary construction for $X$. We define $\overline{T}=\{w_{k}\in V|t_{k}\in T,t_{k}=1\}$ and $I(w)$ the index of word $w$ in $V$. $\overline{T}$ is equivalent to $T$. Remember that $T$ is a variable sampled from a multivariate Bernoulli distribution which is a joint distribution of $|V|$ independent Bernoulli distributions. Each Bernoulli distribution depicts the probability of a word $w$ from $V$ being selected to $\overline{T}$ and is parameterized by $\beta_{I(w)}$. We make such an assumption because there does not exist a clear “one-to-one” relationship between words in a message and words in its proper responses and we have to treat $T$ as a latent variable in training as useful words for forming a proper response to a message can only be partially observed in training data.

$\overline{T}=\overline{T}_{c}\cup\overline{T}_{f}$ where $\overline{T}_{c}$ refers to content words and $\overline{T}_{f}$ refers to function words. Function words guarantee grammatical correctness and fluency of responses. Therefore, there should not be a large variance on $\overline{T}_{f}$ over difference messages. We collect words appearing more than $10$ times in the training data, excluding nouns, verbs, adjectives and adverbs from them, and use the remaining ones to form a function word set $\overline{V}_{f}$ of $V$. $\forall w\in\overline{V}_{f}$, we define $\beta_{I(w)}=1$. Thus, $\overline{T}_{f}=\overline{V}_{f}$ regardless of inputs. In other words, all function words are always sampled in the construction of $\overline{T}$.

Content words, on the other hand, express semantics of responses, and thus should be highly related to the input message. Let $\overline{V}_{c}=\overline{V}\setminus\overline{V}_{f}$ be the full content word set, then $\forall c\in\overline{V}_{c}$, we parameterize $\beta_{I(c)}$ as

where $\sigma$ is a sigmoid function, $h_{t}$ is the last hidden state of the encoder, and $W_{c}$ and $b_{c}$ are parameters. In the construction of $\overline{T}$, $\overline{T}_{c}$ is sampled from $\overline{V}_{c}$ based on $\{\beta_{I(c)}|c\in\overline{V}_{c}\}$.

How to allocate a proper $\overline{T}$ to $X$ is key to the success of DVS2S. $\overline{T}$ should cover enough words that are necessary to generate relevant, informative, and fluent responses for $X$, but cannot be too large for the sake of cost control in decoding. To make sure that we can sample such a $\overline{T}$ with high probability, we consider jointly learning vocabulary construction and response generation from training data, as will be seen in the next section.

Model Training

With a latent variable $T$, the objective of learning can be written as

Equation (9) is difficult to optimize as logarithm is outside the summation. Hence, we instead maximize a variational lower bound of $\sum_{i=1}^{N}\log[p(Y_{i}|X_{i})]$ which is given by

Let $\Theta$ represent the parameters of $L$ and $\frac{\partial L_{i}(\Theta)}{\partial\Theta}$ be the gradient of $L$ on an example $X_{i}\in\mathcal{D}$, then $\frac{\partial L_{i}(\Theta)}{\partial\Theta}$ can be written as

Enumerating all $2^{|V|}$ samples of $T_{i}$ in Equation (11) is intractable. Therefore, we employ the Monte Carlo sampling technique to approximate $\frac{\partial L_{i}(\Theta)}{\partial\Theta}$. Suppose that we have $S$ samples, then the approximation of the gradient can be written as

where $\widetilde{T}_{i,s}\sim\text{a multivariate Bernoulli distribution}(\{\beta_{i}\}^{|V|})$. To reduce variance, we normalize the gradient with the length of the response and introduce a moving average baseline $b_{k}$ to the gradient (?):

where $b_{k}$ is the baseline after $k$-th mini-batch, and $b_{k}$ is updated using the following equation:

We summarize our training algorithm in Algorithm 1 where we initialize $\Theta$ by pre-training an S2S model and a word prediction model to facilitate convergence and use a mini-batch training strategy to update it and the baseline $\{b_{k}\}$. We employ AdaDelta algorithm (?) to train our model with a batch size $64$. We set the initial learning rate as $1.0$ and reduce it by half if perplexity on validation begins to increase. We will stop training if the perplexity on validation keeps increasing in two successive epochs.

One advantage of joint learning is that errors in response prediction in training can be backpropagated to vocabulary construction and signals from response can help calibrate word selection. Therefore, the learning approach can mitigate discrepancy between training and inference in practice. It is easy to extend DVS2S to handle multi-turn response generation by replacing its encoder with one that can model contexts (e.g., the one in (?)), and model learning can also be enhanced using techniques like adversarial learning (?) and reinforcement learning (?) by re-defining the objective function in (Model Training).

Experiment

We compare DVS2S with state-of-the-art response generation models in terms of both efficacy and efficiency.

We use the data in (?) which consists of message-response pairs crawled from Baidu Tiebahttps://tieba.baidu.com/. Messages and responses are tokenized by Standford Chinese word segmenter. There are $5$ million pairs in the training set, $10,000$ pairs in the validation set, and $1,000$ pairs in the test set. Messages in the test data are used to generate responses, and responses in the test data are treated as ground truth to calculate automatic evaluation metrics. Both the message vocabulary and the response vocabulary contain $30,000$ words that cover $98.8\%$ and $98.3\%$ of words appearing in the messages and in the responses respectively in the training data. In this work, we take the response vocabulary in the data as the entire vocabulary for decoding (i.e., $V$).

We implement our model using Theano (?). In our model, we set the word embedding size as $620$ and the hidden vector size as $1024$ in both encoding and decoding. In the Monte Carlo sampling, we set the number of samples $S$ as $5$. We follow the method described in the dynamic vocabulary construction section to construct target vocabularies. There are $701$ function words. In test, we rank content words according to $\{\beta_{i}\}$ and select top $1,000$ words to form a target vocabulary for a message with the function words. This is equivalent to sampling many times and selecting top $1,000$ words according to their frequency in the union of all samples. The strategy does not change the time complexity of decoding and could reduce variance of the model in inference. We set the beam size as $20$ and use the top one response from beam search in evaluation. Code will be released later.

Evaluation Metrics

we evaluate the performance of different models with the following metrics:

Word overlap based metricsAs our model makes prediction on a small vocabulary, perplexity is not a proper metric for evaluation.: following previous work (?; ?), we employ BLEU-1, BLEU-2, and BLEU-3 as evaluation metrics.

Embedding based metrics: following (?; ?), we employ Embedding Average (Average), Embedding Extrema (Extrema), and Embedding Greedy (Greedy) as evaluation metrics. These metrics are based on word embeddings, and they can measure relevance of a response regarding to a message when there is little word overlap between them. According to Liu et al. (?), these metrics have higher correlation with human judgment than BLEUs. We obtain word embeddings by running a public word2vec toolhttps://code.google.com/archive/p/word2vec/ on the $5$ million training data. The embedding size is set as $200$.

Distinct-1 & distinct-2: following (?; ?), we calculate the ratios of distinct unigrams and bigrams in generated responses, and use the metrics to measure how diverse and informative the responses are.

3-scale human annotation: in addition to the automatic metrics, we recruit three human annotators with rich Tieba experience to judge the quality of the generated responses. Responses from different models are pooled and randomly shuffled for each annotator. Each response is rated by the three annotators under the following criteria: +2: the response is not only relevant and natural, but also informative and interesting; +1: the response can be used as a reply to the message, but might not be informative enough (e.g., “Yes, I see” , “Me too”, and “I don’t know”); 0: The response makes no sense, irrelevant, or grammatically broken.

Comparison Methods

S2SA: the standard S2S model with an attention mechanism (?). We use the implementation with Blocks https://github.com/mila-udem/blocks.

S2SA-MMI: the model proposed by Li et al. (?). We implement this baseline by the code published by the authors at https://github.com/jiweil/Neural-Dialogue-Generation.

TA-S2S: the topic-aware sequence-to-sequence model proposed in (?). We implement this baseline by the code published by the authors at https://github.com/LynetteXing1991/TAJA-Seq2Seq.

CVAE: recent work for response generation with a conditional variational auto-encoder (?). We use the published code at https://github.com/snakeztc/NeuralDialog-CVAE

In all the baseline models, we set the parameters as suggested by the existing papers. In addition to these methods, we also compare DVS2S with a simple version of the model. Following (?), we separately learn a generation model and a word prediction model for target vocabulary construction. The procedure is the same as the parameter initialization step in Algorithm 1. The model shares the same embedding size, hidden vector size, target vocabulary size, and the inference process with DVS2S, but differs from DVS2S in that signals from response prediction in training cannot be backpropagated to word prediction for vocabulary construction. We denote the model as S-DVS2S.

Evaluation Results

Table 1 shows the evaluation results on automatic metrics. DVS2S and S-DVS2S significantly outperform the baseline methods on most metrics, demonstrating the effectiveness of the dynamic vocabulary mechanism on response generation for open domain dialogues. Moreover, DVS2S also significantly improves upon S-DVS2S on metrics except BLEU-2 and BLEU-3. The results verify the advantage of joint learning of vocabulary and generation. DVS2S is significantly better than all baseline methods on distinct-1 and distinct-2, indicating that the model can generate more diverse and informative responses. This is because with the dynamic vocabulary mechanism, the model can circumvent the influence from generic patterns when frequent but irrelevant nouns, verbs, adjectives, and adverbs are excluded from decoding, and pay more attention to useful content words in decoding.

Table 2 reports human evaluation results. DVS2S generates much more informative and interesting responses ($2$ responses) and much less invalid responses ( responses) than the baseline methods. The results are consistent with the automatic evaluation results. S-DVS2S is much worse than DVS2S on responses. This is because the gap between training and test in S-DVS2S leads to more grammatical broken and irrelevant responses. Fleiss’ Kappa (?) on all models are around $0.4$, indicating relatively high agreement among labelers. We also conduct a t-test between DVS2S and the baseline models and results show that the improvement from our model is statistically significant (p-value $<0.01$).

In addition to response quality, we also compare DVS2S with baselines on efficiency of decoding. We calculate the average time per word in generating responses for the test messages with a beam size $20$. To make sure that the efficiency comparison is conducted under the setting with which all baselines achieve their best performance on response quality, we use the published codes and the parameters suggested by their papers. S2SA, TAS2S and DVS2S are all implemented on top of Theano, so comparison among them is fair. S2SA-MMI is implemented with Torch, and CVAE is implemented with Tensorflow. Their efficiency might be influenced by the implementation libraries, but they are theoretically not faster than S2SA. We also show their efficiency for reference. The efficiency comparison is conducted on both a GPU environment with a single Tesla K80 and a CPU environment with 6 Intel Xeon CPUs E5-2690 @ 2.6GHz. Figure 2 gives the comparison results. We can see that because of the small target vocabularies, DVS2S can save $40\%$ decoding time on both environments compared to S2SA. TAS2S is better than S2SA on response quality, but it sacrifices efficiency. From the comparisons on both efficiency and efficacy, we can conclude that DVS2S can achieve high quality response generation and fast decoding at the same time.

Discussions

In this section, we give more analysis on DVS2S to help others understand the model.

Dynamic vocabulary coverage. The first problem we investigate is how many words from the ground truth responses (i.e., responses from human) in the test data are covered by the vocabularies allocated by our algorithm in inference. We measure the coverage by this metric:

where $N_{t}$ is the number of instances in the test set (i.e., $1000$), $w$ represents a word, $Y_{i}$ is the ground truth response in the $i$-th instance, $\overline{T}_{i}$ is the target vocabulary predicted by DVS2S, and $|\cdot|$ means the number of elements in a set.

Table 4 reports the metrics varying with respect to the number of content words in the target vocabularies (note that all vocabularies share the same function words). We can see that when selecting top $1000$ content words, the target vocabularies on average can cover about $80\%$ words appearing in the ground truth responses, which is a good balance between efficiency and efficacy. The numbers in the table indicate that useful words can be accurately predicted by the word selection model in DVS2S, and the learning approach generalizes well on the test data. The results are also consistent with the good performance of DVS2S on BLEUs.

Performance across different dynamic vocabulary sizes. Next, we examine how the performance of DVS2S changes with respect to the size of the target vocabularies. We vary the number of content words selected from the entire vocabulary according to $\{\beta_{i}\}$ in a range of $\{0,100,1000,3000,5000,10000\}$, and then check how the embedding based metrics change on the test data. Table 5 shows the results. The results are consistent with our intuition: we may lose important words for response generation when the number of selected words is too small (e.g., less than $100$), but we cannot let the target vocabulary become too large either (e.g., larger than $1000$) because that may involve many irrelevant words into generation. $1000$ is the best choice as the performance of the model reaches its peak.

Case study. Finally, we qualitatively analyze DVS2S with some examples from the test data given in Table 3. In each example, we also list the top three content words according to the estimated multivariate Bernoulli distribution under the response of our model. Because our model can focus on high quality content words given by the word prediction module in decoding, it can avoid safe responses (e.g., Case 3) and promote responses that are more informative (e.g., Case 1) and more relevant (e.g., Case 2) to top position in beam search of decoding.

Conclusion and Future Work

We consider dynamically allocating a vocabulary to an input in the decoding stage for response generation in open domain conversation. To this end, we propose a dynamic vocabulary sequence-to-sequence model, and derive a learning approach that can jointly optimize vocabulary construction and response generation through a Monte Carlo sampling method. Experimental results on large scale conversation data show that DVS2S can significantly outperform state-of-the-art methods in terms of response quality and at the same time accelerate the decoding process. In the future, we will investigate how to apply the dynamic vocabulary technique to multi-turn response generation, and examine if techniques like reinforcement learning and adversarial learning can further enhance the model.

References