Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading

Introduction

While end-to-end neural conversation models (Shang et al., 2015; Sordoni et al., 2015; Vinyals and Le, 2015; Serban et al., 2016; Li et al., 2016a; Gao et al., 2019a, etc.) are effective in learning how to be fluent, their responses are often vacuous and uninformative. A primary challenge thus lies in modeling what to say to make the conversation contentful. Several recent approaches have attempted to address this difficulty by conditioning the language decoder on external information sources, such as knowledge bases Agarwal et al. (2018); Liu et al. (2018a), review posts Ghazvininejad et al. (2018); Moghe et al. (2018), and even images Das et al. (2017); Mostafazadeh et al. (2017). However, empirical results suggest that conditioning the decoder on rich and complex contexts, while helpful, does not on its own provide sufficient inductive bias for these systems to learn how to achieve deep and accurate integration between external knowledge and response generation.

We posit that this ongoing challenge demands a more effective mechanism to support on-demand knowledge integration. We draw inspiration from how humans converse about a topic, where people often search and acquire external information as needed to continue a meaningful and informative conversation. Figure 1 illustrates an example human discussion, where information scattered in separate paragraphs must be consolidated to compose grounded and appropriate responses. Thus, the challenge is to connect the dots across different pieces of information in much the same way that machine reading comprehension (MRC) systems tie together multiple text segments to provide a unified and factual answer (Seo et al., 2017, etc.).

We introduce a new framework of end-to-end conversation models that jointly learn response generation together with on-demand machine reading. We formulate the reading comprehension task as document-grounded response generation: given a long document that supplements the conversation topic, along with the conversation history, we aim to produce a response that is both conversationally appropriate and informed by the content of the document. The key idea is to project conventional QA-based reading comprehension onto conversation response generation by equating the conversation prompt with the question, the conversation response with the answer, and external knowledge with the context. The MRC framing allows for integration of long external documents that present notably richer and more complex information than relatively small collections of short, independent review posts such as those that have been used in prior work (Ghazvininejad et al., 2018; Moghe et al., 2018).

We also introduce a large dataset to facilitate research on knowledge-grounded conversation (2.8M turns, 7.4M sentences of grounding) that is at least one order of magnitude larger than existing datasets (Dinan et al., 2019; Moghe et al., 2018). This dataset consists of real-world conversations extracted from Reddit, linked to web documents discussed in the conversations. Empirical results on our new dataset demonstrate that our full model improves over previous grounded response generation systems and various ungrounded baselines, suggesting that deep knowledge integration is an important research direction.Code for reproducing our models and data is made publicly available at https://github.com/qkaren/converse_reading_cmr.

Task

We propose to use factoid- and entity-rich web documents, e.g., news stories and Wikipedia pages, as external knowledge sources for an open-ended conversational system to ground in.

Formally, we are given a conversation history of turns $X=(\bm{x}_{1},\dots,\bm{x}_{M})$ and a web document $D=(\bm{s}_{1},\dots,\bm{s}_{N})$ as the knowledge source, where $\bm{s}_{i}$ is the $i$th sentence in the document. With the pair $(X,D)$, the system needs to generate a natural language response $\bm{y}$ that is both conversationally appropriate and reflective of the contents of the web document.

Approach

Our approach integrates conversation generation with on-demand MRC. Specifically, we use an MRC model to effectively encode the conversation history by treating it as a question in a typical QA task (e.g., SQuAD (Rajpurkar et al., 2016)), and encode the web document as the context. We then replace the output component of the MRC model (which is usually an answer classification module) with an attentional sequence generator that generates a free-form response. We refer to our approach as CMR (Conversation with on-demand Machine Reading). In general, any off-the-shelf MRC model could be applied here for knowledge comprehension. We use Stochastic Answer Networks (SAN)https://github.com/kevinduh/san_mrc (Liu et al., 2018b), a performant machine reading model that until very recently held state-of-the-art performance on the SQuAD benchmark. We also employ a simple but effective data weighting scheme to further encourage response grounding.

We adapt the SAN model to encode both the input document and conversation history and forward the digested information to a response generator. Figure 2 depicts the overall MRC architecture. Different blocks capture different concepts of representations in both the input conversation history and web document. The leftmost blocks represent the lexicon encoding that extracts information from $X$ and $D$ at the token level. Each token is first transformed into its corresponding word embedding vector, and then fed into a position-wise feed-forward network (FFN) Vaswani et al. (2017) to obtain the final token-level representation. Separate FFNs are used for the conversation history and the web document.

The next block is for contextual encoding. The aforementioned token vectors are concatenated with pre-trained 600-dimensional CoVe vectors McCann et al. (2017), and then fed to a BiLSTM that is shared for both conversation history and web document. The step-wise outputs of the BiLSTM carry the information of the tokens as well as their left and right context.

2 Response Generation

Having read and processed both the conversation history and the extra knowledge in the document, the model then produces a free-form response ${\bf y}=(y_{1},\dots,y_{T})$ instead of generating a span or performing answer classification as in MRC tasks.

We use an attentional recurrent neural network decoder (Luong et al., 2015) to generate response tokens while attending to the memory. At the beginning, the initial hidden state $\bm{h}_{0}$ is the weighted sum of the representation of the history $X$. For each decoding step $t$ with a hidden state $\bm{h}_{t}$, we generate a token $y_{t}$ based on the distribution:

where $\tau>0$ is the softmax temperature. The hidden state $\bm{h}_{t}$ is defined as follows:

Here, $[\cdot+\!\!+\cdot]$ indicates a concatenation of two vectors; $f_{attention}$ is a dot-product attention Vaswani et al. (2017); and $\bm{z}_{t}$ is a state generated by $\text{GRU}(\bm{e}_{t-1},\bm{h}_{t-1})$ with $\bm{e}_{t-1}$ being the embedding of the word $y_{t-1}$ generated at the previous ($t-1$) step. In practice, we use top-$k$ sample decoding to draw $y_{t}$ from the above distribution $p(y_{t})$. Section 5 provides more details about the experimental configuration.

3 Data Weighting Scheme

Dataset

To create a grounded conversational dataset, we extract conversation threads from Reddit, a popular and large-scale online platform for news and discussion. In 2015 alone, Reddit hosted more than 73M conversations.https://redditblog.com/2015/12/31/reddit-in-2015/ On Reddit, user submissions are categorized by topics or “subreddits”, and a submission typically consists of a submission title associated with a URL pointing to a news or background article, which initiates a discussion about the contents of the article. This article provides framing for the conversation, and this can naturally be seen as a form of grounding. Another factor that makes Reddit conversations particularly well-suited for our conversation-as-MRC setting is that a significant proportion of these URLs contain named anchors (i.e., ‘#’ in the URL) that point to the relevant passages in the document. This is conceptually quite similar to MRC data (Rajpurkar et al., 2016) where typically only short passages within a larger document are relevant in answering the question.

We reduce spamming and offensive language by manually curating a list of 178 relatively “safe” subreddits and 226 web domains from which the web pages are extracted. To convert the web page of each conversation into a text document, we extracted the text of the page using an html-to-text converter,https://www.crummy.com/software/BeautifulSoup while retaining important tags such as