Investigating Linguistic Pattern Ordering in Hierarchical Natural Language Generation

Introduction

Spoken dialogue systems that can help users to solve complex tasks have become an emerging research topic in artificial intelligence and natural language processing areas . With a well-designed dialogue system as an intelligent personal assistant, people can accomplish certain tasks more easily via natural language interactions. Today, there are several virtual intelligent assistants, such as Apple’s Siri, Google’s Home, Microsoft’s Cortana, and Amazon’s Alexa, in the market. A typical dialogue system pipeline can be divided into several parts: a recognized result of a user’s speech input is fed into a natural language understanding module (NLU) to classify the domain along with domain-specific intents and fill in a set of slots to form a semantic frame . A dialogue state tracking (DST) module predicts the current state of the dialogue by means of the semantic frames extracted from multi-turn conversations. Then the dialogue policy determines the system action for the next step given the current dialogue state. Finally the semantic frame of the system action is then fed into a natural language generation (NLG) module to construct a response utterance to the user .

As a key component to a dialogue system, the goal of NLG is to generate natural language sentences given the semantics provided by the dialogue manager to feedback to users. As the endpoint of interacting with users, the quality of generated sentences is crucial for better user experience. The common and mostly adopted method is the rule-based (or template-based) method , which can ensure the natural language quality and fluency. In spite of robustness and adequacy of the rule-based methods, frequent repetition of identical, tedious output makes talking to a template-based machine unsatisfactory. Furthermore, scalability is an issue, because designing sophisticated rules for a specific domain is time-consuming .

Recurrent neural network-based language model (RNNLM) have demonstrated the capability of modeling long-term dependency in sequence prediction by leveraging recurrent structures . Previous work proposed an RNNLM-based NLG that can be trained on any corpus of dialogue act-utterance pairs without hand-crafted features and any semantic alignment . The following work based on sequence-to-sequence (seq2seq) further obtained better performance by employing encoder-decoder structure with linguistic knowledge such as syntax trees . However, due to grammar complexity and lack of diction knowledge, it is still challenging to generate long and complex sentences by a simple encoder-decoder structure.

To address the issue, previous work attempted separating decoding jobs in a decoding hierarchy, which is constructed in terms of part-of-speech (POS) tags . The original single decoding process is separated into a multi-level decoding hierarchy, where each decoding layer generates words associated with a specific POS set. This paper extends the idea to a more flexible design by incorporating attention mechanisms into the decoding hierarchy. Because prior work designs the decoding hierarchy in a hand-crafted manner based on a subjective intuition , in this work, we experiment on various generating hierarchies to investigate the importance of linguistic pattern ordering in hierarchical language generation. The experiments show that our proposed method outperforms the classic seq2seq model with a smaller model size; in addition, the concept of the hierarchical decoder is proven general enough for various generating hierarchies. Furthermore, this paper also provides the design guidelines and insights of designing the decoding hierarchy.

Hierarchical Natural Language Generation (HNLG)

The framework of the proposed hierarchical NLG model is illustrated in Figure 1, where the model architecture is based on an encoder-decoder (seq2seq) structure with attentional hierarchical decoders . In the encoder-decoder architecture, a typical generation process includes encoding and decoding phases: First, a given semantic representation sequence $\textbf{x}=\{w_{t}\}^{T}_{1}$ is fed into a RNN-based encoder to capture the temporal dependency and project the input to a latent feature space; the semantic representation sequence is also encoded into an one-hot representation as the initial state of the encoder in order to maintain the temporal-independent condition as shown in the left part of Figure 1. The recurrent unit of the encoder is bidirectional gated recurrent unit (GRU) ,

Then the encoded semantic vector, $\textbf{h}_{\text{enc}}$, is fed into an RNN-based decoder as the initial state to decode word sequences, as shown in the right part of Figure 1.

In spite of the intuitive and elegant design of the seq2seq model, it is still difficult to generate complex and decent sequences by a simple encoder-decoder structure, because a single decoder is not capable of learning all diction, grammar, and other related linguistic knowledge at the same time. Some prior work applied additional techniques such as reranker and beam-search to select a better result among multiple generated sequences . However, it is still an unsolved issue to the NLG community.

Therefore, we propose a hierarchical decoder to address the above issue, where the core idea is to allow the decoding layers to focus on learning different types of patterns instead of learning all relevant knowledge together. The hierarchical decoder is composed of several decoding layers, each of which is only responsible for learning a portion of the required knowledge. Namely, the linguistic knowledge can be incorporated into the decoding process and divided into several subsets.

We use part-of-speech (POS) tags as the additional linguistic features to construct the decoding hierarchy in this paper, where POS tags of the words in the target sentence are separated into several subsets, and each layer is responsible for decoding the words associated with a specific set of POS patterns. An example is shown in the right part of Figure 1, where the first layer at the bottom is in charge of decoding nouns, pronouns, and proper nouns, and the second layer is for verbs, and so on. The prior work manually designed the decoding hierarchy by considering the subjective intuition about how children learn to speak : infants first learn to say keywords, which are often nouns. For example, when an infant says “Daddy, toilet.”, it actually means “Daddy, I want to go to the toilet.”. Along with the growth of the age, children learn more grammars and vocabulary and then start adding verbs to the sentences, further adding adverbs, and so on. However, the hand-crafted linguistic order may not be optimal, so we experiment and analyze the model on various generating linguistic hierarchies to deeply investigate the effect of linguistic pattern ordering.

In the hierarchical decoder, the initial state of each GRU-based decoding layer $i$ is the extracted feature $\textbf{h}_{\text{enc}}$ from the encoder, and the input at every step is the last predicted token $\textbf{y}^{i}_{t-1}$ concatenated with the output from the previous layer $\textbf{y}^{i-1}_{t}$,

where $\textbf{h}^{i}_{t}$ is the $t$-th hidden state of the $i$-th GRU decoding layer and $\textbf{y}^{i}_{t}$ is the $t$-th outputted word in the $i$-th layer. We use the cross entropy loss as our training objective for optimization, where the difference between the predicted distribution and target distribution is minimized. To facilitate training and improve the performance, several strategies including scheduled sampling, a repeat input mechanism, curriculum learning, and an attention mechanism are utilized.

2 Scheduled Sampling

Teacher forcing is a strategy for training RNN that uses model output from a prior time step as an input, and it works by using the expected output at the current time step $\hat{\textbf{y}}_{t}$ as the input at the next time step, rather than the output generated by the network. The teacher forcing techniques can also be triggered only with a certain probability, which is known as the scheduled sampling approach . We adopt scheduled sampling methods in our experiments. In the proposed framework, an input of a decoder contains not only the output from the last step but one from the last decoding layer. Therefore, we design two types of scheduled sampling approaches – inner-layer and inter-layer.

Inner-layer schedule sampling is the classic teacher forcing strategy:

Inter-layer schedule sampling uses the labels instead of the actual output tokens of the last layer:

3 Curriculum Learning

The proposed hierarchical decoder consists of several decoding layers, the expected output sequences of upper layers are longer than the ones in the lower layers. The framework is suitable for applying the curriculum learning , of which core concept is that a curriculum of progressively harder tasks could significantly accelerate a network’s training. The training procedure is to train each decoding layer for some epochs from the bottommost layer to the topmost one.

4 Repeat-Input Mechanism

The concept of the hierarchical decoding is to hierarchically generate the sequence, gradually adding words associated with different linguistic patterns. Therefore, the generated sequences from the decoders become longer as the generating process proceeds to the higher decoding layers, and the sequence generated by a upper layer should contain the words predicted by the lower layers. To facilitate the behavior, previous work designs a strategy that repeats the outputs from the last layer as inputs until the current decoding layer outputs the same token, so-called the repeat-input mechanism . This approach offers at least two merits: (1) Repeating inputs tells the decoder that the repeated tokens are important to encourage the decoder to generate them. (2) If the expected output sequence of a layer is much shorter than the one of the next layer, the large difference in length becomes a critical issue of the hierarchical decoder, because the output sequence of a layer will be fed into the next layer. With the repeat-input mechanism, the impact of length difference can be mitigated.

5 Attention Mechanism

In order to model the relationship between layers in a generating hierarchy, we further design attention mechanisms for the hierarchical decoder. The proposed attention mechanisms are content-based, which means the weights are determined based on hidden states of neural models:

where $\textbf{h}^{l}_{i}$ is the hidden state at the current step, $\textbf{h}^{l-1}_{j}$ are the hidden states from the previous decoder layer, and $W$ is a learned weight matrix. At each decoding step, attention values $\alpha^{l}_{i,j}$ are calculated by these methods and then used to compute the weighted sum as a context vector, which is then concatenated to decoder inputs as additional information.

6 Training

The objective of the proposed model is to optimize the conditional probability $p(\mathbf{y}\mid\mathbf{x})$, so that the difference between the predicted distribution and the target distribution, $q(\mathbf{\hat{y}}_{\text{k}}=z\mid\mathbf{x})$, can be minimized:

where $n$ is the number of samples and the labels $\mathbf{\hat{y}}$ are the word labels. Each decoder in the hierarchical NLG is trained based on curriculum learning with the objective.

Experiments

The E2E NLG challenge dataset http://www.macs.hw.ac.uk/InteractionLab/E2E/ is utilized in our experiments, which is a crowd-sourced dataset of 50k instances in the restaurant domain. Our models are trained on the official training set and verified on the official testing set. As shown in Figure 1, the inputs are semantic frames containing specific slots and corresponding values, and the outputs are the associated natural language utterances with the given semantics. For example, a semantic frame with the slot-value pairs “name[Bibimbap House], food[English], priceRange[moderate], area [riverside], near [Clare Hall]” corresponds to the target sentence “Bibimbap House is a moderately priced restaurant who’s main cuisine is English food. You will find this local gem near Clare Hall in the Riverside area.”.

The data preprocessing includes trimming punctuation marks, lemmatization, and turning all words into lowercase. To prepare the labels of each layer within the hierarchical structure of the proposed method, we utilize spaCy toolkithttps://spacy.io/ to perform POS tagging for the target word sequences. Some properties such as names of restaurants are delexicalized (for example, replaced with a symbol “RESTAURANT_NAME”) to avoid data sparsity. In our experiments, we perform six different generating linguistic orders, in which each hierarchy is constructed based on different permutations of the POS tag sets: (1) nouns, proper nouns, and pronouns (2) verbs (3) adjectives and adverbs (4) others.

The probability of activating inter-layer and inner-layer teacher forcing is set to 0.5, the probability of teacher forcing is attenuated every epoch, and the decaying ratio is 0.9. The models are trained for 20 training epochs without early stop; when curriculum learning is applied, only the first layer is trained during first five epochs, the second decoder layer starts to be trained at the sixth epoch, and so on. To evaluate the quality of the generated sequences regarding both precision and recall, the evaluation metrics include BLEU and ROUGE (1, 2, L) scores with multiple references .

2 Results and Analysis

In the experiments, we borrow the idea of hierarchical decoding proposed by the previous work and investigate various extensions of generating hierarchies. To examine the effectiveness of hierarchical decoders, we control our model size to be smaller than the baseline’s. Specifically, the decoder in the baseline seq2seq model has hidden layers of size 400, while our models with hierarchical decoders have four decoding layers of size 100 for fair comparison.

Table 1 compares the performance between a baseline and proposed models with different generating linguistic orders. For all generating hierarchies with different orders, simply replacing the decoder by a hierarchical decoder achieves significant improvement in every evaluation metrics; for example, the topmost generating hierarchy in Table 1 has 49.25% improvement in BLEU, 30.03% in ROUGE-1, 96.48% in ROUGE-2, and 25.99% in ROUGE-L respectively. In other words, separating the generation process into several phases is proven to be a promising method. Performing curriculum learning strategy offers a considerable improvement, take the topmost generating hierarchy in Table 1 for example, this method yields a 102.07% improvement in BLEU, 48.26% in ROUGE-1, 144.8% in ROUGE-2, and 39.18% in ROUGE-L. Despite that applying repeat-input mechanism alone does not offer benefit, combining these two strategies together further achieves the best performance. Note that these methods do not require any additional parameters.

2.2 Effectiveness of Attention Mechanism

Unfortunately, even some of the attentional hierarchical decoders achieve the best results in the generating hierarchies (Table 2). Mostly, the additional attention mechanisms are not capable of bringing benefit for model performance. The reason may be that the decoding process is designed for gradually importing words in the specific set of linguistic patterns to the output sequence, each decoder layer is responsible of copying the output tokens from the previous layer and insert new words into the sequence precisely. Because of this nature, a decoder needs explicit information of the structure of a sentence rather than implicit high-level latent information. For instance, when a decoder is trying to insert some Verb words into the output sequence, knowing the position of subject and object would be very helpful.

2.3 Analysis of Linguistic Orders

The above results show that among these six different generating hierarchy, the generating order: (1) verbs $\rightarrow$ (2) nouns, proper nouns, and pronouns $\rightarrow$ (3) adjectives and adverbs $\rightarrow$ (4) the other POS tags yields the worst performance. Table 3 shows that the gap of average length of target sequences between the first and the second decoder layer is the largest among all the hierarchies; in average, the second decoder needs to insert up to 8 words into the sequence based on 3.62 words from the first decoder layer in this generation process, which is absolutely difficult. The essence of the hierarchical design is to separate the job of the decoder into several phases; if the job of each phase is balanced, it is intuitive that it is more suitable for applying curriculum learning and improve the model performance.

The model performance is also related to linguistic structures of sentences: the fifth and the sixth generating hierarchies in Table 1 have very similar trends, where the length of target sentences of each decoder layer is almost identical as shown in Table 3. However, the model performance differs a lot. An adverb word could be used to modify anything but nouns and pronouns, which means that the number of adverbs used for modifying verbs would be a factor to determine the generating order as well. In our cases, almost all adverbs in the dataset are used to describe adjectives, indicating that generating verbs before inserting adverbs to sequences may not provide enough useful information; instead, it would possibly obstruct the model learning. We can also find that in all experiments, inserting adverbs before verbs would be better.

In summary, the concept of the hierarchical decoder is simple and useful, separating a difficult job to many phases is demonstrated to be a promising direction and not limited to a specific generating hierarchy. Furthermore, the generating linguistic orders should be determined based on the dataset, and the important factors include the distribution over length of subsequences and the linguistic nature of the dataset for designing a proper generating hierarchy in NLG.

Conclusion

This paper investigates the seq2seq-based model with a hierarchical decoder that leverages various linguistic patterns. The experiments on different generating linguistic orders demonstrates the generalization about the proposed hierarchical decoder, which is not limited to a specific generating hierarchy. However, there is no universal decoding hierarchy, while the main factor for designing a suitable generating order is the nature of the dataset.

Acknowledgements

We would like to thank reviewers for their insightful comments on the paper. This work was financially supported by Ministry of Science and Technology (MOST) in Taiwan.

References