Ask the GRU: Multi-Task Learning for Deep Text Recommendations

Introduction

Text recommendation is an important problem that has the potential to drive significant profits for e-businesses through increased user engagement. Examples of text recommendations include recommending blogs, social media posts , news articles , movies (based on plot summaries), products (based on reviews) and research papers .

Methods for recommending text items can be broadly classified into collaborative filtering (CF), content-based, and hybrid methods. Collaborative filtering methods use the user-item rating matrix to construct user and item profiles from past ratings. Classical examples of this include matrix factorization methods which completely ignore text information and rely solely on the rating matrix. Such methods suffer from the cold-start problem – how to rank unseen or unrated items – which is ubiquitous in most domains. Content-based methods , on the other hand, use the item text or attributes, and make recommendations based on similarity between such attributes, ignoring data from other users. Such methods can make recommendations for new items but are limited in their performance since they cannot employ similarity between user preferences . Hybrid recommendation systems seek the best of both worlds, by leveraging both item content and user-item ratings . Hybrid recommendation methods that consume item text for recommendation often ignore word order , and either use bags-of-words as features for a linear model or define an unsupervised learning objective on the text such as a topic model . Such methods are unable to fully leverage the text content, being limited to bag-of-words sufficient statistics , and furthermore unsupervised learning is unlikely to focus on the aspects of text relevant for content recommendation.

In this paper we present a method leveraging recurrent neural networks (RNNs) to represent text items for collaborative filtering. In recent years, RNNs have provided substantial performance gains in a variety of natural language processing applications such as language modeling and machine translation . RNNs have a number of noteworthy characteristics: (1) they are sensitive to word order, (2) they do not require hand-engineered features, (3) it is easy to leverage large unlabeled datasets, by pretraining the RNN parameters with unsupervised language modeling objectives , (4) RNN computation can be parallelized on a GPU, and (5) the RNN applies naturally in the cold-start scenario, as a feature extractor, whenever we have text associated with new items.

Due to the extreme data sparsity of content recommendation datasets , regularization is also an important consideration. This is particularly important for deep models such as RNNs, since these high-capacity models are prone to overfitting. Existing hybrid methods have used unsupervised learning objectives on text content to regularize the parameters of the recommendation model . However, since we consume the text directly as an input for prediction, we can not use this approach. Instead, we provide regularization by performing multi-task learning combining collaborative filtering with a simple side task: predicting item meta-data such as genres or item tags. Here, the network producing vector representations for items directly from their text content is shared for both tag prediction and recommendation tasks. This allows us to make predictions in cold-start conditions, while providing regularization for the recommendation model.

We evaluate our recurrent neural network approach on the task of scientific paper recommendation using two publicly available datasets, where items are associated with text abstracts . We find that the RNN-based models yield up to 34% relative-improvement in Recall@50 for cold-start recommendation over collaborative topic regression (CTR) approach of Wang and Blei and a word-embedding based model model , while giving competitive performance for warm-start recommendation. We also note that a simple linear model that represents documents using an average of word embeddings trained in a completely supervised fashion , obtains competitive results to CTR. Finally, we find that multi-task learning improves the performance of all of the models significantly, including the baselines.

Background and Related Work

This paper focuses on the task of recommending items associated with text content. The $j$-th text item is a sequence of $n_{j}$ word tokens, $X_{j}=(w_{1},w_{2},\ldots,w_{n_{j}})$ where each token is one of $V$ words from a vocabulary. Additionally, the text items may be associated with multiple tags (user or author provided). If item $j\in[N_{d}]$ has tag $l\in[N_{t}]$ then we denote it by $t_{jl}=1$ and otherwise.

There are $N_{u}$ users who have liked/rated/saved some of the text items. The rating provided by user $i$ on item $j$ is denoted by $r_{ij}$. We consider the implicit feedback setting, where we only observe whether a person has viewed or liked an item and do not observe explicit ratings. $r_{ij}=1$ if user $i$ liked item $j$ and otherwise. Denote the user-item matrix of likes by $R$. Let $R^{+}_{i}$ denote the set of all items liked by user $i$ and $R^{-}_{i}$ denote the remaining items.

The recommendation problem is to find for each user $i$ a personalized ranking of all unrated items, $j\in R^{-}_{i}$, given the text of the items $\{X_{j}\}$, the matrix of users’ previous likes $\{R_{i}\}$ and the tagging information of the items $\{t_{il}\}$.

2 Latent Factor Models

Latent factor models for content recommendation learn $K$ dimensional vector embeddings of items and users:

3 The Cold Start Problem

In many applications, the factorization (1) is unusable, since it suffers from the cold-start problem : new or unseen items can not be recommended to users because we do not have an associated embedding. This has lead to increased interest in hybrid CF methods which can leverage additional information, such as item content, to make cold-start recommendations. In some cases, we may also face a cold-start problem for new users. Though we do not consider this case, the techniques of this paper can be extended naturally to accommodate it whenever we have text content associated with users. We consider:

Where $f(\cdot)$ is a vector-valued function of the item’s text. For differentiable $f(\cdot)$, (3) can also be trained using (2). Throughout the paper, we will refer to $f(\cdot)$ as an encoder. Existing hybrid CF methods which use item metadata take this form. In such cases, $f(.)$ is a linear function of manually extracted item features. For example, Agarwal and Chen , Gantner et al. incorporate side information through a linear regression based formulation on metadata like category, user’s age, location, etc. Rendle proposed a more general framework for incorporating higher order interactions among features in a factor model. Refer to Shi et al. , and the references therein, for a recent review on such hybrid CF methods.

Our experiments compare to collaborative topic regression (CTR) , a state-of-the-art technique that simultaneously factorizes the item-word count matrix (through probabilistic topic modeling) and the user-item rating matrix (through a latent factor model). By learning low-dimensional (topical) representations of items, CTR is able to provide recommendations to unseen items.

4 Regularization via Multi-task Learning

Typical CF datasets are highly sparse, and thus it is important to leverage all available training signals . In many applications, it is useful to perform multi-task learning that combines CF and auxiliary tasks, where a shared feature representation for items (or users) is used for all tasks. Collective matrix factorization jointly factorizes multiple observation matrices with shared entities for relational learning. Ma et al. seek to predict side information associated with users. Finally, McAuley and Leskovec used topic models and Almahairi et al. used language models on review text.

In many applications, text items are associated with tags, including research papers with keywords, news articles with user or editor provided labels, social media posts with hash-tags, movies with genres, etc. These can be used as features $X_{j}$ in (3) . However, there are considerable drawbacks to this approach. First, tags are often assigned by users, which may lead to a cold-start problem , since new items have no annotation. Moreover, tags can be noisy, especially if they are user-assigned, or too general .

While tag annotation may be unreliable and incomplete as input features, encouraging items’ representations to be predictive of these tags can yield useful regularization for the CF problem. Besides providing regularization, this multi-task learning approach is especially useful in cold-start scenarios, since the tags are only used at train time and hence need not be available at test time. In Section 3.3 we employ this approach.

5 Deep Learning

In our work, we represent the item-to-embedding mapping $f(\cdot)$ using a deep neural network. See for a comprehensive overview of deep learning methods. We provide here a brief review of deep learning for recommendation systems.

Neural networks have received limited attention from the recommendation systems community. used restricted Boltzmann machines as one of the component models to tackle the Netflix challenge. Recently, proposed denoising auto-encoder based models for collaborative filtering which are trained to denoise corrupted versions of entire sparse vectors of user-item likes or item-user likes (i.e. rows or columns of the $R$ matrix). However, these models are unable to handle the cold-start problem. Wang et al. addresses this by incorporating a bag-of-words autoencoder in the model within a Bayesian framework. Elkahky et al. proposed to use neural networks on manually extracted user and item feature representations for content based multi-domain recommendation. Dziugaite and Roy proposed to use a neural network to learn the similarity function between user and item latent factors. Van den Oord et al. , Wang and Wang developed music recommender systems which use features extracted from the music audio using convolutional neural networks (CNN) or deep belief networks. However, these methods process the user-item rating matrix in isolation from the content information and thus are unable to exploit the direct interaction between item content and ratings . Weston et al. proposed a CNN based model to predict hashtags on social media posts and found the learned representations to also be useful for document recommendation. Recently, He and McAuley used image-features from a separately trained CNN to improve product recommendation and tackle cold-start. Almahairi et al. used neural network based language models on review text to regularize the latent factors for product recommendation, as opposed to using topic models, as in McAuley and Leskovec . They found that RNN based language models perform poorly as regularizers and word embedding models Mikolov et al. perform better.

Deep Text Representation for Collaborative Filtering

This section presents neural network-based encoders for explicitly mapping an item’s text content to a vector of latent factors. This allows us to perform cold-start prediction on new items. In addition, since the vector representations for items are tied together by a shared parametric model, we may be able to generalize better from limited data.

As is standard in deep learning approaches to NLP, our encoders first map input text $X_{j}=(w_{1},w_{2},\ldots,w_{n_{j}})$ to a sequence of $K_{w}$-dimensional embeddings , $(e_{1},e_{2},\ldots,e_{n_{j}})$, using a lookup table with one vector for every word in our vocabulary. Then, we define a transformation that collapses the sequence of embeddings to a single vector, $g(X_{j})$.

A simple order-insensitive encoder of the document text can be obtained by averaging word embeddings:

This corresponds exactly to a linear model on a bag-of-words representation for the document. However, using the representation (5) is useful because the word embeddings can be pre-trained, in an unsupervised manner, on a large corpus . Note that (5) is similar to the embedding-based model used in Weston et al. for hastag prediction.

Note that CTR , described in 2.3, also operates on bag-of-words sufficient statistics. Here, it does not have an explicit parametric encoder $g(\cdot)$ from text to a vector, but instead defines an implicit mapping via the process of doing posterior inference in the probabilistic topic model.

2 Order-Sensitive Encoders

Bag-of-words models are limited in their capacity, as they cannot distinguish between sentences that have similar unigram statistics but completely different meanings . As a toy example, consider the research paper abstracts: “This paper is about deep learning, not LDA" and “This paper is about LDA, not deep learning”. They have the same unigram statistics but would be of interest to different sets of users. A more powerful model that can exploit the additional information inherent in word order would be expected to recognize this and thus perform better recommendation.

In response, we parametrize $g(\cdot)$ as a recurrent neural network (RNN). It reads the text one word at a time and produces a single vector representation. RNNs can provide impressive compression of the salient properties of text. For example, accurate translation of an English sentence can be performed by conditioning on a single vector encoding .

The extracted item representation is combined with a user embedding, as in (3), to get the predicted rating for a user-item pair. The model can then be trained for recommendation in a completely supervised manner, using a differentiable cost function such as (2). Note that a key difference between this approach and the existing approaches which use item content , apart from sensitivity to word order, is that we do not define an unsupervised objective (like likelihood of observing bag-of-words under a topic model) for extracting a text representation. However, our model can benefit from unsupervised data through pre-training of word embeddings or pre-training of RNN parameters using language models (our experiments use embeddings).

Traditional RNN architectures suffer from the problem of vanishing and exploding gradients , rendering optimization difficult and prohibiting them from learning long-term dependencies. There have been several modifications to the RNN proposed to remedy this problem, of which the most popular are long short-term memory units (LSTMs) and the more recent gated recurrent units (GRUs) . We use GRUs, which are simpler than LSTM, have fewer parameters, and give competitive performance to LSTMs .

The GRU hidden vector output at step $t$, $h_{t}$, for the input sequence $X_{j}=(w_{1},\ldots,w_{t},\ldots,w_{n_{j}})$ is given by:

The final architecture, shown in Figure 1, consists of two stacked layers of RNN with GRU hidden units. We use a bi-directional RNN at the first layer and feed the concatenation of the forward and backward hidden states as the input to the second layer. The output of the hidden states of the second layer is pooled to obtain the item content representation $g(X_{j})$. In our experiments, mean pooling performs best. Models that use the final RNN state take much longer to optimize. Following (4), the final item representation is obtained by combining the RNN representation with an item-specific embedding $v_{j}$. We now describe the multi-task learning setup.

3 Multi-Task Learning

The encoder $f(\cdot)$ can be used as a generic feature extractor for items. Therefore, we can employ the multi-task learning approach of Section 2.4. The tags associated with papers can be considered as a (coarse) summary or topics of the items and thus forcing the encoder to be predictive of the tags will provide a useful inductive bias. Consider again the toy example of Figure 1. Observing the tag “RNN” but not “LDA” on the paper, even though the term LDA is present in the text, will force the network to pay attention to the sequence of words “not LDA” in order to explain the tags.

where $c_{jl}^{\prime}$ down-weights the cost for predicting the unobserved tags. The final cost is $C(\boldsymbol{\theta})=\lambda C_{R}(\boldsymbol{\theta})+(1-\lambda)C_{T}(\boldsymbol{\theta})$ with $C_{R}$ defined in (2), and $\lambda$ is a hyperparameter.

It is worth noting the differences between our approach and Almahairi et al. , who use language modeling on the text as an unsupervised multi-task objective with the item latent factors as the shared parameters. Almahairi et al. found that the increased flexibility offered by the RNN makes it too strong a regularizer leading to worse performance than simpler bag-of-words models. In contrast, our RNN is trained fully supervised, which forces the item representations to be discriminative for recommendation and tag prediction. Furthermore, by using the text as an input to $g(\cdot)$ at test time, rather than just for train-time regularization, we can alleviate the cold-start problem.

Experiments

Datasets: We use two datasets made available by Wang et al. from CiteULikehttp://www.citeulike.org/. CiteULike is an online platform which allows registered users to create personal libraries by saving papers which are of interest to them. The datasets consist of the papers in the users’ libraries (which are treated as ‘likes’), user provided tags on the papers, and the title and abstract of the papers. Similar to Wang and Blei , we remove users with less than 5 ratings (since they cannot be evaluated properly) and removed tags that occur on less than 10 articles. Citeulike-a consists of 5551 users, 16980 papers and 3629 tags with a total of 204,987 user-item likes. Citeulike-t consists of 5219 users, 25975 papers and 4222 tags with a total of 134,860 user-item likes. Note Citeulike-t is much more sparse (99.90%) than Citeulike-a (99.78%).

Evaluation Methodology: Following, Wang and Blei , we test the models on held-out user-article likes under both warm-start and cold-start scenarios.

Warm-Start: This is the case of in-matrix prediction, where every test item had at least one like in the training data. For each user we do a 5-fold split of papers from their like history. Papers with less than 5 likes are always kept in the training data, since they cannot be evaluated properly. After learning, we predict ratings across all active test set items and for each user filter out the items in their training set from the ranked list.

Cold-Start: This is the task of predicting user interest in a new paper with no existing likes, based on the text content of the paper. The set of all papers is split into 5 folds. Again, papers with less than 5 likes are always kept in training set. For each fold, we remove all likes on the papers in that fold forming the test-set and keep the other folds as training-set. We fit the models on the training set items for each fold and form predictive per-user ranking of items in the test set.

Evaluation Metric: Accuracy of recommendation from implicit feedback is often measured by recall. Precision is not reasonable since the zero ratings may mean that a user either does not like the article or does not know of it. Thus, we use Recall@M and average the per-user metric:

We compare the proposed methods with CTR, which models item content using topic modeling. The approach put forth by CTR cannot perform tag-prediction and thus, for a fair comparison, we modify CTR to do tag prediction. This can be viewed as a probabilistic version of collective matrix factorization . Deriving an alternating least squares inference algorithm along the line of is not possible for a sigmoid loss. Thus, for CTR, we formulate tag prediction using a weighted squared loss instead. Learning this model is a straightforward extension of CTR: rather than performing alternating updates on two blocks of parameters, we rotate among three. We call this CTR-MTL. The word embedding-based model with order-insensitive document encoder (section 3.1) is Embed, and the RNN-based model (section 3.2) is GRU. The corresponding models trained with multi-task learning are Embed-MTL and GRU-MTL.

1.2 Implementation Details

For CTR, we follow Wang and Blei for setting hyperparameters. We use latent factor dimension $K=200$, regularization parameters $\lambda_{u}=0.01,\;\lambda_{v}=100$ and cost weights $a=1,\;b=0.01$. The same parameters gave good results for CTR-MTL. CTR and CTR-MTL are trained using the EM algorithm, which updates the latent factors using alternating least squares on full data . CTR is sensitive to good pre-processing of the text, which is common in topic modeling . We use the provided pre-processed text for CTR, which was obtained by removing stop-words and choosing top words based on tf-idf. We initialized CTR with the output of a topic model trained only on the text. We used the CTR code provided by the authors.

For the Embed and GRU models, we used word embeddings of dimension $K_{w}=K=200$, in order to be consistent with CTR. For GRU models, the first layer of the RNN has hidden state dimension $K_{h1}=400$ and the second layer (the output layer) has hidden state dimension $K_{h2}=200$. We pre-trained the word embeddings using CBOW on a corpus of 440,756 ACM abstracts (including the Citeulike abstracts). Dropout is used at every layer of the network. The probabilities of dropping a dimension are 0.1, 0.5 and 0.3 at the embedding layer, the output of the first layer and the output of the second layer, respectively. We also regularize the user embeddings with weight $0.01$. We do very mild preprocessing of the text. We replace numbers with a token and all words which have a total frequency of less than 5 by . Note that we don’t remove stop words or frequent words. This leaves a vocabulary of 21,129 words for Citeulike-a and 24,697 words for Citeulike-t.

The models are optimized via stochastic gradient descent, where mini-batch randomly samples a subset of $B$ users and for each user we sample one positive and one negative example. We set the weights $c_{ij}$ in (2) to $c_{ij}=1+\alpha\log(1+\frac{|R_{i}|}{\epsilon})$, where $|R_{i}|$ is the number of items liked by user $i$, with $\alpha=10,\;\epsilon=1e-8$. Unlike Wang and Blei we do not weight the cost function differently for positive and negative samples. Since the total number of negative examples is much larger than the positive examples for each user, stochastically sampling only one negative per positive example implicitly down-weights the negatives. We used a mini-batch size of $B=512$ users and used Adam for optimization. We run the models for a maximum of 20k mini-batch updates and use early-stopping based on recall on a validation set from the training examples.

2 Quantitative Results

Table 1 summarizes Recall@50 for all the models, on the two CiteULike datasets, for both warm-start and cold-start. Figure 3, further shows the variation of Recall@M for different values of $M$ for the multi-task learning models.

Next, observe that multi-task learning uniformly improves performance for all models. The GRU model’s recall improves by 7.8% on Citulike-a and by 7.6% on Citeulike-t. This leads to an overall improvement of 19.30% on Citeulike-a and 34.22% on Citeulike-t, over best of the baselines. Comparatively, improvement for CTR is smaller. This is expected since the Bayesian topic model provides strong regularization for the model parameters. Contrary to this, Embed models also benefits a lot by MTL (up to 8.2%). This is expected since unlike CTR, all the $K_{w}\times V$ parameters in the Embed model are free parameters which are trained directly for recommendation, and thus MTL provides necessary regularization.

Tag Prediction: Although the focus of the models is recommendation, we evaluate the performance of the multi-task models on tag prediction. We again use Recall@50 (defined per article) and evaluate in the cold-start scenario, where there are no tags present for the test article. The GRU and Embed models perform similarly. CTR-MTL is significantly worse, which could be due to our use of the squared loss for training or because hyperparameters were selected for recommendation performance, not tag prediction.

3 Interpreting Prediction Decisions

We employ a simple, easy-to-implement tool for analyzing RNN predictions, based on Denil et al. and Li et al. . We produce a heatmap where every input word is associated with its leverage on the output prediction. Suppose that we recommended item $j$ to user $i$. In other words, suppose that $\hat{r}_{ij}$ is large. Let $E_{j}=(e_{j,1},e_{j,2},\ldots,e_{j,n_{j}})$ be the sequence of word embeddings for item $j$. Since $f(\cdot)$ is encoded as a neural network, $\frac{d\hat{r}_{ij}}{de_{j,t}}$ can be obtained by backpropagation. To produce the heatmap’s value for word $t$, we convert $\frac{d\hat{r}_{ij}}{de_{j,t}}$ into a scalar. This is not possible by backpropagation, as $\frac{d\hat{r}_{ij}}{dx_{j,t}}$ is not well-defined, since $x_{j,t}$ is a discrete index. Instead we compute $\lVert\frac{d\hat{r}_{ij}}{de_{j,t}}\rVert$. An application is in Figure 2.

Conclusion & Future Work

We employ deep recurrent neural networks to provide vector representations for the text content associated with items in collaborative filtering. This generic text-to-vector mapping is useful because it can be trained directly with gradient descent and provides opportunities to perform multi-task learning. For scientific paper recommendation, the RNN and multi-task learning both provide complementary performance improvements. We encourage further use of the technique in a variety of application domains. In future work, we would like to apply deep architectures to users’ data and to explore additional objectives for multi-task learning that employ multiple modalities of inputs, such as movies’ images and text descriptions.

Acknowledgment

This work was supported in part by the Center for Intelligent Information Retrieval, in part by The Allen Institute for Artificial Intelligence, in part by NSF grant #CNS-0958392, in part by the National Science Foundation (NSF) grant number DMR-1534431, and in part by DARPA under agreement number FA8750-13-2-0020. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor.

References