Topics to Avoid: Demoting Latent Confounds in Text Classification

Introduction

Text classification systems based on neural networks are biased towards learning frequent spurious correlations in the training data that may be confounds in the actual classification task (Leino et al., 2019). A major challenge in building such systems is to discover features that are not just correlated with the signals in the training data, but are true indicators of these signals, and therefore generalize well.

For example, Kiritchenko and Mohammad (2018) found that sentiment analysis systems implicitly overfit to demographic confounds, systematically amplifying the intensity ratings of posts written by women. Zhao et al. (2017) showed that visual semantic role labeling models implicitly capture actions stereotypically associated with men or women (e.g., women are cooking and men are fixing a faucet), and in cases of higher model uncertainty assign stereotypical labels to actions and objects, thereby amplifying social biases found in the training data.

We focus on the task of native language identification (L1ID), which aims at automatically identifying the native language (L1) of an individual based on their language production in a second language (L2, English in this work). The aim of this task is to discover stylistic features present in the input that are indicative of the author’s L1. However, a model trained to predict L1 is likely to predict that a person is, say, a native Greek speaker, if the texts authored by that person mention Greece, because the training data exhibits such topical correlations (§2).

This problem is the focus of our work, and we address it in two steps. First, we introduce a novel method for representing latent confounds. Recent relevant work in the area of domain adaptation (Ganin et al., 2016) and deconfounding for text classification (Pryzant et al., 2018; Elazar and Goldberg, 2018) assumes that the set of confounds is known a priori, and their values are given as part of the training data. This is an unrealistic setting that limits the applicability of such models in real world scenarios. In contrast, we introduce a new method, based on log-odds ratio with Dirichlet prior (Monroe et al., 2008), for identifying and representing latent confounds as probability distributions (§3). Second, we propose a novel alternating learning procedure with multiple adversarial discriminators, inspired by adversarial learning (Goodfellow et al., 2014), that demotes latent confounds and results in textual representations that are invariant to the confounds (§4).

Note that these two proposals are task-independent and can be extended to a vast array of text classification tasks where confounding factors are not known a priori. For concreteness, however, we evaluate our approach on the task of L1ID (§5). We experiment with two different datasets: a small corpus of student written essays (Malmasi et al., 2017) and a large and noisy dataset of Reddit posts (Rabinovich et al., 2018). We show that classifiers trained on these datasets without any intervention learn spurious topical correlations that are not indicative of style, and that our proposed deconfounded classifiers alleviate this problem (§6). We present an analysis of the features discovered after demoting these confounds in §7.

We introduce a novel method for representing and identifying variables which are confounds in text classification tasks.

We propose a classification model and an algorithm aimed at learning textual representations that are invariant to the confounding variable.

We introduce a novel approach to adversarial training with multiple adversaries, to alleviate the problem of drifting parameters during alternating classifier–adversary optimization.

Finally, we analyze some linguistic features that are not only predictive of the author’s L1 but are also devoid of topical bias.

Motivation

We study the general effect of topical confounds in text classification. To motivate the need to demote them, we introduce as a case study the L1ID task, in which the goal is to predict the native language of a writer given their texts in L2.

We begin with a subset of the L2-Reddit corpus (Rabinovich et al., 2018), consistsing of Reddit posts by authors with 23 different L1s, most of them European languages. Some of the posts come from Europe-related forums (e.g. r/Europe, r/AskEurope, r/EuropeanCulture), whereas others are from unrelated forums. We view the latter as out-of-domain data and use them to evaluate the generalization of our models. We use a subset of this corpus, with only the 10 most frequent L1s, to guarantee a large enough balanced training set. We remove all the posts with fewer than 50 words and sample the dataset to obtain a balanced distribution of labels: from this balanced dataset, we randomly sample 20% of examples from each class and divide them equally to create development and test sets. In total, there are around 260,000 examples in the training set and 32,000 examples each in the development, the in-domain test set, and the out-of-domain test set.

We trained a standard (non-adversarial) classifier, with a bidirectional LSTM encoder followed by two feedforward layers with a $\tanh$ activation function and a softmax in the final layer (full experimental details are given in §5.2). We refer to this model as no-adv. The results are shown in Table 1. Notice the huge drop in accuracy on the out-of-domain data, which indicates that the model is learning topical features.

To further verify this claim, we used log-odds ratio with Dirichlet prior (Monroe et al., 2008)—a common way to identify words that are statistically overrepresented in a particular population compared to others—to identify the top-$K$ words that were most strongly associated with a specific L1 in the training set. (We refer the reader to (Monroe et al., 2008) for the details about the algorithm.) We experimented with $K\in\{20,50,100,200\}$. Table 2 shows the top-$10$ words in each class; observe that almost all of these words are geographical (hence, topical) terms that have nothing to do with the L1.

Next, we masked such topical words (by replacing them with a special token) and evaluate the trained classifier on masked test sets. Accuracy (Table 1) degrades on both the in-domain and out-of-domain sets, even when only $20$ words are removed. The drop in accuracy with the out-of-domain dataset is smaller since these data do not include many instances where the presence of topical words would help in identifying the label. These experiments confirm our hypothesis that the baseline classifier is primarily learning topical correlations, and motivate the need for a deconfounded classification approach which we describe next.

Representing Confounds

Latent Dirichlet allocation (LDA; Blei et al., 2003) is a probabilistic generative model for discovering abstract topics that occur in a collection of documents. Under LDA, each document can be considered a mixture of a small (fixed) number of topics—each represented as a distribution over words—and each word’s presence is assumed to be attributed to one of the document’s topics. More precisely, LDA assigns each document a probability distribution over a fixed number of topics $K$.

LDA topics are known to be poor features for classification (McAuliffe and Blei, 2008), indicating that they do not encode all the topical information. Moreover, they can encode information which is not actually topical and can be a useful L1 marker. Motivated by our case study (§2), we propose a novel method to represent topic distributions, based on log-odds scores (Monroe et al., 2008), and compare it to LDA as a baseline.

Deconfounded Text Classification

We now formalize the task setup and the classification model. We are given $N$ labeled documents in the training set $\{(x_{1},y_{1}),(x_{2},y_{2}),$ $\ldots,(x_{N},y_{N})\}$, where $x_{i}$ is a document with label $y_{i}\in\mathcal{Y}$, where $m=|\mathcal{Y}|$ is the number of labels. For each document $x_{i}$, we represent latent (topical) confounds—domain-specific and superficial document features—as a $K$-dimensional multinomial distribution $t_{i}\in\{(t_{1},\ldots,t_{K})\ |\ \sum_{j=1}^{K}t_{j}=1\}$. In our task, the confounds are topics, so that each $t_{j}$ represents the proportion of document $i$ associated with topic $j$ but these topics are not given a priori. In this work, the number of topics $K$, equals $m$, but the methods presented in this work are valid for any number of topics.

In practice, this optimization is done in an alternating fashion by minimizing the following two quantities:

The training schedule is critical in adversarial setups where the loss has two competing terms (Mescheder et al., 2018; Arjovsky and Bottou, 2017; Roth et al., 2017); here, these terms minimize classification loss while maximizing the topic prediction loss. Algorithm 1 details our proposed alternating learning procedure.

2 Multiple Adversaries

Experimental Setup

We evaluate our topical confound demotion method on the L1ID task. We show experiments with two datasets where L2 is English: the L2-Reddit dataset described in §2, and TOEFL17, a collection of essays authored by non-native English speakers who apply for academic studies in the US (Malmasi et al., 2017). This corpus reflects eleven L1s: Arabic, Chinese, French, German, Hindi, Italian, Japanese, Korean, Spanish, Telugu, and Turkish. The training data include 11,000 authors (1,000 per L1) and the development set has 1,100 essays per L1. We evaluate on the development set. Each essay is also marked with a prompt ID which was given to the authors to write the essay. There are 8 prompts in total, based on which we construct 8 versions of train and test set. In each version, we remove essays marked with one of the prompts from both the train and the development sets, and consider the removed essays from the development set an “out-of-domain” test set. We refer to the version where prompt “P$K$” is out-of-domain as “–P$K$” in the results (Table 3), $K\in\{0,\ldots,7\}$.

2 Implementation Details

3 Baselines

We consider several baselines that are intended to capture the stylistic features of the texts, explicitly avoiding content.

Replicating Goldin et al. (2018), we trained a logistic regression classifier with three types of features: function words, POS trigrams, and sentence length, all of which are reflective of the style of writing. We deliberately avoided using content features (e.g., word frequencies).

Classification with no adversary on masked texts (lo-top-K𝐾K)

Adversarial training with gradient reversal (gr-lo)

LDA topics as confounds (alt-lda)

We trained LDA on the training set and for each example in the training set, generated a probability distribution (over $50$ topics), and used it as topical confound with our proposed learning setup, alternating classifier-adversary training.

Results

We begin with experiments on the TOEFL17 dataset, where predicting L1 is an easier task due to the lower proficiency of the authors. Table 3 reports the accuracy of our proposed model, denoted alt-lo, compared to the logistic regression baseline (lr), and two adversarial baselines: one demotes latent log-odds-based topics via gradient reversal (gr-lo), and another uses our proposed novel learning procedure but demotes baseline LDA topics (alt-lda). We report both in-domain accuracy and out-of-domain results; the latter is obtained by averaging the accuracy of each set “–P$K$” over $K\in\{0,\ldots,7\}$.

Our model strongly outperforms all baselines that demote confounds, in both classification setups. We observe in our experiments that gradient reversal is especially unstable and hyperparameter sensitive: it has been shown to work well with categorical confounds like domain type or binary gender, but in demoting continuous outputs like a topic distribution, we observe it is not effective. The proposed alternating training with multiple discriminators obtains better results, and replacing LDA with log-odds-based topics also improves both in-domain and (much more substantially) out-of-domain predictions, confirming the effectiveness of our proposed innovations.

A vanilla classifier without demoting confounds (denoted in §2 as no-adv) yields in-domain and out-of-domain accuracies of 62.0 and 58.3, respectively. We would expect that the better generalization power of our proposed model would come at a price of lower accuracy in-domain. Our goal is to capture the true signals of L1, rather than superficial patterns that are more frequent in the data and artificially boost the performance in no-adv settings. This is indeed what we observe.

For example, the text “…i agree with you on the prolonged war if the plc heartland (poland proper) was not as rich as it was i dont really see how we would been …” in the dataset is labeled as “Polish” instead of the gold label “Swedish” by the no-adv classifier, likely because of the mention of the term “poland”, but the adv-lo model predicts it correctly since it likely picks on other features that indicate non-fluency, like “we would been”. Such naive classification errors become especially costly in making predictions about people’s demographic attributes: ethnicity, which often correlates with L1, but also gender, race, religion, and others Hardt et al. (2016); Beutel et al. (2017).

2 L2-Reddit Dataset

Next, we experiment with L2-Reddit, a larger and more challenging dataset (since many speakers in the dataset are highly fluent, and the signal of their native language is weaker). The performance of the simple baselines on this dataset is shown in Table 4. The accuracy of the linear classifier is poor (compared to Table 1), perhaps because it fails to capture some contextual features learned by the neural network models. With lo-top-20, the performance on both test sets improves. It slightly degrades when more words are removed, perhaps because some words indicative of L1 are also removed.

Finally, we evaluate the impact of our novel training procedure and the quality of our proposed topical confound identification method. We compare our proposed solution, denoted alt-lo, with two alternatives, as before, one with a different learning setup (gr-lo) and one with a different confound representation (alt-lda). Table 5 summarizes the results: our proposed learning procedure alt-lo performs better than both the alternatives. Unsurprisingly, the model trained with gradient reversal (gr-lo) performs particularly poorly; this was our primary motivation to explore better learning techniques.

To further confirm that the alt-lo model is not learning topical features, we repeat the experiment presented in Table 1—masking the top $K$ topical words (based on log-odds scores) from the test sets, but not retraining the models—now, with our proposed model alt-lo. Table 6 shows that in contrast to standard models that do not demote topical confounds (as in Table 1), there is less degradation in the performance of alt-lo. We conjecture that our model is stable to demoting topics because it learns relevant stylistic features, rather than spurious correlations.

Analysis

We present an analysis of what the models are learning, based on words they attend to for classification. We focus on the L2-Reddit dataset.

Following Pryzant et al. (2018), we generated a lexicon of most attended words by (1) running the model on the test set and saving the attention score for each word; and (2) for each word, computing its average attentional score and selecting the top-$k$ words based on this score.

What emerges from this lexicon (Table 7) is a dramatic difference between the top indicative words in the various models. Whereas in the baseline model all the most indicative words are proper nouns, the alt-lo model highlights exclusively function words. The proper nouns in the baseline model are all geographical terms directly associated with the L1s reflected in the L2-Reddit dataset: they are easy giveaways of the authors’ L1s, but they are meaningless linguistically. In contrast, the function words highlighted in the alt-lo model are mostly prepositions and determiners; it is well known that nonnative speakers are challenged by the use of prepositions (in any L2, English included). The distribution of determiners is also a challenge for nonnatives, and the correct usage of the in particular is quite hard for learners to master. These challenges are evident from the most indicative words of our model. Observe also that the lo-top-$50$ model is somewhere in the middle: it includes some proper nouns (including geographical terms such as eu or us) but also several function words. A more detailed analysis of these observations is left for future work.

Recently, there has been a debate on whether attention can be used to explain model decisions (Serrano and Smith, 2019; Jain and Wallace, 2019; Wiegreffe and Pinter, 2019), we thus present additional analysis of our proposed method based on saliency maps (Ding et al., 2019). Saliency maps have been shown to better capture word alignment than attention probabilities in neural machine translation. This method is based on computing the gradient of the probability of the predicted label with respect to each word in the input text and normalizing the gradient to obtain probabilities. We use saliency maps to generate lexicons similar to the ones generated using attention. As shown in table 8, the top indicative words for baseline and lo-top-50 follow a similar pattern as the ones obtained with attention scores. In line with results in Table 7, salient words for alt-lo are determiners and prepositions. However, saliency maps also reveal that our proposed approach still attends to some geographical terms that were not demoted by our classifier.

Related Work

Controlling for confounds is an active field of research, especially in the medical domain, where the common solution is to do random trials or propensity score matching (Rosenbaum and Rubin, 1985). Paul (2017) tackled the problem of learning causal associations between word features and class labels using propensity matching for the task of sentiment analysis. This method is not scalable to large text datasets as it involves training a logistic regression model for every word type. Tan et al. (2014) built models to estimate the number of retweets of Twitter messages and addressed confounding factors by matching tweets of the same author and topic. Reis and Culotta (2018) proposed a statistical technique called Pearl’s back-door adjustment for text classification (Pearl, 2009). All these works focused on a bag-of-words model with lexical features only.

Adversarial training in text

Much recent work focuses on learning textual representations that are invariant to selective properties of the text. This work used domain adaptation and transfer learning (Ganin et al., 2016; Tzeng et al., 2014; Xie et al., 2017), either to remove sensitive attributes such as demographic information (Li et al., 2018; Elazar and Goldberg, 2018; Beutel et al., 2017; Coavoux et al., 2018), or to understand costumer behavior for social science applications (Pryzant et al., 2018). Most of the work in this area, however, focuses on cases where these confounds are known in advance and their values are given along with the training data. Our presented approach is most closely related to Coavoux et al. (2018) who proposed an alternating optimization method to learn privacy-preserving text representations. This work focuses on demoting binary-valued attributes by maximizing the likelihood of erroneous label using a single adversary network, which we have shown to be inadequate in our experiments. In constrast, we propose a more general method focusing on multinomial distributions which we push towards a uniform distribution with the help of multiple adversaries.

Native language identification

The L1ID task was introduced by Koppel et al. (2005), who worked on the International Corpus of Learner English (Granger, 2003). The same experimental setup was adopted by several other authors (Tsur and Rappoport, 2007; Wong and Dras, 2009, 2011). Since the release of nonnative TOEFL essays by the Educational Testing Service (Blanchard et al., 2013), the task gained popularity and this dataset has been used for two L1ID Shared Tasks (Tetreault et al., 2013; Malmasi et al., 2017).

Malmasi and Dras (2017) report that the state of the art is a linear classifier with character $n$-grams and lexical and morphosyntactic features.

The best accuracy under cross-validation on the TOEFL17 dataset, which includes 11 native languages (with a rather diverse distribution of language families), was 85.2%.

The above works all identify the L1 of learners. Identifying the native language of advanced, fluent speakers is a much harder task. Goldin et al. (2018) addressed this task, using the L2-Reddit dataset with as many as 23 different L1s, all of them European and many which are typologically close, which makes the task even harder. They experimented with a variety of features, using logistic regression as the classifier, and achieved results as high as 69% accuracy with cross-validation; however, when testing their classifier outside the domain it was trained on (Reddit forums focusing on European issues), accuracy dropped to 36%.

Conclusion

We introduced a method to represent unknown confounds in text classification using topic models and log-odds scores, and a new general method with alternating optimization to learn textual representations which are invariant of confounds. We evaluated the proposed solution on the task of native language identification, and showed that it learns to make predictions using stylistic features, rather than focus on topical information.

The learning procedure we presented is general and applicable to other tasks that require learning invariant representations with respect to some attribute of text (some of which are discussed in §8). We plan to evaluate our proposed solution on other tasks where topics can be latent confounds, like predicting gender bias (Voigt et al., 2018). We leave this exploration for future work.

Acknowledgments

The authors acknowledge helpful input from the anonymous reviewers. This work was supported in part by NSF grants IIS-1812327 and IIS-1813153, by grant no. 2017699 from the United States-Israel Binational Science Foundation (BSF), and by grant no. LU 856/13-1 from the Deutsche Forschungsgemeinschaft. Finally, the authors also thank Anjalie Field, Biswajit Paria, Ella Rabinovich, and Gili Goldin for helpful discussions.

References