Challenges in Automated Debiasing for Toxic Language Detection

Introduction

Current hate speech or toxic language detectionWe use hate speech and toxic language interchangeably in this work, though their definitions do not perfectly align. systems exhibit problematic and discriminatory behavior that causes them to have disparate negative impact on minority populations Yasin (2018); Guynn (2020); Kim et al. (2020); Dias Oliva et al. (2020). Tweets simply containing a minority identity mention are commonly flagged as toxic by current systems, in contrast to those containing majority identity mentions, as illustrated in Figure 1.

At the core of the issue are dataset biases, i.e., spurious correlations between surface patterns and annotated toxicity labels (§2), which stem from the data creation process Sap et al. (2019). Previous work has outlined two such biases for hate speech datasets (both shown in Figure 1): lexical bias which associates toxicity with the presence of certain words (e.g., profanities, identity mentions; Dixon et al., 2018; Dinan et al., 2019) and dialectal bias, where toxicity is correlated with surface markers of African American English (aae; Davidson et al., 2019; Sap et al., 2019). When trained on biased datasets, models acquire and exacerbate these biases (e.g., flagging text by Black authors as more toxic than by white authors; Sap et al., 2019; Zhang et al., 2018).

Concurrently, there has been elevated interest in developing debiasing methods for standard natural language understanding (NLU) tasks, i.e., methods that aim to decrease over-reliance on spurious correlations in NLU models Clark et al. (2019); He et al. (2019); Karimi Mahabadi et al. (2020); Bras et al. (2020). This raises a natural question: are current debiasing approaches effective for mitigating biases specific to toxic language detection?

In this work, we address the above question by investigating two classes of debiasing approaches to mitigate lexical and dialectal biases—one that employs additional training objectives for bias removal, and another that filters training instances likely exhibiting spurious biases (§3). Through comprehensive experiments, we show that both approaches face major challenges in mitigating biases from a model trained on a biased dataset (in our case, the dataset from Founta et al., 2018) for toxic language detection. While data filtering results in reduced bias associations in the data, models trained on filtered datasets still pick up on lexical (§4) and dialectal biases (§5). We find that dialectal biases are particularly challenging to address, as has also been shown by Xia et al. (2020). “Debiased” models still disproportionately flag text in certain dialects as toxic. Notably, mitigating dialectal bias through current debiasing methods does not mitigate a model’s propensity to label tweets by Black authors as more toxic than by white authors.

We additionally explore an alternative proof-of-concept study—relabeling supposedly toxic training instances whose automatic translations into a majority dialect are deemed non-toxic by the classifier. To this end, we create a synthetic dataset via few-shot dialect translation system built with GPT-3 Brown et al. (2020). While only an illustrative solution, it nevertheless takes into account the dialectal context of the tweet, resulting in a model less prone to dialectal and racial biases (§6). Overall, our findings indicate that debiasing a model already trained on biased toxic language data can be challenging, compared to relabeling the data to remove existing biases. Our code and data are publicly available on Github.https://github.com/XuhuiZhou/Toxic_Debias

Biases in Toxic Language Detection

We test the use of debiasingOur definition of “bias” is specific to the social biases in toxic language detection datasets, grounded as lexical and dialectal biases; see Blodgett et al. (2020) for a detailed investigation of the term “bias”. methods for the task of toxic language detection, which aims to flag rude, offensive, hateful, or toxic language on the internet, with the goal of moderating online communities Roberts (2019); Vidgen et al. (2019). This task differs in several ways from the natural language understanding (NLU) tasks that debiasing methods have been successful on, such as textual entailment (e.g., SNLI, MNLI; Bowman et al., 2015; Williams et al., 2018) or reading comprehension (e.g., SQuAD; Rajpurkar et al., 2016). First, compared to these NLU tasks where there is one correct label, the toxicity of language is inherently more nuanced, subjective, and contextual, which causes toxic language datasets to have lower agreement in general Ross et al. (2017). Second, the dataset biases in NLU are predominantly artifacts introduced during data creation (e.g., negations, exaggerations; Schwartz et al., 2017; Gururangan et al., 2018), whereas those in toxic language detection are grounded in the social dynamics of the world Spears (1998); Technau (2018). For example, viewing aae as a more toxic or less proper variety of English is a form of linguistic discrimination that upholds racial hierarchies in the United States (Rosa and Flores, 2017).

In this work, we consider two broad categories of toxic language dataset biases—lexical (§2.1) and dialectal (§2.2). Our experiments focus on a single, widely used dataset (§2.3) from Founta et al. (2018).

Current toxic language detection systems often rely on the presence or absence of certain words (e.g., swear words, identity mentions) to make their predictions Dixon et al. (2018); Dinan et al. (2019). While most previous analyses of this bias relied on a simple list of “bad” words Davidson et al. (2019); Dinan et al. (2019),https://tinyurl.com/list-of-bad-words we take a more nuanced view of how lexical items can convey toxicity, inspired by work in pragmatics and sociolinguistics of rudeness (Dynel, 2015; Kasper, 1990, inter alia). Specifically, we manually split our full list of words into three distinct categories depending on the extent to which they carry profane or hateful meanings or are simply associated with hateful contexts.We note, however, that this categorization is in itself subjective. We refer to the full set of words as ToxTrig, for Toxicity Triggers, which is included in our released repository.https://github.com/XuhuiZhou/Toxic_Debias/blob/master/data/word_based_bias_list.csv

refers to descriptive mentions of minoritized demographic or social identities (e.g., gay, female, Muslim). While these mentions are not usually inherently offensive by themselves, they are often found in offensive statements that are hateful towards minorities (Dixon et al., 2018). We detect these identity mentions in text using a list of 26 regular expressions.

Possibly offensive minority identity mentions (OI)

are mentions of minoritized identities that could denote profanity or hate depending on pragmatic and contextual interpretations. This includes slurs and objectifying outdated terms to refer to minority groups, which are usually understood as attacks. Additionally, this includes reclaimed slurs (queer, n*gga), which connote less offensive intent when spoken by in-group members compared to out-group members Croom (2013).

Possibly offensive non-identity mentions (OnI)

contains swear words and other profanities, which are usually offensive but not associated to any social groups (e.g., f*ck, sh*t). Note that the pragmatic interpretation of these words is not necessarily always toxic or offensive Dynel (2012), as they are often used to convey closeness between the speaker and listener or emphasize the emotionality of a statement (e.g., second example in in Figure 1).

2 Dialectal Biases (aae)

Current toxic language detection systems also associate higher toxicity with dialectal markers of African American English (aae; Sap et al., 2019; Davidson et al., 2019). Since aae is a variety of English that is common among African Americans and often signals a cultural identity in the US Green (2002), this dialect-based racial bias causes speech by Black authors to be suppressed more often than non-Black authors Sap et al. (2019), thereby exacerbating racial inequality Rosa (2019).

In our experiments, we estimate the dialect of a tweet using a topic model from Blodgett et al. (2016). This model was trained on 60M tweets, where the dialect of the tweet was inferred from the model coordinates, which yielded a probability of a tweet being in one of four dialects (African-American English, white-aligned English, Hispanic, and other). In this study, we only focus on African-American English (aae) and white-aligned English (wae) tweets; both definitions are based on US English, as per Blodgett et al. (2016).We avoid using disputed terms such as general American English, standard American English, or mainstream US English, which are frequently used for wae, since we believe that no dialect should be privileged with the designation “general”, “standard”, or “mainstream” Rosa (2019). Our experiments either use the probability of a tweet being in these dialects, or assign tweets their estimated-most-probable dialect.

3 Dataset for Toxic Language Detection

We focus our analyses on a widely used hate speech dataset of English tweets Founta et al. (2018). The tweets were collected using a multi-round bootstrapping procedure, and were labeled out of contextOnly the tweet text—no profile information or conversational context—was shown to annotators. for toxic language. We focus on the 86k tweets that are annotated as hateful, abusive, or neither and discard those labelled as spam. We aggregate the abusive and hateful labels into a single toxic category, yielding 32k toxic and 54k non-toxic tweets.We also explored using another widely used hate speech dataset Davidson et al. (2017), which collected tweets using a seed list of swear words and slurs. However, in line with findings by Xia et al. (2020), debiasing led to degenerate behavior due to the data collection process, as discussed in Appendix B.

Debiasing Methods

We consider two types of debiasing methods from current literature. The first type addresses known, pre-defined biases—such as lexical and dialectal biases for hate speech detection, via a model-based approach involving additional training objectives (§3.1). In contrast, the second type is agnostic to prior knowledge about biases, and instead filters out examples that appear “too easy” and might hence contain spurious correlations (§3.2).

We use the Learned-Mixin method of Clark et al. (2019), which achieved high out-of-distribution (OOD) performance on several NLU tasks, for debiased training. This method trains an ensemble containing a bias-only model which only uses pre-defined features corresponding to known biases, and a full model which uses all features. Intuitively, the ensemble encourages the full model to rely more on features unrelated to the biases. Once trained, the bias-only model is discarded, and only the “bias-free” full model is used for inference, following Clark et al. (2019).

Given its effectiveness on bag-of-words (BoW) features, we use an SVM classifier as the lexical-bias-only model. For example, the ToxTrig-only model counts the frequency of ToxTrig words in each tweet. Our dialectal-bias-only model uses the probability of dialects (aae, wae, Hispanic, and other) obtained from a dialect detector Blodgett et al. (2016) as features in a SVM classifier.

Full model

We fine-tune a RoBERTa-large classifier Liu et al. (2019), a state-of-the-art classifier for the toxicity detection task. See Appendix A.1 for more modeling details.

Note that we only consider the Learned-Mixin-OnI and Learned-Mixin-ToxTrig models for lexical debiasing, due to poor accuracies of the bias-only models for nOI and OI.The nOI and OI bias-only models reach 63% and 67% accuracy, respectively, which is empirically hard for the ensemble to use. This is likely due to low coverage in the train set of those categories (4.43% nOI and 4.25% OI).

2 Data Filtering for Spurious Biases

In addition to debiasing methods that handle known biases, we also explore automated approaches which filter out instances exhibiting unspecified, spurious biases. Specifically, we describe below two data selection methods that have shown strong OOD performance.

is an algorithm based on the key intuition that examples predicted correctly by the simplest methods likely exhibit spurious biases. An ensemble of simple linear classifiers is trained and tested on different partitions of the data; test instances which are “predictable”, or classified correctly by most classifiers in the ensemble are discarded. The algorithm is iterative, and is repeated until a target data size is achieved. Models trained on this filtered dataset achieve higher performance on OOD and adversarially constructed test sets, compared to the original model, on several text and image classification datasets. This indicates a reduction in spurious biases in the filtered data.

DataMaps Swayamdipta et al. (2020)

show the presence of distinct regions in a dataset—namely, easy, hard and ambiguous—defined with respect to a given model. These regions are discovered based on the training dynamics of a model, determined by the model’s confidence in the true class, for each example, as well as the variability of this confidence, throughout training epochs. Swayamdipta et al. (2020) show that training exclusively on the hard and ambiguous regions of the data results in high OOD performance, indicating lower prevalance of spurious biases. The easy region is the largest in size for RoBERTa; however, experiments showed that training exclusively on these examples hurt OOD generalization on different NLU tasks. Following this work, we create DataMaps-Easy, DataMaps-Ambiguous, and DataMaps-Hard subsets for our dataset.

Following Swayamdipta et al. (2020), we set the target filtered subset size to 33% of the original training set for both filtering methods, but our filtering additionally preserved the original label proportions. We then fine-tune a RoBERTa-large classifer on these filtered subsets; see Appendix A.2 for more details.

Experiments: Lexical Biases

We investigate the effect of debiasing approaches (§3) on removing lexical biases in hate speech detection. First, we discuss the evaluation framework for measuring bias reduction (§4.1). We present quantitative (§4.2) and qualitative (§4.3) results on lexical bias removal for all debiasing approaches, and OOD evaluation for debiased training methods (§4.4). See Appendix A.3 for hyperparameters and other experimental settings.

We report the performance of all models as overall accuracy and $F_{1}$ with respect to the toxic class. Given that current hate speech systems tend to rely heavily on the presence of nOI, OI, and OnI mentions (§2.1) for labeling text as toxic, we use false positive rate (FPR) over each of these categories to measure the degree of bias in the model, following Hardt et al. (2016) and Xia et al. (2020). Specifically, we report the FPR of a model on tweets containing nOI (FPR${}_{\textsc{nOI}}$), OI (FPR${}_{\textsc{OI}}$), and OnI (FPR${}_{\textsc{OnI}}$), as well the $F_{1}$ corresponding to each of these classes. Intuitively, the lower the FPR∗, the less the model infers lexical associations for toxicity, and hence is less biased.

We additionally consider metrics based on spurious lexical associations for data filtering approaches. This measures prevalence of spurious surface patterns in the filtered datasets, which might propagate to models trained on the data. Specifically, we report the Pearson’s correlation between the gold standard toxicity label and whether or not it contains nOI, OI, or OnI mentions. These correlations are denoted as $R_{\textsc{OnI}}$, $R_{\textsc{nOI}}$, and $R_{\textsc{OI}}$, respectively; lower values indicate reduction in lexical biases.

Baselines

We consider comparison against two natural baselines: a vanilla RoBERTa-large classifier trained on the original dataset (Original). We also consider a baseline trained on a random selection of the training data (Random), for comparison with data filtering methods for debiasing. Each subset is trained on 33% of the training data.

2 Results for Lexical Bias Reduction

First, we measure the reduction in lexical biases in filtered datasets, as given by AFLite and DataMaps. As shown in Table 1, subsets given by AFLite and the ambiguous and hard regions produced by DataMaps reduce the overall associations between ToxTrig words and toxicity, compared to the original and random baselines; DataMaps-Hard has the largest reduction. On the other hand, as expected, DataMaps-Easy shows an increased association between ToxTrig mentions and toxicity, showing that the these examples display overt lexical biases.

Table 2 shows results for lexical bias reduction using both debiased training approaches, as well as models trained on datasets filtered using AFLite and all three regions from DataMaps. Both debiased training approaches, LMixin-OnI and LMixin-ToxTrig, reduce FPR${}_{\textsc{OnI}}$ as well as FPR${}_{\textsc{OI}}$ by a large amount. However, both approaches also hurt in-distribution test performance, indicating that OnI and other ToxTrig features are essential for good performance.When we combine the bias-only model and the full model, we obtain competitive performance (see Appendix A.4). In contrast, the models trained on hard and ambiguous subsets from DataMaps both preserve in-distribution performance, even though they are trained only a third of the original data. They also reduce the rate of falsely predicting nOI mentions as toxic (FPR${}_{\textsc{nOI}}$), while not showing much improvement for OnI and maintaining FPR${}_{\textsc{OI}}$ of the original baseline.

Surprisingly, the model trained on the easy subset from DataMaps shows good bias reduction on the nOI and OnI categories, while matching the random selection baseline for OI. This is despite DataMaps-Easy showing an increased association between ToxTrig mentions and toxicity (Table 1). Notably, the $F_{1}$ for all categories suffers under this model, indicating that it is less competent than the baseline. These results suggest that reduced associations in the data might not necessarily lead to debiased models trained on the same data. Overall, no single approach outperforms all others across different categories for lexical debiasing.

3 Qualitative Analysis

A qualitative study of the Founta et al. (2018) test set shows the presence of many annotation errors. We show three representative annotation errors in Table 3. The first example contains an atypical example of toxicity, towards white folks, which the annotators might have been unaware of. It also contains a link which annotators had access to, but not models. The second contains the word p*ss which the annotators may have relied for their assessment. The third encourages violence/abuse towards an identity which isn’t typically the target of violence. Interestingly, the DataMaps-Easy predictions agree with all the gold standard annotations; perhaps such annotation errors and ambiguity are responsible for the performance discussed in §4.2. These annotation ambiguities might also impair our measurement for models’ performance and debiasing effects, and expose a limitation of these datasets.

4 Adversarial Evaluation: OnI-Adv

To further study the reliance of debiased models on the OnI words, we use the test set from Dinan et al. (2019), which contains 3000 manually created sentences, 300 toxic. The toxic sentences barely contain any OnI mentions, e.g., Cheese is made by french people who smell. We call this test set OnI-Adv (for adversarial) since it challenges models with a reversal in the association between toxicity and offensive non-identity words (e.g., “f*ck”, “sh*t”).

We report $F_{1}$ for all models in Figure 2, which shows how well a model identifies toxicity in offensive tweets that do not contain overtly lexical cues of toxicity. The debiased training approaches improve over the baselines; data filtering methods do not. One reason for this might be that data filtering methods were trained on much less data than both LMixin models. Regardless, none of the models we test are good at predicting subtle, non-overt toxicity.

Experiments: Dialectal and Racial Biases

We test the efficacy of the bias reduction methods from §3 for dialectal bias (§2.2) reduction.

For our dialectal bias experiments, we first infer the dialect of a tweet as described in §2.2. Then, analogous to the lexical bias evaluation, we quantify the dialectal debiasing using the Pearson’s correlation between estimated probabilities of aae and toxicity ($R_{\textsc{aae}}$), and the false positive rates of models on aae tweets (FPR${}_{\textsc{aae}}$). See Appendix A.3 for hyperparameters and other experimental settings.

Results in Table 4 show that almost all data filtering and debiasing methods reduce dialectal biases, with DataMaps-Easy as the exception (consistent with Table 1). Notably, DataMaps-Hard performs the best at dialectal debiasing, both in terms of toxicity-aae correlation ($R_{\textsc{aae}}$) and in terms of false flagging of toxicity (FPR${}_{\textsc{aae}}$). Interestingly, most models’ decrease in false flagging is small, suggesting room for improvement.

2 Racial Biases

To quantify the real-world impact of dialect-based racial bias, we measure the rates of toxicity predicted by models on a corpus of tweets for which the race of authors is available, but not annotations of toxicity. Specifically, we consider the dataset released by Preoţiuc-Pietro and Ungar (2018), which consists of 5.4M tweets, collected from 4,132 survey participants (3,184 White, 374 African American) with self-reported race/ethnicity and Twitter user handles.For efficiency, we randomly select 12k tweets from the dataset as the OOD test set.

We quantify our models’ racial bias by measuring the difference in rates of flagging tweets by African American authors and those by white authors, following Sap et al. (2019).Note that we assume that authors from all races have the same likelihood of writing toxic language.

Listed in Table 5, our results show that automatic debiasing methods do not consistently decrease the racial discrepancy in flagging toxicity. Notably, the toxicity rates on tweets by African American authors—and the diferences compared to white authors—are similar across all debiasing methods and baselines, except for DataMaps-Easy, which shows the most racial bias in toxicity flagging. Surprisingly, DataMaps-Hard, which mitigated dialectal bias the best out of all debiasing methods, also shows high discrepancy between author races. Confirming previous results, this suggests that debiasing these systems requires more than automatic debiasing methods.

Towards Data Relabeling

Based on our quantitative and qualitative analyses, we believe there still is room for improvement in debiasing hate speech detection. Therefore, we turn our attention to the role of label noise in datasets. Partly inspired by our qualitative analyses of debiased models’ predictions, we design a proof-of-concept study where we automatically correct the label of tweets using a(n automatic) dialectal translation of the tweet, inspired by previous work showing that highlighting aae tweets’ dialect led them to be labeled as less toxic Sap et al. (2019). We conclude this study by discussing the limitations and ethical implications of the synthetic data, and cautioning against its real-world application.

Focusing on dialectal bias, our key assumption is that an aae tweet and its corresponding wae version should have the same toxicity label, therefore toxic aae tweets whose wae versions are non-toxic are candidates for label correction.Note that this assumption does not hold for lexical items, because substituting lexical items (e.g., swapping a minority mention for a majority mention) would drastically change the denotational meaning of the sentence.

However, gold-standard translations of aae to wae would require qualified translators, and automatic aae-to-wae translation systems do not exist, to the best of our knowledge. Therefore, we create a proof-of-concept study—we set up a aae to wae “translation” system using the few-shot capabilities of the GPT-3 language model Brown et al. (2020). Under this mechanism, we prompt GPT-3 with four translation pairs (taken from Spears, 1998) and an aae tweet from our training data, and generate its wae “translation”. The list of prompts, as well as further details, are provided in Appendix C. Note that we do not recommend this approach to build large scale parallel data for dialects, as discussed under ethical implications and limitations.

Next, as per our heuristic, we only relabel toxic aae tweets whose wae translation is predicted as non-toxic by either our vanilla classifier trained on the original Founta et al. (2018) dataset, or an identical classifier trained on the wae translated tweets. The resulting dataset (aae-relabeled) is the same size as the original dataset, but with 954 (12%) out of 8260 toxic aae tweets relabeled as non-toxic (examples in Table 6). To assess the validity of the relabeling, the first three authors manually annotated toxicity of 50 randomly selected relabeled tweets. On average, authors agreed with 84% of the relabeling decisions.

Then, we evaluate the dialectal bias of aae-relabeled and quantify the dialect and racial prediction biases from a RoBERTa-large classifier trained on aae-relabeled, following §5. As shown in the last row of Table 4, this relabeling scheme decreases dialectal bias more than any other debiasing method, specifically as measured by the FPR on aae tweets, with one point drop in $F_{1}$ score. The $F_{1}$ score on the “gold” test data (Table 4) are not fully reliable, as test data contain label biases and better performance could come from exploiting these biases. As shown in Table 5, the model trained on aae-relabeled has the lowest racial disparity in toxicity flagging rates compared to all other methods.

These results highlight that debiasing methods are much less effective at mitigating dialectal dataset biases compared to data relabeling. For future investigations, we recommend obtaining human-written aae-wae pairs (e.g., as done by Groenwold et al., 2020). Additionally, to ensure less biased toxicity labeling, we recommend recruiting aae speakers or experts for avoiding over-identification of aae-markers as toxic Spears (1998); Croom (2013). Alternatively, we recommend exploring more holistic representations of social biases or toxicity (e.g., Social Bias Frames; Sap et al., 2020).

The above synthetic setting is meant to illustrate the role of labeling quality on biases in annotations. We strongly caution against using this approach in real-world applications, such as building parallel datasets for dialects. First, due to how its training data was selected, GPT-3 has likely not been exposed to many African American English varieties during training Jo and Gebru (2020). Second, pretrained language models are known to generate toxic language at non-trivial rates Gehman et al. (2020), which could cause differential toxicity in the translations.

Related Work

As the popularity of hate speech and toxic language detection systems has grown, several biases have been found in dataset and models, spurring various debiasing efforts to mitigate these individual biases (e.g., gender bias, racial bias; Park et al., 2018; Sap et al., 2019; Davidson et al., 2019). Some work tackles identity-based biases, e.g., using data re-balancing Dixon et al. (2018), or adversarial feature learning Vaidya et al. (2019). Less work has tackled racial or dialectal bias. Notably, Xia et al. (2020) use adversarial training to prevent the model from associating toxicity with aae, showing only small improvements in fairness. Based on those results, we do not explore adversarial methods, opting instead for ensemble-based methods of predefined bias reduction. In contemporary work, Mozafari et al. (2020) use a re-weighting mechanism, which shows some effects in debiasing racial bias. We leave it for future work to evaluate this method in our setting. In contrast to all previous work, our experiments also measure the effectiveness of bias-agnostic methods.

Other General Debiasing Methods

Several approaches for debiasing NLU tasks have been proposed lately. Some approaches rely on adversarial training to remove protected attributes (e.g. gender or race), from a model’s internal representations (Zhang et al., 2018; Wang et al., 2019; Xia et al., 2020). Other approaches include confidence regularization Utama et al. (2020), as well as other product of expert approaches He et al. (2019); Karimi Mahabadi et al. (2020) similar to the debiased training approach from Clark et al. (2019), which is the only debiased training we employ due to its relatively strong performance.

Conclusion

We investigate whether toxic language detection systems can be debiased using recently introduced methods for debiasing text classification in NLU tasks. Focusing on two types of biases, lexical and dialectal, our experiments show that these methods face significant challenges in reducing the biased behavior in toxicity detectors. This indicates that biases in toxic language detection might be different in nature compared to spurious associations studied in typical NLU settings. We studied a synthetic scheme for relabeling examples with potential dialectal biases; our results indicate that correcting noisy labels results in better bias reduction. Our findings suggest that instead of solely relying on development of automatic debiasing for existing, imperfect datasets, future work focus primarily on the quality of the underlying data for hate speech detection, such as accounting for speaker identity and dialect. Indeed, such efforts could act as an important step towards making systems less discriminatory, and hence safe and usable.

Acknowledgments

We thank the anonymous reviewers and Laura Vianna for helpful comments on this work. This research was supported in part by NSF grants 1813153 and 1714566.

References

Appendix

Appendix A Further Details for Models

The learned-mixin ensemble allows the model to explicitly determine how much to trust the bias given the input:

where $\mathbf{x}_{i}$ is the $i$th input text, $p_{i}$ and $b_{i}$ is the toxicity prediction produced by RoBERTa, and bias-only model respectively, and $g$ is a parametric function, which is defined as $\text{softplus}(\mathbf{w}\cdot\mathbf{h}_{i})$, where $\mathbf{w}$ is a learned vector, $\mathbf{h}_{i}$ is the last hidden layer of the model for example $\mathbf{x}_{i}$, and the $\text{softplus}(x)=\text{log}(1+\exp{x})$. To prevent the learned-mixin ensemble from ignoring $b_{i}$, Clark et al. (2019) add an entropy penalty ($H$) to the loss:

Where $H(z)=-\sum_{j}z_{j}\log z_{j}$ is the entropy and $\alpha$ is a hyperparameter.

A.2 Data Filtering

For the data filtering methods, we first filter data to 50% of the original data as in Swayamdipta et al. (2020). Then we further downsample the dataset to 33% of the original data to control that each training set has the same toxic ratio as the original training set. This step is to avoid confounding our results with different toxic ratio among different training sets.

A.3 Training Settings

For all the experiments, we fine-tune RoBERTa-large Liu et al. (2019) over the corresponding corpus with one GTX2080 Ti. We use the default hyperparameters as provided in the HuggingFace Transformers library (Wolf et al., 2019), with two major changes: we use a learning rate of $10^{-5}$ and 8 batch size in all experiments.

A.4 Prediction Combining with Bias-only Model

To prevent the possibility that our LMixin-ToxTrig/OnI is not well trained, thus resulting in the decrease of models’ in-distribution performance, we use the joint-prediction from the main and bias-only model to infer the in-distribution test set and they obtain 94.15% and 94.17% accuracy, respectively. This is competitive performance as shown in Table 2.

Appendix B Alternative Dataset of Toxic Language

Davidson et al. (2017) collected data from Twitter, starting with 1,000 terms from HateBase (an online database of hate speech terms) as seeds, which the process relies on lexical biases. We find that performing data filtering methods over this dataset leads to degenerate behaviour. Specifically, as shown in Table 7, the easy region demonstrates least spurious correlation due to its heavily skewed class distribution, which further prevent us from downsampling to control the toxic ratio. We also train LMixin-ToxTrig and LMixin-dialect over the dataset. Table 8 shows that FPR of the debiased model increase instead except for the OI category and Table 9’s results behave in-line with Table 4.

Appendix C Few-shot aae-to-wae Translation

Note that we do not recommend the following approach to build large scale parallel data for dialects, as discussed under ethical implications and limitations (§6).

We use GPT-3 Brown et al. (2020) to create a few-shot aae-to-wae translation system, using the following set of example translation pairs drawn from Spears (1998):

aae: Get your triflin’ ass out of here. wae: Get your trifling self out of here. aae: I saw his ass yesterday. wae: I saw him yesterday. aae: His ass is gonna get fried. wae: He is gonna get fried aae: Wassup, nigga? wae: What’s up bro? aae: $\langle$tweet$\rangle$ wae:

Note that Spears (1998) refers to wae as White language varieties, and deals with English prevalent in the United States.

We prepend the formatted example pairs to each aae tweet in our training data, and generate the translation from GPT-3 using top-0.95 nucleus sampling with a temperature of 0.5. Prompts, formatting, and generation parameters were chosen based on manual inspection of the output.