Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality

Introduction

With the growing adoption of algorithmic decision making systems, e.g., AI and machine learning systems, across many real-world decision making scenarios on the Web and elsewhere, there is a pressing need to make sure that such systems do not become vehicles of unfair discrimination, inequality, and social injustice (Barocas and Selbst, 2016; Barocas et al., 2017). Of particular interest in this context is the task of detecting and preventing discrimination or unfair treatment of individuals or groups on the basis of gender, race, religion, etc. Such discrimination is traditionally addressed using one of two legal frameworks: disparate treatment (which aims to enforce procedural fairness, namely, the equality of treatment that prohibits the use of the protected attribute in the decision process); and disparate impact (Barocas and Selbst, 2016) (which aims to guarantee outcome fairness, namely, the equality of outcomes between protected groups relative to other groups). It is clear that enforcing procedural fairness within the disparate treatment framework does not guarantee non-discrimination within the disparate impact framework.

There is growing interest in algorithmic decision making systems that are demonstrably fair (see (Berk et al., 2017) for a review). Much of this literature relies on precise definitions that quantify fairness to avoid discrimination with respect to protected attributes, e.g., race, gender, on the basis of the legal notions of disparate treatment or disparate impact (Barocas and Selbst, 2016) (see (Romei and Ruggieri, 2014; Zliobaite, 2015; Barocas and Selbst, 2016; Berk et al., 2017; Loftus et al., 2018) for reviews). Some examples include: fairness through unawareness (Grgic-Hlaca et al., 2016), individual fairness (Dwork et al., 2012), equalized odds (Hardt et al., 2016; Zafar et al., 2017a), calibration (Chouldechova, 2017), demographic (or statistical) parity (Calders et al., 2009; Kamishima et al., 2012; Kamiran and Calders, 2009; Johndrow and Lum, 2017), the 80% rule (disparate impact) (Feldman et al., 2015; Zafar et al., 2017b), representational fairness (Zemel et al., 2013; Louizos et al., 2015), and fairness under composition (Dwork and Ilvento, 2018).

Answering questions of fairness through the lens of causality entails replacing the question “Is the decision discriminatory with respect to a protected attribute?” by: “Does the protected attribute have a causal effect on the decision?” A practical difficulty in using this approach is that, in general, establishing a causal relationship between a protected attribute and a decision requires the results of experimental manipulation of the protected attribute. Fortunately, however, existing frameworks for determining causal effects from observational data (Pearl, 2009; Imbens and Rubin, 2015) provide a rich set of theoretical results as well as practical tools for elucidating causal effects, and specifically, answering questions about counterfactuals or potential outcomes, i.e., results of hypothetical experimental interventions from observational data, whenever it is possible to do so. Hence, there is a growing body of work (see (Loftus et al., 2018) for a recent review) focused on explicitly causal (as opposed to purely joint distribution based or observational) definitions for fairness (e.g., (Kusner et al., 2017, 2018; Zhang and Bareinboim, 2018; Nabi and Shpitser, 2018; Kilbertus et al., 2017; Chiappa and Gillam, 2018; Zhang et al., 2017; Bonchi et al., 2017; Li et al., 2017; Zhang et al., 2016; Russell et al., 2017; VanderWeele and Robinson, 2014)). While some, e.g., (Zhang et al., 2017), have focused on testing fairness (or conversely, determining whether there is discrimination), others, e.g., (Kilbertus et al., 2017; Kusner et al., 2017) have sought to design machine learning algorithms that yield predictive models that are demonstrably fair. However, most of the existing work on defining fairness in causal terms has focused on variants of individual fairness. Against this background, we focus on robust methods for detecting and quantifying discrimination against protected groups, which is a necessary prerequisite for developing predictive models that are provably non-discriminatory.

We reduce the problem of quantifying discrimination against protected groups to the well-studied problem of estimating the causal effect of some variable(s) on a target (outcome) variable. We introduce two explicitly causal definition of fairness in a population, fair on average causal effect (FACE), and in a protected group, fair on average causal effect on the treated (FACT), both with respect to a protected attribute (e.g., gender, race). We use the Rubin-Neyman potential outcomes framework (Rubin, 1974, 2005; Imbens and Rubin, 2015) for robust estimation of FACE and FACT. We demonstrate the effectiveness of the proposed approach in detecting and quantifying group fairness using synthetic data, as well as two real-world data sets: the Adult income data from the UCI repository (Dheeru and Karra Taniskidou, 2017) (with gender being the protected attribute), and the NYC Stop and Frisk data (with race being the protected attribute). We show that the evidence of discrimination, or lack thereof, obtained by FACE and FACT is often in agreement with other studies. We further show that FACT, being somewhat more nuanced compared to FACE, can yield findings of discrimination that differ from those obtained using FACE.

Fairness: A Causal Perspective

Imagine we are given the hiring data of a company containing demographic information about applicants, as well as $A=$ {male, female} as their gender, and $Y=$ {hired, rejected} as whether they were hired by the company. Our task is to determine whether the company’s hiring decisions are fair on average with respect to gender. FACE contrasts the expected outcomes (i.e., hiring) between men vs. women with the expectation taken over the entire population. FACT contrasts the expected outcomes observed for a specific protected group (e.g., women) and the hypothetical (counterfactually inferred) outcomes for the group had they not been members of the protected group (with the expectation taken only over the members of the protected group), e.g., hiring outcomes for women contrasted with outcomes for the same individuals had their gender been different with all other attributes remaining unchanged. Obviously, such counterfactual outcomes cannot be obtained from observational dataThis is called the Fundamental Problem of Causal Inference (FPCI) from observational data (Holland, 1986). and ought to be estimated.

Estimating FACE and FACT

We use tools offered by the potential outcomes framework (Imbens and Rubin, 2015) to estimate FACE and FACT. These tools rely on the following key assumptions: i) Consistency which requires that for a data point $i$, the potential outcome of $i$ under any level of treatment $a$, i.e., $Y^{a}_{i}$, equals the actual outcome observed for that data point, $Y^{obs}_{i}$, had they been exposed to treatment $a$. Formally, under consistency, $Y_{i}^{obs}=a\,Y_{i}^{(a)}+a^{\prime}\,Y_{i}^{(a^{\prime})}$ would hold for all $i$. This assumption, used in existing literature (Nabi and Shpitser, 2018; Chiappa and Gillam, 2018; Pearl, 2019; Madras et al., 2019), is a rather natural one to make in our setting. ii) Positivity which asserts that the probability $Pr(A=a\,|\,X=x)>0$ for all values of $A$. In our setting, this means each value of the protected attribute has a non-zero probability. iii) Stable Unit Treatment Value Assumption (SUTVA) (Rubin, 1980) which consists of two sub-assumptions: 1) Absence of interference between individuals (Cox, 1958), which means that an individual’s potential outcome is unaffected by the treatment assigned to any other individual. While this assumption is plausible in our setting, it may be violated in some settings, in which case, such violations should be accounted for (Hernan and Robins, 2018). 2) Presence of only one form of treatment (and control). For example, if a treatment involves administering a drug, then all individuals who take the drug, take it in the same form (e.g., injection). This assumption is trivially satisfied in our setting because treatment is simulated by the protected attribute. iv) Unconfoundedness of the treatment mechanism which implies that given a set of observables, the potential outcomes of each individual are jointly independent of the corresponding treatment (Rubin, 1978). Unconfoundedness cannot be verified or contradicted entirely on the basis of observational data. However, sensitivity analysis (Rosenbaum, 2005; Liu et al., 2013) can be a useful tool for analyzing the estimated causal effects under violations of the unconfoundedness assumption. Strong ignorability refers to the combination of unconfoundedness and positivity (Rosenbaum and Rubin, 1983). Strong ignorability is a sufficient condition for the causal effect to be identifiable (Hernan and Robins, 2018) and is equivalent to the back-door criterion (Pearl, 2010), which is required for identifiability of the causal effects in Pearl’s model of causality (Pearl, 2010). In our work, as in the case of existing work on causal definitions of fairness (Nabi and Shpitser, 2018), we assume strong ignorability.

2. Estimating and Interpreting FACT

We use matching to estimate FACT. Consider the example of salary discrimination based on gender. For a woman, we can never observe what the salary would have been, had she been a man (i.e., her counterfactual salary). Hence, we estimate the counterfactual salary as follows (Imbens and Rubin, 2015): 1) Using a suitable matching technique (see Section Matching Methods), we match the woman $i$, to a man $j$ who is closest to $i$ with respect to a distance measure $d(i,j)$. 2) The matching process is repeated as needed until matches are of acceptable quality (see Section Quality of Matches). 3) After matching, we use the salary of the matched man $j$ (i.e., $Y_{j}$), as the counterfactual salary of the woman $i$.

The results of matching depend on the choice of distance measure $d(\cdot,\cdot)$ as well as the matching process. Several matching methods exist (see (Stuart, 2010) for a survey). In what follows, for simplicity and brevity, we refer to individuals with protected attribute set to $A=a$ as the treated individuals and those with the protected attribute set to $A=a^{\prime}$ as the controlled individuals. We used the matching methods implemented within the R package MatchIt (version 3.0.2) (Ho et al., 2011) with all parameters set to their default values unless otherwise noted: (i) Exact Matching (EM); (ii) Nearest Neighbor Matching (NNM) with propensity score (Rosenbaum and Rubin, 1983). Following (Rubin, 2001), we estimated the propensity scores using the logit link and transformed them to the linear scale. Then, we ran NNM with replacement, based on the linear propensity scores, and discarded the data points (both from treated and controlled) that fall outside the support of the distance measure; (iii) Nearest Neighbor Matching with a Propensity Caliper (NNMPC). NNMPC includes only matches within a certain number of standard deviations of the distance measure and discards the rest. In NNMPC, we use the same procedure as in NNM, augmented with a caliper = 0.25 (Rosenbaum and Rubin, 1985), resulting in the matches outside 0.25 times the standard deviation of the (transformed) linear propensity score, being discarded; (iv) Mahalanobis Metric Matching within the Propensity Caliper (Rubin, 2001) (MMMPC). MMMPC determines for each data point, a “donor pool” of available matches within the propensity caliper. Mahalanobis metric matching is then performed among the data points chosen in the previous step mimicking blocking in randomized experiments (Rubin, 2001). We ran MMMPC with caliper, replacement, and discarding strategy as described above in NNM; and (v) Full Matching (FM) (Rosenbaum, 1991). We used the same distance measure and discarding strategy as described above in NNM.

Quality of Matches

To ensure accurate estimation of FACT, it is crucial to measure the “goodness-of-match.” If the data points are well matched, then one can proceed to estimate FACT. Common diagnostics for examining the quality of match include both numerical and graphical criteria. Among the numerical criteria, following (Rubin, 2001), we compare the standardized difference in the means of the treated and the controlled data points in terms of the distance measure. We denote the absolute value of this difference in means on the original, and matched data, by $\overline{D}_{a,a^{\prime}}$, and $\overline{D}^{m}_{a,a^{\prime}}$, respectively. For the match to be of good quality, $\overline{D}^{m}_{a,a^{\prime}}$ has to be close to . Among the graphical criteria, we use quantile-quantile (QQ), and jitter plots recommended by (Stuart, 2010; Ho et al., 2011).We avoid the commonly used hypothesis tests for assessing feature balance in diagnosing the quality of matches because such tests have been shown to be misleading in general (Imai et al., 2008).

Outcome Analysis After Matching

Interpreting γ^^𝛾\hat{\gamma} as a Measure of FACT

Impact of Unmeasured Confounders on γ^^𝛾\hat{\gamma}

What if the strong ignorability assumption (i.e., no hidden confounders) is violated? In the absence of unmeasured confounding, matching estimators are unbiased if the matching model is specified correctly, i.e., if balance is achieved over the observed attributes. However, it is conceivable that the results of matching could change in the presence of unobserved confounders (i.e., hidden bias). We perform sensitivity analysis (Rosenbaum, 2005; Liu et al., 2013) to investigate the degree to which the unmeasured confounders impact $\hat{\gamma}$. Let $\Gamma$ be the odds ratio of matched (using any matching method) data points $i$ and $j$ receiving a treatment. Sensitivity analysis proceeds by first assuming $\Gamma=1$ (i.e., no hidden bias). Then, it increases $\Gamma$ (e.g., $1,\ldots,5$), thus mimicking the presence of hidden bias, and examines the resulting changes to statistical significance of $\hat{\gamma}$. The $\Gamma$ at which the significance of the upper bound for the p-value would change (e.g., from $<0.05$ to $>0.05$) is the point at which $\hat{\gamma}$ is no longer robust to hidden bias. We ran sensitivity analysis using the R package rbounds (version 2.1) (Keele, 2010).

Experiments and Results

We tested our approach on a synthetic data set (where the discrimination based on a protected attribute can be varied in a controlled fashion), and two real-world data sets that have been previously used in studies of fairness. In each case, we designated a protected attribute and estimated FACE and FACT as measures of discrimination based on that attribute. We run all of our statistical significance tests with $\alpha=0.05$. We proceed to describe the data sets, experiments, as well as our FACE and FACT analyses in detail.

The Adult data set

The Adult income data set (Kohavi, 1996)https://archive.ics.uci.edu/ml/datasets/adult, contains information about individuals as well as their salaries. The data set includes 48842 individuals each with 14 attributes, 6 continuous and 8 categorical, including demographic and work-related information such as age, gender, hours of work per week, etc. We examined whether there is gender-based discrimination in salaries by designating gender as the sensitive attribute. We encoded categorical variables using one-hot-encoding and removed data records with missing values, yielding a data set with 46033 individuals and 45 features (excluding gender, the protected attribute). We designated the outcome $Y$ to be a binary variable denoting whether the person’s annual salary is >\50K$(Y=1), or$\leq\$50K$ (Y= 0).

The NYC Stop and Frisk (NYCSF) data set

We retrieved the publicly available stop, search, and frisk data from The New York Police Department (NYPD)https://www1.nyc.gov/site/nypd/stats/reports-analysis/stopfrisk.page website which serves demographic and other information about drivers stopped by the NYC police force. Our question is whether the arrests made after stops have been discriminatory with respect to race. Following (Kusner et al., 2017), we restricted our experiment to the year 2014 yielding a total of 45787 records. We selected the subset of records corresponding to only Black-Hispanic and White men. We designated race as the protected attribute with $A=1$ denoting Black-Hispanic and $A=0$ denoting White. We dropped the data records with missing values and encoded categorical variables with one-hot-encoding. The resulting data consist of 7593 records each with 73 features (excluding race, the sensitive attribute). The outcome $Y$ denotes whether an arrest was made ($Y=1$), or not ($Y=0$).

2. FACE Check: Fairness Analysis Using FACE

We report our analysis of fairness using FACE, for the synthetic, Adult, and NYCSF data sets. The estimated FACE ($\hat{\beta}$) are shown in Table 1. In all cases, the null hypothesis is $H_{0}:\beta=0$. In the case of synthetic data, we find insufficient evidence to reject $H_{0}$, suggesting the outcome is fair with respect to the protected attribute (an expected conclusion given the design of the generative model in Section 4.1, which ensures that the outcome is fair with respect to the protected attribute). In the case of Adult data, we reject $H_{0}$ and find that $\hat{\beta}$, the average causal effect of gender on salaries, is $-1.069$. This means that on average, over the entire population, the odds of women having a salary >\50K$a year is$\exp(-1.069)\approx 0.34$times that of men, suggesting gender-based discrimination against women as measured by FACE. This finding is in agreement with the conclusions reported in (Nabi and Shpitser, 2018; Li et al., 2017). In the case of NYCSF data, we reject$H_{0}$and find that$\hat{\beta}$is$0.273$which means that on average, the odds of Black-Hispanics being arrested after a stop by the police, is$\exp(0.273)\approx 1.31$ times that of Whites, suggesting possible racial bias against non-Whites.

3. FACT Check: Fairness Analysis Using FACT

We report our analysis of fairness using FACT, for the synthetic, Adult, and NYCSF data sets.

Because the quality of matched pairs used to estimate FACT impacts the conclusions that can be drawn using it, we compare the FACT estimates obtained using several widely-used matching methods described in Section 3.2. We present some analyses to verify that the generated matches are of sufficiently high quality for estimating FACT.

We observe that before matching, $\overline{D}_{a,a^{\prime}}$ is 1.6400, 3.3508, and 1.1616, on the synthetic, Adult, and NYCSF data sets, respectively. The matching methods dramatically reduced $\overline{D}_{a,a^{\prime}}$ on all of the data sets (see Table 2). Overall, NNM and FM achieved the lowest $\overline{D}^{m}_{a,a^{\prime}}$ as compared to other matching methods on all data sets. The greater the number of pairs that are matched, the harder it is to achieve balance, and the trade-off between the two can be application dependent. We observed that considering the trade-off between the number of matches and $\overline{D}^{m}_{a,a^{\prime}}$, FM yields higher quality matches on all data sets as compared to other methods.

The QQ plots are generated for each feature in each data set. In Figure 1 we show the QQ plots before and after FM for the first three features of the synthetic data set. The features lie far away from the 45 degree line before FM. After FM, the features are much better aligned to the diagonal line showing a more desirable feature balance. We also show the jitter plots of FM on all data sets in Figure 2. It is clear that the distribution of propensity scores of the treated and controlled data points are very similar to each other after matching. Having verified that the results of matching are of adequate quality, we proceed to use them for estimating FACT.

FACT Estimates

The results of FACT analyses on the synthetic, Adult, and NYCSF data sets are summarized in Table 2 (Note that EM did not yield any matches and hence is omitted from Table 2). In all cases, the null hypothesis is $H_{0}:\gamma=0$. In the case of synthetic data, FACT analyses show that for NNM and MMMPC, there is not enough evidence to reject $H_{0}$. The p-values in the case of NNMPC and FM are $<0.05$, but the magnitude of the estimated $\hat{\gamma}$ is close to zero. We conclude that the synthetic data set is fair on average with respect to FACT. On the Adult data, we can reject $H_{0}$, suggesting that salaries of women are significantly lower than those of men who match them on the non-protected attributes. For example, using FM, we find that $\hat{\gamma}=-0.573$, thus the odds of women earning >\50K$a year, is$\exp(-0.573)\approx 0.56$times that of men. We conclude that in the Adult data, there is evidence of gender-based discrimination in salary, on average, against women. On the NYCSF data, interestingly, FACT analyses show that$H_{0}$ cannot be rejected, suggesting a lack of evidence for racial bias, on average, in arrests after stops (when Black-Hispanics are compared with Whites who match them on non-protected attributes). This conclusion contradicts the finding of racial bias based on counterfactual fairness analysis (Supplementary Material S6 in (Kusner et al., 2017)) which suggests discrimination against individuals, as well as FACE analysis (see Section 4.2). We conjecture that the apparent discrepancy can be explained by noting that (i) fairness (or discrimination) on average does not necessarily imply individual-level fairness (or individual-level discrimination), and (ii) FACT compares the observed outcomes of members of a protected group with the hypothetical (counterfactual) outcomes they would have experienced had they not been members of the protected group (with all non-protected attributes remaining unchanged), whereas FACE compares such counterfactual outcomes on the entire population.

Impact of Unmeasured Confounders

We ran sensitivity analysis of our estimates of FACT for $\Gamma=1,\ldots,10$ (where larger values of $\Gamma$ correspond to greater bias introduced by hidden confounders) on the Adult and NYCSF data sets. We find that all of our estimates obtained with various matching methods are quite robust to hidden confounder bias. Specifically, on the Adult data set, for all matching methods except FM, the estimates are robust to such bias, and for FM, they are robust up to $\Gamma=4.5$, which corresponds to a fairly large amount of bias. On the NYCSF data set, estimates obtained via NNM and MMMPC are robust to hidden confounder bias, and NNMPC and FM are robust up to $\Gamma$ equals 8.5, and 3, respectively. These results mean that our FACT estimates (and hence our findings of discrimination on the basis of protected attributes, or lack thereof) are fairly robust to hidden confounder bias.

Summary and Discussion

We have approached the problem of detecting whether a group of individuals that share a sensitive attribute, e.g., race, gender, have been subjected to discrimination in an algorithmic decision-making system, through the lens of causality. We have introduced two explicitly causal definitions of group fairness: fair on average causal effect (FACE), and fair on average causal effect on the treated (FACT). We have shown how to robustly estimate FACE and FACT, and use the resulting estimates to detect and quantify discrimination based on specific attributes (e.g., gender, race). The results of our experiments on synthetic data show that our proposed methods are effective at detecting and quantifying group fairness. Our analyses of the Adult data set for evidence of gender-based discrimination in salary, and of the NYCSF data set for evidence of racial bias in arrests after traffic stops, yield evidence of discrimination, or lack thereof, that is often in agreement with other studies.The regression and matching-based methods we employed to estimate FACE and FACT adjust for covariates that might be potential confounders of the protected attribute, which although necessary in general, may be unnecessary in the case of gender and race, because they are unlikely to be caused by any other covariate. Consequently, the reported estimates of FACE and FACT are likely to represent direct causal effects as opposed to total causal effects. We show on the real-world data that our estimates of FACE and FACT are robust to unmeasured confounding. Our results further show on the real-world data that FACE and FACT based findings do not always agree. Our FACT analyses also demonstrate that group-fairness (or discrimination) does not necessarily imply individual-level fairness (or individual-level discrimination).

Some directions for further research include: relaxing the assumption that the data are independent and identically distributed (i.i.d.) in settings where individuals are related to each other through family ties or other relationships; examining the relationships between different causal notions of fairness; and designing automated decision support systems that are demonstrably non-discriminatory with respect to given outcome(s) and protected attribute(s).

This work was funded in part by grants from the NIH NCATS through the grant UL1 TR000127 and TR002014 and by the NSF through the grants 1518732, 1640834, and 1636795, the Edward Frymoyer Endowed Professorship in Information Sciences and Technology at Pennsylvania State University and the Sudha Murty Distinguished Visiting Chair in Neurocomputing and Data Science funded by the Pratiksha Trust at the Indian Institute of Science (both held by Vasant Honavar). The content is solely the responsibility of the authors and does not necessarily represent the official views of the sponsors.

References