A Variational Approach to Privacy and Fairness

I Introduction

Currently, many systems rely on machine learning algorithms to make decisions and draw inferences. That is, they use previously existing data in order to shape some stage of their decision or inference mechanism. Usually, this data contains private or sensitive information, e.g., the identity of the person from which a datum was collected or their membership to a minority group. Thus, an important problem occurs when the data used to train such algorithms leaks this information to the system, contributing to unfair decisions or to a privacy breach.

When the content of the private information is arbitrary and the task of the system is not defined, the problem is reduced to learning private representations of the data; i.e., representations that are informative of the data (utility), but are not informative of the private information. Then, these representations can be employed by any system with a controlled private information leakage. If the informativeness is measured by the mutual information, the problem of generating private representations is known as the privacy funnel (PF) .

When the task of the system is known, then the aim is to design strategies so that the system performs such a task efficiently while employing or leaking little sensitive information. The field of algorithmic fairness has extensively studied this problem, especially for classification tasks and categorical sensitive information, c.f., . An interesting approach is that of learning fair representations , where similarly to their private counterparts, the representations are informative of the task, but contain little sensitive information.

There is a compromise between information leakage and utility when designing private representations . Similarly, it has been shown empirically and theoretically that there is a trade-off between fairness and utility.

In this work, we investigate the trade-off between utility and privacy and between utility and fairness in terms of mutual information (details about the choice of mutual information in Appendix A). More specifically, we aim at maintaining a certain level of the information about the data (for privacy) or the task (for fairness) that is not shared by the senstive attributes, while minimizing all the other information. We name these two optimization problems the conditional privacy funnel (CPF) and the conditional fairness bottleneck (CFB) due to their similarities with the PF , the information bottleneck (IB) , and the recent conditional entropy bottleneck (CEB) .

We tackle both optimization problems with a variational approach based on their Lagrangian. For the privacy problem, we show that the minimization of the Lagrangians of the CPF and the PF is equivalent (see Appendix B-A), meaning that the proposed approach also attempts at solving the PF. Moreover, this approach improves over current variational approaches to the PF by respecting the problem’s Markov chain in the encoder distribution.

Finally, the resulting approaches for privacy and fairness can be implemented with little modification to common algorithms for representation learning like the variational autoencoder (VAE) , the $\beta$-VAE , the variational information bottleneck (VIB) , or the nonlinear information bottleneck . Therefore, it facilitates the incorporation of private and fair representations in current applications (see Appendices C and D for an extended related section and a guide on how to modify these algorithms).

We demonstrate our results both in the Adult dataset and a toy dataset based on MNIST . Further experiments on the COMPAS dataset can be found in Appendix E.

II Methods

In this section we present our approach. First, we introduce the proposed models for the privacy and fairness problems. Then, we show a suitable Lagrangian formulation. Finally, we describe a variational approach to solve both problems.

Suppose we want to share some data $X\in\mathcal{X}$ so that third-parties can make statistical analyses and draw inferences from them. However, these data $X$ contains information about some private attributes $S\in\mathcal{X}$ that we want to protect. For this reason, we encode the data $X$ into the representation $Y\in\mathcal{Y}$, forming the Markov chain $S\leftrightarrow X\leftrightarrow Y$, and then share $Y$.

This encoding, characterized by the conditional probability distribution $P_{Y|X}$, is designed so that the representation $Y$ keeps a certain level $r$ of the information about the data $X$ that is not shared with the private attributes $S$ (i.e., the light gray area in Figure 1a), while minimizing the information it keeps about the private attributes $S$ (i.e., the dark gray area in Figure 1a). That is,

This formulation reassembles the privacy funnel (PF) , since both minimize the information the representation $Y$ keeps about the private attributes $S$. Nonetheless, in the PF, the encoding is designed so that the representation $Y$ keeps a certain level $r^{\prime}$ of information about the data $X$, disregarding if this information is also shared by the private attributes $S$. Hence, the optimization of the PF Lagrangian may lead to representations $Y$ that filter private information arbitrarily. In contrast, the CPF avoids this issue with an inherent additional constraint on the Lagrange multipliers (see Appendix B-A).

II-A2 Fairness: the conditional fairness bottleneck (CFB)

Suppose we want to use (or share) some data $X\in\mathcal{X}$ to draw inferences or make decisions about a task $T\in\mathcal{T}$. However, these data $X$ and the task $T$ contain information about some sensitive attributes $S\in\mathcal{S}$, and we do not want our inferences to be influenced by these sensitive attributes $S$. For this reason, we encode the data $X$ into a representation $Y\in\mathcal{Y}$ and then use $Y$ to draw inferences about the task $T$. Therefore, the Markov chains $S\leftrightarrow X\rightarrow Y$ and $T\leftrightarrow X\rightarrow Y$ hold.

This encoding, characterized by the conditional probability distribution $P_{Y|X}$, is designed so that the representation $Y$ keeps a certain level $r$ of the information about the task $T$ that is not shared by the sensitive attributes $S$ (i.e., the light gray area in Figure 1b), while minimizing the information it keeps about the sensitive attributes $S$ and the information about the data $X$ that is not shared with the task $T$ (i.e., the dark and darker gray areas in Figure 1b, respectively). That is,

This formulation differs from other approaches to fairness in two main points: (i) Similarly to the IB , the CFB does not only minimize the information the representation keeps about the sensitive attributes $S$, but also the information about the data $X$ that is irrelevant for the task $T$. That is, the CFB seeks a representation that is both fair and relevant, thus avoiding the risk of keeping nuisances and harming its generalization capability. (ii) Similarly to the CEB , the CFB aims to produce a representation $Y$ that maintains a certain level $r$ of the information about the task $T$ that is not shared by the sensitive attributes $S$. This differs from formulations that aim to keep a certain level $r^{\prime}$ of the information about the task $T$, disregarding if it is also shared by the sensitive attributes $S$.

II-B The Lagrangians of the problems

A common approach to solving optimization problems such as the CPF or the CFB is to minimize the Lagrangian of the problem. The Lagrangian is a proxy of the trade-off between the function to optimize and the constraints on the optimization search space [23, Chapter 5]. Particularly, the Lagrangians of the CPF and the CFB are, ommiting the constant term $\lambda r$ in the optimization w.r.t. $P_{Y|X}$, respectively,

In the following propositions, proved in Appendix B-B, we present two alternative Lagrangians that are equivalent to the original Lagrangians, are more tractable, and exhibit similar properties and structure in the privacy and fairness problems.

Minimizing $\mathcal{L}_{\textnormal{CPF}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{J}_{\textnormal{CPF}}(P_{Y|X},\gamma)$, where $\gamma=\lambda+1$ and

Minimizing $\mathcal{L}_{\textnormal{CFB}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{J}_{\textnormal{CFB}}(P_{Y|X},\beta)$, where $\beta=\lambda+1$ and

The minimization of $\mathcal{J}_{\textnormal{CPF}}$ and $\mathcal{J}_{\textnormal{CFB}}$, by means of $\gamma$ and $\beta$, trades off the level of compression of the representation Y with the information it keeps, respectively, about the data $X$ and the task $T$ that is not shared by the sensitive attributes $S$.

II-C The variational approach

We consider the minimization of $\mathcal{J}_{\textnormal{CPF}}$ and $\mathcal{J}_{\textnormal{CFB}}$ to solve the CPF and CFB problems. Furthermore, we assume that the probability density (or mass if $|\mathcal{Y}|$ is countable) functions $p_{Y|X}$ that describe the conditional probability distribution $P_{Y|X}$ exist and are parameterized by $\theta$, i.e., $p_{Y|X}=p_{Y|X,\theta}$.

The Markov chains of the CPF and the CFB characterize the densities $p_{S,X,Y|\theta}=p_{S,X}\cdot p_{Y|X,\theta}$ and $p_{S,T,X,Y|\theta}=p_{S,T,X}\cdot p_{Y|X,\theta}$. The densities $p_{S,X}$ and $p_{S,T,X}$ can be inferred from the data and the density $p_{Y|X,\theta}$ is to be designed.

The term $I(X;Y)$ depends on the density $p_{Y|\theta}$, which is usually intractable. Similarly, the terms $I(X;Y|S)$ and $I(T;Y|S)$ depend on the densities $p_{X|S,Y,\theta}$ and $p_{T|S,Y,\theta}$, respectively, which are also usually intractable. Therefore, an exact optimization of $\theta$ is prohibitively computationally expensive. For this reason, we introduce the variational density approximations $q_{Y|\theta}$, $q_{X|S,Y,\phi}$, and $q_{T|S,Y,\phi}$, where the generative and inference densities are parametrized by $\phi$.

Then, as previously done in, e.g., , we leverage the non-negativity of the relative entropy to bound $\mathcal{J}_{\textnormal{CPF}}$ and $\mathcal{J}_{\textnormal{CFB}}$ from above. More precisely, we bound $I(X;Y)$ from above and $I(X;Y|S)$ and $I(T;Y|S)$ from below, i.e.,

In practice, if we have a dataset of $N$ samples $\left(x^{(i)},s^{(i)}\right)$ for the CPF or $N$ samples $\left(x^{(i)},s^{(i)},t^{(i)}\right)$ for the CFB, we minimize, respectively, the following cost functions:

where the expectation over $E\in\mathcal{E}$ is usually estimated with a naive Monte Carlo of a single sample.

An a posteriori interpretation of this approach is that if the encoder compresses the representation $Y$ assuming that the decoder will use both $Y$ and the private or sensitive attributes $S$, then the encoder will discard the information about $S$ contained in the original data $X$ in order to generate $Y$.

The resulting cost functions for the CPF and the CFB ressemble those of the VAE , the $\beta$-VAE , the VIB , or the nonlinear IB . Consider the (common) case that the decoder density is estimated with a neural network. If such a network is modified so that it receives as input both the representation and the private or sensitive attributes instead of only the representation, then the optimization of these algorithms results in private and/or fair representations (see Appendix D for the details).

III Results

In this section, we present experiments on two datasets to showcase the performance of the presented variational approach to the privacy and fairness problems. First, we show its performance in a dataset commonly used for benchmarking both tasks. Second, we show the performance on high-dimensional data on a toy dataset designed for this purpose. The encoder density is modeled with an isotropic Gaussian distribution, i.e., $p_{Y|X,\theta}=\mathcal{N}(Y;\mu_{\textnormal{enc}}(X;\theta),\sigma_{\theta}^{2}I_{d})$, where $\mu_{\textnormal{enc}}$ is a neural network and $d$ is the dimension of the representation. The marginal density of the representation is also modeled as an isotropic Gaussian $q_{Y|\theta}=\mathcal{N}(Y;0,I_{d})$. Finally, the decoder density, $q_{X|S,Y,\phi}$ or $q_{T|S,Y,\phi}$, is modeled with a product of categorical (for discrete data) and/or isotropic Gaussians (for continuous data), e.g., $q_{X|S,Y,\phi}=\textnormal{Cat}(X_{1};\rho_{\textnormal{dec}}(Y,S;\phi))\mathcal{N}(X_{2};\mu_{\textnormal{dec}}(S,Y;\phi),\sigma_{\phi}^{2})$ if $X$ consists of a discrete variable $X_{1}$ and a continuous variable $X_{2}$. These and additional experiments are detailed in Appendix E.

The Adult dataset (available at the UCI machine learning repository ) contains $45,222$ samples from the 1994 U.S. Census. Each sample comprises 15 features such as, e.g., gender, age, or income level (binary variable stating if the income level is higher than \50,000$). We followed the experimental set-up from , where the sensitive attribute$S$is the gender, the task$T$is the income level, and the data$X$ is the rest of the features.

The MNIST dataset is a collection of $70,000$ grayscale $28\times 28$ images of hand-written digits from to $9$. The Colored MNIST is a modification of the former dataset where each digit is randomly colored in either red, green, or blue. We considered that the data $X$ are the $3\times 28\times 28$ digit images, the sensitive attribute $S$ is the digit’s color, and the task $T$ is digit’s number.

III-A Privacy

The proposed approach is able to control the trade-off between a private and an informative representation for both the Adult and the Colored MNIST datasets. We minimized (8) for different values of $\gamma\in$, thus controlling the trade-off between the compression $I(X;Y)$ and the informativeness of the representation independent of the private data $I(X;Y|S)$ (see Figures 2a and 2b). Hence, as suggested by Proposition 3, the multiplier $\gamma$ also controls the amount of private information the representation keeps (see Figures 2e and 2f).

As an illustration, we constructed a representation with the same dimension to the digit images by minimizing (8) with $\gamma=1$ (Figure 3b). The representation is both informative and private; e.g., the 2D UMAP vectors of the representation are mingled with respect to the digits’ color, as opposed to the UMAP vectors of the original images, where the vectors are clustered by the color of the digits (see Figures 3c and 3d).

Compared to other variational approaches to the PF like the [16, PPVAE] and [27, VFAE], the proposed approach performed better in terms of information leakage $I(S;Y)$ and accuracy of non-linear attackers (see Table I). More specifically, the leaked information is about an order of magnitude lower than the obtained in the other methods and the accuracy of a random forest attacker is below the prior probability (0.67), while the other methods allow successful attacks of 0.8 – 0.9 accuracy. An explanation for this phenomenon is that some private information is leaked to the representations $Y$ of the PPVAE and the VFAE via their encoding density $p_{Y|(S,X,\theta)}$, which does not respect the Markov chain $S\leftrightarrow X\rightarrow Y$.

III-B Fairness

The proposed variational approach is also able to control the trade-off between fair and accurate representations. We minimized (9) for different values of $\beta\in$, thus controlling the trade-off between the compression $I(X;Y)$ and the informativeness of the representations independent of the sensitive attributes $I(T;Y|S)$ (see Figures 2c and 2d). Thus, as hinted by Proposition 4, the multiplier $\beta$ also controls the amount of private information leaked (see Figures 2g and 2h).

Furthermore, in the Adult dataset, the Lagrange multiplier $\beta$ allows us to control the behavior of different utility and group fairness indicators (defined in Appendix F), namely the accuracy, the error gap, and the discrimination (or demographic parity gap). That is, the higher the value of $\beta$, the higher the accuracy and the discrimination, and the lower the error gap (see Figures 4a, 4b, and 4d). The behavior of the discrimination is enforced by the minimization of $I(S;Y)$, as discussed in Remark 4 from Appendix F. However, there is no clear indication of the effect of $\beta$ on the accuracy of adversarial predictors on the sensitive data (which is still below the prior probability of the biased training dataset) and on the equalized odds (see Figures 4e and 4c). The equalized odds are not optimized in this scenario since the resulting representation $Y$ is such that $I(S;Y;T)<0$ as discussed in Remark 5 from Appendix F. An example where the equalized odds gap is minimized is presented in Appendix E.

The representation generated from the proposed method is more robust to non-linear adversaries (as shown in Table II by the smaller accuracy on $S$ and smaller discrimination for random forest adversaries) than the standard baseline from [7, LFR]. Compared with other state-of-the-art methods, either those employing variational inference [28, FFVAE] or adversarial training [9, CFAIR], the proposed method for generating fair representations achieves similar results (see Table II), albeit having an easier cost function and requiring to train less networks, respectively (see Appendix C).

IV Discussion

In this article, we studied the problem of mitigating private or sensitive information $S$ leakage into data-driven systems through the training data $X$. We formalized the trade-off between the relevant information for the system that is not shared by the private or sensitive attributes $S$ and the remaining information as a constrained optimization problem. When the task of the system is unknown, the problem is referred as learning private representations and the formalization is the conditional privacy funnel (CPF); and when it is kwnon the problem is referred as learning fair representations and the formalization is the conditional fairness bottleneck (CFB).

To solve these problems, we proposed a variational approach based on the Lagrangians of the CPF and the CFB. This approach leads to a simple structure highlighting the similarities between private and fair representation learning. Moreover, in practice, private and fair representations can be learned with little modification to the implementation of common algorithms such as the VAE , the $\smash{\beta}$-VAE , the VIB , or the nonlinear IB . Namely, modifying the decoder neural network so it receives both the representation $Y$ and the private or sensitive attributes $S$ as an input. Then, the learned representations can be fed to any algorithm of choice. For this reason, the efforts for reducing unfair decisions and privacy breaches will be small for many practitioners.

The proposed formulation has some limitations due to the non-convexity of the CPF and the CFB (see Appendix G). Moreover, the proposed approach also faces limitations due to its variational nature. In Appendix H, we reflect on these limitations and propose future directions to overcome them.

References

Appendix A Motivation of mutual information as utility and privacy/fairness metric

The main reason to choose mutual information (and conditional mutual information) as the measure of utility and privacy and/or fairness is the tractability of this metric and the fact that it allowed us to (i) draw connections between the privacy and fairness problems and (ii) derive an algorithm that could be easily incorporated to current approaches to representation learning.

Other than that, even though there are some caveats of using mutual information as a measure of privacy (see ) or fairness, this metric has several operational meanings. Namely:

Utility. The conditional mutual information $I(X;Y|S)$ (or $I(T;Y|S)$) is a measure of the relevance of the representation $Y$ to explain $X$ (or $T$). This is in line with the information bottleneck and other common representation learning algorithms , .

An intuition would be that if we want to keep the average distortion smaller than a certain quantity, and we consider the distortion to be the log-loss, then we want that the (conditional) mutual information is greater than a, different, certain quantity (see e.g. [2, Section III-B]).

Privacy. The mutual information $I(S;Y)$ is the average cost gain by an adversary with the self-information or log-loss cost function [2, Lemma 1]. This has also been used in other works such as . Moreover, an upper bound on mutual information limits both the Bayesian and minimax risks that can be achieved in any inference based on the data; see e.g. [33, Chapter 15.3 on Fano’s method].

Fairness. As explained in Remark 3 minimizing $I(S;Y)$ encourages demographic parity and minimizing $I(S;Y|T)$ encourages equalized odds.

Appendix B Equivalences of the Lagrangians

In this section of the appendix, we show how minimizing the Lagrangians of the CPF and the CFB problems is equivalent to minimizing other Lagrangians. First, in B-A we show that minimizing the Lagrangian of the CPF is equivalent to minimizing the Lagrangian of the PF, meaning that the conditional probability distributions $P_{Y|X}$ obtained using the Lagrangian of the CPF could have been obtained through the Lagrangian of the PF, too. Then, in B-B we show that minimizing the CPF and CFB Lagrangians is equivalent to minimizing the Lagrangians that are used in the variational approach we propose in this paper.

The privacy funnel is defined in a similar way to the CPF. It is an optimization problem that tries to design an encoding probability distribution $P_{Y|X}$ such that the representation $Y$ keeps a certain level $r^{\prime}$ of information about the data of interest $X$, while minimizing the information it keeps about the private data $S$ . That is,

Therefore, the Lagrangian of the privacy funnel problem is

where $\alpha\geq 0$ is the Lagrange multiplier of $\mathcal{L}_{\textnormal{PF}}(P_{Y|X},\alpha)$. This multiplier controls the trade-off between the information the representations keep about the private and the original data. If $\alpha=1$, then $\mathcal{L}_{\textnormal{PF}}(P_{Y|X},1)=-I(X;Y|S)$, for which optimal values of the encoding distribution $P_{Y|X}$ can filter private information arbitrarily. If $\alpha>1$ this problem is even more pronounced. If $\alpha=0$, trivial encoding distributions like a degenerate distribution $P_{Y|X}$ with density $p_{Y|X}=\delta(Y)$ are minimizers of the Lagrangian. Therefore, in practice one might need to restrict $\alpha$ to the range $(0,1)$.

Minimizing $\mathcal{L}_{\textnormal{CPF}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{L}_{\textnormal{PF}}(P_{Y|X},\alpha)$, where $\alpha=\lambda/(\lambda+1)$.

If we manipulate the expression of the CPF Lagrangian we can see how the minimizing $\mathcal{L}_{\textnormal{CPF}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{L}_{\textnormal{PF}}(P_{Y|X},\alpha)$, where $\alpha=\lambda/(\lambda+1)$. More specifically,

where $\mathcal{P}$ is the set of probability distributions over $\mathcal{Y}$ such that if $P_{Y|X=x}\in\mathcal{P}$ for all $x\in\mathcal{X}$, then the Markov chain $S\leftrightarrow X\rightarrow Y$ holds. ∎

We note how the relationship $\alpha=\lambda/(\lambda+1)$ maintains $\alpha\in[0,1)$ for $\lambda\geq 0$. This showcases how the CPF poses a more restrictive problem, in the sense that as long as $\lambda<\infty$ there are no solutions of the problem that filter private information arbitrarily.

B-B Equivalence of the Lagrangians used for the minimization

If we manipulate the expression of the CPF Lagrangian we can see how minimizing $\mathcal{L}_{\textnormal{CPF}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{J}_{\textnormal{CPF}}(P_{Y|X},\gamma)$, where $\gamma=\lambda+1$. More specifically,

where $\mathcal{P}$ is the set of probability distributions over $\mathcal{Y}$ such that if $P_{Y|X=x}\in\mathcal{P}$ for all $x\in\mathcal{X}$, then the Markov chain $S\leftrightarrow X\rightarrow Y$ holds. ∎

If we manipulate the expression of the CFB Lagrangian we can see how minimizing $\mathcal{L}_{\textnormal{CFB}}(P_{Y|X},\lambda)$ is equivalent to minimizing $\mathcal{J}_{\textnormal{CFB}}(P_{Y|X},\beta)$, where $\beta=\lambda+1$. More specifically,

where $\mathcal{P}$ is the set of probability distributions over $\mathcal{Y}$ such that if $P_{Y|X=x}\in\mathcal{P}$ for all $x\in\mathcal{X}$, then the Markov chains $S\leftrightarrow X\rightarrow Y$ and $T\leftrightarrow X\rightarrow Y$ hold. ∎

Appendix C Extended Related Work

If the secret information $S$ is the identity of the samples or their membership to a certain group, the field of differential privacy (DP) provides a theoretical framework for defining privacy and several mechanisms able to generate privacy-preserving queries about the data $X$ and explore such data, see, e.g., . If, on the other hand, the secret information is arbitrary, variants of DP such as [35, Bounded DP] and [36, Attribute DP] or the theoretical framework introduced in are commonly adopted. The privacy funnel is a special case of the latter, when the utility and the privacy are measured with the mutual information.

The original greedy algorithm to compute the PF assumes the data is discrete or categorical and do not scale. For this reason, methods that attempt at learning non-parametric encoding densities $p_{Y|X}$ such as and other approaches that take advantage of the scalability of deep learning emerged. For instance, and learn the representations through adversarial learning but are limited to $S\in\{0,1\}$ and do not offer an information theoretic interpretation. Similar to us, in the the privacy preserving variational autoencoder (PPVAE) and the unsupervised version of the variational fair autoencoder (VFAE) they learn such representations with variational inference.

At their core, the PPVAE and the unsupervised VFAE end up minimizing the cost functions

and $\mathcal{J}_{\textnormal{VFAE}}(\theta,\phi,\delta)=\mathcal{J}_{\textnormal{PPVAE}}(\theta,\phi,1)+\delta\mathcal{J}_{\textnormal{MMD}}(\theta,\phi)$, where $\mathcal{J}_{\textnormal{MMD}}(\theta,\phi)$ is a maximum-mean discrepancy term. Even though the resulting function to optimize is similar to ours, it is important to note that the encoding density in these works is $p_{Y|(S,X,\theta)}$, which does not respect the problem’s Markov chain $S\leftrightarrow X\rightarrow Y$. Therefore, the optimization search space includes representations $Y$ that contain information about the private data $S$ that is not even contained in the original data $X$. Moreover, the private data $S$ is needed to generate the representations $Y$, which is problematic since it might not be available during inference.

C-B Fairness

The field of algorithmic fairness is mainly dominated by the notions of individual fairness, where the sensitive data $S$ is the identity of the data samples, and group fairness, where $S$ is a binary variable that represents the membership of the data samples to a certain group. There are several approaches that aim at producing classifiers that ensure either of these notions of fairness; e.g., discrimination-free naive Bayes , constrained logistic regression, hinge loss, and support vector machines , or regularized logistic regression through the Wasserstein distance .

Other lines of work on algorithmic fairness are based on causal inference and data massaging , where the values of the labels of the training data are changed so that the training data is fair.

The notion of fair representations, introduced by , boosted the advances on algorithmic fairness due to the expressiveness of deep learning. These advances are mainly dominated by adversarial learning , even though there are recent variational approaches, too .

The main differences with the variational approach from are our simple cost function (which does not require to train an additional adversary discriminator) and that we discard the information that is not necessary to draw inferences about $T$. They generate two representations, $Y_{\textnormal{sens}}$ and $Y_{\textnormal{non-sens}}$, that contain the information about the sensitive data and the original data, respectively, without taking into account the task at hand. At inference time, the sensitive representations $Y_{\textnormal{sens}}$ are corrupted with noise or discarded, and thus the non-sensitive representations $Y_{\textnormal{non-sens}}$ from serve a similar purpose to the representations $Y$ obtained with our approach. Compared to the variational fair autoencoder , our encoding density does not require the sensitive information $S$, which might not be available during inference, thus not breaking the Markov chain $S\leftrightarrow X\rightarrow Y$.

Appendix D Modification of common algorithms to obtain private and/or fair representations

In this section of the appendix, we discuss the simple changes needed to common representation learning algorithms to implement our proposed variational approach. First, we show how common unsupervised learning algorithms can be modified to the variational approach to the CPF, thus generating private representations. Then, we show how common supervised learning algorithms can be modified to the variational approach to the CFB, thus generating fair representations.

The cost function of the $\beta$-VAE and the VIB (when the target variable is the identity of the samples) is

where $\eta$ is a parameter that controls the trade-off between the compression of the representations $Y$ and their ability to reconstruct the original data $X$. Similarly, the VAE cost function is $\mathcal{F}_{\textnormal{uns}}(\theta,\phi,1)$.

In these usupervised learning algorithms the decoding (or generative) density is parametrized with neural networks, e.g., $q_{X|Y,\phi}=\textnormal{Cat}(X;\rho_{\textnormal{dec}}(Y;\phi))$ if $X$ is discrete and $q_{X|Y,\phi}=\mathcal{N}(X;\mu_{\textnormal{dec}}(Y;\phi),\sigma_{\textnormal{dec}}(Y;\phi)^{2}I_{d_{\textnormal{dec}}})$ if $X$ is continuous, where $\rho_{\textnormal{dec}},\mu_{\textnormal{dec}}$, and $\sigma_{\textnormal{dec}}$ are neural networks and $d_{\textnormal{dec}}$ is the dimension of $X$. In this work, the decoding density can also be parametrized with neural networks, e.g., $q_{X|Y,\phi}=\textnormal{Cat}(X;\rho^{\prime}_{\textnormal{dec}}(S,Y;\phi))$ if $X$ is discrete and $q_{X|Y,\phi}=\mathcal{N}(X;\mu^{\prime}_{\textnormal{dec}}(S,Y;\phi),\sigma^{\prime}_{\textnormal{dec}}(S,Y;\phi)^{2}I_{d_{\textnormal{dec}}})$ if $X$ is continuous, where $\rho^{\prime}_{\textnormal{dec}},\mu^{\prime}_{\textnormal{dec}}$, and $\sigma^{\prime}_{\textnormal{dec}}$ are neural networks. Therefore, if the decoding density neural networks from are modified so that they take the private attributes $S$ as an input, then the resulting algorithm is the one proposed in this paper.

The cost function of the VIB and the nonlinear IB is

where $\eta$ is a parameter that controls the trade-off between the compression of the representations $Y$ and their ability to draw inferences about the task $T$.

The argument is analogous to the one for the modifications of unsupervised learning algorithms to obtain private representations. The only modification required in these supervised learning algorithms is to modify the decoding density neural networks to receive the sensitive attributes $S$ as an input as well as the representations $Y$.

In all these works and ours, the first (or the compression) term is usually calculated assuming that the encoder density is parametrized with neural networks, e.g., $p_{Y|X,\theta}=\mathcal{N}(Y;\mu_{\textnormal{enc}}(X;\theta),\sigma_{\textnormal{enc}}(X;\theta)^{2}I_{d})$, which allows the representations to be constructed using the reparametrization trick, e.g., $Y=\mu_{\textnormal{enc}}(X;\theta)+\sigma_{\textnormal{enc}}(X;\theta)E$, where $E\sim\mathcal{N}(0,I_{d})$, $d$ is the dimension of the representations, and $I_{d}$ is the $d$-dimensional identity matrix. Then, the marginal density of the representations is set so that the Kullback-Leibler divergence has either a closed expression, a simple way to estimate it, or a simple upper bound, e.g., $q_{Y|\theta}=\mathcal{N}(Y;0,I_{d})$ or $q_{Y|\theta}=\frac{1}{N}\sum_{i=1}^{N}p_{Y|X=x^{(i)},\theta}$, where $x^{(i)}$ are the input data samples. Moreover, the loss function applied to the output of the decoding density and the optimization algorithm, e.g., stochastic gradient descent or Adam , can remain the same in these works and ours, too.

The aforementioned modifications can also be introduced in other algorithms with cost functions with additional terms to $\mathcal{F}_{\textnormal{uns}}$ and $\mathcal{F}_{\textnormal{sup}}$. For example, adding a maximum-mean discrepancy (MMD) term on the representation priors to avoid the information preference problem like in the InfoVAE ; adding an MMD term on the encoder densities to enforce privacy or fairness like in the VFAE ; or adding a total correlation penalty to the representation’s marginal to enforce disentangled representations like in the Factor-VAE, the $\beta$-TCVAE, or the FFVAE , .

Appendix E Details of the experiments

In this section of the appendix, we include an additional experiment on the COMPAS dataset and describe the details of the experiments performed to validate the approach proposed in this paper. The code is at https://github.com/burklight/VariationalPrivacyFairness.

The ProPublica COMPAS dataset Available in the Kaggle website. contains $6,172$ samples of different attributes of criminal defendants in order to classify if they will recidivate within two years or not. These attributes include gender, age, or race. In both tasks, we followed the experimental set-up from and considered $S$ to be a binary variable stating if the defendant is African American and $X$ to be the rest of attributes. For the fairness task, we considered $T$ to be the binary variable stating if the defendant recidivated or not. Since this dataset was not previously divided between training and test set, we randomly splitted the dataset with $70$% of the samples ($4,320$) for training and the rest ($1,852$) for testing.

Similarly to the previous experiments, the proposed approach controls the trade-off between private and informative representations and between fair and accurate representations. In Figure 5 we see how the trade-off between the compression level $I(X;Y)$ and the informativeness of the representations independent of the private data $I(X;Y|S)$ and between the compression level $I(X;Y)$ and the predictability of the representations without the sensitive data $I(X;Y|S)$ is controlled by the private and the fair representations, respectively. Moreover, we can also see how the amount of information the representations keep about the private or the sensitive data is commanded by the Lagrange multipliers $\gamma$ and $\beta$.

Compared with other variational approaches to the PF [16, PPVAE] and [27, VFAE], as happened with the Adult dataset, the proposed approach controlled better the information the representations $Y$ contained about the sensitive attribute $S$. In particular, the PPVAE contained between 0.63 and 1 bit of information about $S$ for $\eta^{-1}\in$, and the VFAE contained between 0.68 and 1 bit for $\delta\in[N_{\textnormal{batch}},1000N_{\textnormal{batch}}]$. Note that 1 bit is the maximum information that $Y$ can contain about $S$, since $H(S)=1$ in this scenario. With respect to membership attacks, our method was a slightly weaker to linear attackers than the PPVAE and the VFAE, allowing an accuracy in the range $[0.54,0.60]$, compared to their respective ranges of $[0.46,0.62]$ and $[0.45,0.59]$. That is, considering the prior probability of $0.51$, the best parameter of our method conceded the attacker a $3\%$ accuracy that the other methods did not allow. However, once the attacker employed more sophisticated attacks, such as a random forest, our method maintained a range of $[0.52,0.56]$, while the PPVAE and the VFAE allowed almost a perfect recovery of the group membership, with respective accuracy ranges between $0.96$ and $0.99$ and between $0.96$ and $1.0$, as was suggested by the amount of bits of information their representations contained about $S$. As before, this behavior can be explained due to the Markov chain violation of the encoder densities $p_{Y|S,X,\theta}$ of these approaches.

Furthermore, the Lagrange multiplier $\beta$ also allows us to control the behavior of the accuracy, the error gap, and the discrimination for the COMPAS dataset (Figures 6a, 6d, and 6b). Moreover, in this scenario, as shown in Figures 6c and 6e, an increase of $\beta$ also increased the equalized odds level and the accuracy on $S$ of adversarial classifiers (even though they remained below their values obtained with the original data $X$ for all the $\beta$ tested). These results on the equalized odds, even though not generalizable since we have the counter-example of the Adult dataset, indicate that in some situations this quanitty can be controlled with our approach. More specifically, we believe this happens when we can guarantee that $I(S;Y;T)$ is non-negative as explained in Remark 5.

Finally, we also observe, as for the Adult dataset, how the proposed method’s representations are more robust against non-linear adversaries than the representations obtained with the baseline from [7, LFR] (see Table III). Moreover, the method performs similarly as state-of-the-art methods based on adversarial learning [9, CFAIR] and variational inference [28, FFVAE], as shown in Table III.

E-B Experimental details

In all the experiments performed, we modeled the encoding density as an isotropic Gaussian distribution, i.e., $p_{Y|X,\theta}=\mathcal{N}(Y;\mu_{\textnormal{enc}}(X;\theta),\sigma_{\theta}^{2}I_{2})$, so that $Y=\mu_{\textnormal{enc}}(X;\theta)+\sigma_{\theta}E$, where $E\sim\mathcal{N}(0,I_{2})$, $\mu_{\textnormal{enc}}$ is a neural network, $\sigma_{\theta}$ is also optimized via gradient descent but is not calculated with $X$ as an input, and where the representations have $2$ dimensions. The neural networks in each experiment were:

For the Adult dataset, $\mu_{\textnormal{enc}}$ was a multi-layer perceptron with a single hidden layer with 100 units and ReLU activations.

For the Colored MNIST dataset, $\mu_{\textnormal{enc}}$ was the convolutional neural network CNN-enc-1 for both the privacy and fairness experiments, and the convolutional neural network CNN-enc-2 for the example from Figure 3. Both architectures are described in Table IV.

For the COMPAS dataset, $\mu_{\textnormal{enc}}$ was a multi-layer perceptron with a single hidden layer with 100 units and ReLU activations.

Moreover, the marginal density of the representations was modeled as an isotropic Gaussian of unit variance and zero mean; i.e., $q_{Y|\theta}=\mathcal{N}(Y;0,I_{2})$.

For the Adult dataset, the decoding neural network was a multi-layer perceptron with a single hidden layer with 100 units and ReLU activations. For the fairness task, the output was 1-dimensional with a Sigmoid activation function. For the privacy task, the output was $121$-dimensional. The input of the network was a concatenation of $Y$ and $S$.

For the Colored MNIST dataset and the fairness task, the decoding neural network was also a multi-layer perceptron with a single hidden layer with 100 units, ReLU activations, and a 1-dimensional output with a Sigmoid activation function. For the privacy task, the decoding neural network was the CNN-dec-1 for the normal experiments and the CNN-dec-2 for the example of Figure 3. The input linear layers took as an input a concatenation of $Y$ and $S$ and in the convolutional layers $S$ was introduced as a bias.

For the COMPAS dataset, the decoding neural network also was a multi-layer perceptron with a single hidden layer with 100 units and ReLU activations. For the fairness task, the output was 1-dimensional with a Sigmoid activation function. For the privacy task, the output was $19$-dimensional. The input of the network was a concatenation of $Y$ and $S$.

The hyperparameters employed in the experiments to train the encoder and decoder networks are displayed in Table V, and the optimization algorithm used was Adam . All random seeds were set to 2020.

All experiments were run with PyTorch , NumPy , and scikit-learn on a Nvidia Tesla P100 PCIE GPU of 16Gb of RAM.

The input data $X$ for the Adult and the COMPAS dataset was normalized to have 0 mean and unit variance. The input data for the Colored MNIST dataset was scaled to the range $$.

The mutual information $I(X;Y)$ and the conditional entropy $I(X;Y|S)$ and $I(T;Y|S)$ were calculated with the bounds from (5), (6), and (7), respectively. Since $H(X|S)$ was not directly obtainable, $-H(X|S,Y)$ was calculated and displayed instead. The mutual information $I(S;Y)$ was calculated using the mutual information neural estimator (MINE) with a moving average bias corrector with a an exponential rate of 0.1 ; the resulting information was averaged over the last 100 iterations. The neural networks employed were a 2-hidden layer multi-layer perceptron with 100 ReLU6 activation functions for all the datasets and tasks, except from the example on the Colored MNIST dataset, where the CNN-mine from Table IV was used. In all tasks the input was a concatenation of $S$ and $Y$, except from the example on the Colored MNIST dataset, where in all convolutional layers the private data $S$ was added as a bias and in all linear layers $S$ was concatenated to the input. The hyperparameters used to train the networks are displayed in Table VI.

The accuracy on $T$ and $S$, the discrimination, and the error and equalized odds gaps were calculated using both the input data $X$ and the generated representations $Y$. They were calculated with a Logistic regression (LR) classifier and a random forest (RF) classifier with the default settings from scikit-learn . The prior displayed on the accuracy on $T$ and $S$ figures is the accuracy of a classifier that only infers the majority class of $T$ and $S$, respectively, from the training dataset.

We implemented the algorithm as described in the original paper. We used the same neural network architecture (i.e., multi-layer perceptron with a single hidden layer with 100 units and ReLU activations) and training hyperparameters than in our method in order to provide a fair comparison. The original method was not prepared to handle other data that was not continuous, so we expanded the method assuming categorical, and Bernoulli output distributions for the non-continuous data. We studied 30 linearly equiespaced values of the hyperparameter $\eta^{-1}$ from (10) ranging from 1 to 50.

We implemented the algorithm as described in the original paper. We used the same neural network architecture (i.e., multi-layer perceptron with a single hidden layer with 100 units and ReLU activations) and training hyperparameters than in our method in order to provide a fair comparison. We studied 30 linearly equiespaced values of the hyperparameter $\delta$ multiplying the MMD term ranging from $N_{\textnormal{batch}}$ to $N_{\textnormal{batch}}1000$. We calculated the MMD using random kitchen sinks with $D=500$ and $\gamma=1$ as in the original paper.

We implemented the algorithm as described in the original paper. We used 10 prototypes ($K=10$) and adjusted the hyperparemeters $A_{x}$, $A_{y}$, and $A_{z}$ so that the L-BFGS found a feasible solution in $150,000$ iterations with a tolerance of $10^{-5}$. We observed that different values of the hyperparameters (as noted by ) produced almost equivalent results, so we only report here the results for $A_{x}=10^{-4}$, $A_{y}=0.1$, and $A_{z}=500$.

We implemented the algorithm as described in the original paper. We used the same neural network architecture (i.e., multi-layer perceptron with a single hidden layer with 100 units and ReLU activations) and training hyperparameters than in our method in order to provide with a fair comparison. Since the sensitive variable has dimension 1, there was no need for an adversary discriminator and therefore the $\gamma$ parameter was not explored. We performed experiments with different values of $\alpha$ ranging from $1$ to $400$ as in the original paper and we observed that the value of the hyperparameter did not modify much the results, as shown in Table VII. We selected $\alpha=200$ for the comparisons since the results for that $\alpha$ were the ones with the better trade-off between accuracy and fairness.

We implemented the algorithm as described in the original paper. For the experiments in the Adult dataset, we used the same neural network architecture (i.e., multi-layer perceptron with a single hidden layer with 100 units and ReLU activations) than in our method for the encoder, decoder, and the adversarial decoders, which are equipped with a gradient reversal layer . Also, we employed the same training hyperparameters to ensure a fair comparison between the two methods. We performed experiments with different values of $\lambda$ ranging from $0.1$ to $1000$ as in the original paper. However, for the experiments in the COMPAS dataset, we could not obtain good results with our architecture and hyperparameters nor could we replicate their results with their architecture and hyperparameters. Hence, for this dataset, we report here the results displayed in the original paper.

Appendix F Group fairness and utility indicators

In this section of the appendix, we define and put into perspective a series of metrics, employed in this article, that indicate the predicting and group fairness quality of a classifier.

A common metric to evaluate the performance (utility) of a classifier $w$ on a dataset is its accuracy, which measures the fraction of correct classifications of $w$ on such a dataset.

The accuracy of a classifier $w$ on a dataset $D$ is

An ideally fair classifier $w$ would maintain demographic parity (or statistical parity) and accuracy parity, which, respectively, mean that $w(X)\perp S$ (or, equivalently if $w$ is deterministic, that $\hat{P}_{S=0}(w(X)=1)=\hat{P}_{S=1}(w(X)=1)$) and that $\hat{P}_{S=0}(w(X)\not=T)=\hat{P}_{S=1}(w(X)\not=T)$ . In other words, if a classifier has demographic parity, it means that it gives a positive outcome with equal rate to the members of $S=0$ and $S=1$. However, demographic parity might damage the desired utility of the classifier , [11, Corollary 3.3]. Accuracy parity, on the contrary, allows the existance of perfect classifiers . The metrics that assess the deviation of a classifier from demographic and accuracy parities are the discrimination or demographic parity gap and the error gap .

The discrimination or demographic parity gap of a classifier $w$ to the sensitive variable $S$ on a dataset $D$ is

The error gap of a classifier $w$ with respect to the sensitive variable $S$ on a dataset $D$ is

The equalized odds gap of a classifier $w$ with respect to the sensitive variable $S$ on a dataset $D$ is

In the particular case of learning fair representations, the classifier $w:\mathcal{X}\rightarrow\mathcal{T}$ consists of two stages: an encoder $w_{\textnormal{enc}}:\mathcal{X}\rightarrow\mathcal{Y}$ and a decoder $w_{\textnormal{dec}}:\mathcal{Y}\rightarrow\mathcal{T}$, where the intermediate variable $Y=w_{\textnormal{enc}}(X)$ is the fair representation of the data. Therefore:

Minimizing $I(S;Y)$ encourages demographic parity, since

Minimizing $I(S;Y|T)$ encourages equalized odds, since

Based on Remark 3, we note that the variational approach to the CFB and the CPF for generating private and/or fair representations encourages demographic parity, since the minimization of the Lagrangians of such problems, $\mathcal{L}_{\textnormal{CFB}}$ and $\mathcal{L}_{\textnormal{CPF}}$, indeed minimizes $I(S;Y)$.

Contrary to Remark 4 concerning the minimization of the demographic parity gap, we cannot say that the variational approach to the CFB and the CPF minimizes the equalized odds gap.

Even though $I(S;Y)=I(S;Y|T)+I(S;Y;T)$, since $I(S;Y;T)$ can be negative , then $I(S;Y)$ is not necessarily greater than $I(S;Y|T)$ and thus there is no guarantee that minimizing $I(S;Y)$ will minimize $I(S;Y|T)$ as well.

Therefore, the minimization of $\mathcal{L}_{\textnormal{CFB}}$ and $\mathcal{L}_{\textnormal{CPF}}$, which minimizes $I(S;Y)$, also minimizes the equalized odds gap when $I(S;Y;T)\geq 0$ and therefore $I(S;Y)\geq I(S;Y|T)$. That is, the CFB and the CPF minimize the equalized odds gap when there is no synergy between $X$ and $Y$ to learn about $S$; i.e., $I(S;Y)+I(S;X)\geq I(S;(X,Y))$.

Continuing with the reflection on Remarks 3 and 5, a constrained optimization formulation a la CPF that leads to a minimization of the equalized odds gap is possible. Namely,

which minimizes the dark and darker gray areas from Figure 1a, which correspond to the sensitive and irrelevant data, respectively, except the intersection between the sensitive data $S$, the representations $Y$, and the target $T$, which corresponds to $I(S;Y;T)$. In this formulation, it is ensured that the representations $Y$ also maintain a certain level of the information $r$ of the target that is not shared in the sensitive data $I(T;Y|S)$.

Appendix G Non-convexity of the CPF and the CFB

In this section of the appendix, we show how both the CPF and the CFB as defined in (1) and (2) are non-convex optimization problems.

Let $X\in\mathcal{X}$, $S\in\mathcal{S}$, $Y\in\mathcal{Y}$, and $T\in\mathcal{T}$ be random variables. Then,

If the Markov chain $S\leftrightarrow X\rightarrow Y$ holds and the distributions of $X$ and $S$ are fixed, then $I(X;Y)$, $I(S;Y)$, and $I(X;Y|S)$ are convex functions with respect to the density $p_{Y|X}$.

If, additionally, the Markov chain $T\leftrightarrow X\rightarrow Y$ holds and the distributions of $X$, $S$, and $T$ are fixed, then $I(X;Y|S,T)$ and $I(T;Y|S)$ are also convex functions with respect to the density $p_{Y|X}$.

We start the proof leveraging [57, Theorem 2.7.4], which, in our setting, tells us that:

$I(X;Y)$ is a convex function of $p_{Y|X}$ if $p_{X}$ is fixed.

$I(S;Y)$ is a convex function of $p_{Y|S}$ if $p_{S}$ is fixed.

$I(X;Y|S)$ is a convex function of $p_{Y|X,S}$ if $p_{X|S}$ is fixed.

$I(X;Y|S,T)$ is a convex function of $p_{Y|X,S,T}$ if $p_{X|S,T}$ is fixed.

$I(T;Y|S)$ is a convex function of $p_{Y|S,T}$ if $p_{T|S}$ is fixed.

Let us consider that the distributions of $S$ and $X$ are fixed and that the conditional distribution $P_{Y|X}$ has a density $p_{Y|X}$. Then, the CPF optimization problem is not convex.

From Lemma 1 we know that $I(S;Y)$ and $I(X;Y|S)$ are convex functions with respect to $p_{Y|X}$ for fixed $p_{S}$ and $p_{X|S}$. Hence, the constraint $I(X;Y|S)\geq r$ is concave. ∎

Let us consider that the distributions of $S$, $T$, and $X$ are fixed and that the conditional distribution $P_{Y|X}$ has a density $p_{Y|X}$. Then, the CFB optimization problem is not convex.

From Lemma 1 we know that $I(S;Y)$, $I(X;Y|S,T)$, and $I(T;Y|S)$ are convex functions with respect to $p_{Y|X}$ for fixed $p_{S}$, $p_{X|S,T}$, and $p_{T|S}$. Hence, the constraint $I(T;Y|S)\geq r$ is concave. ∎

Appendix H Limitations and future directions

The CPF (1) and the CFB (2) are non-convex optimization problems with respect to $P_{Y|X}$ (see Appendix G). Therefore, (i) the optimal conditional distribution $P^{\star}_{Y|X}$ that minimizes the Lagrangian might not be achieved through gradient descent, and (ii) even if $P^{\star}_{Y|X}$ is achieved, it could be a sub-optimal value for (1) or (2), since the problems are not strongly dual [23, Section 5.2.3]. A possible solution could be the application of a monotonically increasing concave function $u$ to $I(X;Y|S)$ or $I(T;Y|S)$ in the CPF or CFB Lagrangians, respectively, so that $u(I(X;Y|S))$ or $u(I(T;Y|S))$ is concave (and hence the Lagrangian is convex) in the domain of interest. For some $u$, this approach might allow to attain the desired $r$ in (1) or (2) with a specific value of the Lagrange multiplier; see for an example of this approach for the IB.

The proposed approach entails two limitations that are common in variational attempts at solving an optimization problem. Namely: (i) it approximates the decoding and the marginal distributions and (ii) it considers parametrized densities. The first issue restricts the search space of the possible encoding distributions $P_{Y|X}$ to those distributions with a decoding and marginal distributions that follow the restrictions of the variational approximation. The second issue further limits the search space to the obtainable encoding distributions with densities $p_{Y|X,\theta}$ with a parametrization $\theta$. For this reason, richer encoding distributions and marginals, e.g., by means of normalizing flows , are a possible direction to mitigate these issues.