Towards Efficient Data Valuation Based on the Shapley Value

Introduction

Data analytics using machine learning (ML) is an increasingly common practice in modern science and business. The data for building an ML model are often provided by multiple entities. For instance, Internet enterprises analyze various users’ data to improve product design, customer retention, and initiatives that help them earn revenue. Furthermore, the quality of the data from different entities may vary widely. Therefore, a key question often asked by stakeholders of a ML system is how to fairly allocate the revenue generated by a ML model to the data contributors.

This question is also motivated by a system we are building together with one of the largest hospital in the US. In the system, patients submit part of their medical records onto a “data market,” and analysts pay a certain amount of money to train a ML model on patients’ data. One of the challenges in such data markets is how to distribute the payment from analysts back to the patients.

A natural way of tackling the data valuation problem is to adopt a game-theoretic viewpoint, where each data contributor is modeled as a player in a coalitional game and the usefulness of data from any subset of contributors is characterized via a utility function. The Shapley value (SV) is a classic method in cooperative game theory to distribute the total gains generated by the coalition of all players, and has been applied to problems in various domains, ranging from economics , counter-terrorism , environmental science , to ML . The reason for its broad adoption is that the SV defines a unique profit allocation scheme that satisfies a set of properties with appealing real-world interpretations, such as fairness, rationality, and decentralizability.

Despite the desirable properties of the SV, computing the SV is known to be expensive; the number of utility function evaluations required by the exact SV calculation grows exponentially in the number of players. This poses a radical challenge to using the SV in the context of data valuation—how to calculate, or approximate the SV over millions or even billions of data points, a scale that is rare in previous applications of the SV, but not uncommon for real-world data valuation tasks. Even worse, for ML tasks, evaluating the utility function itself (e.g., testing accuracy) is already computationally expensive, as it requires training a model. Due to the computational challenge, the application of the SV to data valuation has thus far been limited to stylized examples, in which the underlying utility function of the game is simple and the resulting SV can be represented as a closed-form expression . The state-of-the-art method to estimate the SV for a black-box utility function is based on Monte Carlo simulations , which still requires evaluating ML models for $\mathcal{O}(N^{2}\log N)$ many times in order to compute the SV of $N$ data points and is thus clearly impracticable. In this paper, we attempt to answer the question of whether it is possible to efficiently estimate the SV while achieving the same performance guarantee as the state-of-the-art method.

Theoretical Contribution We first study this question from a theoretical perspective. We show that, to approximate the SV of $N$ data points with provable error guarantees, it is possible to design an algorithm with $\mathcal{O}(N(\log N)^{2})$ model evaluationsSee the technique note by Wang and Jia [wang2023note] for an improved version of this algorithm.. We achieve this by enabling proper information sharing between different model evaluations. Moreover, if it is reasonable to assume that the utility function is monotone and the SV is “sparse” in the sense that only few data points have significant values, then we are able to further reduce the number of model training to $\mathcal{O}(\log\log N)$, when the model can be incrementally maintained. It is worth noting that these two algorithms are agnostic to the context wherein the SV is computed; hence, they are also useful for the applications beyond data valuation.

Practical Contribution Despite the improvements from a theoretical perspective, retraining models for multiple times may still be unaffordable for large datasets and ML models. We then introduce two practical SV estimation algorithms specific to ML tasks by introducing various assumptions on the utility function. We show that if a learning algorithm is uniformly stable , then uniform value division produces a fairly good approximation to the true SV. In addition, for an ML model with smooth loss functions, we propose to use the influence function to accelerate the data valuation process. However, the efficiency does not come for free. The first algorithm relies on the stability of a learning algorithm, which is difficult to prove for complex ML models, such as deep neural networks. The compromise that we have to make in the second algorithm is that the resulting SV estimates no longer have provable guarantees on the approximation error. Filling the gap between theoretical soundness and practicality is important future work.

Table 1 summarizes the contributions of this paper. In the rest of the paper, we will elaborate on the idea and analysis of these algorithms, and further use them to compute the data values for various benchmark datasets.

Related Work

Originated from game theory, the SV, in its most general form, can be $\mathsf{\#P}$-complete to compute . Efficiently estimating SV has been studied extensively for decades. For bounded utility functions, Maleki et al. described a sampling-based approach that requires $\mathcal{O}(N\log N)$ samples to achieve a desired approximation error in $l_{\infty}$ norm and $\mathcal{O}(N^{2}\log N)$ in $l_{2}$ norm. Bachrach et al. also leveraged a similar approach but focused on the case where the utility function has binary outputs. By taking into account special properties of the utility function, one can derive more efficient approximation algorithms. For instance, Fatima et al. proposed a probabilistic approximation algorithm with $\mathcal{O}(N)$ complexity for weighted voting games. The game-theoretic analysis of the value of personal data has been explored in , which proposed a fair compensation mechanism based on the SV like ours. They derived the SV under simple data utility models abstracted from network games or recommendation systems, while our work focuses on more complex utility functions derived from ML applications. In our case, the SV no longer has closed-form expressions. We develop novel and efficient approximation algorithms to overcome this hurdle.

Using the SV in the context of ML is not new. For instance, the SV has been applied to feature selection . While their contributions have inspired this paper, many assumptions made for feature “valuation” do not hold for data valuation. As we will see, by studying the SV tailored to data valuation, we can develop novel algorithms that are more efficient than the previous approaches .

Despite not being used for data valuation, ranking the importance of training data points has been used for understanding model behaviors, detecting dataset errors, etc. Existing methods include using the influence function for smooth parametric models and a variant for non-parametric ones. Ogawa et al. proposed rules to identify and remove the least influential data in order to reduce the computation cost when training support vector machines (SVM). One can also construct coresets—weighted data subsets—such that models trained on these coresets are provably competitive with models trained on the full dataset . These approaches could potentially be used for valuing data; however, it is not clear whether they satisfy the properties desired by data valuation, such as fairness. We leave it for future work to understand these distinct approaches for data valuation.

Problem Formulation

Consider a dataset $D=\{z_{i}\}_{i=1}^{N}$ containing data from $N$ users. Let $U(S)$ be the utility function, representing the value calculated by the additive aggregation of $\{z_{i}\}_{i\in S}$ and $S\subseteq I=\{1,\cdots,N\}$. Without loss of generality, we assume throughout that $U(\emptyset)=0$. Our goal is to partition $U_{\text{tot}}\triangleq U(I)$, the utility of the entire dataset, to the individual users; more formally, we want to find a function that assigns to user $i$ a number $s(U,i)$ for a given utility function $U$. We suppress the dependency on $U$ when the utility is self-evident and use $s_{i}$ to represent the value allocated to user $i$.

The SV is a classic concept in cooperative game theory to attribute the total gains generated by the coalition of all players. Given a utility function $U(\cdot)$, the SV for user $i$ is defined as the average marginal contribution of $z_{i}$ to all possible subsets of $D=\{z_{i}\}_{i\in I}$ formed by other users:

The formula in (1) can also be stated in the equivalent form:

where $\pi\in\Pi(D)$ is a permutation of users and $P_{i}^{\pi}$ is the set of users which precede user $i$ in $\pi$. Intuitively, imagine all users’ data are to be collected in a random order, and that every user $i$ receives his data’s marginal contribution that would bring to those whose data are already collected. If we average these contributions over all the possible orders of users, we obtain $s_{i}$. The importance of the SV stems from the fact that it is the unique value division scheme that satisfies the following desirable properties.

1. Group Rationality: The value of the entire dataset is completely distributed among all users, i.e., $U(I)=\sum_{i\in I}s_{i}$.

2. Fairness: (1) Two users who are identical with respect to what they contribute to a dataset’s utility should have the same value. That is, if user $i$ and $j$ are equivalent in the sense that $U(S\cup\{i\})=U(S\cup\{j\}),\forall S\subseteq I\setminus\{i,j\}$, then $s_{i}=s_{j}$. (2) Users with zero marginal contributions to all subsets of the dataset receive zero payoff, i.e., $s_{i}=0$ if $U(S\cup\{i\})=0$ for all $S\subseteq I\setminus\{i\}$.

3. Additivity: The values under multiple utilities sum up to the value under a utility that is the sum of all these utilities: $s(U,i)+s(V,i)=s(U+V,i)$ for $i\in I$.

The group rationality property states that any rational group of users would expect to distribute the full yield of their coalition. The fairness property requires that the names of the users play no role in determining the value, which should be sensitive only to how the utility function responds to the presence of a user’s data. The additivity property facilitates efficient value calculation when data is used for multiple applications, each of which is associated with a specific utility function. With additivity, one can decompose a given utility function into an arbitrary sum of utility functions and compute utility shares separately, resulting in transparency and decentralizability. The fact that the SV uniquely possesses these properties, combined with its flexibility to support different utility functions, leads us to employ the SV to attribute the total gains generated from a dataset to each user.

Efficient SV Estimation

Note that for an ML task, we can write the utility function $U(S)=U_{m}(A(S))$, where $A(\cdot)$ represents a learning algorithm that maps a dataset $S$ onto a model and $U_{m}(\cdot)$ is some measure of model performance, such as test accuracy. Typically, a substantial part of computational costs associated with the utility evaluation lies in $A(\cdot)$. Hence, it is useful to examine the efficiency of an approximation algorithm in terms of the number of model training required. In general, one utility evaluation would need to re-train a model. Particularly, when $A(\cdot)$ is incrementally trainable, one pass over the entire training set allows us to evaluate $\phi_{i}$ for all $i=1,\cdots,N$. Hence, in this case, the number of model training needed to achieve an $(\epsilon,\delta)$-approximation is the same as $m_{\text{perm}}=\mathcal{O}(N\log N)$.

2 Group Testing-Based Approach

We now describe an algorithm that makes the same assumption of bounded utility as the baseline algorithm, but requires significantly fewer utility evaluations than the baseline.

Our proposed approximation algorithm is inspired by previous work applying the group testing theory to feature selection . Recall that group testing is a combinatorial search paradigm , in which one wants to determine whether each item in a set is “good” or “defective” by performing a sequence of tests. The result of a test may be positive, indicating that at least one of the items of that subset is defective, or negative, indicating that all items in that subset are good. Each test is performed on a pool of different items and the number of tests can be made significantly smaller than the number of items by smartly distributing items into pools. Hence, the group testing is particularly useful when testing an individual item’s quality is expensive. Analogously, we can think of SV calculation as a group testing problem with continuous quality measure. Each user’s data is an “item” and the data utility corresponds to the item’s quality. Each “test” in our scenario corresponds to evaluating the utility of a subset of users and is expensive. Drawing on the idea of group testing, we hope to recover the utility of all user subsets from a small amount of customized tests.

Let $T$ be the total number of tests. At test $t$, a random set of users is drawn from $I$ and we evaluate the utility of the selected set of users. If we model the appearance of user $i$ and $j$’s data in a test as Boolean random variables $\beta_{i}$ and $\beta_{j}$, respectively, then the difference between the utility of user $i$ and that of user $j$ is

where $U(\beta_{1},\cdots,\beta_{N})$ is the utility evaluated on the users with the Boolean appearance random variable equal to $1$.

Using the definition of the SV, one can derive the following formula of the SV difference between any pair of users.

For any $i,j\in I$, the difference in SVs between $i$ and $j$ is

Due to the space limitation, we omit all the proofs of the paper to our supplemental materials. The key idea of the proposed algorithm is to smartly design the sampling distribution of $\beta_{1},\cdots,\beta_{N}$ such that the expectation of (3) mirrors the Shapley difference in (4). This will enable us to calculate the Shapely differences from the test results with a high-probability error bound. The following Lemma states that if we can estimate the Shapley differences between all data pairs up to $(\epsilon/\sqrt{N},\delta/N)$, then we will be able to recover the SV with the approximation error $(\epsilon,\delta)$.

Suppose that $C_{ij}$ is an $(\epsilon/(2\sqrt{N}),\delta/(N(N-1)))$-approximation to $s_{i}-s_{j}$. Then, any solutions to the feasibility problem

is an $(\epsilon,\delta)$-approximation to $s$ with respect to $l_{2}$-norm.

Algorithm 1 presents the pseudo-code of the group testing-based algorithm, which first estimates the Shapley differences and then derives the SV from the Shapley differences by solving a feasibility problem.

The following theorem provides a lower bound on the number of tests $T$ needed to achieve an $(\epsilon,\delta)$-approximation.

Algorithm 1 returns an $(\epsilon,\delta)$-approximation to the SV with respect to $l_{2}$-norm if the number of tests $T$ satisfies T\geq 8\log\frac{N(N-1)}{2\delta}/\big{(}(1-q_{tot}^{2})h\big{(}\frac{\epsilon}{Zr\sqrt{N}(1-q_{tot}^{2})}\big{)}\big{)}, where $q_{tot}=\frac{N-2}{N}q(1)+\sum_{k=2}^{N-1}q(k)[1+\frac{2k(k-N)}{N(N-1)}]$, $h(u)=(1+u)\log(1+u)-u$, $Z=2\sum_{k=1}^{N-1}\frac{1}{k}$, and $r$ is the range of the utility function.Wang and Jia [wang2023note] gives an improved version of this result.

Using the Taylor expansion of $h$, it can be proved that when $N$ is large, $T$ is $\mathcal{O}(N(\log N)^{2})$. Since only one utility evaluation is required for a single test, the number of utility evaluations is at most $\mathcal{O}(N(\log N)^{2})$. On the other hand, in the baseline approach, the number of utility evaluations is $\mathcal{O}(N^{2}\log N)$. Hence, the group testing requires significantly fewer model evaluations than the baseline.

3 Exploiting the Sparsity of Values

We now present an algorithm inspired by our empirical observations of the SV for large datasets. This algorithm can produce an $(\epsilon,\delta)$-approximation to the SV with only $\mathcal{O}(N\log(N)\log(\log(N)))$ utility evaluations.

Figure 2 illustrates the distribution of the SV of the MNIST dataset, from which we observed that the SV is “approximately sparse”—most of values are concentrated around its mean and only a few data points have significant values. In the literature, the “approximate sparsity” of a vector $s$ is characterized by a small error of its best $K$-term approximation:

This observation opens up a vast collection of tools from compressive sensing for the purpose of calculating the SV.

It has been shown in that every $k$-sparse vector $s$ can be recovered by solving a convex optimization problem

if $\delta_{2s}(A)<1/3$. This result can also be generalized to noisy measurements . Drawing on the ideas of compressed sensing, we present Algorithm 2, termed compressive permutation sampling.

Suppose that $U(\cdot)$ is monotone. There exists some constant $C^{\prime}$ such that if $M\geq C^{\prime}(K\log(N/(2K))+\log(2/\delta))$ and $T\geq\frac{2r^{2}}{\epsilon^{2}}\log\frac{4M}{\delta}$, except for an event of probability no more than $\delta$, the output of Algorithm 2 obeys

for some constants $C_{1,K}$ and $C_{2,K}$.

Therefore, the number of utility evaluations (and model training) required for achieving the approximation error guarantee in Theorem 4 is $NT=\mathcal{O}(N\log(\log(N)))$. Particularly, when the utility function is defined with respect to an incrementally trainable model, only $\log\log(N)$ full model training is needed for achieving the error guarantee.

4 Stable Learning Algorithms

A learning algorithm is stable if the model learned by the algorithm is insensitive to the removal of an arbitrary point in the training dataset . More specifically, an algorithm $G$ has uniform stability $\gamma$ with respect to the loss function $l$ if $\|l(G(S),\cdot)-l(G(S^{\setminus i}),\cdot)\|_{\infty}\leq\gamma$ for all $i\in\{1,\cdots,|S|\}$, where $S$ denotes the training set and $S^{\setminus i}$ denotes the one by removing $i$th element of $S$. Indeed, a broad variety of learning algorithms are stable, including all learning algorithms with Tikhonov regularization. Stable learning algorithms are appealing as they enjoy provable generalization error bounds . Assume that the model is trained via a stable learning algorithm and training data’s utility is measured in terms of the testing loss. Due to the inherent insensitivity of a stable learning algorithm to the training data, we expect that the SV of each training point is similar to one another. The following theorem confirms our intuition and provides an upper bound on the SV difference between any pair of training data points.

For a learning algorithm $A(\cdot)$ with uniform stability $\beta=\frac{C_{\text{stab}}}{|S|}$, where $|S|$ is the size of the training set and $C_{\text{stab}}$ is some constant. Let the utility of $D$ be $U(D)=M-L_{\text{test}}(A(D),D_{\text{test}})$, where $L_{\text{test}}(A(D),D_{\text{test}})=\frac{1}{N}\sum_{i=1}^{N}l(A(D),z_{\text{test},i})$ and $0\leq l(\cdot,\cdot)\leq M$. Then, $s_{i}-s_{j}\leq 2C_{\text{stab}}\frac{1+\log(N-1)}{N-1}$ and the Shapley difference vanishes as $N\rightarrow\infty$.

By Lemma 2, if $2C_{\text{stab}}\frac{1+\log(N-1)}{N-1}$ is less than $\epsilon/(2\sqrt{N})$, uniformly assigning $\frac{U_{\text{tot}}}{N}$ to each data contributor provides an $(\epsilon,0)$-approximation to the SV.

5 Heuristic Based on Influence Functions

Largest-$S$ Approximation. One practical heuristic of using influence functions is to consider a single subset $S$ for computing $s_{i}$, namely, $I\setminus\{i\}$. With this heuristic, we can simply take a trained model on the whole dataset, and calculate the influence function for each data point. For logistic regression models, the first and second derivations enjoy closed-form expressions and the change in parameters after removing one point $z=(x,y)$ can be approximated by -\big{(}\sum_{i=1}^{N}\sigma(x_{i}^{T}\hat{\theta}^{N})\sigma(-x_{i}^{T}\hat{\theta}^{N})x_{i}x_{i}^{T}\big{)}^{-1}\sigma(-yx_{i}^{T}\hat{\theta}^{N})yx where $\sigma(u)=1/(1+\exp(-u))$ and $y\in\{-1,1\}$. The fact that largest-$S$ influence only considers a single subset makes it impossible to satisfy the group rationality and additivity properties simultaneously.

Consider the value attribution scheme that assigns the value $\hat{s}(U,i)=C_{U}[U(S\cup\{i\})-U(S)]$ to user $i$ where $|S|=N-1$ and $C_{U}$ is a constant such that $\sum_{i=1}^{N}\hat{s}(U,i)=U(I)$. Consider two utility functions $U(\cdot)$ and $V(\cdot)$. Then, $\hat{s}(U+V,i)\neq\hat{s}(U,i)+\hat{s}(V,i)$ unless $V(I)[\sum_{i=1}^{N}U(S\cup\{i\})-U(S)]=U(I)[\sum_{i=1}^{N}V(S\cup\{i\})-V(S)]$.

Experimental Results

We first compare the proposed approximation methods that only require mild assumptions on the ML models (e.g., bounded or differentiable utility), including (a) the permutation sampling baseline, (b) the group testing-based method, (c) using influence functions to approximate all marginal contributions, and (d) approximating the SV with only the influence function to the largest subset. The last two methods are hereinafter referred to as all-$S$ influence and largest-$S$ influence, respectively. We use a small-scale dataset, iris, and use (a) to estimate the true SV for a regularized logistic regression up to $\epsilon=1/N$. Figure 6(d) shows that the approximations produced by (a)-(c) are closest to each other. The result of the largest-$S$ influence is correlated with that of the other techniques, although it cannot recover the true SV.

We implement the SV calculation techniques on a machine with 16 cores (Intel Xeon CPU E5-2620 v4 @ 2.10GHz) and compare the runtime of different techniques on a two-class dog-vs-fish dataset of size $900$ constructed from the ImageNet dataset. To evaluate the runtime for training sizes above $900$, we concatenate duplicate copies of the dog-vs-fish dataset. For each training data point, we first pre-compute the 2048-dimensional inception features and then train a logistic regression using the stochastic gradient descent for $150$ epochs. The utility function is the negative testing loss of the logistic regression model. For the largest-$S$ influence and the all-$S$ influence, we use the method in to compute the influence function. The runtime of different techniques in logarithmic scale is displayed in Figure 3 (b). We can see that the group testing-based method outperforms the permutation sampling baseline by several orders of magnitude for a large number of data points. By exploiting influence function heuristics and the stratified sampling trick in Section 4.5, the computational costs can be further reduced. Due to the fact that the largest-$S$ influence heuristic only focuses on the marginal contribution of each training data point to a single subset, it is much more efficient than the permutation sampling, group testing and the all-$S$ influence, which compute the marginal contributions to a large number of subsets.

When it is plausible to assume the SV of a training set is sparse, we could employ the idea of compressive sensing to recover the SV with fewer samples. Figure 4 compares the sample efficiency of the baseline permutation sampling and the compressive permutation sampling method on a size-$1000$ dataset sampled randomly from MNIST. For a given approximation error, the compressive permutation requires significantly fewer samples and model valuations than the baseline approach. The superiority of the compressive permutation becomes less evident at the large sample regime.

Our theoretical result in Section 4.4 shows that the SV of training data tends to be uniform for a stable learning algorithm, which has a small stability parameter $\beta$. We empirically validate this result by training a ridge regression on the diabetes dataset and varying the strength of its regularization term. In , it is shown that the stability parameter $\beta$ of the ridge regression $\min_{\theta}\frac{1}{N}\sum_{i=1}^{N}l(\theta,z_{i})+\lambda\|\theta\|^{2}$ is proportional to $\sigma^{2}/\lambda$, where $\sigma$ is the Lipschitz constant of the loss function with respect to the model parameter $\theta$ and equal to $2|x_{i}^{T}\theta-y_{i}|\cdot|x_{i}|$. When the model fits the training data well, the change in $\sigma$ is small; therefore, applying more regularization leads to a more stable learning algorithm, which has lower variance in the training data values as illustrated in the shaded area of Figure 5. On the other hand, if the model no longer fits the data well due to excessive regularization, then $\sigma$ will dominate the stability parameter. In this case, since $\sigma$ increases with the regularization strength, $\beta$ and thereby the variance of the SV also increase. Note that the variance of the SV is identical to the approximation error of a uniform value division scheme.

Differential privacy has emerged as a standard privacy notation and is often achieved by adding noise that has a magnitude proportional to the desired privacy level. On the other hand, noise diminishes the usefulness of data and thereby degrades the value of data. We construct a training set using the MNIST, and divide the training dataset into two halves, one half containing normal images and the other half containing noisy ones. The testing accuracy on normal images is used as the utility function. Figure 5(b) illustrates a clear tradeoff between privacy and data value - the SV decreases as data becomes noisier.

Mixing adversarial examples with benign examples in the training dataset, or adversarial training, is an effective method to improve the adversarial robustness of a model. In practice, we measure the robustness in terms of the testing accuracy on a dataset containing adversarial examples. We expect that the adversarial examples in the training dataset become more valuable as more adversarial examples are added into the testing dataset. Based on the MNIST, we construct a training dataset that contains both benign and adversarial examples and synthesize testing datasets with different adversarial-benign mixing ratios. Two popular attack algorithms, namely, Fast Gradient Sign Method (FGSM) and the Carlini and Wagner (CW) attack are used to generate adversarial examples. Figure 6(a, b) compares the average SV for adversarial examples and for benign examples in the training dataset. The negative testing loss for logistic regression is used as the utility function. We see that the SV of adversarial examples increases as the testing data becomes more adversarial and contrariwise for benign examples. This is consistent with our expectation. In addition, the adversarial examples in the training set are more valuable if they are generated from the same attack algorithm for testing adversarial examples.

Conclusion

ML has opened up exciting opportunities to tackle a wide variety of problems; nevertheless, very few works have attempted to understand the value of data used for training models. A principled way of data valuation is the key to stimulating data exchange, enabling the development of more sophisticated and robust ML models. We adopt the SV, a classic concept from cooperative game theory, for data valuation. The SV has many unique properties appealing to data valuation. However, the lack of efficient methods to compute the SV has prevented it from being adopted in the past. We develop a repertoire of techniques for estimating the SV in different scenarios.

For future work, We wish to continue exploring the connection between ML and game theory and develop efficient valuation methods for ML models. It is also critical to understand other concepts from cooperative game theory (e.g., stable coalition) in the context of data valuation. Last but not least, we hope to apply the techniques to real-world applications and revolutionize the way of data collection and dissemination.

We would like to thank Tan Pin Lin and Feng Mingling for helping correct the complexity calculation of the group testing-based approximation algorithm and the assumption for proving the complexity of the compressive permutation sampling algorithm in the earlier version of the paper. We would also like to thank Jiachen T. Wang for enhancing the paper’s readability and strengthening some proofs.

This work is supported in part by the Republic of Singapore’s National Research Foundation through a grant to the Berkeley Education Alliance for Research in Singapore (BEARS) for the Singapore-Berkeley Building Efficiency and Sustainability in the Tropics (SinBerBEST) Program. This work is also supported in part by the CLTC (Center for Long-Term Cybersecurity); FORCES (Foundations Of Resilient CybEr-Physical Systems), which receives support from the National Science Foundation (NSF award numbers CNS-1238959, CNS-1238962, CNS-1239054, CNS1239166); and the National Science Foundation under Grant No. TWC-1518899. CZ and the DS3Lab gratefully acknowledge the support from Mercedes-Benz Research & Development NA, MeteoSwiss, Oracle Labs, Swiss Data Science Center, Swisscom, Zurich Insurance, Chinese Scholarship Council, and the Department of Computer Science at ETH Zurich.

References

Appendix A Proof of Lemma 1

For any $i,j\in I$ and $i\neq j$, the difference in Shapley values between $i$ and $j$ is

Loosely speaking, the proof distinguishes subsets $S$ which include neither $i$ nor $j$ (such that the subset utility $U(S)$ of the marginal contribution directly cancels) and subsets including either $i$ or $j$. In the latter case, $S$ can be partitioned to a mock subset $S^{\prime}$ by excluding the respective point from S such that a common sum over $S^{\prime}$ again eliminates all terms other than $U(S^{\prime}\cup\{i\})-U(S^{\prime}\cup\{j\})$.

Appendix B Proof of Lemma 2

Suppose that $C_{ij}$ is an $(\epsilon/(2\sqrt{N}),\delta/(N(N-1)))$-approximation to $s_{i}-s_{j}$. Then, the solution to the feasibility problem

must exist, and any feasible solutions are an $(\epsilon,\delta)$-approximation to $s$ with respect to $l_{2}$-norm.

To see the existence of the feasible solution, the true Shapley value $\hat{s}_{i}=s_{i}$ is a feasible solution given the condition.

For the second part of the theorem, let $\epsilon^{\prime}=\epsilon/(2\sqrt{N})$. Assume, for contradition, $\hat{s}_{i}-s_{i}>\epsilon/\sqrt{N}$. Let $\hat{s}_{i}-s_{i}=c\epsilon^{\prime}$ where $c>2$.

Since $C_{i,j}$ is an $(\epsilon^{\prime},\delta/(N(N-1)))$-approximation to $s_{i}-s_{j}$, we have that with probability at least $1-\delta/(N(N-1))$,

Moreover, the inequality (12) implies that

with probability at least $1-\delta/(N(N-1))$. By the assumption that $\hat{s}_{i}-s_{i}=c\epsilon^{\prime}$ and $c>2$, we have

which further implies that $\hat{s}_{j}-s_{j}>0$ for some $j\neq i$. Thus, with probability $1-\delta/N$, we have $\hat{s}_{j}-s_{j}>0$ for all $j\neq i$.

Since $\sum_{j=1}^{N}s_{j}=U_{\text{tot}}$, it follows that $\sum_{j=1}^{N}\hat{s}_{j}>U_{\text{tot}}$, which contradicts with the fact that $\hat{s}_{j}$ ($j=1,\ldots,N$) is a solution to the feasibility problem (11) and (12).

The contradiction can be similarly established for $s_{i}-\hat{s}_{i}=c\epsilon^{\prime}$. Therefore, we have that with probability at least $1-\delta/N$, $|s_{i}-\hat{s}_{i}|\leq 2\epsilon^{\prime}$ for some $i$. This in turn implies that with probability at least $1-\delta$, $\|\hat{s}-s\|_{\infty}\leq 2\epsilon^{\prime}=\epsilon/\sqrt{N}$. Moreover, since $\|\hat{s}-s\|_{2}\leq\sqrt{N}\|\hat{s}-s\|_{\infty}=\epsilon$, we have that $\|\hat{s}-s\|_{2}\leq\epsilon$ with probability at least $1-\delta$.

Appendix C Proof of Theorem 3

We prove Theorem 3, which specifies a lower bound on the number of tests needed for achieving a certain approximation error. Before delving into the proof, we first present a lemma that is useful for establishing the bound in Theorem 3.

Given independent zero-mean random variables $X_{1},\cdots,X_{n}$ satisfying the condition $|X_{i}|\leq a$ for all $i$, let $\sigma^{2}=\sum_{i=1}^{n}\sigma_{i}^{2}$ be the total variance where $\sigma_{i}^{2}=Var(X_{i})$. Then for any $t\geq 0$,

where $h(u)=(1+u)\log(1+u)-u$ and $S_{n}=\sum_{i=1}^{n}X_{i}$.

We now restate Theorem 3 and proceed to the main proof.

Algorithm 1 returns an $(\epsilon,\delta)$-approximation to the Shapley value with respect to $l_{2}$-norm if the number of tests $T$ satisfies T\geq 8\log\frac{N(N-1)}{2\delta}/\big{(}(1-q_{tot}^{2})h\big{(}\frac{\epsilon}{Zr\sqrt{N}(1-q_{tot}^{2})}\big{)}\big{)}, where $q_{tot}=\frac{N-2}{N}q(1)+\sum_{k=2}^{N-1}q(k)[1+\frac{2k(k-N)}{N(N-1)}]$, $h(u)=(1+u)\log(1+u)-u$, $Z=2\sum_{k=1}^{N-1}\frac{1}{k}$, and $r$ is the range of the utility function.

By Lemma 1, the difference in Shapley values between points $i$ and $j$ is given as

Let $\beta_{1},\cdots,\beta_{N}$ denote $N$ Boolean random variables drawn with the following sampler:

Sample the “length of the sequence” $\sum_{i=1}^{N}\beta_{i}=k\in\{1,2,\cdots,N-1\}$, with probability $q(k)$.

Uniformly sample a length-$k$ sequence from $N\choose k$ all possible length-$k$ sequences

Then the probability of any given sequence $\beta_{1},\cdots,\beta_{N}$ is

Now, we consider any two data points $x_{i}$ and $x_{j}$ where $i,j\in I=\{1,\cdots,N\}$ and their associated Boolean variables $\beta_{i}$ and $\beta_{j}$, and analyze

Consider the expectation of $\Delta$. Obviously, only $\beta_{i}\not=\beta_{j}$ has non-zero contributions:

for $k=1,\cdots,N-1$. The value of $Z$ is given by

Note that $\Delta=0$ when $\beta_{i}=\beta_{j}$. If $P[\beta_{i}=\beta_{j}]$ is large, then the variance of $\Delta$ will be much smaller than its range.

Now, we analyze the variance of $\Delta$. By the law of total variance,

Recall $\Delta\in[-r,r]$. Then, the first term can be bounded by

where the last inequality follows from the fact that if a random variable is in the range $[m,M]$, then its variance is bounded by $\frac{(M-m)^{2}}{4}$.

By letting $\epsilon=\epsilon^{\prime}/T$,

Therefore, the number of tests $T$ we need in order to get an $(\epsilon/(2\sqrt{N}),\delta/(N(N-1)))$-approximation to the difference of two Shapley values for a single pair of data points is

And the statement above holds true for any pair of $i$ and $j$. By Lemma 2, we approximate the Shapley value up to $(\epsilon,\delta)$ with $(\epsilon/(2\sqrt{N}),\delta/(N(N-1)))$ approximations to all $N(N-1)/2$ pairs of data points.

It can be shown that $q_{\text{tot}}=1-\frac{2}{Z}$ and so

The Taylor expansion of $h(u)$ centered at is $\frac{u^{2}}{2}+\cdots$. Thus, we have

Since $Z\leq 2(\log(N-1)+1)$, we have $\mathcal{O}(NZ\log N)=\mathcal{O}(N(\log N)^{2})$. ∎

Appendix D Proof of Theorem 4

Suppose that $U(\cdot)$ is monotone. There exists some constant $C^{\prime}$ such that if $M\geq C^{\prime}(K\log(N/(2K))+\log(2/\delta))$ and $T\geq\frac{2r^{2}}{\epsilon^{2}}\log\frac{4M}{\delta}$, except for an event of probability no more than $\delta$, the output of Algorithm 2 obeys

for some constants $C_{1,K}$ and $C_{2,K}$.

Let $s=\Delta s+\bar{s}$. Thus, $P[\|A(\bar{s}+\Delta s)-\bar{y}\|_{2}\leq\epsilon]$ holds with probability at least $\delta/2$ provided

By the random matrix theory, the restricted isometry constant of $A$ satisfies $\delta_{2K}\leq C_{\delta}=0.465$ with probability at least $1-\delta/2$ if

Applying the Theorem 2.7 in , we obtain that the output of Algorithm 2 satisfies

with probability at least $1-\delta$ provided that (31) holds and $M\geq C^{\prime}(K\log(N/(2K))+\log(2/\delta))$ for some constant $C^{\prime}$. ∎

Appendix E Proof of Theorem 5

For the proof of Theorem 5 we need the following definition of a stable utility function.

A utility function $U(\cdot)$ is called $\lambda$-stable if

Then, Shapley values calculated from $\lambda$-stable utility functions have the following property.

If $U(\cdot)$ is $\lambda$-stable, then for all $i,j\in I$ and $i\neq j$

Recall the bound on the harmonic sequences

For a learning algorithm $A(\cdot)$ with uniform stability $\beta=\frac{C_{\text{stab}}}{|S|}$, where $|S|$ is the size of the training set and $C_{\text{stab}}$ is some constant. Let the utility of $D$ be $U(D)=M-L_{\text{test}}(A(D),D_{\text{test}})$, where $L_{\text{test}}(A(D),D_{\text{test}})=\frac{1}{N}\sum_{i=1}^{N}l(A(D),z_{\text{test},i})$ and $0\leq l(\cdot,\cdot)\leq M$. Then, $s_{i}-s_{j}\leq 2C_{\text{stab}}\frac{1+\log(N-1)}{N-1}$ and the Shapley difference vanishes as $N\rightarrow\infty$.

Combining the above inequality with Proposition 7 proves the theorem. ∎

Appendix F Proof of Theorem 6

Consider the value attribution scheme that assign the value $\hat{s}(U,i)=C_{U}[U(S\cup\{i\})-U(S)]$ to user $i$ where $|S|=N-1$ and $C_{U}$ is a constant such that $\sum_{i=1}^{N}\hat{s}(U,i)=U(I)$. Consider two utility functions $U(\cdot)$ and $V(\cdot)$. Then, $\hat{s}(U+V,i)\neq\hat{s}(U,i)+\hat{s}(V,i)$ unless $V(I)[\sum_{i=1}^{N}U(S\cup\{i\})-U(S)]=U(I)[\sum_{i=1}^{N}V(S\cup\{i\})-V(S)]$.

Consider two utility functions $U(\cdot)$ and $V(\cdot)$. The values attributed to user $i$ under these two utility functions are given by

where $C_{U}$ and $C_{V}$ are constants such that $\sum_{i=1}^{N}\hat{s}(U,i)=U(I)$ and $\sum_{i=1}^{N}\hat{s}(V,i)=V(I)$. Now, we consider the value under the utility function $W(S)=U(S)+V(S)$:

Then, $\hat{s}(U+V,i)=\hat{s}(U,i)+\hat{s}(V,i)$ if and only if $C_{U}=C_{V}=C_{W}$, which is equivalent to

Appendix G Theoretical Results on the Baseline Permutation Sampling

Let $\pi_{t}$ be a random permutation of $D=\{z_{i}\}_{i=1}^{N}$ and each permutation has a probability of $\frac{1}{N!}$. Let $\phi_{i}^{t}=U(P_{i}^{\pi_{t}}\cup\{i\})-U(P_{i}^{\pi_{t}})$, we consider the following estimator of $s_{i}$:

Given the range of the utility function $r$, an error bound $\epsilon$, and a confidence $1-\delta$, the sample size required such that

The first inequality follows from the union bound and the second one is due to Hoeffding’s inequality. Since $\|\hat{s}-s\|_{2}\leq\sqrt{N}\|\hat{s}-s\|_{\infty}$, we have

Setting $2N\exp(-\frac{T\epsilon^{2}}{2Nr^{2}})\leq\delta$ yields

The permutation sampling-based method used as baseline in the experimental part of this work was adapted from Maleki et al. and is presented in Algorithm 3.