Data Poisoning Attacks on Neighborhood-based Recommender Systems

Introduction

With the booming of information, it is a big challenge for users to find valuable items that satisfy their preference. To address this challenge, most of the existing recommender systems studies utilize users’ historical information, social network information and items’ content to model users’ preference on items. Among these studies, one of the most widely used technique in recommender systems is collaborative filtering (CF), which focuses on the users’ historical information. Depending on the methods utilized to learn the correlation between users’ historical information, CF recommender systems can be divided into four categories: matrix-factorization-based methods , graph-based methods , association-rule-based methods and neighbourhood-based methods . Numerous commercial companies (e.g., Netflix https://www.netflix.com, YouTube https://www.youtube.com, eBay https://www.ebay.com and Taobao https://www.taobao.com) have already applied CF recommender systems to their products, such as web and app, to alleviate the information overload, improve users’ experience, and bring them tremendous economic benefits.

Recommender systems have the advantages of matching users personal interest, however, the overall recommendation result is less robust. This problem is a non-ignorable downside. Several previous studies have already pointed out that CF methods are vulnerable to data poisoning attacks inducing the recommender system to a security risk. The collaborative recommendation system will personalize the recommendation to the user based on the historical data of similar users, when it works normally. However, this may not be the case. For example, an attacker with bad intentions (or a profit motive) quietly injects malicious data with elaborate construction into recommender systems so that he can control the recommendation result as he desires and destory the personal recommendation list.

Early data poisoning studies have applied handcraft rules to generate fake users, which usually achieve suboptimal attack performance even if in the case of possessing the access to input data and knowing the recommendation algorithm. For example, random attack randomly chooses filler-items for each fake user and assigns ratings to the filler-items from the normal distribution of all the rating data. Average attack selects filler-items just the same as the random attack. The difference is that the rating score assigned to the filler-item is based on the normal distribution of the rating data of the filler-item. These methods can’t test the robust of recommender systems completely. In recent years, several studies about optimal data poisoning attack against a certain type of recommender systems have been proposed. However, how to design an optimal data poisoning attack based on neighborhood-based recommender systems remains a challenging problem.

Despite some advanced recommender systems have been proposed, neighborhood-based collaborative filtering remains one of the most common and effective recommender systems and can be deployed by businesses companies, e.g., Amazons . In addition, the recently proposed research points out that neighborhood-based collaborative filtering outperforms than some advanced deep the collaborative filtering models, (e.g., NCF ). Thus, proposing an optimal data poisoning attack to test its robustness is an urgent and important issue. With this motivation, we propose an optimal data poisoning attack based on neighborhood-based recommender systems, namely UNAttack. Neighborhood-based recommender systems are mainly based on the similarity between users or items. The user’s preference on an item is learned through the user’s or the item’s $K$ nearest neighbours. In our method, the characteristics of the users’ $K$ nearest neighbours will be utilized to promote target items (push attack ). For example, in recommender systems, target items will be recommended to as many normal users as possible after we inject a few fake users into the recommender systems. In data poisoning attack, the items which fake users rate is named as filler-items. Due to the limited resources and to keep our attack hard to be detected, some constraints are defined as follows: 1) in each attack, attackers can only inject $j$ fake users and each fake user can only rate $z$ filler-items at most, 2) the rating scores user give to the items must be integer. We regard data poisoning attack as an optimization problem with the aforementioned constraint in order to get the filler-items and the rating scores of each fake user. To demonstrate its effectiveness, extensive experiments are conducted on three real-world datasets. The experimental results show that our UNAttack performs far better than conventional attack methods and other state-of-the-art attack methods. In addition, we observe that the fake users we generate can attack the state-of-the-art CF recommender systems, such as NCF and BPRMF. It means that UNAttack is still effective even if the attackers do not know the details of the recommended systems, namely the black-box attack .

In summary, the main contributions of this paper are as follows

We propose a general and mathematical framework for optimal data poisoning attack against CF recommender systems.

We encode data poisoning attack against neighborhood-based methods into our framework as an optimization problem. Then, we present the solution to this problem so as to generate more effective fake users.

To demonstrate how our method works, we conduct a series of experiments on three public datasets. Both quantitative and qualitative analysis justify that our attack methods can achieve good attack performance, not only in neighborhood-based recommender systems, but also in other collaborative filtering algorithms such as BPRMF and NCF.

Related work

The existing research about data poisoning attack has attracted the attention of many people. propose a data poisoning attack against autoregressive models using the optimal methods. introduce data poisoning strategies to test knowledge graph embedding robustness. design data poisoning attack algorithms targeting objective and output. Existing research about data poisoning attack (also called as shilling attacks) against the recommender system studied from more than 10 years ago. Data poisoning attacks aim to make target items be recommended to as more users as possible. Specifically, when performing data poisoning attacks, attackers firstly registers a few fake accounts in the service. Then they control each fake account to assign well-esigned rating scores to a carefully chosen subset of items. proposed two kinds of data poisoning attack methods: random attack and average attack. Both these attack methods randomly chose filler-items for each fake user. In , the authors proposed the bandwagon attack, which associated the filler-items with popularity. Different from the random attack, bandwagon attack selected a part of the popular items together with some randomly selected items as filler-items.

In recent years, a few data poisoning attack methods, which uses the optimization technique to get the filler-items and the assigned ratings of fake users, have been proposed. A study proposed a data poisoning attack against matrix-factorization-based recommender systems making the root-mean-squared-error become larger than its original value. Fake co-visitationis a injection attack against association-rule-based recommender systems by constructing attack as a constrained linear optimization problem. proposed an optimized data poisoning attack for graph-based recommender systems.

Besides, the paper proposed a new type of attack method named profile pollution attack against recommender system and other web services, in which attackers aimed to pollute users’ profile, such as browsing and clicking session, via cross-site request forgery . However, they only can perform the attack on a small scale. In recently, leakage of private information has attracted more and more people’s attention and especially the edge computing and fog computing are used, which can collect more sensitive information than the remote cloud. Privacy attack mainly contains the item inference attacks and attribute inference attacks. The work proposed privacy attacks to infer the items that a target user has rated via utilizing the publicly available reviews of users. The authors have already proved that these methods can be performed on several popular webs such as Amazon https://www.amazon.com/, LibraryThing https://www.librarything.com/ and Lastfm https://www.last.fm/zh/. Attribute inference attacks are the technique aiming to acquire the users’ privacy attributes with the users’ interaction information. Generally, users’ privacy attributes (e.g., gender, political view, interests, and location) will be reflected by the users’ interaction information (e.g., buy, rate and click items). After collecting a few users’ interaction information and their privacy attributes, the attacker trained a classifier model with interaction information as input and privacy attributes as the prediction. This classifier model will be applied to predict the attributes of users who are unwilling to make their privacy attributes public. These methods have been demonstrated to be feasible by several studies.

Method

We first introduce the formulation of neighborhood-based recommender systems. Then, we define data poisoning attacks as an optimization problem and approximate this optimization problem. Lastly, the solution to this problem will be introduced to generate fake users.

Neighborhood-based recommender systems are the earliest CF recommender systems method including user-based CF and item-based CF. Both of them have been widely applied in various web services. In the user-based CF, when the model recommends items to the user, it selects the top-K nearest-neighbors of the user via calculating the similarity between the user and others, and then predicts the preference of the user on items based on the preference of top-K nearest-neighbours. For the item-based CF, the model considers the similarity based on the items. In this paper, we mainly focus on using our methods against user-based CF by injecting fake users into the recommender systems. We assume there is a set of $m$ users, $U=\{u_{1},u_{2},...,u_{m}\}$, a set of $n$ items, $I=\{i_{1},i_{1},.....,i_{n}\}$ and a sparse matrix $R_{m\times n}$. $X_{u}$ represents the n-dimensional item rating vector for user $u$. Each element $X_{ui}$ in the vector $X_{u}$ denotes the explicit preference of user $u$ on item $i$. The predicted preference of user $u$ on item $i$ based on user-based CF can be calculated as follows:

where $S(u,K)$ is the set of top-K nearest neighbours of user $u$, $U_{i}^{+}$ denotes a set of users who rate item $i$. $s_{uv}$ is the similarity between user $u$ and $v$.

2 Problem Definition

According to the intent, data poisoning attack can be divided into two categories: target attacks and non-target attacks . In non-target attacks, the intent could be to deteriorate the recommendation quality of all users. proposed a data poisoning attack against matrix-factorization-based recommender systems making the root-mean-squared-error become larger than its original value. Our work will focus on the target attack. We consider the case where an attacker generates a few of realistic-looking fake users with the desires of promoting a group of items from himself company into target users’ top lists or removing a group of items from competitor company from target users’ top lists. Figure 1 illustrates how data poisoning attacks work against the recommender systems. Thus, the performance of our data poisoning attack method can be measured via $HitRatio$, and the $HitRatio$ of item $i$ is defined as follows:

Let $L_{u}$ denote the set of top-N items list that user-based CF recommend to user $u$. $U_{i}^{-}$ denotes a set of users who do not rate item $i$, namely target users, and $|U_{i}^{-}|$ is the number of target users.

Considering the attacker’s intent in the paper is to promote target items into as many target users’ top-N lists as possible. Demoting target items out of target users’ top-N lists is the special case of promoting. For concreteness, attackers can promote other items so as to push target items out of target users’ top-N lists. Therefore, the key challenge for us is how to generate an n-dimensional item rating vector for each fake user who can maximize the $HitRatio$ of the target item. In reality, attackers often suffer from some constraints in poisoning the data such as the cost of generating fake users and detection avoidance. As such, the number of fake users, the number of filler-items of each fake user and the range of rating scores will be restricted. Let $X_{f}$ be the n-dimensional item rating vector for fake user $f$, in which $X_{fi}$ is the score that fake user $f$ gives to the item $i$. Under the consideration above, we describe data poisoning attack task as the following optimization problem:

where $\{f_{1},f_{2},...,f_{j}\}$ denotes the set of $j$ fake users, $|X_{f}|^{{}^{\prime}}$ denotes the number of filler-items for each fake user ($|X_{f}|^{{}^{\prime}}\leq z$). The rating scores of items are in the interval $\{0,1,...,r_{max}\}$. It is worth noting that this framework in Eq.3 is suitable for target data poisoning attack against any recommender systems. In this paper, we focus on encoding the neighborhood-based recommendation into this framework.

3 Problem Approximation

It is clearly intractable to find the optimal solution to the optimized problem in Eq.3 due to the following two reasons. Firstly, the item rating vector $X_{f}(f\in\{f_{1},f_{2},...,f_{j}\})$ of the fake user has the implicit and complicated dependency with $HitRatio$. Secondly, the rating score $X_{fi}$ is a discrete value, which means that it can not be optimized via gradient-based methods such as gradient descent.

To overcome the difficulties mentioned above, several approximation techniques have been proposed. Firstly, we will optimize the fake users one by one based on the new data (including the normal users and the fake users generated before). Secondly, we borrow the strategy from the ranking problem to construct pairwise loss function, in which after training item with a higher value means the user is more likely to buy. Through this way, we obtain the filler-items of the fake user by selecting top-$z$ items with the highest scores after training. Thirdly, the filler-items will be assigned integer rating scores to mimic normal user behaviours.

If attackers want to maximize the $HitRatio$, he should make the target item $t$ appear in the top-N recommended list $L_{u}$ as more as possible for each user $u\in U_{t}^{-}$. To achieve this purpose, the objective function must be converted into the loss function that is convenient to be optimized. For user $u$, if $t$ appears in the recommended list $L_{u}$, then users’ rating scores to $t$ must be higher than item $i$ (${i\in L_{u}}$), which can be expressed as $p_{ut}>p_{ui}$. However, according to Eq.1, we can know that the user-based CF evaluates the user’s preference through his top-K nearest neighbors’ preference. Therefore, if the attacker wants to impose influence on $p_{ut}$ and $p_{ui}$, he must make the fake user $f$ be in the top-K nearest neighbours of user $u$, which can be expressed as $s_{uf}>s_{uv}$.

More specifically, to approximate the objective function $HitRatio$, the loss function must satisfy a condition when it becomes smaller, $s_{uf}$ and $p_{ut}$ become higher than $s_{uv}$ and $p_{ui}$, respectively. We borrow the idea from BPR pairwise loss and formulate the user $u$’s loss function as follows:

where $\sigma(x)=\frac{1}{1+e^{-x}}$ is the sigmoid function. $\lambda\in\left[0,1\right]$ is the trade-off parameters. $\lambda$ controls the relative importance of $loss_{1}$ and $loss_{2}$. By minimizing $loss_{u}$, we can promote the fake user into the top-K nearest neighbours $S(u,K)$ and promote the target item into the user’s top-N items $L_{u}$ list.

For all the normal users in $U_{t}^{-}$, the loss function is the sum of their single loss function:

With the loss function, we can consider the optimization problem as follows:

4 Fake Users Generation

We elaborate the following specific steps on how to solve the optimized problem in Eq.6 and generate fake users. 1) choosing the optimal filler-items for fake users. In this step, the stochastic gradient descent will be chosen to solve the problem in Eq.6 due to its ease in deriving the update strategy. Discrete value is hard to be optimized by gradient descent, so $X_{fi}$ is relaxed as a continuous value in training phrase. In this section,we choose the Cosine Similarity to calculate the similarity between users as the illustrative example. In iteration $t$, $X_{f}(t)$ will be updated as follows:

where $Project(x)$ is the project function that cuts each $X_{fi}$ into the range $[0,1,..r_{max}]$. The gradient of $F(X_{f})$ (with respect to $X_{f}$) are as follows:

where $Q=s_{uv}-s_{uf}$, $P=p_{ui}-p_{ut}$ and $W=(S(u,k)\cap U_{i}^{+})$

Before calculating the $\frac{\partial F(X_{f})}{\partial X_{f}}$, we must solve the $\frac{\partial s_{uf}}{\partial X_{f}}$. $s_{uf}$ denotes the similarity between user $u$ and $f$. In Eq.9 and 10, the Cosine Similarity will be used and the gradient $\frac{\partial s_{uf}}{\partial X_{f}}$ can be computed as follows:

2) assigning integer rating scores to filler-items. After finishing the training of $X_{f}$, we use the following several strategies to convert the continuous value into integer rating score: first of all, let $X_{ft}=r_{max}$. In other words, we give the target items the maximum ratings. Secondly, inspired by the ranking problem, all items will be ranked according to $X_{fi}$, and top-$z$ items with the highest values will be chosen as the filler-items. Because we consider that after training the items with higher value means the fake user is more likely to buy them. Finally, the rating score assigned to each filler-item is drawn from a normal distribution of the normal users’ rating data of this item. After that, we can keep our fake user mimic the normal user’s behaviours as well as avoid potential detection. Algorithm 1 shows the detailed procedure of our solution. The time cost of fake users generation is $j\times z$ , where $j$ is the number of fake users and $z$ is the number of filler items.

[!htbp] UNAttack Input: Matrix $R_{m\times n}$ Parameter: $\lambda,K,N,z,j$ Output: $j$ fake users {algorithmic} \Foreach fake user f \StateSolve the problem in Eq.6 with current rating matrix $R$ to get $X_{f}$ \StateLet $X_{ft}=r_{max}$ \StateSelect $z$ items with highest value in $X_{fi}$ as filler items. \StateFor each filler-items j, $X_{fj}\sim\mathcal{N}(\mu_{j},\sigma_{j}^{2})$ \State$R_{m\times n}=R_{m\times n}\cup X_{f}$ \EndFor

Experiments

In this section, we empirically evaluate the performance of our proposed method, UNAttack, with all the baselines on three real-world datasets. Then, we explore the impact of hyper-parameters on the UNAttack. Finally, the performance of transferability on the UNAttack will be introduced.

Dataset description. We conduct extensive experiments on three real-world datasets: FilmTrust , Movielens https://grouplens.org/datasets/movielens/, and Amazons Video http://jmcauley.ucsd.edu/data/amazon/. Table 1 summarizes their detailed statistics. For each dataset, 80% of interactions information between users and items will be split into the training set and the remaining 20% as the test set. Then, we randomly select aside 10% of the training data as validation set for tuning hyper-parameters of the recommendation model. The word average in the Table 1 means the average rating number of each users. The definition of sparsity in Table 1 is as follows:

Compared methods. We compared our method with several data poisoning attack methods using the $HitRatio@N$ ($HR@N$).

None. This represents the situation that recommendation is normal and do not suffer any attack.

Random attack . This method randomly chooses filler-items for each fake user, and assigns ratings to the filler-items from the normal distribution of all the rating data.

Average attack . It selects filler-items just the same as the random attack. However, the rating score assigned to the filler-item is based on the normal distribution of the rating data of the filler-item.

Bandwagon attack . Bandwagon attack associates the filler-items with popularity. In our experiments, for each fake user, we select $z\times 20\%$ items which get high average scores and randomly choose $z\times 80\%$ as filler-items, and assign rating to the filler-items from the normal distribution of the whole rating data.

Co-visitation attack . Fake Co-visitation injection attack is designed for the association-rule-based recommender systems. Our experiment only considers the attack with injecting fake users. Therefore, we use the method in Fake Co-visitation injection attack to choose the filler-items for fake users. Moreover, if the item $i$ is frequently rated with the target item $t$ at the same time, it has a high probability to be selected as filler-items. The way we assign the rating score for filler-items is the same as the average attack.

Parameter Setting. The details of our parameter are as follows. The best $\lambda$ values will be used for each dataset. $z$ will be set to the average rating number of normal users. Without specification, in FilmTrust, $\lambda=0.6$, $K=30$, $N=20$, $z=23$. In Movielens, $\lambda$ = 0.5, $K=30$, $N=20$, $z=106$. In Amazons, $\lambda=0.3$, $K=30$, $N=20$, $z=10$. Besides, the number of fake users (attack size) is set from $0.5\%$ to $2\%$ of the number of normal users. By default, we assume that the similarity between users is calculated by Cosine Similarity. To demonstrate the performance of our method, a part of items are first selected randomly as random items. Secondly, we consider the items whose rating data is less than 5 and which are almost not recommended to normal users at all (None performance less than 0.001) as cold-start target items. Moreover, the items with high possibility are recommended to normal users before the attack will be regarded (None performance higher than 0.1) as warm-start items. We select 10 target items for each type of target items (eg. random items, cold-start items and warm-start items) to calculate the average $HR@N$. All of the methods run on an Intel Core i7 with 2.2 GHz , 2080 Ti GPU,128GB RAM, 64 bit system.

2 Experiment Results

Overall performance. Table 2 summarizes the overall performance of baselines and our attack method. From the table, we can obtain the following observations:

Our attack method can promote the $HitRatio$ in different types of target items effectively by injecting a few fake users. For example, when injecting $1\%$ fake users, in FilmTrust datasets, our attack method increases the $HitRatio$ 15.2 times as much as None in random target items. In Amazons dataset, the $HitRatio$ of our attack method is 35.9 times higher than None in random target items. For the warm-start items, our method improves $HitRatio$ by $20.1\%$ than None in the Filmtrust dataset.

our UNAttack achieves the highest threat in the same attack size comparing with other methods. For example, in FilmTrust datasets, when injecting $1\%$ fake users, the $HitRatio$ of our attack method attacks random target items is 10 times higher than Random attack, Average attack, and Bandwagon attack, and 8.6 times than Co-visitation attack. If we attack warm-start items, our method can increase the $HitRatio$ by 20.1%, and the Co-visitation method can only increase the $HitRatio$ by 4.4%.

The improvement of $HitRatio$ when attacking the cold-start target items is more obvious than random target items. For example, when injecting $1\%$ fake users, in FilmTrust datasets, the $HitRatio$ of our attack method is 15.2 times and 827 times higher than None in random target items and cold-start target items, respectively. In Amazons dataset, our attack method increases the $HitRatio$ 35.9 times and 287 times as compared with None in random target items and cold-start target items, respectively. In addition, the increment of $HitRatio$ in warm-start target items is less effective than in random target items. For instance, the $HitRatio$ of UNAttack is 1.3 and 15.2 times higher than None in warm-start target items and random target items, respectively.

Data poisoning attack can achieve better performance in the sparse dataset than the dense dataset. For example, when injecting $2\%$ fake users and attacking the random target items, in Amazons dataset, the $HitRatio$ of our attack method achieves the highest performance (0.1794), and in the dense dataset, Movielens, the $HitRatio$ is the lowest (0.0911).

We can find that the time cost of fake users generation with UNAttack is small. In the Movielens dataset, attackers only spend 8m56s if they generate the number of fake users equivalent to 2% of the number of normal users. In the Filmtrust and Amazons dataset, they spend 12m8s and 28m27s respectively.

Impact of the type of similarity measurement method. User-based CF can take different types of similarity measurement methods to calculate the similarity between users. Figure 2 shows the result of all attack methods against the User-based CF with different similarity measurements, where the attack size = $2\%$ and we attack random target items. The similarity measurement methods includes Cosine Similarity, Euclidean Distance-based Similarity and Pearson Similarity. We can observe an interesting insight that the performance of all attack methods on Cosine Similarity and Pearson Similarity is better than that on Euclidean Distance-based Similarity. The reason is that Euclidean Distance-based Similarity focuses on the distance of two vectors in space, while Cosine and Pearson Similarity focus on the direction of the two vectors in space. The experimental results demonstrate that fake users are more similar in direction with real users and it is hard to reduce the distance between them. In other words, it is difficult for attack method to promote $HitRatio$ if user-based CF uses the Euclidean Distance-based similarity. Therefore, the user-based CF with Euclidean Distance-based similarity has better robustness.

Impact of the number of filler-items. Figure 3 (a), (b) and (c) show the impact of the number of filler-items $z$ on our attack method, where the attack size = $2\%$ and we attack random target items. As we can see, in FilmtTrust and Amazons datasets when $z$ is close to the average rating number of normal users (23 in FilmTrust, 8 in Amazons), the attack can achieve better performance. If $z$ is larger than the average rating number, it is more difficult for fake users to imitate the behaviour of real users, which decreases the performance of similarity calculation between fake users and real users. In the dense dataset Movielens, it can be clearly seen that our attack method achieves more competitive and stable performances if $z$ is set higher than 50.

Impact of the number of recommended items. We fix the attack size = $2\%$, attack random target items and set the number of recommended items $N$ to $\left\{1,5,10,15,20,25,30,35,40,45,50\right\}$. In Figure 3 (d), the smaller $N$ represents the harder the neighborhood-based recommender system is to be attacked. From the figure, we can find that when $N$ is small, our attack method can also achieve good performance. For instance, in Amazons dataset, when $N=5$, our method can still obtain 0.089 $HitRatio$ (close to the $HitRatio$ of warm-start items), which means our method can effectively promote the target items ranked in top-5 recommendation list.

Impact of the number of users’ nearest neighbours. In Figure 3 (e), the number of nearest neighbours of users $K$ is set to $\{1,5,10,15,20,25,30,35,40,45,50\}$, where the attack size = $2\%$ and we attack random target items. In FilmtTrust and Amazons datasets, we can observe that the $HitRatio$ increases as user-based CF considers more nearest neighbours of users. For instance, when the fake user is the 25th nearest neighbour of user $u$, if user $u$ only considers the top-20 nearest neighbours, he can avoid the attack; if the user considers top-30 nearest neighbours, he will be effected. However, in the Movielens, we notice that the $HitRatio$ decreases after $K$ is set higher than 30. The possible reason is that Movielens is a dense dataset. Setting higher $K$ for user $u$ may contain more normal users who have rated the target items, thus pushing the fake users out of top-K nearest neighbours.

Impact of the weight of different loss functions. Figure 3 (f) shows the performance of our attack method depending on the value of $\lambda$, where the attack size = $2\%$ and we attack random target items. $\lambda$ controls the relative importance of $loss_{1}$ and $loss_{2}$. Setting $\lambda$ to 1 or 0 means we only consider the $loss_{1}$ or $loss_{2}$ respectively in the overall loss function. It can be noticed that the $loss_{1}$ imposes higher impact on the performance of our method. Besides, we can see that our attack method in different datasets achieves the best performance with different $\lambda$ values (e.g. $\lambda$ = 0.6 for FilmTrust, $\lambda$ = 0.5 for Movielens, $\lambda$ = 0.3 for Amazons).

3 Transferability

Previous sections demonstrate the performance of our method in the white-box setting. In addition, adversarial samples usually have another important property, namely transferability . Adversarial samples that are generated based on a certain model can also successfully fool other models with the same task. This case of attacks is named black-box attacks. Nowadays, web and apps show and collect information through a terminal machine. Then, a model deployed on cloud servers will handle this information. In this case, an attacker is able to acquire data through crawlers, but the structure and parameters of the model are hard to acquire because the model is deployed on cloud servers. Therefore, we consider that data poisoning attack in the black-box setting is generally more realistic and common than in the white-box setting.

To explore the performance of our attack method in the black-box setting, we inject fake users generated by UNAttack to against other CF recommender systems and explore the robustness of them. Two state-of-the-art CF recommender systems, i.e., BPRMF and NCF, are chosen as illustrative examples. Firstly, the fake user will be generated based on the neighborhood-based recommender system. In this experiment, K and N are set to 30 and 20 respectively, and the number of filler-items is set to the average rating number of normal users. Note that in this experiment, the Cosine Similarity is chosen in training the fake user’s phrase. In fact, other similarity measurement methods also can be applied such as Euclidean Distance-based Similarity and Pearson Similarity. Secondly, for the model BPRMF and NCF, we fix the embedding size to 20 and N to 20, and the learning rate is set to $\left\{0.001,0.005,0.01\right\}$. The original $HitRatio$ of target items will be calculated after the training based on the original data $R_{m\times n}$. We define the $R_{m\times n}$ added with fake users as $R_{m\times n}^{{}^{\prime}}$ and train the model based on the $R_{m\times n}^{{}^{\prime}}$ to obtain the new $HitRatio$.

From the experimental results shown in Table 3, we can obtain the following observations:

As can be observed, all data poisoning attack methods can be transferred to attack both BPRMF and NCF, and if attackers inject enough fake users into recommender systems, they can improve the $HitRatio$ upon the case of None. For example, in the FilmTrust dataset, when attacking BPRMF and NCF by injecting $2\%$ fake users, the HitRatio of UNAttack achieves 0.2565 and 0.2223, which is 213 times and 199 times higher than the None. This phenomenon can be explained by the fact that almost all the CF recommender systems will learn the correlation between users’ historical information, which will be utilized to generate a personalized ranking list for users. The correlation learned by different CF recommender systems may be similar. Therefore, the fake users generated by UNAttack also can be transferred to attack BPRMF and NCF effectively.

In the dense dataset, it is harder for the attacker to achieve the improvement of $HitRatio$. For example, when the attack size is set to $0.5\%$, in the Movielens dataset, UNAttack only improves $HitRatio$ 4.7 times over the None lower than 15 times in the white-box setting. This phenomenon is similar in attacking the neighborhood-based CF.

Among all methods, UNAttack largely outperforms the traditional attack methods and the state-of-the-art methods Co-visitation whether attacking the BPRMF or NCF on the three datasets. For instance, in the Amazons dataset, when attacking NCF by injecting $2\%$ fake users, the best HitRatio of Co-visitation reaches 0.1656 and our UNAttack can obtain 0.3260. This roughly 1 times relative improvement demonstrates the advantage of combining optimal technique with data poisoning.

NCF has better robustness against the data poisoning method than BPRMF. For example, in the Amazons dataset, when attacking NCF by injecting $2\%$ fake users, UNAttack achieves 59.4 times improvement compared with the None, whereas it can achieve 122.8 times improvement in attacking BPRMF. The reason lies in that NCF unifies the linearity of MF and the non-linearity of multi-layer perceptron (MLP).

Conclusion

This paper proposes a novel target attack framework against recommmender systems. Data poisoning attack against neighborhood-based methods is encoded into our framework from the perspective of optimization. The generated fake users are injected into original data when attacking the neighborhood-based recommendation so that we can improve the $HitRatio$ of the target items. Further, a cold-start item can be changed to a warm-start item. In addition, attack on BPRMF and NCF are conducted to demonstrate that our method can work in the black-box setting. The results of the experiments on three real-world datasets indicate the effectiveness and transferability of our method via comparing with the state-of-the-art methods when targeting different types of target items.

In the future, how to achieve the good performance of our method will be studied, even if we only know part of the dataset. With the rapid development of deep learning based recommender systems, we would like to design data poisoning attacks against the recommender systems based on deep learning. Moreover, we are interested in building an effective defence model to detect the data poisoning attack.

Acknowledgments

This work supported by the National Key Research and Development Program (2017YFB0202201), the National Natural Science Foundation of China (61702568,U1711267), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2017ZT07X355) and the Fundamental Research Funds for the Central Universities under Grant (17lgpy117).

References