A federated graph neural network framework for privacy-preserving personalization

Graph neural network (GNN) is effective in modeling high-order interactions and has been widely used in various personalized applications such as recommendation. However, mainstream personalization methods rely on centralized GNN learning on global graphs, which have considerable privacy risks due to the privacy-sensitive nature of user data. Here, we present a federated GNN framework named FedPerGNN for both effective and privacy-preserving personalization. Through a privacy-preserving model update method, we can collaboratively train GNN models based on decentralized graphs inferred from local data. To further exploit graph information beyond local interactions, we introduce a privacy-preserving graph expansion protocol to incorporate high-order information under privacy protection. Experimental results on six datasets for personalization in different scenarios show that FedPerGNN achieves 4.0% ~ 9.6% lower errors than the state-of-the-art federated personalization methods under good privacy protection. FedPerGNN provides a promising direction to mining decentralized graph data in a privacy-preserving manner for responsible and intelligent personalization.


INTRODUCTION
Graph neural network (GNN) is widely used by many personalized recommendation methods in recent years [12,31,36], since it can capture high-order interactions between users and items on the user-item graph to enhance the user and item representations [2,37,38]. For example, Berg et al. [2] proposed to use graph convolutional autoencoders to learn user and item representations from the user-item bipartite graph. Wang et al. [31] proposed to use a three-hop graph attention network to capture the high-order interactions between users and items. These existing GNN-based recommendation methods usually necessitate centralized storage of the entire user-item graph to learn GNN models and the representations of users and items, which means that the user-item interaction data needs to be centrally stored, as shown in Fig. 1(a). However, user-item interaction data is highly privacy-sensitive, and its centralized storage can lead to the privacy concerns of users and the risk of data leakage [24]. Moreover, under the pressure of strict data protection regulations such as GDPR 1 , online platforms may not be able to centrally store user-item interaction data to learn GNN models for recommendation in the future.
An intuitive way to tackle the privacy issue of user-item interaction data is to locally store the raw data on user devices and learn local GNN models based on it, as shown in Fig. 1(b). However, in this scenario, it is very difficult to train an accurate GNN model for recommendation due to following reasons. First, for most users the volume of interaction data on their devices is too small to locally train accurate GNN models. Thus, a unified framework that coordinates a large number of user clients to collectively learn an accurate global GNN model from decentralized user data is required. Second, the local GNN model trained on local user data may convey private information, and it is challenging to protect user privacy when synthesizing the global GNN model from the local ones. Third, the local user data only contains first-order user-item interactions, and users' interaction items cannot be directly exchanged due to privacy restrictions. Thus, it is very challenging to exploit the high-order user-item interactions without privacy leakage.
In this paper, we propose a federated framework named FedGNN for privacy-preserving GNN-based recommendation, which can effectively exploit high-order user-item interaction information by collectively training GNN models for recommendation in a privacypreserving way. Since user interaction data is highly decentralized, there is no global user-item graph. Thus, in our method each user device locally learns a GNN model and the embeddings of user and items based on the user-item graph inferred from the local useritem interaction data on this device. The user devices compute the gradients of models and user/item embeddings and upload them to a central server, which aggregates the gradients from a number of users and distributes them to user devices for local updates. However, both the items with non-zero gradients and the GNN model gradients contain private information. Thus, we propose a privacy-preserving model update method to protect user-item interaction data without locally memorizing the full item set during model training. More specifically, we apply local differential privacy (LDP) techniques to the local gradients computed by user clients to protect user privacy. In addition, in order to protect the real items that user interacted with when uploading the gradients of item embeddings, we generate random embedding gradients of a certain number of randomly sampled pseudo interacted items. Besides, to exploit high-order information of the user-item graph without leaking user privacy, we propose a privacy-preserving user-item graph expansion method that aims to find the neighbors of users with co-interacted items and exchange their embeddings to expand their local user-item graph. In this way, high-order information of the user-item graph can be exploited by the GNN model to enhance user and item representations, and the private user-item interaction data do not leak. We conduct massive experiments on six widely used benchmark datasets for recommendation, and the results show that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile effectively protect user privacy.
The major contributions of this paper are summarized as follows: • We propose a novel federated framework for privacy-preserving GNN-based recommendation that can exploit highly decentralized user data to collectively train GNN models. • We propose to protect model gradients in model training with local differential privacy and propose a pseudo interacted item sampling technique to protect the items that users have interactions with. • We propose a privacy-preserving user-item graph expansion method to exploit high-order user-item interactions from decentralized user data.
• Extensive experiments and analysis on six benchmark datasets show that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile protect user privacy.

RELATED WORK 2.1 GNN for Recommendation
Graph neural networks are preferred by many recommendation methods to model high-order relations between users and items [2, 5, 7, 10, 27, 30-34, 36, 37]. For example, Berg et al. [2] proposed a graph convolutional matrix completion (GC-MC) approach. GC-MC uses a graph convolutional encoder to learn user and item representations from the user-item bipartite graph, and then predicts unknown ratings via a bilinear decoder. Ying et al. [36] proposed a graph convolutional neural network based method for recommendation named PinSage. It learns item representations from an item-item graph via 2-hop graph convolutions, and uses these representations in downstream recommendation tasks. Wang et al. [31] proposed a neural graph collaborative filtering (NGCF) approach that uses 3-hop graph neural networks to learn user and item embeddings from the user-item bipartite graph. Besides the user-item graph, several GNN-based recommendation methods also incorporate other kinds of graphs into recommendation, such as useritem-entity graph [29] and user-user-item graph [5]. For example, Wang et al. [30] proposed a knowledge-graph enhanced recommendation approach based on Knowledge Graph Attention Network (KGAT). They use a 3-hop graph-attention network to learn user, item and entity representations from a heterogeneous graph, which is formed by linking entities in knowledge graphs with items in the user-item graph. Fan et al. [5] proposed a social recommendation approach named GraphRec. They use graph attention networks to learn user and item embeddings from the user-item bipartite graph and the user-user social graph. However, these methods need centralized storage of users' interactions with items to form the entire user-item graph, which may arouse users' privacy concerns and the risk of private data leakage. Different from them, in our FedGNN method the raw user data never leaves the local user devices. In addition, FedGNN leverages a privacy-preserving model update method to protect private gradients and a privacy-preserving useritem graph expansion method to incorporate high-order user-item interactions. Thus, FedGNN can employ GNN models for grasping high-order information in a privacy-preserving way.

Federated Learning
Federated learning is a machine learning technique to collectively learn intelligent models based on decentralized user data in a privacy-preserving manner [14,17]. Different from existing machine learning methods based on centralized storage of user data, in federated learning the user data is kept locally on user devices [35]. Each device maintains a local model and computes local model updates based on the user data stored on this device. The local model updates from a number of users are uploaded to a central server that coordinates the model training process. These updates are aggregated into a unified one to update the global model maintained by this server. The updated model is further distributed to Table 1: Comparison of different methods in terms of high-order user-item interaction modeling and privacy protection. "Cen." and "Local" represent centralized and decentralized data storage, respectively.

PMF SVD++ GRALS sRGCNN GC-MC PinSage NGCF FCF FedMF FedGNN High-order user-item interaction
Cen. Cen. Cen. Cen. Cen. Cen. Cen. Local Local Local all user devices to update the local models. This process is iteratively executed until the model converges. Since the model updates usually contain much less private information and the raw user data never leaves the devices, the risk of privacy leakage can be effectively reduced [8].
The framework of federated learning has been applied to personalized recommendation [1,3,6,9,22]. For example, Ammad et al. [1] proposed a federated collaborative filtering (FCF) approach. In FCF, each user device locally computes the gradients of the user and item embeddings based on the personal ratings stored on this device. The user embeddings are locally updated, and the gradients of item embeddings are uploaded to a central server. The server aggregates the item gradients from massive clients to update the global item embeddings it maintains. The updated item embeddings are further distributed to user clients for local embedding updates. However, in this method the gradients of item embedding may leak some information on the private ratings [39]. To solve this problem, Chai et al. [3] proposed a federated matrix factorization (FedMF) method, where the item embeddings are protected by homomorphic encryption techniques. However, these methods do not consider the high-order interactions between users and items, which may not be optimal in learning accurate user and item representations. In addition, they mainly focus on protecting the private ratings given by users and cannot protect the raw user-item interaction data unless they locally maintain the full item set on each device, which is impractical due to the heavy storage and communication costs. Different from these methods, our approach can capture high-order interactions between users and items by our proposed privacy-preserving user-item graph expansion method. In addition, our method can protect the raw user-item interaction data during the model training process in an effective and efficient way. To better demonstrate the advantage of our approach, we summarize the comparison between FedGNN and existing methods on exploiting high-order user-item interactions and privacy protection in Table 1.

METHODOLOGY
In this section, we first present the problem definitions in our federated GNN-based recommendation framework (FedGNN), then introduce the details of our FedGNN approach for privacy-preserving recommendation, and finally provide some discussions and analysis on privacy protection.

Problem Formulation
Denote U = { 1 , 2 , ..., } and T = { 1 , 2 , ..., } as the sets of users and items respectively, where is the number of users and is the number of items. Denote the rating matrix between users and items as Y ∈ R × , which is used to form a bipartite useritem graph G based on the observed ratings Y . We assume that the user has interactions with items, which are denoted by [ ,1 , ,2 , ..., , ]. These items and the user can form a first-order local user-item subgraph G . The ratings that given to these items by user are denoted by [ ,1 , ,2 , ..., , ]. To protect user privacy (both the private ratings and the items a user has interactions with), each user device locally keeps the interaction data of this user, and the raw data never leaves the user device. We aim to predict the unobserved ratings ( ∈ Y\Y ) based on the interaction data G locally stored on user devices in a privacy-preserving way. Note that there is no global user-item interaction graph in our approach and local graphs are built and stored in different device, which is very different from existing federated GNN methods [11,18] that require the entire graph is built and stored together in at least one platform or device.

FedGNN Framework
Next, we introduce the framework of our FedGNN method for privacy-preserving GNN-based recommendation. It can leverage the highly decentralized user interaction data to learn GNN models for recommendation by exploiting the high-order user-item interactions in a privacy-preserving way. The framework of FedGNN is shown in Fig. 2. It mainly consists of a central server and a large number of user clients. The user client keeps a local subgraph that consists of the user interaction histories with items and the neighbors of this user. Each client learns the user/item embeddings and the GNN models from its local subgraph, and uploads the gradients to a central server. The central server is responsible for coordinating these user clients in the model learning process by aggregating the gradients received from a number of user clients and delivering the aggregated gradients to them. Next, we introduce how they work in detail. The local subgraph on each user client is constructed from the user-item interaction data and the neighboring users that have cointeracted items with this user. The node of this user is connected to the nodes of the items she interacted with, and the node of her neighboring users. An embedding layer is first used to convert the user node , the item nodes  can be locally tuned during model training, while the embeddings of neighboring users are fixed. 2 Next, we apply a graph neural network to these embeddings to model the interactions between nodes on the local first-order subgraph. Various kinds of GNN network can be used in our framework, such as graph convolution network (GCN) [13], gated graph neural network (GGNN) [16] and graph attention network (GAT) [28]. The GNN model outputs the hidden representations of the user and item nodes, which are denoted as h , , respectively. Then, a rating predictor module is used to predict the ratings given by the user to her interacted items (denoted by [ˆ, 1 ,ˆ, 2 , ...,ˆ, ]) based on the embeddings of items and this user. These predicted ratings are compared against the gold ratings locally stored on the user device to compute the loss function. For the user , the loss function L is computed as We use the loss L to derive the gradients of the models and embeddings, which are denoted by g and g , respectively. These gradients will be further uploaded to the server for aggregation.
The server aims to coordinate all user devices and compute the global gradients to update the model and embedding parameters in these devices. In each round, the server awakes a certain number of user clients to compute gradients locally and send them to the 2 We find this method slightly outperforms using trainable neighboring user embeddings (shown in experiments). Thus, we prefer fixed ones to reduce computational and communicational costs of model training. server. After the server receiving the gradients from these users, the aggregator in this server will aggregate these local gradients into a unified one g. 3 Then, the server sends the aggregated gradients to each client to conduct local parameter update. 4 Denote the parameter set in the -th user device as Θ . It is updated by Θ = Θ − g, where is the learning rate. This process will be iteratively executed until the model converges. We summarize the framework of our FedGNN method in Algorithm 1. We will then introduce two modules for privacy protection in FedGNN, i.e., a privacy-preserving model update module (corresponding to Lines 9-11 in Algorithm 1) for protecting gradients in the model update and a privacy-preserving user-item graph expansion module (corresponding to Line 15 in Algorithm 1) to protect user privacy when modeling high-order user-item interactions.

Privacy-Preserving Model Update
If we directly upload the GNN model and item embedding gradients, then there may be some privacy issues due to following reasons. First, for embedding gradients, only the items that a user has interactions with have non-zero gradients to update their embeddings, and the server can directly recover the full user-item interaction history based on the non-zero item embedding gradients. Second, besides the embedding gradients, the gradients of the Select a subset S from the user set U randomly 7: for each user client ∈ S do 9: if |S| < · then 10: else 12: Use neighboring user embeddings 25: end if 26: Compute GNN model gradients g and embedding gradients g on N 27: g ← (g , g ) 28: return g GNN model and rating predictor may also leak private information of user histories and ratings [39], because the GNN model gradients encode the preferences of users on items. In existing methods such as FedMF [3], homomorphic encryption techniques are applied to gradients to protect private ratings. However, in this method the user device needs to locally memorize the embedding table of the entire item set T and upload it in every iteration to achieve user interaction history protection, which is impractical due to the huge storage and communication costs during model training.
To tackle these challenges, we propose two strategies to protect user privacy in the model update process. The first one is pseudo interacted item sampling. Concretely, we sample items that the user has not interacted with 5 , and randomly generate their gradients g using a Gaussian distribution with the same mean and co-variance values with the real item embedding gradients. The real embedding gradients g are combined with the pseudo item embedding gradients g , and the unified gradient of the model and 5 There are many sampling methods such as using the displayed items that have no interaction with a user. In our experiments we randomly sample items from the full item set for simulation. In addition, needs to be larger than to protect user privacy. ... , Figure 3: The framework of the privacy-preserving useritem graph expansion method.
embeddings on the -th user device (Line 26 in Algorithm 1) is modified as g = (g , g , g ). The second one is local differential privacy. Following [21], we clip the local gradients on user clients based on their L∞-norm with a threshold , and apply a local differential privacy (LDP) [4] module with zero-mean Laplacian noise to the unified gradients to achieve better user privacy protection, which are formulated as follows: where is the strength of Laplacian noise. 6 The protected gradients g are uploaded to the server for aggregation.

Privacy-Preserving User-Item Graph Expansion
Then, we introduce our privacy-preserving user-item graph expansion method that aims to find the neighbors of users and extend the local user-item graphs in a privacy-preserving way. In existing GNN-based recommendation method based on centralized graph storage, high-order user-item interactions can be directly derived from the global user-item graph. However, when user data is decentralized, it is a non-trivial task to incorporate high-order user-item interactions without violating user privacy protection. To solve this problem, we propose a privacy-preserving user-item graph expansion method that finds the anonymous neighbors of users to enhance user and item representation learning, where user privacy does not leak. Its framework is shown in Fig. 3. We introduce its details as follows.
The central server that maintains the recommendation services first generates a public key, and then distributes it to all user clients for encryption. After receiving the public key, each user device applies homomorphic encryption [3] to the IDs of the items she Algorithm 2 privacy-preserving user-item graph expansion 1: PrivacyPreservingGraphExpansion(): 2: Server sends a public key to user clients 3: User clients encrypt item IDs with 4: User clients upload the user embedding and encrypted item IDs to a third-party server 5: Third-party server distributes neighboring user embeddings to user clients 6: User clients extend local graphs interacted based on this key because the IDs of these items are privacy-sensitive. 7 The encrypted item IDs as well as the embedding of this user are uploaded to a third-party server (do not necessarily to be trusted). This server finds the users who interacted with the same items via item matching, and then provides each user with the embeddings of her anonymous neighbors. In this stage, the server for recommendation never receives the private information of users, and the third-party server cannot obtain any private information of users and items since it cannot decrypt the item IDs. 8 We connect each user node with its neighboring user nodes. 9 In this way, the local user-item graph can be enriched by the high-order user-item interactions without harming the protection of user privacy. We summarize the process of our privacy-preserving user-item graph expansion method in Algorithm 2.

Analysis on Privacy Protection
The user privacy is protected by four aspects in our FedGNN approach. First, in FedGNN the recommendation server never collects raw user-item interaction data, and only local computed gradients are uploaded to this server. Based on the data processing inequality [17], we can infer that these gradients contain much less private information than the raw user interaction data. Second, the third-party server also cannot infer private information from the encrypted item IDs since it cannot obtain the private key. However, if the recommendation server colludes with the third-party server by exchanging the private key and item table, the user interaction history will not be protected. Fortunately, the private ratings can still be protected by our privacy-preserving model update method. Third, in FedGNN we propose a pseudo interacted item sampling method to protect the real interacted items by sampling a number of items that are not interacted with a user. Since gradients of both kinds of items have the same mean and co-variance values, it is difficult to discriminate the real interacted items from the pseudo ones if the number of pseudo interacted items is sufficiently larger than the number of real interacted items. The average degree of privacy protection is proportional to 1+ | | [26]. Thus, the number of pseudo interacted items can be relatively larger to achieve better privacy protection as long as the computation resources of user devices permit. Fourth, we apply the LDP technique to the gradients locally computed by the user device, making it more difficult to recover the raw user consumption history from these gradients. It 7 We choose homomorphic encryption because the server cannot match the items hashed by many other salted encryption methods. 8 We assume that they do not collude with each other. 9 The neighboring user nodes are not connected to the co-interacted items for better user privacy protection under Byzantine attack. is shown in [21] that the upper bound of the privacy budget is 2 , which means that we can achieve a smaller privacy budget by using a smaller clipping threshold or a larger noise strength . 10 However, the accuracy of model gradients will also be affected if the privacy budget is too small. Thus, we need to properly choose both hyperparameters to balance model performance and privacy protection.

EXPERIMENTS 4.1 Dataset and Experimental Settings
In our experiments, following [2] we use six widely used benchmark datasets for recommendation, including MovieLens 11 (100K, 1M, and 10M), Flixster, Douban, and YahooMusic. We use the preprocessed subsets of the Flixster, Douban, and YahooMusic datasets provided by [20]. 12 We denote the three versions of MovieLens as ML-100K, ML-1M and ML-10M respectively, and we denote Ya-hooMusic as Yahoo. The detailed statistics of these datasets are summarized in Table 2. In our experiments, we use graph attention network (GAT) [28] as the GNN model, and use dot product to implement the rating predictor. The user and item embeddings and their hidden representations learned by graph neural networks are 256-dim. The epoch threshold is 2. The gradient clipping threshold is set to 0.1, and the strength of Laplacian noise in the LDP module is set to 0.2 to achieve 1-differential privacy. The number of pseudo interacted items is set to 1,000. The number of users used in each round of model training is 128, and the total number of epoch is 3. The ratio of dropout [25] is 0.2. SGD is selected as the optimization algorithm, and its learning rate is 0.01. The splits of datasets are the same as those used in [2], and these hyperparameters are selected according to the validation performance. The metric used in our experiments is rooted mean square error (RMSE), and we report the average RMSE scores over the 10 repetitions.

Performance Evaluation
First, we compare the performance of our FedGNN approach with several recommendation methods based on centralized storage of user data and several privacy-preserving ones based on federated learning, including: • PMF [19], probability matrix factorization, which is a widely used recommendation method;  • SVD++ [15], another popular recommendation method based on a variant of singular value decomposition; • GRALS [23], a collaborative filtering approach with graph information; • sRGCNN [20], a matrix completion method with recurrent multi-graph neural networks; • GC-MC [2], a matrix completion method based on graph convolutional autoencoders; • PinSage [36], a recommendation approach based on 2-hop GCN networks; • NGCF [31], a neural graph collaborative filter method; • FCF [1], a privacy-preserving recommendation approach based on federated collaborative filtering; • FedMF [3], another privacy-preserving recommendation approach based on secure matrix factorization.
The recommendation performance of these methods is summarized in Table 3. We have several findings from Table 3. First, we observe that the methods which incorporate high-order information of the user-item graph (e.g., GC-MC, PinSage and NGCF) achieve better performance than those based on first-order information only (PMF). This is probably because modeling the highorder interactions between users and items can enhance user and item representation learning, and thereby improves the accuracy of recommendation. Second, compared with the methods based centralized user-item interaction data storage like GC-MC and NGCF, our FedGNN approach can achieve comparable or even better performance. It shows that our approach can protect user privacy and meanwhile achieve satisfactory recommendation performance. Third, among the compared privacy-preserving recommendation methods, FedGNN achieves the best performance. This is because FedGNN can incorporate high-order information of the user-item graphs, while FCF and FedMF cannot. Besides, our approach can protect both ratings and user-item interaction histories, while FCF and FedMF can only protect ratings.

Model Effectiveness
Then, we validate the effectiveness of incorporating high-order information of the user-item graphs as well as the generality of our approach. We compare the performance of FedGNN and its variants with fully trainable neighbor user embeddings or without high-order user-item interactions. In addition, we also compare their results under different implementations of their GNN models (GGNN, GCN and GAT). The results are shown in Fig. 4, which reveals several findings. First, compared with the baseline performance reported in Table 3, the performance of FedGNN and its variants implemented with other different GNN models is satisfactory. This result shows that our approach is compatible with different GNN architectures. Second, FedGNN slightly outperforms its variants based on GCN and GGNN. This may be because the GAT network can more effectively model the importance of the interactions between nodes than GCN and GGNN, which is beneficial for user and item modeling. Third, the variants that can utilize the high-order information by using our FedGNN framework perform better than those without high-order information. It validates the effectiveness of our approach in incorporating high-order information of the user-item graph into recommendation. Fourth, we find that using fixed neighbor user embeddings that are trained for a certain number of iterations is slightly better than using fully trainable ones that are updated in each iteration. This may be because the neighboring user embeddings may not be accurate at the beginning of model training, which is not beneficial for learning precise user and item representations.

Hyperparameter Analysis
Finally, we explore the influence of three important hyperparameters, i.e., the gradient clip threshold , the strength of the Laplacian noise in the LDP module, and the number of the pseudo interacted items. We first compare the performance of our FedGNN approach by varying both hyperparameters, and the results are plotted in Fig. 5. 13 According to these results, we find that the difference between the model performance under = 0.1 and = 0.2 is quite marginal. However, if we clip the gradients with a smaller threshold such as 0.05, the prediction error will substantially increase. Thus, we prefer to set = 0.1 because we can achieve better privacy protection without much sacrifice of model performance.
In addition, the model performance declines with the growth of the noise strength , while the performance loss is not too heavy if is not too large. Thus, a moderate value of such as 0.2 is preferable to achieve a good balance between privacy protection and recommendation accuracy. 14 We also compare the performance and communication cost 15 of FedGNN w.r.t. different in Fig. 6. From Fig. 6, we observe that the performance is the best if is 0, but the user-item interaction histories cannot be protected. According to the discussions in Section 3.5, if is too small the user privacy cannot be well-protected. In addition, the performance also declines because the randomly generated gradients will influence the accuracy of item gradients. By comparing the results on the three MovieLens datasets, we find 13 A larger and smaller means smaller budget , i.e., better privacy protection. 14 We achieve 1-differential privacy under = 0.1 and = 0.2. 15 We use the number of parameters to be exchanged in each iteration during model training to measure the communication cost. that the rating matrix is sparser 16 , may need to be larger to keep good recommendation performance. This ie because when is relatively large, the random gradients of pseudo interacted items will be better counteracted after aggregation and their influence will be mitigated. However, as shown in Fig. 6, the communication cost is also proportional to and it will be very heavy if is too large. Therefore, we choose as 1,000 to achieve good privacy protection and recommendation performance under reasonable communication cost.

CONCLUSION
In this paper, we propose a federated framework for privacy-preserving GNN-based recommendation, which aims to collectively train GNN models from decentralized user data by exploiting high-order useritem interactions in a privacy-preserving manner. Concretely, we locally train GNN model in each user client based on the local user-item graph stored on this device. Each client uploads the locally computed gradients to a server for aggregation, which are further sent to user clients for local updates. In addition, to protect user-item interaction data during model training, we apply local differential privacy techniques to the local gradients to enhance user privacy protection. Moreover, we sample pseudo interacted items to protect the embeddings of items that users have interactions with. Besides, to incorporate high-order user-item interaction information into model learning, we propose a privacy-preserving user-item graph expansion method that can find neighboring users with cointeracted items and exchange their embeddings for extending their local graphs. Massive experiments on six benchmark datasets show that our approach can achieve competitive performance with existing methods based on centralized storage of user-item interaction data and meanwhile effectively protect user privacy.