Introduction

Collective behaviors have been investigated for decades and have been proved to be regular more than random. Especially in recent years, thanks to the information technology and the Internet, physicists and sociologists have uncovered many remarkable statistical properties and patterns of collective behaviors with massive data. The human mobility has been found following reproducible and predictable patterns1,2,3. The communication pattern shows the bursty nature, exhibiting heavy-tailed distribution for the inter-event time4,5,6,7. Many models have been proposed to describe the patterns and fundamental mechanisms of those collective behaviors, such as the task-based queuing model4,8,9 and the interest-driven model10 describing the origin of the bursty nature, the radiation model11 describing the migration and mobility patterns. Numerous scientists are going further and deeper on the road of understanding the collective behavior's mechanism.

Among these collective behaviors, the online user behavior got more and more attention12,14,15,16,17,18,19,20,21,22,23 with the rapid development of Internet. In addition to the bursty nature of inter-event time, efforts also have been paid to investigate the behavior itself, such as the social influence of the decision making when installing online applications12,13 and the anchoring bias of online rating14. Although the collective behavior's inter-event time exhibits high-burstiness-low-memory property24, evidences of memory effect of the online user behavior itself have been brought out. The Markovian process is widely used to model the users' web browsing patterns15,16, assuming that, the user's next action depends only on his/her current action. Actually, we can consider this kind of Markovian type patterns having short memories because there are correlations between every two continuous actions of a user. Furthermore, using the method of detrended fluctuation analysis, Rybski et al.17,18 found the long-term memory of users' online communicating frequency.

Nowadays, the Internet does lots of favor for our daily life. The most frequent action of our online behavior is selecting, which is a reflection of our online preference, such as selecting commodities, music or movies. Thus, the memory effect representing the predictability of users' online preference is of great significance for developing the recommendation systems25,26,27 and providing better online service. But the question is, what is the mechanism governing our online preference? A related work refers to the recent study of the users' commodity-browsing behavior19. Zhao et al. found that, when browsing commodities, the intervals of users staying in a single catalog present a power-law distribution, which indicates the users tend to continuously browse similar commodities. On the other hand, users may return to historical catalogs after some specific intervals. This kind of online commodity-browsing behavior has memory effect and is quite predictable which is similar to the human mobility behavior3. In addition to the memory effect of transferring from a catalog to another, are there any correlations between every two continuous actions or how do the users feel like their selections? Our goal here is to uncover and model the local correlations of the dynamics of online user preference when selecting and evaluating objects. As many online systems allow users to deliver ratings on objects which could largely reflect how the user feels about the objects28, we adopt another method besides the catalogs to evaluate objects in a more detailed way.

Data

Two datasets, namely MovieLens and Amazon, are investigated in this paper. The MovieLens data consists 698054 ratings delivered by 5547 users on 5850 movies during 1686 days. The Amazon data consists 1406147 ratings of 624271 users rated on 86087 items. We uniformly call both the movies in the MovieLens data and the items in the Amazon Data as objects in this paper. Each rating in both datasets is an integer ranging from 1 to 5 reflecting how the user feels like the objects' worth, taste and so on. The higher a rating is, the better the user evaluates the object. Then, we could also define the quality of an object as the average value over all ratings the object got in the whole dataset. Thus, we can get two messages from the user's behavior: the quality of the object he/she selects and the value of the rating. Those two messages can be regarded as the user's selecting behavior and rating behavior respectively. Consequently, we investigate the Selecting Series (SS) which is the sequence of the object qualities of each user's choice (a typical example is shown in Fig. 1 (d)) and the Rating Series (RS) which is the sequence of each user's ratings (a typical example is shown in Fig. 1 (a)). These two series are sequences of users' behavior in order of time. We will firstly uncover and model the self-correlation of users' SS and RS ignoring the exact inter-event time and then discuss the inter-event time's effect on the self-correlation. It should be noted that, while the MovieLens data is ordered by seconds, the Amazon data is ordered by days. Thus, a few records of Amazon data involved in the situation that, at the same day, a user selects several objects which cannot be ordered by time according to the data. Against this kind of situations, we arrange the records of a specific user occurring on the same day in random order. Furthermore, the activity level L of each user, i.e. the number of objects the user has selected, is different. In order to ensure the accuracy of the results, we only take users with L ≥ 100 into consideration.

Figure 1
figure 1

A typical user's RS, SS and duration distribution p(λ) from the MovieLens data.

This user's activity level is L = 464 and his/her memory effect is MRS = 0.216 and MSS = 0.471 for RS and SS respectively. (a), (d) A part of the user's whole RS and SS respectively. While the values in RS are integers, SS consists of float values ranging from 1 to 5. Regarding those values which is greater than the mean value as positive bars and those which is less than the mean value as negative bars, one can get the subplot (b) and (e) for RS and SS respectively. While every user may have many memories, we define in this paper that, a memory ends only when the bar in his/her series changing from positive (negative) to negative (positive). Then, the memory's duration λ is the length of the series involved in that memory, as typical marks shown in subplot (b) and (e). (c), (f) The memory duration distribution p(λ) of the typical user for RS and SS respectively.

Full size image

Methods

To evaluate the memory effect of the user's SS and RS, we use the method of Correlation Coefficient which is also used by Goh and Barabási24. The memory M is defined as

where ri is the ith value in the user's series and m1(m2) and σ1(σ2) are the mean and standard deviation of the sample series () respectively. Actually, the correlation coefficient method could measure the correlation between two continuous values in a single sequence. Thus, the correlation coefficient could well describe the local memory effect. With this definition, the memory M has a value in the range (−1, 1) and is positive when a high(low) value in the series tends to be followed by a high(low) one and it is negative when a high(low) value in the series tends to be followed by a low(high) one. Note that, the present paper aims to study the users' memory effect on the values of ratings and qualities, not the inter-event times24 and the object catalogs19.

There are also other methods evaluating the memory effect of series, for example, by using detrended fluctuation analysis17,18,29 to detect the Hurst exponent30 of the series. However, the length of series should reach 104 to guarantee the estimation of Hurst exponent31 and furthermore, the Hurst exponent is to describe the long-term memory. In this paper, the length of series is of scale 102 and we mainly focus on the local correlation, thus, we didn't use the method of Hurst exponent.

Using the correlation coefficient method, the user shown in Fig. 1 has memory effect MRS = 0.216 and MSS = 0.471 for RS and SS respectively. The positive value of this user's memory effect MRS and MSS means that, when the user selected a high(low)-quality object, he/she would continuously select high(low)-quality objects and further, when he/she delivered a high(low) rating, he/she would continuously deliver high(low) ratings. In other words, a memory of this user's preference may last for several rounds of actions. But the question is, how long this kind of memory could last? We define in this paper that, when the value in a user's series changing from less(greater)-than-mean to greater(less)-than mean, the user's current memory ends and his/her next memory begins. Thus, a user may has many memories and we further define the memory duration λ as each memory's length, i.e., the number of continuous qualities or ratings which are greater(less) than the mean value of the user's SS or RS, as shown in the Fig. 1 (b) and (e). To give an example, for a specific user, one of his/her memory's duration is λ = 4 for RS means that, he/she continuously delivered 4 ratings which are all greater than the mean value of his/her RS. Note that, the summation of the memory durations over all of a user's memory equals to the length of the behavior sequence , where λi is the duration of the user's ith memory and ku is the number of user u's memories. Counting every memory duration, we can calculate the probability p(λ) of a memory with duration λ and the duration distributions for RS and SS of the typical user are reported in Fig. 1 (c) and (f).

Empirical Results

For each user, we calculate the memory M of his/her SS and RS respectively according to Eq. (1). The distribution of memory M is shown in Fig. 2 together with that of the null model. The null model is the case in which we shuffle each user's series into random order to remove the temporal behavior, so that the users' selecting and rating patterns no longer exist. The mean value of the memory M of RS and SS is 0.19 and 0.36 respectively for the MovieLens data and is 0.18 and 0.17 respectively for the Amazon data. This result indicates the existence of the memory effect of users' both rating and selecting behavior. On the other hand, the memory M of the null model has a mean value of 0. We can conclude from the comparisons between the null model and the empirical data that, the memory effect of users' selecting and rating behavior comes from their own temporal behavior patterns not the random mechanism.

Figure 2
figure 2

The distributions of memory M for the MovieLens and the Amazon datasets.

Subplots (a) and (b) show the distributions of RS and SS for the MovieLens data respectively, in which, the memory 〈MRS〉 = 0.19 and 〈MSS〉 = 0.36. Subplots (c) and (d) show the distributions of RS and SS for the Amazon data respectively, in which, the memory 〈MRS〉 = 0.18 and 〈MSS〉 = 0.17. As to the null model, the memory M distribute in a narrow range and the mean value 〈M〉 = 0. Comparisons between the empirical data and the null model report the existence of the memory effect.

Full size image

As shown in Fig. 1 (c) and (f), the duration distributions p(λ) approximately exhibit power-law decays on the individual level. That is to say, there are probabilities occurring memories with very long duration. For the typical user in Fig. 1, whose activity is L = 464, the memories may last for about 30 rounds of actions for both rating and selecting behavior. Actually, on the individual level, users with different activity levels L all approximately have the power-law duration distribution (Fig. S1). On the collective level, the duration distribution p(λ) also exhibits the power-law form, as shown in Fig. 3. Thinking of the totally random case, the probability of a user selecting a high(low)-quality object at each time is 0.5. Thus, the theoretical duration ditribution should be p(λ) = 0.5λ which is the exponential form. As shown in Fig. 3, the duration distributions of the shuffled series for both the MovieLens and the Amazon data are similar to that of the totally random case, following the exponential form. That is to say, once the temporal pattern is removed, users' online behaviors would perform high randomness. On the other hand, the empirical results are very different with the totally random case, with power-law form duration distribution which suggests that, the sequence of users' selecting and rating behavior is self-correlated rather than random. Note that, although the probability of the memory having a long duration is small (for example, p(λ > 10) = 0.042 for MovieLens' SS), the amount of user's actions involved in a long duration memory is apparent. About 28.3% of the users' actions involved in memories with duration larger than 10, which are quite long memories (Fig. S2).

Figure 3
figure 3

Memory duration distributions p(λ) of the empirical data, the null model and the totally random case for the MovieLens data (above) and the Amazon data (bottom).

While duration λ of the empirical data follows the power-law distribution, that of the null model approximately follows the exponential distribution, which is very similar with the totally random case in which p(λ) = 0.5λ. Note that, the shuffled RS partly deviates from the totally random case in both datasets. The reason lies in the fact that, the values in RS are integers ranging from 1 to 5, with only 5 options and some users tend to deliver the same ratings to whatever he/she selected without consideration. Thus, some users' RS may consist of large amount of same values which would lead to the continuous same values for the shuffled RS.

Full size image

Preference Model

Results from the empirical data reported the memory effect of users' selecting and rating behaviors. Modelling the mechanism is crucial for understanding the dynamic of users' online preference. However, the classical models of human dynamics in the literatures mainly focused on the mechanism of the heavy-tailed inter-event time distribution. One of the most well-known models is the task-based queuing model4. While the model well explained the origin of the heavy tails of the inter-event time distribution, it cannot generate the memory effect observed in the present paper (See Supplementary Information for detail). It is reasonable that, the queuing model aims to describe the time distribution and the human behavior's time distribution in general has no memory effect24. Thus, to gain insights to the memory effect and the power-law duration distribution of online users' preference, we develop a preference model in which the users' selecting and rating behaviors are modeled in two steps.

1) Selecting behavior. Suppose a user's next selection depends on the current selection, which results in the memory effect of the user's SS. Thus, we use the Markovian process to model the selecting behavior mechanism. The Markovian process in users' selecting behavior is the process in which the users' current selection with quality qi transfers to the next selection with quality qi+1. The empirical statistics of the bias δSS = qi+1 − qi are shown to follow Gaussian forms with expectations μSS = 0 (Fig. S3). Furthermore, the standard deviations are and for the MovieLens and the Amazon data respectively. Thus, the transition probability distribution of the Markovian process should be a Gaussian distribution with expectation μ = qi and standard deviation σSS. Then, the quality of the user's next selection qi+1 comes from this Gaussian distribution, i.e. the probability of the next object's quality being qi+1 is given by

2) Rating behavior. We assume that, while the selecting behavior is self-correlated between current and next actions, the memory effect of the rating behavior origins in the selecting behavior. The value of rating ri only depends on the quality qi of the selected object. Similar with the selecting bias, the rating bias δRS = ri − qi also exhibit Gaussian distributions for both the MovieLens and the Amazon data (Fig. S3). The expectations are and and the standard deviations are and for the MovieLens and the Amazon data respectively. Thus, we can also give the probability of the rating the user deliver on a object with quality qi being ri as

The quality of an object could be a float value ranging from 1 to 5 and the rating ri is an integer. In those two stochastic processes, we continuously get random values until the value locates in the range [1, 5]. In addition, for the RS, we round the value into integer. When generating series, with respect to the empirical data, we generate series with length Li for each of the user i, i.e. we remain the number of users and the activity levels of each of them. As to the initial condition q0, we give a mid-value of the range of quality [1, 5], i.e., q0 = 3.

There are two free parameters left in the preference model, that is the standard deviations of the two Gaussian distributions σSS and σRS. Despite that the statistics have shown the empirical value of the standard deviation σSS and σRS for the MovieLens and Amazon data (Fig. S3), those two parameters could control the strength of the memory effect and the distributions of the memory duration p(λ). We simulate with different values of σSS and σRS for the MovieLens system and the distributions of the memory duration p(λ) is shown in Fig. 4. When those two parameters are small, the memory duration of both SS and RS exhibit a well power-law distribution which is similar to the empirical pattern. As the parameters increase, the distributions gradually change from the power-law form to the exponential form and become more and more similar with the totally random case. The reason lies in the fact that, when the standard deviation is small, a burst jump of the values has little chance to occur, i.e. the memory effect is stronger. On the other hand, if the standard deviation is large enough, the Gaussian distribution could be approximately considered as the uniform distribution which leads to the totally random case in which p(λ) = 0.5λ. Results of Fig. 4 indicate that, the standard deviations of the SS's and RS's bias indeed could control the distribution of the memory duration. When simulating with the empirical values of σSS and σRS for the MovieLens and Amazon data respectively, the memory duration distributions of the simulation consist with that of the empirical data (Fig. S4). It means that, the preference model can well describe the emergence of heavy-tailed distribution in the memory duration. But on the other hand, the preference model has deviation from the empirical data in the distributions of memory M for users' both SS and RS. The comparison between the empirical data and the preference model indicates that, users' selecting behavior is approximately a Markovian process and that the rating behavior mainly depends on the selecting behavior. Briefly speaking, when a user selects an object with quality qi at this time, the quality of the next object qi+1 is probably around qi and the user further tend to deliver a rating which is also around the quality of the object. In addition, the memory effect of users' rating behavior origins in the selecting behavior.

Figure 4
figure 4

Memory duration distributions p(λ) of SS and RS from the simulations of preference model with different values of parameter σSS and σRS respectively.

When simulate the preference model for the RS, we arrange the standard deviation of SS's bias as σSS = 0.5. As the results show, for both SS and RS, the smaller the standard deviation is, the heavier the distribution's tail would be. On the other hand, as the standard deviation increases, the distribution gradually changes from the power-law form to the exponential form and is more and more similar to the distribution of the totally random case.

Full size image

We can reproduce any duration distributions from power-law to exponential form with just one parameter for each of the selecting and rating behavior. The standard deviation could largely explain the users' memory effect and the power-law decay of duration distribution. However, as users with different activity levels have in general different properties20,32,33,34, the parameter may be various from users to users. Actually, the activity level indeed affects the memory effect (Fig. S5). In the MovieLens system, high-activity users generally have stronger memory effect and lower deviation parameter σSS and σRS. But in the Amazon system, the activity level is uncorrelated with the users selecting behavior. For the rating behavior, high activity level would approximately leads to strong memory effect and small deviation parameter σRS. Overall, we just need to fit different parameters σSS and σRS to describe and reproduce different users' selecting and rating behavior.

Effect of the Inter-event Time

Numerous investigations proved that, the heavy-tailed distribution of inter-event time was one of the most important properties of collective behavior4,10,20. As the present paper aims to study the correlations between online users' current and next actions, the inter-event time between those two actions may has important influence. As Fig. 5 (a) and (d) show, the distributions of inter-event time for both MovieLens and Amazon system exhibit power-law forms which have been observed in many classical researches. For a user whose activity level is L, there would be L − 1 inter-event times τ. After averaging each user's inter-event times, we show the correlation between user's average inter-event time 〈τ〉 and their memory M as shown in subplot (b) and (e). In MovieLens dataset, users who have long average inter-event time generally have weak memory effect and those whose average inter-event times are short have strong memory effect. It is easy to understand that, while there would be a strong correlation between two actions, if one occurred immediately after another, the correlation would be very weak if two actions have very long inter-event time. However, for Amazon system, the users' memory effect is approximately uncorrelated with their average inter-event time. To uncover the time effect on the preference model, we calculate the conditional standard deviation σSS(τ). Each pair of selecting behaviors with inter-event time τ has bias δSS(τ) and σSS(τ) is the standard deviation of δSS(τ). Figure 5 (c) and (f) shows the correlation between σSS(τ) and τ. It is surprising to find that, the MovieLens and Amazon system has different reactions toward the inter-event time in the parameter σSS(τ). As we reported, σSS(τ) could reflect the strength of the memory effect. Subplot (c) shows that, the longer the inter-event time is, the weaker the memory effect would be in the MovieLens system. But for the Amazon system which is shown in subplot (f), the situation is totally different that, the longer the inter-event time is, the stronger the memory effect would be. The difference of the inter-event time's effect between those two datasets may lies in the fact that, MovieLens is a system in which users watch and rate movies, but Amazon is a system in which users buy items. In the Amazon dataset, we observed the phenomenon that users might select many same objects. Those repeated objects could be consumables that, users may buy a new one after a specific time interval. Furthermore, the evaluation of a user to a very object is in general similar. This may be the reason that, long inter-event time brings strong memory effect.

Figure 5
figure 5

Inter-event time distribution p(τ) and its effects on the memory effect M for the MovieLens and the Amazon data.

Subplots (a) and (d) report the inter-event time distributions on the collective level. As has also been reported in many classical researches, the inter-event time exhibits heavy-tailed power-law distributions. Subplots (b) and (e) show the correlation between memory effect M and the user's average inter-event time 〈τ〉. Subplots (c) and (f) show the correlation between the deviation parameter σSS and inter-event time τ.

Full size image

Discussion

Considering the significance of the online user preference for the understanding of collective behavior pattern and the developing of online systems, we investigated the memory effect of users' selecting and rating behavior with the method of Correlation Coefficient. The mean value of memories M for SS and RS of the empirical data is 0.36 and 0.19 for the MovieLens dataset and 0.17 and 0.18 for the Amazon dataset respectively, indicating that, the complex online user preference dynamics have memory effect. Furthermore, we found the distribution of the memory duration, which was used to describe the memory's length, exhibiting scaling law with heavy-tailed power-law form. To model the pattern and the fundamental mechanism of the online user preference, we utilized a Morckovian process to model the selecting behavior and supposed the rating behavior totally depending on the selecting behavior. The distribution of the memory duration of the preference model coincided with the empirical data. Just one parameter for each of the RS and SS is needed to reproduce any duration distribution ranging from the power-law to exponential form.

Results in this paper indicated that, the Markovian process could largely explain the memory effect of users' online selecting behavior and the memory effect of the rating behavior origins in the selecting behavior. In a recent study14, Yang et al. found a correlation between the quality of the former selection qi−1 and the current rating ri, that, when qi−1 is very high(low), ri would also be high(low). This anchoring bias phenomena can be explained by the present paper. Actually, when qi−1 is high(low), the quality of the current selection qi would also be high(low) according to the memory effect. Furthermore, as the rating behavior largely depends on the selecting behavior, the current rating ri would consequently also be high(low).

However, we cannot ensure the effect of inter-event time and the activity level. As active users have in general shorter inter-event times, we could not know whether the activity level or the inter-event time or both of them lead to the heterogenous memory effect M shown in Fig. 5 (b) and (e) and Fig. S5 (a) and (d). In addition, the MovieLens and the Amazon datasets exhibit differences in some results such as the activity's and inter-event time's effect on the memory and the preference model. What caused the different patterns for those two systems is an important question. As we have introduced that, the MovieLens is a system in which users watch and rate movies, but the Amazon is a system in which users buy items. While watching movies is an entertainment behavior, buying items would cost money. Users' selecting and rating behaviors are based on different purposes and considerations in these two systems. Thus, the question is, do the patterns vary for different types of behaviors? Are there several universal classes of systems for the users' online preference?

Absolutely, our effort is still far from totally understanding the online user behavior patterns. When modeling the users' selecting behavior, we utilized the Morckovian process in which the user's next selection depends only on his/her current selection. We didn't consider the possibility that, the next selection might be correlated with not only the current one but also former several ones. This is a kind of very short and local correlation, but whether there is a long-term correlation and whether it is possible for a user to repeat some selecting behavior fragments are still open questions. In addition, many other factors could affect user's selecting and rating behavior, such as the social influence12,13 or the recommendation list. Another problem is, the quality (average rating) is still not enough to describe all the aspects of an object. It's reasonable to consider other aspects seriously such as the popularity.