The Effect of Memory in Inducing Pleasant Emotions with Musical and Pictorial Stimuli

Abstract

Music is known to evoke emotions through a range of mechanisms, but empirical investigation into the mechanisms underlying different emotions is sparse. This study investigated how affective experiences to music and pictures vary when induced by personal memories or mere stimulus features. Prior to the experiment, participants were asked to select eight types of stimuli according to distinct criteria concerning the emotion induction mechanism and valence. In the experiment, participants (N = 30) evaluated their affective experiences with the self-chosen material. EEG was recorded throughout the session. The results showed certain interaction effects of mechanism (memory vs. stimulus features), emotional valence of the stimulus (pleasant vs. unpleasant), and stimulus modality (music vs. pictures). While effects were mainly similar in music and pictures, the findings suggest that when personal memories are involved, stronger positive emotions were experienced in the context of music, even when the music was experienced as unpleasant. Memory generally enhanced social emotions specifically in pleasant conditions. As for sadness and melancholia, stimulus features did not evoke negative experiences; however, these emotions increased strongly with the involvement of memory, particularly in the condition of unpleasant music. Analysis of EEG-data corroborated the findings by relating frontomedial theta activity to memory-evoking material.

Introduction

The induction of affect is a multidisciplinary topic that intrigues scholars from psychology, cognitive neuroscience, and art. Due to its complexity, affect induction requires approaches ranging from laboratory experiments to subjective evaluations of personal meanings, to generate comprehensive knowledge of affect induction principles. In the studies of emotions induced by art objects, however, the participants’ personal memories, experiences, and meanings constitute an area that often remains beyond reach in laboratory settings. Furthermore, while some modality specific features have been identified in the perception of auditory and visual stimuli1, little is known about cross-modal differences of affective processing involving personal meanings and memories. This study aims to find out (a) whether emotions involving personal memories impact the affective experience and (b) whether the potential influence of memory is similar in distinct modalities (music and pictures) and in pleasant and unpleasant experiences.

Pleasant and Unpleasant Emotions

In the studies scrutinizing positive and negative emotions, valence refers to the intrinsic attractiveness (positive valence) or aversiveness (negative valence) evoked by an object. Affect, considered as an umbrella term, covers all evaluative (positive/negative) states, such as mood, and aesthetic evaluation (see2) – or to put it more generally, all the positive and negative evaluative states3. The term valence is used here to describe specific emotions on a dimension ranging from pleasurable to unpleasant. More broadly, valence is part of a two-dimensional model of emotions4 that proposes that all affective states arise from independent neurophysiological systems, one related to valence and another related to arousal. The model is widely adopted in affective sciences5,6, albeit not the only candidate for the emotions7.

There are several competing hypotheses as to why distinct modalities, such as music and pictures, may evoke strong positively valenced experiences. For instance, peak sensations that are typically studied through self-reports, physiological responses, and by neural measurements, have been studied in the context of music and have been associated with the release of dopamine via the neural reward system8,9,10. Related sensations are often depicted as chills, shivers, or goose bumps, and when music is involved, they seem to correspond, for instance, to unexpected changes or gradual expansions in the dynamics, structure, and volume of music (see, for example11,12,13,14,15,16). Assuming a close link between pleasant feelings and affective arousal in the past studies, one assumes that self-reports and EEG-measures reliably evince the respondents’ affective engagement with music and pictures.

The experiences related to pleasure as characterized by extreme positive valence could be characterized in several ways; top-down influences such as activation of specific memories and symbolic construction are assumed to be important generators of pleasure17,18,19,20. In such instances, emotional experience cannot be explained by reducing them to stimulus properties, such as musical structure or visual elements21,22. The simultaneous presence of distinct emotions may further result in mixed affective experiences. Exposure to negative emotions occurring in art reception is often associated with positive emotions, constituting a so-called paradox of art23. For example, sadness, that is typically experienced in the context of music24,25 – defined as a negative emotional state in psychology25 (see also26) – may be experienced as pleasant when associations and memories are strongly involved27 (further details about enjoyment of negative emotions across different fields of art, see e.g.28). Instead, when the stimulus is conceived as purely negative it activates defensive motivation systems found to enhance sensory, perceptual, and memory processes29,30, leading to displeasure, also frequently elicited by unfamiliar or violent music31 (for visual arts, see32).

Emotion Induction Mechanisms

Emotion induction mechanisms cover a range of processes, such as perception, memory, imagination, and somatic experience, as well as those behavioral processes that result from the elicitation of emotion. The term induction mechanism is used in the present study to broadly refer to all information processing that leads to the induction of emotions through the perception of the object3.

Previous studies have identified several distinct induction mechanisms, some focusing on fewer possibilities, such as cognitive appraisal33, and others presenting an extensive set of mechanisms and emotion induction moderators14,34,35. The empirical investigation of these mechanisms is sparse; some mechanisms, such as expectancy, appear to be invoked less often than other mechanisms, such as episodic memory2. This may relate to the fact that implicit mechanisms remain unacknowledged in the studies based on self-reports. Also, some mechanisms may be more suitable for inducing particular emotions; music-induced sadness, for instance, may use the feature-based mechanism of emotional contagion while happiness may be induced by memories36. Another assumed asymmetry between emotions and induction mechanisms relates to emotions that are strongly social such as tenderness. Social emotions are likely to relate to social connectedness and belonging37,38,39,40. It is expected that such emotions are linked with different induction mechanisms, although no such studies have yet been carried out.

Comparison of Distinct Modalities

Most studies of pleasurable music have focused on sedentary listening. In that respect these findings may relate to pictures as well, and indeed, emotion induction theory14 assumes that music evokes emotions through mechanisms that are not unique to music. There is, however, relatively little prior research directly comparing music and pictures as modalities for emotional experiences. While music has been actively studied as a source for distinct emotions and daily pleasure, research on pictures has rather focused on aesthetic appreciation of visual stimuli as cognitive-perceptual judgments41.

Previous research suggests some modality-based qualitative differences in the elicited emotional experiences. Rottenberg and her colleagues42 proposed the following key dimensions that differentiate emotion induction: (1) emotional intensity, (2) complexity, (3) levels of attentional capture, (4) demand characteristics, (5) standardization, (6) temporal considerations, and (7) ecological validity. The dimensions may vary in terms of their relevance depending upon the variable under scrutiny. For example, music may score high in ecological validity through the activation of associated memories, the elicitation of imagery, and conditioned emotional responses14; regarding pictorial materials, the personal relevance and ecological validity dimensions may be lower due to the related approach-avoidance motives to a minor extent, than real-life interactions43. This suggests that emotions evoked by visual arts may be perceived as less similar to emotions experienced in other everyday life situations.

The present study focuses on comparable aspects of the phenomenon (emotion, valence, mechanism) that are present and hold relevance for both modalities, yet we assume that there are fundamental variations between modalities due to the differences in the way they relate to space (picture) and time (music), and the generic conventions of practice. Despite the issue of comparability, both music and images are able to evoke a diverse set of emotions and are found to be highly effective for inducing emotional states43,44,45,46, driven by hedonic expectations and reward47,48,49,50, or more generally, positive affect44. To our knowledge, past research has not attempted to clarify whether emotions induced by images tap into similar mechanisms as emotions induced by music. Furthermore, if the targeted emotion is the same across the domains, such as pleasure or displeasure, do the two domains have distinct ways of inducing these states?

Rationale

The aim of the study was to explore the interaction effects between stimulus valence, emotion induction mechanism, and modality. The specific focus of interest was to investigate the role of memory as a mechanism for inducing pleasant emotions. This focus can be divided into two research questions related to memory as the emotion induction mechanism:

  1. (1)

    Do episodic memories contribute to emotional intensity? Are the induced emotions stronger if they carry personal memories when compared to emotions induced by stimulus features51,52?

  2. (2)

    Do episodic memories lead to similar emotional valence in both domains? In other words, is there a three-way interaction among the emotion induction mechanism, stimulus valence, and stimulus modality?

Study design and hypotheses

Differences between pleasant and unpleasant emotional experiences of music and pictures representing four stimuli types that differed in terms of emotion induction mechanism (memory vs. stimulus feature) and valence (pleasant vs. unpleasant), were examined using self-reports and neurophysiological measures. All stimuli were participant-selected (see e.g.8,10), and a within-subject design was utilised.

Our hypothesis concerning emotion induction mechanisms is that emotions involving memories are more intense than experiences based on stimulus features. Art reception involves interactions of stimulus characteristics, individual differences, and context53,54,55,56. Research on highly intense emotional responses to art has revealed a close connection between personal memories and emotional responses10,15,16,27,57,58,59,60. Such responses may indicate high levels of emotional intensity, high levels of aesthetic enjoyment and high memorability as associated with artworks, the processing of which may also include negative affect. It is expected that emotions induced by memory enhance their intensity more than stimulus features2,22. There are also implications of a connection between particular emotions, such as sadness, and strong involvement of memories and associations61.

The second hypothesis concerns the modality and valence of emotions. We do not expect a main effect of modality, but we do expect to observe an interaction between valence, mechanism and modality. Since both visual and auditory stimuli have been found to elicit strong positive and negative emotions62,63, priority for either of the modalities is not assumed. Instead, valence-specific biases to modality are expected to occur. Emotions elicited by negatively valenced (unpleasant) stimuli may differ based on distinctions between modalities. This might be expected due to the knowledge of high ecological validity of music through the activation of associated memories14, specifically in the context of negatively valenced emotions63. In contrast, looking at pictures may activate motives (e.g., an abstract scene, or Christ on a cross) to a minor extent than real everyday transactions, proposing probable lower personal relevance43. Thus, while memories have been identified as an important element for music-induced sadness, the same argument cannot be drawn based on the extant literature on pictures, although some past studies have documented the occurrence of both sadness64,65 and the involvement of self-referential memorizing66,67 in the context of emotions induced by visual art.

Methods

Participants

Thirty participants were recruited through University mailing lists. Those with previously diagnosed mental disorders and those currently under medication for any mental health or mood disorders were excluded. The participants were asked to report about such disorders and/or medication in the beginning of recruitment process, and again in the questionnaire that they filled before the laboratory experiment. Regarding the sample, 77% were women and the mean age was 29 years (SD = 5.84), ranging from 21 to 45 years. The participants were given two movie tickets for their participation and provided informed consent prior to the study.

Stimuli

Stimulus selection strategies were designed to optimise ecological validity and experimental control. Participants were instructed to choose eight stimuli before the experiment: Four pictures and four pieces of music. The emotion induction mechanism was manipulated by altering the stimulus selection principle to rely on (a) personal memory or (b) purely stimulus properties. Valence was varied by asking them to choose either (a) pleasurable or (b) unpleasant stimuli. Both requisites were applied to music and pictures. Therefore, the stimulus instructions were as follows:

Choose a piece of music….

  • that evokes pleasure which is based on your personal memories

  • that evokes pleasure which is based purely on how it sounds

  • that evokes unpleasantness/aversive emotions based on your personal memories

  • that evokes unpleasantness/aversive emotions based purely on how it sounds

Choose a picture

  • that evokes pleasure which is based on your personal memories

  • that evokes pleasure which is based purely on how it looks

  • that evokes unpleasantness/aversive emotions based on your personal memories

  • that evokes unpleasantness/aversive emotions based purely on how it looks

These materials had to represent an object that could be available for anyone, excluding, for instance, personal photos, and non-published musical productions. The difference between perceived and experienced emotions was explained to the participants before the stimulus selection and again at the beginning of the experiment. They were instructed to focus on their experience rather than on what the music or picture is representing. Participants were instructed to give detailed information about the stimuli to ensure that the researchers found the exact same stimuli. Musical excerpts were downloaded by the experimenters and edited into 40-second excerpts. The segments were chosen by a music expert to be a representative and memorable section of the song. Visual materials were downloaded and each picture was presented in the experiment for 40 seconds to eliminate confounds created by varying durations between the two domains.

Stimulus self-reports for validating stimulus selection concerning valence

In the experiment, participants rated verbally the valence of each stimuli during the breaks between stimulus presentation. A scale ranging from −3 (extremely unpleasant) to 3 (extremely pleasant) was used. The ratings were subjected to repeated-measures ANOVA to check assumed pleasantness and unpleasantness across the Valence, Mechanisms, and Modality. Significant effects of Valence (F (1,29) = 435.5, p < 0.001), Modality (F = 9.68, p < 0.05), and Mechanism (F = 9.00, p < 0.05) were observed. Ratings for positively valenced stimuli were higher (M = 1.98, SD = 0.83) than for negatively valenced stimuli (M = −1.47, SD = 1.14). These analyses were essentially a manipulation check. The results corroborate that the stimulus selection principles operated according to intentions.

Stimulus properties

The stimulus properties were analysed with computational models. The musical and acoustic qualities of the excerpts were estimated using MIR toolbox68 by focusing on a limited set of acoustic features (mean and standard deviation of dynamics, tempo, pulse clarity, register, major-minor, rhythmic fluctuation) that have been shown to be relevant for emotional expression (e.g.69). The extraction parameters were like those reported in past studies and the windowed analyses of the features were aggregated using means and standard deviations; these were entered into a two-way repeated-measures ANOVA with Valence and Mechanisms as the factors. Only one feature, spectral flux, showed significant main effects of Valence (F (1,29) = 4.34, p < 0.05) and Mechanism, F (1,29) = 4.47, p < 0.05). The excerpts chosen to represent stimulus feature category (in contrast to memory) in the unpleasant condition were considerably higher in spectral flux; this was also evident in the type of music genres and bands chosen for the unpleasant and unfamiliar condition (typically heavy rock and related genres). The rest of the features did not differ across the selections.

The 120 images were analysed in terms of their overall colour profiles and its complexity. For this, the perceptual hue, saturation, and brightness (HSV) were extracted from the RBG values. The HSV profiles of the images were summarised by the mean and entropy for saturation and brightness histograms, which were binned to 50 values within saturation and brightness distributions. The mean hue values were estimated after conversion into degrees and taking the mean angle of the resulting distribution. The means and entropies of the profiles were subjected to repeated-measures ANOVA across Valence and Mechanism factors, yielding non-significant main effects (F < 2.06 for all six features). In sum, the surface characteristics of the images do not distinguish the sample categories. Next, the images were subjected to automatic content analysis where the tags describing the images were retrieved using an online service (Clarifai). The top 10 tags for each image were then subjected to sentiment analysis using vocabulary approach70 based on 13,915 English words rated for Valence, Arousal and Dominance. Each image obtained a score in each dimension based on the average of the matching keywords (8–11 keywords/image). A repeated-measures ANOVA for Valence score revealed significant effects for Valence (F (1,29) = 7.93, p < 0.01) but not for Mechanism (F = 0.46), showing higher Valence scores (M = 6.32, SD = 0.44) for Pleasant images than for Unpleasant images (M = 6.05, SD = 0.54). The Arousal scores exhibited a similar pattern where unpleasant images scored significantly (F = 5.89, p < 0.05) higher Arousal scores (M = 4.22, SD = 0.36) than pleasant images (M = 4.07, SD = 0.34). The dominance scores did not differ across the Valence and Mechanism (F < 1.7). Although the analysis provides an expected summary of the affective themes in the images, it was not sensitive to all culturally appropriate meanings in the images.

Measures

Questionnaire

Self-reports of emotions evoked were collected through a questionnaire prior to the laboratory experiment using 10 emotion concepts for each picture and music excerpt. The selection of concepts was based on factors that resulted from an analysis of both a pilot survey (N = 109) designed to explore the affective characteristics of everyday emotions of music and pictures, and previous studies on emotional responses to music71,72. Factor analyses were executed for music and pictures separately, and factors that occurred for both modalities were included in the present study. The factors were labeled as joy, strength, sadness, relaxation, tenderness, eroticism, melancholia, spiritualness, curiosity, and kinship. Participants were instructed to rate the felt intensity of each emotion on a 7-point Likert scale for each of the eight stimuli. High values are indicative of increased intensity of emotional experience (Appendix 1).

EEG recordings and analyses

64-channel EEG (BioSemi Active II amplifier system utilizing Active View 6.05 recording software) with a sampling rate of 500 Hz was recorded continuously using active Ag/AGCl electrodes (BioSemi Headcap). As per recommendations of the manufacturer, the electrode voltage offsets were kept below 25 mV. Analyses were executed with Brain Vision Analyzer software (Brain Products) and custom written Matlab scripts. Bad channels were removed and the remaining EEG signals were first re-referenced to the average of all channels and then band-pass filtered from 0.5 to 30 Hz. A segment of 60 s of spontaneous EEG from the early phase of the session was fed into independent components analysis (ICA) to recalculate the data, and minimize the effects of eye blinks and eye movements. ICA (Infomax algorithm) was set to produce as many components as there were channels. Among the first components, there were, in most cases, two components corresponding to eye blinks (clear frontal distribution) and lateral eye movements (sinks and source bilaterally in the anterior part of the head); their contribution to the data were calculated away. The obtained signal was then visually compared with the raw signal to verify that this procedure removed stereotypical artefacts.

The EEG signal was segmented based on the event types. Fast Fourier transformation was applied to an EEG epoch consisting the time window of 2–28 s after the onset of the stimulus. Fourier transformed signals were then averaged across event types. For further analyses, frontomedial (C1, C2, Cz, F1, F2, FC1, FC2, Fz), left frontal (C3, C5, F3, F5, F7, FC3, FC5, FT7) and right frontal (C4, C6, F4, F6, F8, FC4, FC6, FT8) electrode pools were formed by averaging the frequency distributions of these signals. The mean magnitude of theta (4–8 Hz) and alpha (9–13 Hz) frequency band activity was calculated for each participant during each event type. The data of two participants were excluded due to problems related to EEG-recordings.

Procedure

All experimental protocols were approved by the University of Jyvaskyla Ethics Committee. The methods were carried out in accordance with the ethical principles of research in the humanities and social and behavioural sciences defined by the National Advisory Board on Research Ethics in Finland (TENK). Informed consent was obtained from all participants. Participants were given a personal identification code after they provided the stimulus materials; with this code they received access to the questionnaire that was asked to be completed in the two days before neurophysiological measures. Upon arriving for the experiment, participants signed the consent form. They were then seated, facing a monitor. Audio stimuli were presented through stereo speakers and visual stimuli on a PC monitor. All the tasks and instructions were provided through e-Prime 2.0 Professional software operated by the researcher. EEG-data were collected using ActiView lab software. The experiment comprised eight blocks that contained 40-sec presentations of each stimulus type; within each block the order of stimulus presentation was randomized. After each 40-sec excerpt the participants reported orally the level of valence evoked by the stimuli to minimally disrupt ongoing EEG-measurement using scales ranging from −3 (extremely unpleasant) to 3 (extremely pleasant). Each stimulus type was evaluated eight times during the experiment.

Appendix

Research data is available at Harvard Dataverse open access https://doi.org/10.7910/DVN/ZZR7WX.

Results

The statistical analysis of the questionnaire data focused on main effects and interactions of the main variables. An overall view on how the particular emotions were rated throughout different conditions was first summarized by mean values. The lowest-scoring emotions (eroticism, spirituality, curiosity) were omitted from further analyses, except sadness, which is in the scope of interest of this study. The emotions relaxation (mean 3.29, SD 0.53), strength (mean 3.27, SD 0.67), joy (mean 3.23, SD 0.64), tenderness (mean 3.16, SD 0.65), kinship (mean 2.97, SD 0.90), melancholia (mean 2.94, SD 0.79), and sadness (mean 2.66, SD 0.69), were retained for further analyses. Then, a four-way MANOVA was executed for identifying significant main effects of all variables. A three-way MANOVA was then performed separately for each emotion to explore their main effects and interactions between mechanism, modality, and stimulus valence. EEG-data were analysed to verify the effect of induction mechanism on distinct conditions. Self-evaluations of stimulus valence during the EEG ratings were compared by the original ratings and means per factor between modalities and stimulus types. In addition, both stimulus types were analysed in terms of their acoustic, visual and semantic content that revealed the expected differences across valence (see Methods/Supporting Information).

Variable main effects and interactions

The repeated four-way MANOVA for modality, mechanism, valence, and emotion indicated a significant main effect for all variables with several variable interactions (Table 1). All two- and three-way interactions with emotion were significant, and thereby constituted grounds for exploring the results separately over singular emotions.

Table 1 Summary of ANOVA analyses across the four factors.

Main effects and interactions over specific emotions

Three-way MANOVA analyses for modality, mechanism, and valence were performed separately for each emotion. In the following sections emotions are presented in groups labeled as positive (joy, strength, relaxation), social (kinship, tenderness), and negative (sadness, melancholia) emotions. The thematic division is based on theoretical consideration of the functional roles of the emotions, for instance, social emotions relates to social connectedness and belonging37,38,39,40,73. The division was further supported by observations of similarities in how the interactions were distributed in the data.

Positive emotions

For joy, significant main effects were observed with modality (F (1, 29) = 5.24; p = 0.030), valence (F (1, 29) = 306.49; p < 0.001), and mechanism (F (1, 29) = 5.63; p = 0.024). Joy was experienced more to music than to pictures, more through memory than stimulus features, and more to pleasant than unpleasant stimuli. A significant interaction was observed for modality × valence (F (1, 29) = 4.56; p = 0.041), with the predominance of positively valenced stimuli as a source of joy being even clearer in music than in pictures. Significant interaction was also found for mechanism × valence (F (1, 29) = 5.66; p = 0.024) with the prevalence of positively valenced stimuli as the source of joy being even clearer when memories were the induction mechanism.

Considering strength, significant main effects of modality (F (1, 29) = 8.96; p = 0.006), and valence (F (1, 29) = 484.58; p < 0.001) were observed. Strength was experienced stronger when evoked by music, and more typically when related to positively valenced stimuli. Significant interactions were not found.

Over relaxation, a significant main effect of valence (F (1, 29) = 368.31; p < 0.001) was found, with relaxation being experienced more strongly with pleasant than unpleasant stimuli. A significant interaction of modality × mechanism (F (1, 29) = 4.67; p = 0.039) was also observed: memory was a more prevalent mechanism in inducing relaxation in musical than in pictorial stimuli. Mean ratings for positive emotions across modality, valence, and mechanism are depicted in Fig. 1.

Figure 1
figure1

Mean ratings across the Mechanisms for positive emotions (Joy, Strength, and Relaxation).

Social emotions

For kinship, significant main effects were found for valence (F (1, 29) = 135.16; p < 0.001), and mechanism (F (1, 29) = 26.77; p < 0.001): kinship was induced particularly by memories and pleasant stimuli. Significant interaction was found for modality × valence (F (1, 29) = 4.43; p = 0.044) with the prevalence of pleasant stimuli as the source of kinship being slightly more prominent for music than pictures. Significant interaction was also found for mechanism × valence (F (1, 29) = 11.8; p = 0.002), with the effect of memory inducing kinship being particularly clear when the stimuli were pleasant.

Regarding tenderness, significant main effects of valence (F (1, 29) = 264.86; p < 0.001), and mechanism (F (1, 29) = 19.10; p < 0.001) were observed: similarly to kinship, tenderness was particularly induced by memories and by pleasant stimuli. A significant interaction for modality × mechanism (F (1, 29) = 10.59; p = 0.003) showed that the prevalence of memory as the induction mechanism was more pronounced with musical than pictorial stimuli. Mean ratings for social emotions across modality, valence, and mechanism are depicted in Fig. 2.

Figure 2
figure2

Mean ratings across the Mechanisms for social emotions (Kinship and Tenderness).

Negative emotions

Significant main effects for sadness were found for all variables: modality (F (1, 29) = 6.24; p = 0.018), valence (F (1, 29) = 30.39; p < 0.001), and mechanism (F (1, 29) = 16,15; p < 0.001). The experience of sadness was generally more prevalent when evoked by pictorial stimuli, and when induced by memories rather than stimulus features. Negative stimuli compounded the occurrence of sadness. Significant interaction was found for mechanism × valence (F (1, 29) = 20.66; p < 0.001): the impact of memory was particularly prevalent in inducing sadness when the stimuli were unpleasant. A significant 3-way interaction of modality × mechanism × valence (F (1, 29) = 5.03; p = 0.033) indicated that when evoked by pleasant musical stimuli, sadness was induced predominantly through stimulus features, while when evoked by unpleasant musical stimuli or any visual stimuli sadness was prevalently induced by memories

With regards to melancholia, significant main effects of valence (F (1, 29) = 10.33; p = 0.003), and mechanism (F (1, 29) = 24.17; p < 0.001) were found. In contrast to sadness, melancholy was experienced more typically in the context of pleasant stimuli. Similar to sadness, memory was more effective than stimulus features in inducing melancholia. Significant interactions were found for modality × valence (F (1, 29) = 5.93; p = 0.021), mechanism × valence (F (1, 29) = 6.07; p = 0.020), and modality × mechanism × valence (F (1, 29) = 4.84; p = 0.036). In pictures, memory was more effective in inducing melancholia than stimulus features regardless of the stimulus’ valence. In contrast, for music, the effect of memory was clear only in the context of unpleasant stimuli. It thus appeared that the role of memory in inducing melancholia was particularly pronounced in unpleasant musical stimuli. Mean ratings for negative emotions across modality, valence, and mechanism are depicted in Fig. 3.

Figure 3
figure3

Mean ratings across the Mechanisms for negative emotions (Sadness and Melancholia).

Results of EEG-measures

The EEG data were utilized to substantiate the effects of emotion induction mechanisms observed in self-evaluations. A repeated-measures ANOVA was executed to examine the possible interactions between Modality, Valence, and Mechanism in the theta oscillations from frontal midline electrodes. In addition, the effects on laterality (left vs. right) on modality, valence and mechanism were explored by looking at the frontal alpha oscillations.

Frontomedial theta

The main results are largely in support of the behavioral results; The modality × valence × mechanism analysis yielded a significant main effect of mechanism (F (1,28) = 5.10; p < 0.05, \({\eta }_{p}^{2}\) = 0.154), indicating stronger theta activity during memory-related stimulus material, and a significant main effect of modality (F (1,28) = 39.93; p < 0.001; \({\eta }_{p}^{2}\) = 0.588), indicating more theta activity during picture viewing. No significant main effect of valence nor interactions were observed (Fig. 4).

Figure 4
figure4

Mean magnitude of theta frequency band of EEG activity.

Valence × mechanism interactions regarding music demonstrated no significant effects, but a trend (p = 0.062, \({\eta }_{p}^{2}\) = 0.119) for more theta activity during memory condition was found. Also, for pictures, valence × mechanism interactions did not cause a significant effect, but the observed significance level (p = 0.052, \({\eta }_{p}^{2}\) = 0.128) indicate that elevated levels of theta activity might be specifically linked to pleasant memorising.

Frontal alpha (central locations omitted in order to evaluate laterality effects)

For frontal alpha activity, the hemisphere × modality × valence × mechanism showed a significant main effect of valence (F (1,28) = 6.09; p < 0.05; p_eta2 = 0.18) indicating stronger alpha oscillations during pleasant events. Also, a significant interaction effect of modality × valence (F (1,28 = 6.22; P < 0.05; \({\eta }_{p}^{2}\) = 0.18), and location × mechanism (F (1,28) = 4.61; p < 0.05; \({\eta }_{p}^{2}\) = 0.14) was found, suggesting more alpha oscillations during music (Fig. 5). To further explore the laterality effect of valence, we ran separate analyses comparing F3-F4 and F7-F8 electrode pairs instead of pooled left vs. right signals; results showed no lateral effect on valence.

Figure 5
figure5

Mean magnitude of alpha frequency band of EEG activity.

For music, a significant main effect of valence occurred in location × valence × mechanism interaction (F (1,28) = 6.96; p < 0.05; \({\eta }_{p}^{2}\) = 0.20). There was also an location × valence interaction approaching significance (F (1,28) = 4.00; p = 0.055, \({\eta }_{p}^{2}\) = 0.13). Elevated levels of alpha activity might be specifically linked to lateralisation in specific frontal brain locations during pleasant music. To eliminate the possibility that alpha range oscillations occur due to participants having their eyes closed, the number of eye blinks during pleasant music were calculated (valence: p = 0.224; mechanism: p = 0.176; valence × mechanism: p = 0.461). The number of eye blinks remained similar throughout distinct conditions. In terms of the number of eye blinks, no differences were found between music and pictures suggesting that pleasant music may, indeed, cause relaxation. The repeated-measures ANOVA on location × valence × mechanism interactions during pictures did not show any significant effects.

Discussion

This study investigated how affective experiences to pleasant and unpleasant music and pictures vary if induced by personal memories or stimulus features. We particularly tested whether memory, in comparison to stimulus features, would play a major role in inducing different types of emotions (positive, negative, social), and whether this impact of memory would be similar in the case of both music and pictures, and pleasant and unpleasant stimuli. Confirming our hypothesis concerning the prevalence of memory as an induction mechanism, the results showed that the role of memory indeed is indeed central in inducing emotional experiences: stimuli chosen based on memories were more inductive of joy, kinship, tenderness, sadness, and melancholia than stimuli chosen based on the stimulus features. Only relaxation and strength did not show a main effect of mechanism. Overall, it can be concluded that memory plays an important role in the induction of all types of emotions, but this effect seems to be even more pervasive for the social and negative emotions in comparison to general positive emotions.

Main effects of modality on emotions were not expected, but we did observe that music led to slightly higher ratings of joy and strength, while pictures evoked more sadness than music. Concerning our exploration of the role of memory across the domains, the results generally confirmed that memory is the primary induction mechanism regardless of the stimulus domain. Yet, some distinct asymmetries related to the domain and emotion induction mechanism could also be observed. The pronounced role of memory as an induction mechanism was generally even more evident in music than in pictures. This could reflect the fact that music engagement is closely linked to everyday life events, as music can serve as the background soundtrack for a range of such events. Also, our assumption that memories would play a stronger role for inducing negative affects in music than in pictures received some support: for the induction of melancholia and sadness, memory contributed more highly to these emotions in comparison to stimulus features, and this was particularly clear in the case of unpleasant musical stimuli. Interestingly, the stimulus features were more inductive of sadness in the case of pleasant musical stimuli.

The neural measures confirmed the validity of the setting by indicating the presence of memory processing during the memory conditions: Increased theta oscillations that occurred during the memory condition bolster the earlier observations relating theta to memory tasks74. The low arousal pleasant experiences were demonstrated in neural responses as increased alpha oscillations in the memory condition. For instance, pleasant sadness is associated with social signals and functions of music75, and the function of distinct neural areas in contrast to those activated by psychological pain, such as posterior cingulate cortex, cerebellum, and parahippocampal gyrus76. In future research, however, the study design could capture the effects of observable stimulus features in more detail (e.g. lyrics, genre, familiarity). For further differentiating the types of pleasant and unpleasant experiences based on mechanism and modality, one appealing direction is to alter the conditions to have an effect of the function of particular emotions (e.g. social).

As with all research, the present findings need to be considered in the light of inherent limitations; as participants were exposed to their own choices of music and pictures, it might be argued that it is not the stimulus itself, but the memories associated with it that induced the emotions. More detailed information about the memories could further benefit in the analysis as well. On the one hand, we did not collect data on how much exposure the participants had to the stimuli beforehand, or whether they just thought about this while rating their emotions for the questionnaire. If the latter is true, such retrospective responses can be biased and unreliable. On the other hand, concurrent exposure (Cf.77) to the stimuli may create further memories related to the process of filling in the questionnaire, which might influence the experience in various ways (see74). To address this, one could for instance utilise the Fading Affect bias to normalize the effects of memories from the distant past78. Another option is to ask people to rate the same 4 popular songs and 4 popular pictures and indicate for each whether they have a memory associated with them. This way all participants are being exposed to the same stimuli and any confounds related to familiarity are minimised. One could also carry out correlational research where one predicts the intensity of emotions from scales expressing familiarity and memorability. In the present study, the relatively small sample size and bias in gender distribution restricts the generalization of the results beyond the current sample. While the exclusion of potential confounding variables certainly improves internal validity, further research with larger and more representative samples is certainly warranted before reaching any general conclusions.

Numerous unanswered questions remain in emotion research, particularly concerning mechanisms in general and the complex, social emotions. The present study took a novel step in investigating the interaction between emotion induction mechanisms, emotional valence, and stimulus modality by combining self-reports and objective measures (EEG) using self-selected stimuli. Overall, the present findings suggest that future research on complex emotions should consider the role of memories as a crucial element of the investigation. Deconstructing the central elements of the memories by focussing on familiarity, autobiographical salience, or timeframe of them might be the most effective strategy to pursue this topic (e.g.51). Another line of research is to explore in more detail the interactive effects of memories and stimulus properties in paradoxical affects such as the simultaneous experience of pleasantness and sadness. Furthermore, despite the inherent differences in the stimulus properties between images (spatial) and music (temporal), the concurrent use of multiple modalities offers exciting prospects to further our knowledge of affective experiences regardless of the domain.

Data Availability Statement

The datasets used in this study are published in Harvard Dataverse https://doi.org/10.7910/DVN/ZZR7WX.

References

  1. 1.

    Soto-Faraco, S. & Spence, C. Modality-specific auditory and visual temporal processing deficits. Quart J of Experim Psychol. 55, 23–14 (2002).

    Article  Google Scholar 

  2. 2.

    Juslin, P. N. From everyday emotions to aesthetic emotions: Towards a unified theory of musical emotions. Phys Life Rev. 10, 235–66, https://doi.org/10.1016/j.plrev.2013.05.008 (2013).

    ADS  Article  PubMed  Google Scholar 

  3. 3.

    Juslin, P. N. & Sloboda, J. A. (eds). Handbook of music and emotion: theory, research, applications. New York: Oxford University Press (2010).

  4. 4.

    Russell, J., Weiss, A. & Mendelsohn, G. Affect grid: A singleitem scale of pleasure and arousal. J. Pers Soc Psychol. 57, 493–502, https://doi.org/10.1037/0022-3514.57.3.493 (1989).

    Article  Google Scholar 

  5. 5.

    Oatley, K. Dacher, K. & Jenkins, J. M. Understanding Emotions (2nd ed.). Wiley-Blackwell.

  6. 6.

    Gross, J. J. Emoton regulation: taking stock and moving forward. Emotion. 13, 359–65, https://doi.org/10.1037/a0032135 (2013).

    Article  PubMed  Google Scholar 

  7. 7.

    Scherer, K. R. Emotion theories and concepts in The Oxford companion to emotion and the affective sciences (eds Sender, D. & Scherer, K. R.) 145–151 (New York Oxford University Press, 2010).

  8. 8.

    Blood, A. J. & Zatorre, R. J. Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. PNAS 98, 11818–11823, https://doi.org/10.1073/pnas.191355898 (2001).

    ADS  CAS  Article  PubMed  Google Scholar 

  9. 9.

    Gebauer, L., Kringelbach, M. L., Vuust, P., Cohen, A. J. & Stewart, L. Ever-changing cycles of musical pleasure: The role of dopamine and anticipation. Psychomusicology: Music, Mind, and Brain 22, 152–167, https://doi.org/10.1037/a0031126 (2012).

    Article  Google Scholar 

  10. 10.

    Salimpoor, V., Benovoy, M., Larcher, K., Dagher, A. & Zatorre, R. Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nat Neurosc. 14, 257–355, https://doi.org/10.1038/nn.2726 (2011).

    CAS  Article  Google Scholar 

  11. 11.

    Gomez, P. & Danuser, B. Relationships between musical structure and psychophysiological measures of emotion. Emotion. 7, 377–387, https://doi.org/10.1037/1528-3542.7.2.377 (2007).

    Article  PubMed  Google Scholar 

  12. 12.

    Grewe, O., Kopiez, R. & Altenmüller, E. Chills as an indicator of individual emotional peaks. ANAS 1169, 351–354, https://doi.org/10.1111/j.1749-6632.2009.04783.x (2009).

    Article  Google Scholar 

  13. 13.

    Guhn, M., Hamm, A. & Zentner, M. Physiological and musico-acoustic correlates of the chill response. Mus Perc. 24, 472–483, https://doi.org/10.1525/mp.2007.24.5.473 (2007).

    Article  Google Scholar 

  14. 14.

    Juslin, P. N. & Västfjäll, D. Emotional responses to music: The need to consider underlying mechanisms. Behav.Brain Sc. 31, 559–575, https://doi.org/10.1017/S0140525X08005293 (2008).

    Article  Google Scholar 

  15. 15.

    Panksepp, J. The emotional sources of “chills” induced by music. Mus Perc. 13, 171–207, https://doi.org/10.2307/40285693 (1995).

    Article  Google Scholar 

  16. 16.

    Rickard, N. S. Intense emotional responses to music: A test of the psychological arousal hypothesis. Psychol Mus. 32, 371–388, https://doi.org/10.1177/0305735604046096 (2004).

    Article  Google Scholar 

  17. 17.

    Cacioppo, J. T., Gardner, W. L. & Berntson, G. G. The affect system has prallel and integrative processing components: Form follows function. J. Pers Soc Psychol. 76, 839–55, https://doi.org/10.1037/0022-3514.76.5.839 (1999).

    Article  Google Scholar 

  18. 18.

    Frijda, N. H. The laws of emotion. Am Psychol. 43, 349–58, https://doi.org/10.1037/0003-066x.43.5.349 (1988).

    CAS  Article  PubMed  Google Scholar 

  19. 19.

    Larsen, R. J. & Prizmic, Z. Regulation of emotional well-being in The science of subjective well-being (eds Eid, M. & Larsen, R. J.) 258‒89 (Guilford Press, 2008).

  20. 20.

    Vaish, A., Grossmann, T. & Woodward, A. Not all emotions are created equal: The negativity bias in social- emotional development. Psychol Bul. 134, 383–403, https://doi.org/10.1037/0033-2909.134.3.383 (2008).

    Article  Google Scholar 

  21. 21.

    Huron, D. Sweet anticipation: Music and the psychology of expectation (Cambridge, MA: MIT Press, 2006a).

  22. 22.

    Juslin, P. N., László, H. & Eerola, T. What makes music emotionally significant? Exploring the underlying mechanisms. Psychol Mus. 42, 599–623 (2014).

    Article  Google Scholar 

  23. 23.

    Hume, D. Of tragedy in Four Dissertations, 185–200 (A. Millar, 1757).

  24. 24.

    Eerola, T. & Vuoskoski, J. K. A comparison of the discrete and dimensional models of emotion in music. Psychol Mus. 39, 18–49, https://doi.org/10.1177/0305735610362821 (2011).

    Article  Google Scholar 

  25. 25.

    Garrido, S. & Schubert, E. Moody melodies: Do they cheer us up? A study of the effect of sad music on mood. Psychol Mus. 43, 244–261, https://doi.org/10.1177/0305735613501938 (2015).

    Article  Google Scholar 

  26. 26.

    Bonnano, G. A., Goorin, L. & Coifman, K. C. Sadness and Grief in Handbook of Emotions (eds Lewis, M. J, Haviland-Jones, M. & Barrett, L. F.) 3rd ed., 797–810 (New York: The Guilford Press, 2008).

  27. 27.

    Peltola, H. R. & Eerola, T. Fifty Shades of Blue: Classification of music-evoked sadness. Mus Sc. 20, 84–102, https://doi.org/10.1177/1029864915611206 (2016).

    Article  Google Scholar 

  28. 28.

    Menninghaus, W. et al. The distancing-embracing model of the enjoyment of negative emotions in art reception. Behav Brain Sci. 40, https://doi.org/10.1017/S0140525X17000309 (2017).

  29. 29.

    Kensinger, E. A. Remembering emotional experiences: The contribution of valence and arousal. Rev Neurosc. 15, 241–251 (2004).

    Article  Google Scholar 

  30. 30.

    Phelps, E. A., Ling, S. & Carrasco, M. Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol. Sc. 17, 292–299 (2006).

    Article  Google Scholar 

  31. 31.

    Thompson, W. F., Greeves, A. M. & Olsen, K. N. Who enjoys listening to violent music and why? Psychol. Popular Media Culture. Advance online publication, https://doi.org/10.1037/ppm0000184 (2018).

  32. 32.

    Gerger, G., Pelowski, M. & Leder, H. Erratum to: Empathy, Einfühlung, and aesthetic experience: The effect of emotion contagion on appreciation of representational and abstract art using fEMG and SCR. Cognitive Processing. 19, https://doi.org/10.1016/j.actpsy.2014.06.008 (2017).

    Article  Google Scholar 

  33. 33.

    Scherer, K. R. Appraisal theories in Handbook of cognition and emotion (eds Dalgleish, T. & Power, M.) 637–663 (Chichester, UK: Wiley, 1999).

  34. 34.

    Scherer, K. R. & Zentner, M. R. Emotional effects of music: Production rules in Music and emotion: Theory and research (eds Juslin, P.N. & Sloboda, J. A.) 361–392 (Oxford, UK: Oxford University Press, 2001).

  35. 35.

    Juslin, P. N., Liljeström, S., Västfjäll, D. & Lundqvist, L.-O. How does music evoke emotions? Exploring the underlying mechanisms in Handbook of music and emotion: Theory, research, applications (eds Juslin, P. N. & Sloboda, J.) 605–642 (New York: Oxford University Press, 2010).

  36. 36.

    Juslin, P. N. Barradas, G. & Eerola, T. From sound to significance: Exploring the mechanisms underlying emotional reactions to music. Am J Psychol. 128, 281–304 (2015).

    Article  Google Scholar 

  37. 37.

    Huron, D. Is music and evolutionary adaptation? In The origins of music (eds Wallin, N.L., Merker, B. & Brown, S.) 57–75 (Cambridge, MA: MIT Press 2006b).

    Google Scholar 

  38. 38.

    Fitch, T. The biology and evolution of music: A comparative perspective. Cognition 100, 173–215, https://doi.org/10.1016/j.cognition.2005.11.009 (2006).

    Article  PubMed  Google Scholar 

  39. 39.

    Clayton, M. The social and personal functions of music in cross-cultural perspective in The Oxford handbook of music psychology (eds Hallam, S., Cross, I., Thaut, M.) 35–44 (Oxford: Oxford University Press 2010).

  40. 40.

    Ruud, E. Music and the quality of life. Nord J Music Ther. 6, 86–97, https://doi.org/10.1080/08098139709477902 (1997).

    Article  Google Scholar 

  41. 41.

    Tiihonen, M., Brattico, E., Maksimainen, J., Wikgren, J. & Saarikallio, S. The constituents of music and visual-art related pleasure – A critical integrative literature review. Frontiers in Psychology, Published Online, Jul 20 (2017).

  42. 42.

    Rottenberg, J., Ray, R. D. & Gross, J. J. Emotion elicitation using films in Handbook of emotion elicitation and assessment (eds Coan, J. A. & Allen, J. J. B.) 9–28 (New York, NY: Oxford University Press, 2007).

  43. 43.

    Lang, P. J., Greenwald, M. K. & Bradely, M. M. Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophys. 30, 261–273 (1993).

    CAS  Article  Google Scholar 

  44. 44.

    Gross, J. J. & Levenson, R. W. Emotion elicitation using films. Cog Emot. 9, 87–108 (1995).

    Article  Google Scholar 

  45. 45.

    Lang, P. J., Bradley, M. M. & Cuthbert, B. N. International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual. Technical Report no A-6. Gainseville: University of Florida (2005).

  46. 46.

    Westermann, R., Spies, K., Stahl, G. & Hesse, F. Relative effectiveness and validity of mood induction procedures: A meta-analysis. Eur J Soc Psychol. 26, 557–580 (1996).

    Article  Google Scholar 

  47. 47.

    Berenbaum, H. Varieties of joy-related pleasurable activities and feelings. Cog Emot. 16, 473–94, https://doi.org/10.1080/0269993014000383 (2002).

    Article  Google Scholar 

  48. 48.

    Dubé, L. & Le Bel, J. L. The content and structure of laypeople’s concept of pleasure. Cog Emot. 17, 263–95, https://doi.org/10.1080/02699930244000309 (2003).

    Article  Google Scholar 

  49. 49.

    Knobloch-Westerwick, S. & Keplinger, C. Mystery appeal: Effects of uncertainty and resolution on the enjoyment of mystery. Media Psychol. 8, 193–212, https://doi.org/10.1207/S1532785xmep0803_1 (2006).

    Article  Google Scholar 

  50. 50.

    Scherer, K. R. What are emotions? And how can they be measured? Soc. Sc. Inf. - Sur Les Sciences Sociales 44, 695–729, https://doi.org/10.1177/0539018405058216 (2005).

    Article  Google Scholar 

  51. 51.

    Routledge, C., Wildschut, T., Sedekides, C., Juhl, J. & Arndt, J. The power of the past: Nostalgia as a meaning-making resource. Memory. 20, 452–60, https://doi.org/10.1080/09658211.2012.677452 (2012).

    Article  PubMed  Google Scholar 

  52. 52.

    Barrett, F. S. et al. Music-evoked nostalgia: Affect, memory, and personality. Emotion. 10, 390–403, https://doi.org/10.1037/a0019006 (2010).

    Article  PubMed  Google Scholar 

  53. 53.

    Bullot, N. J. & Reber, R. A psycho-historical research program for the integrative science of art. Behav Brain Sc. 36, 163–80, https://doi.org/10.1017/S0140525X12002464 (2013).

    Article  Google Scholar 

  54. 54.

    Chatterjee, A. The aesthetic brain: How we evolved to desire beauty and enjoy art (Oxford University Press, 2013).

  55. 55.

    Jacobsen, T. Bridging the arts and sciences: A framework for the psychology of aesthetics. Leonardo 39, 155–62, https://doi.org/10.1162/leon.2006.39.2.155 (2006).

    Article  Google Scholar 

  56. 56.

    Leder, H., Belke, B., Oeberst, A. & Augustin, M. D. A model of aesthetic appreciation and aesthetic judgments. Brit J. Psychol. 95, 489–508, https://doi.org/10.1348/0007126042369811 (2004).

    Article  PubMed  Google Scholar 

  57. 57.

    Benedek, M. & Kaernbach, C. Physiological correlates and emotional specificity of human piloerection. Bio. Psychol. 86, 320–29, https://doi.org/10.1016/j.biopsycho.2010.12.012 (2011).

    Article  Google Scholar 

  58. 58.

    Goldstein, A. Thrills in response to music and other stimuli. Phys Psychol. 8, 126–29, https://doi.org/10.3758/BF03326460 (1980).

    Article  Google Scholar 

  59. 59.

    Grewe, O., Nagel, F., Kopiez, R. & Altenmuller, E. Listening to music as a re-creative process: Physiological, psychological, and psychoacoustical correlates of chills and strong emotions. Mus Perc. 24, 297–314, https://doi.org/10.1525/mp.2007.24.3.297 (2007).

    Article  Google Scholar 

  60. 60.

    Salimpoor, V. N., Benovoy, M., Longo, G., Cooperstock, J. R. & Zatorre, R. J. The rewarding aspects of music listening are related to degree of emotional arousal. PLoS ONE 4, e7487, https://doi.org/10.1371/journal.pone.0007487 (2009).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  61. 61.

    Eerola, T. & Peltola, H.-R. Memorable experiences with sad music - Reasons, reactions and mechanisms of three types of experiences. PLoS One 11, e0157444 (2016).

    Article  Google Scholar 

  62. 62.

    Wagner, V., Menninghaus, W., Hanich, J. & Jacobsen, T. Art schema effects on affective experience: The case of disgusting images. PACA 8, 120–29, https://doi.org/10.1037/a0036126 (2014).

    Article  Google Scholar 

  63. 63.

    Goldstein, T. R. The pleasure of unadulterated sadness: Experiencing sorrow in fiction, nonfiction, and “in person”. PACA 3, 232–37, https://doi.org/10.1037/a0015343 (2009).

    Article  Google Scholar 

  64. 64.

    Israeli, N. Affective reactions to painting reproductions: A study in the psychology of esthetics. J Appl Psychol. 12, 125, https://doi.org/10.1037/h0070445 (1928).

    Article  Google Scholar 

  65. 65.

    Rowold, J. Instrument development for esthetic perception assessment. J Media Psychol. 20, 35–40, https://doi.org/10.1027/1864-1105.20.1.35 (2008).

    Article  Google Scholar 

  66. 66.

    Hagtvedt, H., Hagtvedt, R. & Patrick, V. M. The perception and evaluation of visual art. Empir Stud Arts. 26, 197–218, https://doi.org/10.2190/EM.26.2.d (2008).

    Article  Google Scholar 

  67. 67.

    Hager, M., Hagemann, D., Danner, D. & Schankin, A. Assessing aesthetic appreciation of visual artworks - The construction of the Art Reception Survey (ARS). PACA. 6, 320–33, https://doi.org/10.1037/a0028776 (2012).

    Article  Google Scholar 

  68. 68.

    Lartillot, O., Toiviainen, P. & Eerola, T. A Matlab toolbox for music information retrieval. http://citeseerx.ist.psu.edu, 10.1.1.706.2450 (2007).

  69. 69.

    Eerola, T. Are the motions expressed in music genre-specific? An audio-based evaluation of datasets spanning classical, fil, pop and mixed genres. J. New Mus Res. 40, 394–366 (2011).

    Google Scholar 

  70. 70.

    Warriner, A. B., Kuperman, V. & Brysbaert, M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav Res Methods 45, 1191–1207 (2013).

    Article  Google Scholar 

  71. 71.

    Juslin, P. N. & Laukka, P. Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. J. New Mus Res. 33, 217–238 (2004).

    Article  Google Scholar 

  72. 72.

    Zentner, M., Grandjean, D. & Scherer, K. R. Emotions evoked by the sound of music: Characterization, classification, and measurement. Emotion 8, 494–521, https://doi.org/10.1037/1528-3542.8.4.494 (2008).

    Article  PubMed  Google Scholar 

  73. 73.

    Leary, M. R. Affect, cognition, and the social emotions in Feeling and Thinking: The Role of Affect in Social Cognition (eds Forgas, J. P., Oatley, K., Manstead, A.) 331–356 (Cambridge University Press 2005).

  74. 74.

    Hsieh, L. T. & Ranganath, C. Frontal midline theta oscillations during working memory maintenance and episodic encoding and retrieval. Neuroim. 85, 721–729, https://doi.org/10.1016/j.neuroimage.2013.08.003 (2014).

    Article  Google Scholar 

  75. 75.

    Koelsch, S. et al. The roles of superficial amygdala and auditory cortex in music-evoked fear and joy. Neuroim. 81, 49–60, https://doi.org/10.1016/j.neuroimage (2013).

    Article  Google Scholar 

  76. 76.

    Meerwijk, E. L., Ford, J. M. & Weiss, S. J. Brain regions associated with psychological pain: implications for a neural network and its relationship to physical pain. Brain Im Behav. 7, 1–14, https://doi.org/10.1007/s11682-012-9179-y (2013).

    Article  Google Scholar 

  77. 77.

    Rivas, C. R. et al. Seeing music: The perception of melodic ‘ups and downs’ modulates the spatial processing of visual stimuli. Neuropsychol. 117, 67–74, https://doi.org/10.1016/j.neuropsychologia.2018.05.009 (2018).

    Article  Google Scholar 

  78. 78.

    Ritchie, T. D., Skowronski, J. J., Hartnett, J., Wells, B. & Walker, W. R. The fading affect bias in the context of emotion activation level, mood, and personal theories of emotion change. Memory 17, 428–444 (2009).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Kone Foundation (grant no. 331d5b).

Author information

Affiliations

Authors

Contributions

Study design: J.W., T.E., J.M., S.S. Ethical approval: S.S., J.M. Data collection – E.E.G.: J.M. Data collection - self-evaluations: J.M., S.S. Data curation: J.W., J.M. Data analysis: J.W., S.S., J.M. Analysis of musical & visual materials: T.E. Data publishing – T.E., J.W., J.M. Writing - original draft: J.M., J.W., T.E., S.S.

Corresponding author

Correspondence to Johanna Maksimainen.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Maksimainen, J., Wikgren, J., Eerola, T. et al. The Effect of Memory in Inducing Pleasant Emotions with Musical and Pictorial Stimuli. Sci Rep 8, 17638 (2018). https://doi.org/10.1038/s41598-018-35899-y

Download citation

Keywords

  • Emotion Induction
  • Stimulus Features
  • Personal Memories
  • Induction Mechanism
  • Social Emotions

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing