Introduction

User experience mapping (UEM) is an analysis tool of the relationships among users, products, and the environment in terms of requirements (Li et al., 2019). It is increasingly being utilized to capture user experience requirements (Mark et al., 2017). UEM is widely used in cultural and creative product design (Wang and Xu, 2021), interactive interface design (Li et al., 2017), directional sign design (Yang et al., 2018; Wei and Ren, 2020), and other domains.

In order to conduct design research that is more focused, many academics innovate by analyzing UEM for specific research objects. (Wu and Li, 2018: Lee and Park, 2016) visualized user experience maps through data modeling and information analysis. (Yang et al., 2018; Li et al., 2019) prioritized user needs through user experience map methods combined with the Kano model and hierarchical analysis methods. They all proposed an effective user experience map innovation method for mining user needs. However, further studies are needed to figure out how to accurately express and quantify users’ emotional needs.

In this study, the automobile human–machine interface (HMI) was taken as the research object. A quantitative user experience map was proposed by combining a user experience map with Kansei Engineering to quantify drivers’ emotions. Users’ emotional needs could be accurately expressed and quantified using this method. Firstly, the semantic differential (SD) method was used to investigate users’ emotional perceptions of the automobile interface through a questionnaire. Secondly, user emotion curves were drawn to obtain the design pain points of the interface. Finally, design pain points were identified and iterative design was carried out. A validation study was implemented to ensure the method’s validity. The study provides a method for design iteration to improve user emotional experience.

Literature review

HMI structure

HMI has recently emerged as a global hub for vehicle engineering research and a new force behind the expansion of the automotive industry. They are currently a part of the intelligent transportation systems in many developed countries. “Intelligent” and “emotional” have emerged as major trends in the development of automobiles. The “Automotive User Experience 2020 Design Trend Report” (Huawei UCD Center, 2020) mentioned that smart cars will be developed with “intelligence”, “interaction” and “emotional experience” as the core in the future. The interactive mode and interface effects displayed in the car will furtherly affect the driver’s experience. The three modes of functional layout, visual effects, and interactive behavior of the smart car’s HMI are linked to user emotions. The emotional design of the car interface is mainly reflected in the driver’s five senses and the car’s interior environment. The driver’s emotional experience will be enhanced by improvements in touch, perception, vision, hearing, and smell. The in-vehicle HMI mainly involves the driver’s vision, operation, and hearing. The literature involves voice assistance (Julien et al., 2015), icon clicks (Tao et al., 2018), and visual perception (Talia et al., 2010) in the driver’s five senses. Based on the view that the pursuit of complexity and careful management of complexity is essential for the embodiment of the Norman emotional experience demand, Tan et al. (2015a) evaluated the usability of the service model and its user interface in the smart car music system. In the same year, Tan et al. (2015b) mentioned the usability evaluation tool of an automotive HMI and studied the emotional experience of automotive HMI in “Automotive HMI Design”. Zhong (2018) established a model of perceivable characteristics of automotive HMI products users’ emotional feeling-satisfaction through the James-Lange theory. Kim et al. (2015) proposed that the in-vehicle interface combines multiple modes such as gestures and voice to enhance the user’s emotional experience.

Kansei engineering

Professor Mitsuo Nagamachi of Hiroshima University, Japan, proposed Kansei Engineering. Kansei Engineering accurately grasps people’s “Kansei” or emotion toward products, establishes the connection between “Kansei”, and transforms users’ perceived needs, which are difficult to quantify into product design elements (Yuan et al., 2017). With the development of the Internet, Kansei Engineering based on user needs has gradually shifted from the appearance of physical objects to analyzing the virtual interaction interface of products. The quantification of emotions has become an important method for mining user interaction interface requirements. Zhou et al. (2013) analyzed the Kansei imagery structure of interface esthetics, proposed esthetic indicators, constructed an interface esthetics computing system, and quantified the esthetic emotional experience of the interface. Cao and Wan (2020) used Kansei Engineering’s quantitative reasoning to establish a mapping relationship between the elements of interface design and the perceived needs of users. Ren et al. (2021) proposed a task-structure-iterative fusion strategy that makes use of structural decomposition to examine the layout of the initial operating interface and create mapping relationships between modules and design points for iterative design.

One of the most fundamental methods for emotion quantification in perceptual engineering is the semantic differential method (SD method), which can objectively measure the emotional content of a vocabulary. A set of adjectives and their antonyms are placed at the left and right ends of an attitude scale using the SD method. It has a total of seven intervals, with choice 1 indicating the most tendency toward negative affective perception, choice 7 indicating the most tendency toward positive affective perception, and choice 4 indicating a neutral attitude. Using the SD method, this study created a 7-level Likert scale and a questionnaire to measure users’ affective perceptions of the in-vehicle HMI.

User experience map

In service design, user experience map (UEM) is a useful emotion description tool. It investigates the change of satisfaction during the experience through the subjective feeling of users. UEM can analyze the change in a user’s emotional experience during the use of the interface, and it describes the user’s experience emotion and user satisfaction in stages during the complete experience in chronological order, oriented to the user’s needs (Wei and Wu 2019). The user experience map is mainly composed of several elements: user behavior, behavioral touchpoints, user emotions, design pain points, and design opportunity points. By analyzing each constituent element in the experience phase to understand user needs, user emotions are visualized as curves to find design pain points (Wu et al., 2019). Tan et al. (2015) mentioned the combination of automotive products and experience process in “Automotive HMI Design”, and in a subsequent study, they proposed the experience model of an in-vehicle music system and its user interface usability based on Norman’s emotional experience needs.

In this paper, a user experience map was combined with Kansei Engineering, and user satisfaction curves were drawn based on the mean value of Kansei imagery word evaluation on each behavioral touchpoint of the users.

Through the literature review, the theoretical and applied research on the emotional design of automotive interfaces is still at the stage of continuous subdivision and exploration. From the perspective of the emotional experience process, there is still much interspace for research on the accurate expression and precise quantification of emotion in smart cars. The design method of the user experience map can present the changes in user emotion perception. Through the study of behavioral and emotional experiences of experience touchpoints, design pain points and opportunity points can be found, helping designers to identify design problems. However, in practical applications, there are some shortcomings in the accuracy of the emotional expression of user experience maps and the accuracy of the emotional design imagery words extracted during the experience. As a result, the following research issues are addressed in this study.

· Is the quantitative user experience map effective for automotive HMI?

· Does continuous emotion quantification help improve visual perception and functional layout effects?

The accurate description of the emotional touchpoints in the user experience map was improved by filtering emotional vocabulary of touchpoints through Kansei Engineering. Additionally, the expression accuracy of emotional change process was improved by quantitatively scoring the user experience process touchpoints. The improved user experience map provides designers with a reliable guarantee for design iteration by accurately finding design pain points. It also provides an evaluation method for iterative design validation. The original and improved user experience maps are shown in Figs. 1 and 2, respectively.

Fig. 1: User experience map.
figure 1

The figure shows a commonly used user experience map. This figure is not covered by the Creative Commons Attribution 4.0 International License. Reproduced with permission of the author; copyright © Faren Huo, all rights reserved.

Full size image
Fig. 2: Improved user experience map.
figure 2

The figure shows an improved user experience map by filtering the emotional vocabulary of the user interface experience process touchpoints through Kansei Engineering. The picture shows a commonly used user experience map This figure is not covered by the Creative Commons Attribution 4.0 International License. Reproduced with permission of author; copyright © Faren Huo, all rights reserved.

Full size image

Methods

Design 1

Through user action trajectory, user behaviors are divided into three stages, namely observation experience, interface level experience, and overall experience evaluation. The pre-stage process (observation experience) is to open the car driving interface and observe the style, color, and layout of the home interface. The middle process (interface level experience) is to click on each interface and feel and observe the visual effect and functional layout of the secondary interface. The latter process (overall experience evaluation) involves finishing the process of operating the interface and evaluating the overall feeling of using it.

Participants

During the preparation stage of experimental materials, we assembled a six-member design team. All of them had rich professional knowledge and design experience. Based on the collected interface samples, they discussed and screened the Kansei imagery word pairs to ensure the validity and professionalism of experimental materials. In the later questionnaire phase, participants aged from 20 to 45 years old were selected. All participants had driving experience and used the in-vehicle HMI. Prior to the questionnaire, each participant was asked to sign the informed consent and paid for their participation.

Materials

The experimental materials mainly included the samples of in-vehicle HMI collected from various channels and the research target sample, Geely Star HMI (home page and secondary interface). The in-vehicle HMI included parts of the entertainment system (multimedia), navigation system, communication system, car adjustment system, and general settings. As a result, as shown in Fig. 3, the interface design objects primarily selected the main page of the HMI, the basic setting interface, the main air conditioning interface, and adjustment interface, the main navigation interface and its sub-interfaces, and the main multimedia interface and its sub-interfaces.

Fig. 3: Smart car central control interface before revision.
figure 3

The figure shows Geely Star central control interface used in the test for improvement. This figure is not covered by the Creative Commons Attribution 4.0 International License. Reproduced with permission of the author; copyright © Geely Auto.

Full size image

Procedure

Collecting samples of intelligent vehicle central control interface

18 sample images of in-vehicle HMI were collected through literature, websites, and field photos. To ensure the validity and usability of the test, a screening procedure was performed by removing the samples with low clarity and high similarity. Finally, 10 representative interface design samples were selected.

Collecting Kansei imagery word

In Kansei engineering studies, Kansei imagery words are normally extracted by users’ perception of product design features, by way of brainstorming (Nagamachi, 1995, 2002), interviews (Luo et al., 2012). A method for kansei words selection based on a literature review of previous kansei engineering studies was proposed in Wang et al. (2018). Based on the general methods previously studies used, 196 Kansei imagery words in Chinese for in-vehicle HMI were collected through books, magazines, related websites, literature, and interviews with vehicle users.

Screening Kansei imagery word

Firstly, cluster analysis was conducted to exclude Kansei imagery words with similar meanings. Secondly, the Kansei imagery words that are more literal and used to describe the interface style, layout function, and overall perception in the three experience steps were filtered out. Finally, 61 Kansei imagery words were identified.

The 61 imagery words used to describe the Kansei imagery were rather vague due to the wide range of initial screening. In-vehicle HMI design characteristics should be represented by selecting imagery words from a second round of screening. The design theme of the Geely Star commercial is “futuristic, intelligent, technological, new standard”. The above-mentioned Kansei imagery words close to the theme were integrated and summarized, and 20 Kansei imagery words were screened out.

Formation of the final Kansei imagery word pairs

The 20 Kansei imagery words in Chinese were once more screened using the KJ method, and the sample was divided into four groups. The KJ method is a means of identifying key points by categorizing them according to shared characteristics through discussions with designers and users. They completely removed the problems and views in the collected Kansei imagery words utilizing their professional knowledge and design expertize. A focus group of six experts with backgrounds in design and driving experience was assembled for this study. Firstly, they were asked to conduct a tendency count of the Kansei imagery words for in-vehicle HMI design. Next, a mapping relationship between the Kansei imagery words and the behavioral touchpoints of user experience was formed. The emotional design touchpoints of the HMI included the color, style, and layout of the home page and secondary interface. The ten Kansei imagery words were matched with their antonyms to finally form ten pairs of Kansei imagery word pairs, as shown in Table 1.

Table 1 Perceptual imagery words of smart car central control interface.
Full size table

Questionnaire design

The questionnaire consisted of basic user information and a scale of Kansei imagery words derived from the sematic differential method. Each interface was evaluated by a corresponding seven-point scale from minus three to three. For each interface in the first two stages, the Kansei imagery word pairings were balanced-unbalanced, simple-complex, technological-outdated, compact-decentralized, clear-vague, colorful-uniform in color. The final Kansei imagery word pairs for the overall perception stage were decentralized-compact, balanced-unbalanced, clear-vague, convenient-difficult, simple-complex, calming-restless, technological-outdated, cold-warm and individual-ordinary.

Data acquisition

163 questionnaires were collected following the semantic differential method questionnaire survey of the interface sample. 161 valid questionnaires were obtained after the questionnaires with similar scale selections were excluded.

According to Table 2, the reliability coefficient of the questionnaire data was 0.741 higher than 0.7, indicating that the data’s quality was high and could be further analyzed.

Table 2 Cronbach reliability analysis.
Full size table

KMO and Bartlett’s tests were used to evaluate the questionnaire’s validity. The KMO value was 0.919, which was greater than 0.6, and the p of Bartlett’s sphericity test was 0.000 < 0.050, indicating that the questionnaire had a validity structure, as shown in Table 3.

Table 3 Validity analysis.
Full size table

The mean value was obtained by calculating the Kansei imagery words on the scale, which represented the user satisfaction of each touchpoint, and the quantitative graph of user experience was drawn, as shown in Fig. 4.

Fig. 4: Before the revision, the smart car central control interface user experience map score statistics results.
figure 4

The figure shows the calculating results of the Kansai imagery words on the scale, which represented the user satisfaction of each touchpoint, and the quantitative graph of user experience before revision. This figure is not covered by the Creative Commons Attribution 4.0 International License. Reproduced with permission of the author; copyright © Faren Huo, all rights reserved.

Full size image

Result

Interface visual perception

The quantitative user experience map created using scale score statistics makes it plain that there is significant variability and fluctuation in the scores for each touchpoint. Among the emotional touchpoints under the satisfaction dividing line, the lowest value is the multimedia interface. The mismatch between its color scheme, product positioning, and customers’ emotional need for the multimedia interface is increasingly pronounced. Overall, the low emotional touchpoints for users indicate that the studied interface is inconsistent with the car theme and lacks a sense of technology. This is followed by inconsistent colors and a lack of personalization.

Interface function layout

The pain point analysis results demonstrate that users are less emotionally satisfied than is typical as a result of the complexity and irrationality of the interface function layout. The main page icon layout is not eye-catching enough and has the lowest evaluation value among the interface function layouts. The user satisfaction scores are higher than evaluation values for the simplicity of the main interface, the air conditioning adjustment interface, and the multimedia interface layout. The interval distance between icons in the air conditioning interface makes the layout of the whole air conditioning adjustment interface more spacious.

To sum up, the user’s emotional satisfaction with Geely Starry’s HMI design could be summed up as follows. 9 out of 41 pain points are performing below average. The analysis results mainly concentrated on these 9 pain points are shown in Table 4.

Table 4 Specific analysis of design pain points.
Full size table

Iterative design

In the iterative design, interviews were conducted with users consisting of designers and design professionals. The interviewed users suggested changes to design pain points and design themes, which were aggregated and analyzed to inform the redesign. Combining the opinions of the ten interviewees, Tania Schlatter & Deborah Levinson’s (2013) interface visual principles (consistency, hierarchy, and personalization), and the designers’ own design experience, several design pain points were redesigned in terms of interface visuals and functional layout.

Design analysis

In terms of visual perception of the interface, the pain points were analyzed around the automotive theme using Kansei Engineering. When an obvious emotional low point is presented in the color, the initial main color scheme impression is obtained by pointing the technological sense style to the blue tone according to “Research on color scheme and strategy and its application in user interface” (Hu, 2015). Finally, the final color scheme is determined according to the color matching strategy in coordinated color balance and similar color management-related theories.

For the color, technology, and personalization of touchpoint 2, touchpoint 3 and touchpoint 9 (P2, P3, and P9), the designer extracted the main color of this model through product positioning, reduced its saturation, increased the sense of technology, formed the color tone of the whole HMI, and then enhanced the personalization of the overall interface through the independent design of functional color blocks.

In terms of interface functional layout, according to the driving visual interference theory and Sun Bowen et al.’s “Study on the layered design of in-vehicle HMI”, the layered design of touch interaction needs to follow the three principles of logic, descendance, and suggestibility, and combine with the simple requirements of pain point analysis (Sun et al., 2019). The iterative layout design was carried out based on the continuation of the functional layout of the original interface version. In response to the problem of an overly complex main interface raised by the low-rated touchpoints 1 and 4 (P1 and P4), the designer refined and simplified the functional division boundary of the main page. The functional layout was divided into large color blocks to reduce cognitive difficulties during driving. Through previous questionnaires on users’ functional needs, the navigation system with the greatest user demand will occupy a larger module area. The rest functions will be sorted by color block size according to a level of demand. The rest auxiliary touchpoints were mainly focused on the layout, and the overall layout was simply designed to reduce driving burden and fatigue.

Through the design analysis of the visual effect and functional layout, the interface design was improved by combining the design theme of the car research object. The design effect was shown in Fig. 5 as follows.

Fig. 5: Smart car central control interface after revision.
figure 5

The figure shows Geely Star central control interface used in the test after revision. This figure is not covered by the Creative Commons Attribution 4.0 International License. Reproduced with permission of the author; copyright © Faren Huo, all rights reserved.

Full size image

Design 2—Validation test of improved in-vehicle HMI

Participants

Participants were selected who did not overlap Experiment 1, including professional designers working on interface design, and experienced vehicle users aged from 20 to 45. They were given the post-design optimization evaluation questionnaire. All participants signed an informed consent form prior to the questionnaire and paid a fee for their time.

Materials

The improved interface was furtherly validated with a questionnaire, to verify whether the design approach was effective in improving the user affective experience of the in-vehicle HMI. Each improved interface and corresponding pair of Kansei imagery words were rated by the 7-point Likert semantic differential scale in the questionnaire. Each Kansei imagery word pair was graded on a scale based on the participants’ emotional experience.

Data acquisition

For the 109 questionnaires that were received, the data were processed (see Supplementary Questionnaire). The semantic differential method was used to determine the mean ratings for each sector (before and after improvement). 108 valid questionnaires were obtained, excluding those with inadequate response times. Statistical descriptions of evaluation scores on before and after revisions were presented in Table 5. A paired t-test was made to clarify the difference on level 0.01 and 0.05, respectively.

Table 5 Evaluation results of before and after revisions.
Full size table

Results

As Table 5 showed, most touchpoints showed higher values under the perceptual imagery word mapping than they did before the redesign. The graph demonstrates that the contacts in the setting interface that correspond to “technologically-outdated” have undergone the greatest improvement. As determined by the design evaluation method combining user experience mapping and perceptual engineering, the average value of the emotional experience for each touchpoint is higher than before the redesign. The scores of perceptual imagery words like “technological” and “simple” in the interface layout were significantly higher than before the redesign in terms of visual effects. In addition, the scores for “technological-outdated,” “conspicuous-vague,” and “simple-complex” in the theme of “technology, future, intelligent, new standard” were all higher than those in the previous version. The graphs demonstrate that the revised interface design received higher ratings than the previous version for the nine previously suggested touchpoints, achieving the desired effect.

Discussion

The main purpose of this study is to construct a quantitative map of user emotional experience based on Kansei Engineering and verify its feasibility and validity. Nine of the 41 touchpoints of each level of the Geely Starry car’s interface were evaluated with satisfaction levels below the average in the study. The design effect was significantly improved by the iterative design of the low-satisfaction experience touchpoints in combination with relevant principles of interface visual design. The layout and style of the multimedia interface are followed by the style of the settings interface as the most significant effect. According to the results of comparative study, it can be verified that the method of quantitative evaluation of automotive human–machine interface by quantitative user experience map is effective.

The quantitative user experience map was effective for quantifying emotional experience

The subjects in Experiment 1 presented clearer pain points in the evaluation of emotional satisfaction with the visual effects of the interface under each touchpoint. The description of the Kansei imagery word in terms of style and color of each interface provided better design directions for designers.

In Experiment 1, the user questionnaire for visual perception and functional layout of the interface was used to create a quantitative curve representing the user’s emotional experience, and it was found that 9 touchpoints fall short of the average. Targeted iterative design was carried out in experiment 2 according to the quantitative results of Experiment. And the quantification was verified again. The quantitative results showed that the low emotional experience touchpoints were significantly improved. The reliability and validity tests of the two questionnaire results were better, indicating that the emotional experience quantification map was effective for emotional experience quantification and could better measure users’ emotional perception.

The quantified user experience map could visualize the quantified user experience results

The user experience quantification map accurately reflects each emotional experience touchpoint, as demonstrated in experiment 1. The emotional experience curve and user interface emotional experience can be visualized using the questionnaire statistics. It can quickly identify both emotional touchpoints with high and low emotional touchpoints. The changes in emotional experience touchpoints can be compared using the user emotional experience curves produced by Experiment 1 and Experiment 2. The visualized emotional experience map can help designers discover interface design strengths and weaknesses, and thus guide them to improve the weaknesses and maintain the strengths.

The user emotional experience evaluation at each stage fluctuates especially between interface levels, and it can be seen in Experiment 1 and 2 that the emotional difference in the Kansei image words representing the functional layout at different stages is more obvious. It can be seen that the functional layout of the interface can influence the users’ affective experience when performing tasks. The user’s cognitive load when processing information can be reduced in particular by the interface’s overall recognizability and simplicity of the functional layout. The highlighting of elements in the interface hierarchy and the emphasis on articulation can enhance recognition and browsing fluency, thus improving users’ affective experience.

Limitations

This study has the following limitations,

The evaluation of emotional experience differs to some extent because the automobile-related experimental environment is more complex and the stationary driving and driving environments are vastly different.

The process of user emotional experience is quantified by the quantitative diagram of user emotional experience. The current study focuses primarily on the visual level, but further investigation at the interaction level is required.

Only the relationship between the main interface and the first-level interface is examined in this study; the evaluation of the deeper-level interface still requires further investigation.

Conclusion

The design method of a quantitative user experience map based on Kansei Engineering can effectively quantify and visualize user emotional experience. Firstly, it can accurately describe the emotional expression of user’s behavioral touchpoints. Secondly, it is capable of accurately quantifying and visualizing the emotional experience of the user. Lastly, it can effectively assist designers in creating targeted interface iterations. It has the potential to significantly enhance the design effect and the emotional experience of users. Since not all design touchpoints can be considered in the design, this design approach will effectively improve user experience by identifying and improving low-rated emotional touchpoints.

A quantitative design method for UX emotions has been scientifically established by combining the UEM and Kansei Engineering to construct a quantitative UEM. For the UX map touchpoints, Kansei Engineering is used to extract and score Kansei imagery words. This provides designers with a set of tools to quantify and visualize user emotions. The emotional needs of the user can be better understood by designers, which makes it easier to optimize and iterate on interface design. The research methodology serves as a guide for making decisions regarding the emotional design of HMIs.

Based on this method, the future study intends to concentrate the user experience process on user behavior observation. In order to investigate the evaluation of emotional experience in the dynamic process of users and further enhance the design experience, quantitative analysis will be carried out through behavioral observation and user interviews.