Introduction

The “Maker Movement” has sparked interest worldwide among stakeholders involved in the operation and management of K12 educational institutions (Hsu et al., 2022; Schad and Jones, 2020). The exponential growth of maker spaces can be attributed to technological advances (Fu et al., 2022; Godhe et al., 2019; Hwang, 2023; Pei, 2018). The utilization of open-source programs and equipment contributes explicitly to the technical assistance in the rise of Maker space (Pei, 2018). The defining characteristic of a Maker space is not the state-of-the-art equipment it exhibits but rather the Maker environment that values autonomy, open-source principles, and an environment of collaboration within the environment (Fu et al., 2022). Maker culture fosters a community of individuals who freely exchange ideas, engage in collective learning, and utilize open-source software and hardware resources to transform concepts into tangible creations (Davies et al., 2023; Hwang, 2023; Pei, 2018; Tabarés and Boni, 2023). Maker education is a kind of technology-based, hands-on, and creation-oriented learning (Serdyukov, 2017) that is regarded as a new educational mode to develop core literacies and innovation abilities (Hsu et al., 2017; Jaatinen and Lindfors, 2019; Sungur Gül and Ateş, 2022; Wang et al., 2018). Maker education originates from the theory of constructivism and Dewey’s concept of “learning by doing” and is also profoundly influenced by the cognitive theory emphasizing the study of active constructive, situational authenticity, and social interaction (He, 2022; Mayer, 1998). Furthermore, Maker education allows learners to construct knowledge by solving real-world problems and developing mental representations (Sawyer, 2006). Adhering to the concepts of “innovation” and “creation” promoted by the Maker movement, Maker education focuses on cultivating students’ innovative thinking (Zhan and Niu, 2023). This kind of education has the potential value to develop the core qualities of the 21st century, such as critical thinking, communication, problem-solving, and collaboration abilities (Hughes and Kumpulainen, 2021; Iwata et al., 2020; Veldhuis et al., 2021).

As Maker education research progresses, Maker activity assessment has become a prominent problem that has hampered the development of Maker education (Nikou, 2023). Existing educational assessments mainly evaluate learners’ learning results according to specific academic standards (Dixson and Worrell, 2016). While Maker education represents project-based, hands-on, learner-centered, and iterative learning, the traditional assessment methods may not be suitable for measuring the higher-order skills associated with Maker education (Murai et al., 2019). Therefore, such a gap significantly hinders the evaluation of student Maker-learning outcomes. So consequently, there is a necessity to do research in the primary student context.

Maker education facilitates interactive learning by promoting the exchange of knowledge and creative thoughts (Tabarés and Boni, 2023). In the present context, the learning tasks and ideas associated with Maker education align harmoniously with those of classroom instruction. The present education system, combined with Maker education, which is closely linked to STEM learning, is an instructional style that emphasizes problem-oriented and project-centered training (Tabarés and Boni, 2023). It depends on practical, frequently interactive learning activities to resolve real-world issues. Maker education offers a practical approach to instructing and acquiring new knowledge among elementary kids in China (Xu et al., 2024).

In recent years, maker education assessment has attracted great scholarly attention. For example, it has been argued that the evaluation of Maker teaching comprises three parts: process, understanding, and work (Yokana, 2015). Lin et al. (2020) analyzed the role of hands-on practice in learning and concluded that the learning outcomes of Makers are divided into three components: influence (emotional attitude in the Maker process), cognition (understanding and constructing knowledge), and participation and collaboration (Lindberg et al., 2020). The Mount Royal University (MRU) Maker Space has developed an evaluation scale with four dimensions: critical thinking, innovation and creation, collaboration, and work habits (Makerspace, 2022). Furthermore, previous studies have used quantitative methods based on students’ design schemes, works, and self-rating scales, among others, to measure cognitive development, behavioral performance, and student attitudes in Maker activities (Blikstein et al., 2017; Chamrat, 2018; Lui et al., 2019; Murai et al., 2019; Yin et al., 2020).

The term “Maker contribution research gap” is a recognized insufficiency or shortcoming within the current archive of the scholarly literature pertaining to the contributions made by Maker education. This statement implies that further scholarly research and exploration are necessary to examine the precise mechanisms via which Maker education contributes to other domains, including education, innovation, and workforce development. However, thus far, most of the frameworks presented in the literature only provide basic indicators for Maker activities assessment and are too general to capture the multi-dimensional performance of students in making artifacts. Thus, we set out to develop in the current research a new framework for Maker activities. This framework can invoke “a holistic engagement” that encompasses various practical abilities embedded in technology, collaborative working, design thinking, and emotional attitude. Moreover, the proposed framework is not just philosophically aligned and coherent with the needs of Chinese teachers, policymakers, and researchers alike, it is also especially developed for their implementation.

The Maker Movement increases primary school children’s enthusiasm and drive by teaching them how to produce things and care for the resources of the globe around them (Chen et al., 2023; Fu et al., 2022). The study we conducted builds upon existing literature and highlights the significance of Maker education. Furthermore, Maker education is an effective method for fostering strong interactions with pupils (Soomro et al., 2023). It entails accompanying learners as they develop into discerning thinkers. It is truly remarkable to provide children with the opportunity and resources to recognize their ability to generate and explore ideas independently (Rehman et al., 2023).

The purposes of this study are as follows: First, it aims to develop a Maker literacy assessment model for primary school students in the People’s Republic of China, which is a collection of essential characters, key abilities, and values for primary school students to accomplish Maker activities (Maraschin et al., 2022). Second, the current study aims to identify the specific weights of indicators from the proposed research model. The resulting model can be used to provide theoretical and practical guidance for Maker education. For example, it is helpful to judge whether teaching has achieved the expected effects in order to identify the gaps between students, guide teachers in determining the teaching objectives, and facilitate the process of content selection.

Establishing a literacy-oriented education evaluation system is the trend of international education research. Related to this, enriching and developing evaluation theory and innovating the evaluation practice model of Maker literacy is an emerging approach to strengthening Maker education, which has attracted increasing attention in recent years (Kumpulainen et al., 2020; Marsh et al., 2018). In particular, “Maker literacy” is a social practice that entails making and remaking artifacts using various materials and technologies (Kim et al., 2022). In the People’s Republic of China, Maker education has brought great opportunities for educational innovation (Jia et al., 2021).

The notion of the “Maker contribution research gap” highlights the need for undertaking more comprehensive and nuanced investigations that delve into the specific contributions and outcomes associated with Maker education. This instructional approach will ultimately improve our understanding of its potential benefits and limitations. The possible improvement of this gap can offer useful insights to educators, policymakers, and other pertinent stakeholders, empowering them to improve Maker’s educational initiatives for the advantage of learners and the broader community (Demirata and Sadik, 2023). Educators are increasingly acknowledging students’ aptitude for excelling in design, creation, and idea dissemination, which are crucial for our economy and the current global landscape (Davies et al., 2023). Currently, there is a dearth of studies on the structural aspects of Maker literacy among primary school students in China. Thus, it is imperative to investigate indicators representing the Maker literacy hierarchy based on Chinese local Maker education practices and related regulations.

Methods

Design

As shown in Fig. 1, the creation of the Maker literacy assessment model for elementary schools on the mainland was completed through a sequence of phases. We included two phases for the proposed framework. In the first phase, we included a conceptual framework (literature review) for Maker literacy evaluation. The first framework considered China’s Maker education policy backdrop and the key literacy development requirements for mainland students (Dan, 2021). In the second phase, we used the Delphi technique (two rounds survey) to analyze and improve the domains and indicators of the Maker literacy evaluation. Finally, we developed a professional agreement on the Maker literacy assessment model for Chinese primary school students (Fig. 1).

Fig. 1
figure 1

Proposed conceptual framework of a Delphi method approach to formulating a Maker literacy assessment model for Chinese primary school students.

Full size image

Literature review (content analysis)

The literature coding was performed to screen key indicators and construct a theoretical structure model of Maker literacy assessment for Chinese primary school students. Around two hundred published papers about Maker activities, the design thinking process, and innovation education from the fields of teaching theory, psychology, and philosophy of technology were retrieved across the electronic databases (i.e., Web of Science and China National Knowledge Infrastructure [through back-to-back translation method]).

The literature review text systematically follows the procedures of content analysis (Fraenkel et al., 2017). Finally, we established a conceptual framework reflecting the potential dimensions and indicators of Maker literacy assessment for Chinese primary school students. At this point, the first version of the web-based Delphi questionnaire was finalized.

Performance evaluation theory

Stone developed performance-based assessment theory, and Wolf proposed standards for qualitative assessment (Stiggins, 1994). After the 1990s, performance evaluation continued to develop and became an essential theoretical direction in educational assessment, combined with curriculum integration, teacher development, and educational equity (Herman, 1992). The main content of performance evaluation theory includes assessing students’ abilities in actual or simulated situations (Palm, 2019; Wiggins, 1998). Performance evaluation emphasizes the need to set up actual or simulated evaluation scenarios to evaluate students’ ability to apply knowledge and skills to solve problems in real situations rather than being limited to using paper-and-pen tests to assess their level of memory of conceptual knowledge (Darling-Hammond et al., 2010). This situational evaluation can better evaluate students’ comprehensive abilities when facing complex situations and has higher predictive validity (Lievens and Coetsier, 2002).

The concepts and techniques used to evaluate the efficacy, development, and results of educational programs centered around Maker culture are referred to as performance evaluation theory in the context of Maker education (Lin et al., 2020). Maker education focuses on practical, hands-on learning where students use various tools, materials, and technology to design, develop, and build physical creations. Within the realm of Maker education, the philosophy of performance evaluation encompasses assessing both the process and the result of students’ creative endeavors (Pei, 2018). This assessment often surpasses conventional standardized examinations and assessments, emphasizing a comprehensive evaluation of students’ aptitude, ingenuity, problem-solving competence, teamwork, and analytical reasoning (Y.-C. Hsu et al., 2017). It may encompass process-oriented evaluation: In one example, it involves examining all aspects, including the actions individuals take, the techniques they utilize, the problems they experience, and how they iterate and enhance their designs. It also includes an assessment of the product. For example, it evaluates students’ final outputs or artworks based on their creativity, inventiveness, usefulness, and compatibility with assignment aims and ambitions.

Evidence-centered design theory

The evidence-centered design theory (ECD) was proposed by Mislevy et al. from the University of Illinois in the late 1990s and is based on theories such as cognitive psychology, cultural psychology, and evaluative psychology (Mislevy et al., 2003; Mislevy and Riconscente, 2011). It emphasizes that when designing educational evaluations, evidence related to the assessment should be systematically considered, and interpretable evidence should be generated around the learner’s knowledge, skills, and attitudes (Mislevy and Haertel, 2006). At the beginning of the 21st century, Mislevy established a theoretical framework and proposed the concepts of learner, evidence, and task models (Mislevy et al., 2002; Mislevy and Riconscente, 2011). The inspiration of the Evidence Center Design Theory for this study: Firstly, when designing the Maker literacy structure model, this study needs to construct a learner model and clarify which knowledge, skills, and attitudes students should be tested in literacy evaluation (Mislevy and Riconscente, 2011). The learner model should describe the knowledge dimension (technical knowledge), skill dimension (hands-on practical ability), and attitude dimension (innovative consciousness) contained in Maker literacy, providing a basis for subsequent refinement of literacy elements and model construction (Gierl et al., 2008; Han et al., 2022). Secondly, it is necessary to establish an evidence model to analyze the different behaviors that students may exhibit when completing Maker tasks, which can provide a basis for extracting elements of literacy. The evidence model can identify the behavior behaviors that students may exhibit in problem-solving, hands-on production, and collaborative communication when carrying out different Maker practice activities (Ferrara et al., 2008). By analyzing these behaviors, key elements reflecting Maker literacy can be determined. The ECD theory helps to improve the explanatory and inferential validity of literacy evaluation results(Ogata et al., 2024). Because model evaluation enhances the correlation between tasks and evidence, comprehensively examining the overall performance of students in terms of knowledge, skills, attitudes, etc., the evaluation results can more accurately reflect their literacy level. Within the framework of Maker education, the philosophy of ECD theory describes a method for creating learning activities and evaluations based on data demonstrating student comprehension (Ogata et al., 2024). This notion is essential in Maker education, which emphasizes active, practical learning and competence building via the development of physical products.

Participants

The volunteers in this survey were chosen based on their area of study and expertise. The inclusion criteria were as follows: employment as school staff in elementary or secondary academic institutions, geographic location restrictions (scholars actually living in the People’s Republic of China and already carrying out work here), and qualifications in the fields of STEAM education, information technology education, labor education, and science education, among others. A group of professionals in the current research was formed with the help of established professors and scholars. The selected specialists have over five years of experience in the research context, so they served as excellent sources from whom we gathered relevant information. A formal invitation request that also contained information about our research background and objective was e-mailed to 18 professionals, along with a volunteer participation agreement and consent form. All experts replied to the e-mail, but two experts were unable to participate due to some reservations. Thus, a total of 16 professionals expressed a desire to cooperate in the study on implementing a Maker literacy assessment model explicitly designed for Chinese primary school children (Table 1).

Table 1 Basic information of experts.
Full size table

Participant demographic profile

All of the participants were citizens of China, of which 44% were male, and 56% were female. In terms of age, 25% were between 31 and 40, 38% were between 41 and 50, and 38% were over 50 years old. Concerning education, 19% held a bachelor’s degree, 6% had postgraduate qualifications, and 75% had a doctorate. Regarding teaching experience, 6% of the participants were less than 5 years, 13% were between 6 and 10 years, 13% were between 11–15 years, and 69% were more than 15 years. Professors made up 56% of the participants, while associate professors made up 44%. The characteristics of the participants are listed in Table 2 (Table 2).

Table 2 Participants profile.
Full size table

Delphi study

The Delphi technique utilizes a sequence of continuous surveys combined with responses in order to promote group engagement, organize the information exchange, disseminate the information, and gather general professional opinions (Donohoe et al., 2012; Skulmoski et al., 2007; Wang et al., 2021). For the data collection, we developed a questionnaire survey through a professional platform called “Wenjuanxing” (a website offering survey features similar to Amazon Mechanical Turk). Delphi requires at least two stages before a conclusion is derived (Keeney et al., 2001). Thus, we also followed previous studies’ approaches in conducting the data collection in two stages: the first round of the survey focused on the exploratory themes and variables, while the second round validated the first-round specification. The same group of professional panelists participated in both rounds. The data-gathering process consisted of a two-round questionnaire delivered using online surveys from May 2022 to August 2022, respectively. The initial round of the questionnaire was circulated from May to June 2022, while the second phase was circulated from July to August 2022. We used an established platform named “Wenjuanxing,” which has survey tools similar to Amazon Mechanical Turk, to make a questionnaire survey for the purpose of gathering data.

Procedure for the first round

The first wave of questionnaires was distributed between May and June 2022. All professionals were required to submit the first-round detailed questionnaires within two weeks. A follow-up email was sent every four days. The respondents were asked whether the indicators and domains of the Maker literacy evaluation for Chinese primary school students should be dropped, continued, or modified. Additionally, a 5-point Likert scale was used to encourage them to evaluate the indicators and domains based on their importance to the research. The dimensions and variables were modified to ensure a simplified Maker literacy assessment model, which we then utilized to design the second round of surveys in accordance with the description of assessments from the first round.

Procedure for the second round

The second wave of questionnaires was fielded between July and August 2022. This second phase aimed to provide inputs on the outcomes of the first round, re-evaluate the relevance of the indicators and dimensions of the modified Maker literacy assessment model, and establish consensus. To determine the actual weights of items throughout the proposed model, the respondents were requested to undertake pair-wise evaluations and importance ranking of indicators of a similar level using the judging matrices. This step helped establish the respective significance of every indicator.

Consensus requirements and data analysis

Here, the following values were used: interquartile range (IQR) ≤ 1, coefficient of variation (CV = (SD/M) 100) = ≤ 18%, and the median ≥4. These criteria were utilized to determine an opinion (Mao et al., 2020; Diamond et al., 2014). If an opinion was established on a certain issue, it was preserved in an overall questionnaire However, if one aspect of the Maker literacy assessment model fails to receive an opinion, then the professional group assessment is not worthy of further justification. If an item does not fit one of the acceptance criteria, the professional group evaluates and debates on it to determine whether it should be eliminated or retained. Next, the analytic hierarchy process (AHP) was utilized to calculate the precise weights of the components inside the Maker literacy model (Wang et al., 2020). YAAHP 10 software was used to calculate the weights. Furthermore, the additional quantitative analysis involving the median, (Interquartile range) IQR, and the percentage was conducted using IBM SPSS 24.0.

Results

Development of the preliminary maker literacy assessment structure

According to empirical and analytical literature, “Maker literacy” represents a state of holistic understanding, wherein individuals utilize innovative ideas to understand a problematic scenario, engage actively with equipment and materials, and creatively address the presented challenges. In other words, Maker literacy refers to a set of skills acquired and developed by a student upon receiving Maker education and by living an innovative and creative lifestyle. Moreover, through textual analysis, we assembled a set of indicators that were used in both Delphi rounds. These indicators describe four dimensions: design thinking, technology application, creation practice, and responsibility awareness (Table 3).

Table 3 Preliminary maker literacy assessment structure.
Full size table

Establishment of a maker literacy assessment model for Chinese primary school students

In the questionnaire, all the respondents involved in the study acknowledged the importance of creating an assessment methodology for Maker literacy in Chinese primary schools. The professionals recommended certain additions, deletions, adjustments, classifications, and descriptions of indicators and domains, as shown in Table 4 (Table 4).

Table 4 Statistical table of expert opinions.
Full size table

The professional group convened to consider the recommendations and remarks of the specialists. First, in terms of domains, after comprehensive consideration, “Creation practice” was modified to “Materialized practice,” which was merged with “Technology application.” “Scheme selection” was changed to “Scheme formulation” in order to incorporate the ideas of engineering design ideas and system thinking. In addition, the description of design thinking was also modified. As for the indicators, “Material technology” and “Tool understanding” were merged into “Technical understanding,” while “Visual representation” was incorporated into “Prototype construction.” “Technical inquiry” was merged with “Craftsman spirit” and then changed to “Maker spirit.” “Ethical responsibility” was changed to “Maker responsibility.” Finally, “Self-monitoring” belonged to metacognition, which was condensed in the process of “Technology application and materialized practice.”

In the second round, the revised assessment model was compiled into a questionnaire survey and sent to the same group of experts once again. After the second round of consultation, the average score of the three domains was above 4.6, and the full score of “Design thinking” was 93.75%. The average value of the 12 indicators is above 4.3, among which the full marks of “Creative idea,” “Knowledge integration,” “Maker spirit,” and “Maker responsibility” are all above 80%. Overall, the experts believe that the assessment model can reflect the key abilities and values of Chinese primary school students in Maker activities. Furthermore, there is no need to introduce other changes, except for the description of some indicators. In terms of the concentration of expert opinions, the IQR values of the indicators are all less than 1.8, indicating that expert opinions are highly concentrated. Among them, the concentrations on “Design thinking,” “Creative thinking,” “Knowledge integration,” “Maker spirit,” and “Maker responsibility” are very high.

Notably, the full score rates of the three indicators of “Iterative optimization,” “Technical understanding,” and “Information gathering” are low, but their average is between important and very important (M > 4). Thus, given that the concentration of expert opinions is good, they can be retained. Overall, the coefficients of variation (Cv) of the three domains and twelve indicators meet the requirements of coefficients of variation (Cv<0.18). Eventually, after two rounds of expert consultation, three domains and twelve indicators are finally determined, as shown in Table 5 (Table 5).

Table 5 Components of primary school students’ maker literacy.
Full size table

Specific weights of items in the maker literacy assessment model of chinese primary school students

The Analytic hierarchy process (AHP) combines quantitative and qualitative analysis, decomposes complex problems into hierarchical problems, and transforms qualitative problems into quantitative ones (Li, 2022; Wedley, 1990). In the current study, the YAAHP software was used through the AHP technique to determine the weights of the domains of the proposed model. The outcomes of the AHP indicate that the weights of “Design thinking,” “Technology application and materialized practice,” and “Maker spirit and responsibility” are 0.365, 0.282, and 0.348, respectively. Based on these results, it makes sense that “Design thinking” (0.365) is the central variable within the Maker literacy assessment model. In addition, all the scores for the twelve indicators are shown in Table 6 (Table 6).

Table 6 Weight assignment of primary school students’ maker literacy.
Full size table

Discussion

The findings of this study have led to the selection of several measures for assessing Maker literacy in Chinese primary schools. The previous research has been limited to the context of a Maker literacy assessment model in Chinese primary school students (6–12 years old) context. In comparison, our research clearly indicates the importance of primary school students’ level in terms of thinking innovatively, thus addressing the research gap. It originates from the current situation of Maker education practice and its related policies in the People’s Republic of China, which, in turn, determine the quality of Maker education programs and Maker activities in the country. They cover components, including “Design thinking,” “Technology application and materialized practice,” and “Maker spirit and responsibility.” The efficiency of Maker education programs, the influence of Maker activities on skill development, the incorporation of Maker concepts into traditional schooling, and the wider societal ramifications of cultivating a Maker mindset are just a few examples of the several facets that may fall under this gap. Academic scholars may exhibit interest in investigating the impact of Maker education on the development of critical skills, creativity, problem-solving proficiencies, and its efficacy in equipping individuals for forthcoming difficulties. Our research expands upon the existing literature by further investigating the notions related to entrepreneurship among Chinese primary students (Lundberg and Rasmussen, 2018). Our research corroborated the prior findings about Maker education as a means of offering a pertinent framework for learning (Gratani et al., 2023; Halverson and Sheridan, 2014; Lundberg and Rasmussen, 2018). Maker Education places significant emphasis on recognizing students’ inherent capabilities, valuing their unique qualities, and equipping them to confront real-world challenges while deriving valuable lessons from their errors.

Design thinking

“Design thinking” refers to the process of using ideas and methods in the field of design. It includes analysis, generalization, prototype, engineering design, and others, and is actually a process of solving problems creatively. Design thinking is composed of five components: Requirement definition, Creative idea, Scheme formulation, Prototype construction, and Iterative optimization, which have weights of 0.295, 0.222, 0.193, 0.179, and 0.111, respectively. Overall, design thinking has a weight value of 0.365, which is the highest weight in primary school students’ Maker literacy.

Many scholars believe that the cultivation of thinking for adolescents has become the mainstream trend (Gursoy and Bağ, 2018; Zeniali Khorchani et al., 2019). Experts ascribe high weight to design thinking for the following two reasons. On the one hand, it is the premise of organizing and implementing Maker activities and has a direct impact on the learning effect of Makers (Veldhuis et al., 2021). On the other hand, in technology-supported learning activities based on creativity (Saher and Uslu, 2017), tools and technologies do not determine students’ learning outcomes; rather, students’ thinking and awareness of applying technology do.

“Requirement definition” is a process of understanding a problem situation by listening, observing, or discussing from the perspective of “seeking understanding.” “Creative idea” entails a process of using associative, divergent, questioning, and other thinking skills to propose a unique and feasible concept that conforms to the design principles. There are two reasons why experts give the greatest weight to creative ideas: (1) because the essence of Maker education is rooted in the cultivation of learners’ innovation ability (Liu et al., 2022), and (2) the creative idea emphasizes the generation of rich ideas for solving a certain problem and provides more options for a prototype (Dosi et al., 2020; Stahl et al., 2014). In the era of the knowledge economy, the cultivation of innovative thinking is the most important element in nurturing talent (Dou et al., 2021).

“Scheme formulation” is a process of using the planning thinking of system analysis and comparison to analyze a given task in its entirety and form a problem solution. Scheme formulation guarantees the realization of creative ideas, so it is reasonable that it ranks second in terms of weight. “Prototype construction” refers to a process of using visual methods, such as charts, 3D models, or digital technology, to transform invisible thinking processes, design intentions, and ideas into tangible results. “Iterative optimization” is a continuous microcirculation of testing models, improving creative artifacts, and modifying schemes according to the feedback. Experts believe that innovation belongs to higher-order thinking, while requirement definition, prototype construction, and iterative optimization belong to the emotional, technology application, and consciousness levels, respectively. Thus, it is reasonable for them to have low weights.

Technology application and materialized practice

“Technology application and materialized practice” refers to the process of selecting appropriate tools, materials, and equipment; acquiring and integrating relevant knowledge and skills; effectively managing the learning process; and conducting technical practice to solve problems in a creative manner. This domain is composed of five components, namely, Technical understanding, Knowledge integration, Making, Information gathering, and Cooperation and communication. The weight of “Technology application and materialized practice” is 0.282, which is the lowest weight given by experts among the three domains. This can be due to several reasons. First, Maker activities entail the process of turning creative ideas into reality through hands-on production. Therefore, technology application and materialized practice ability are important contents in Maker literacy (Dou et al., 2021). Notably, however, the ultimate goal of Maker education is not to use these techniques and tools but to develop design thinking and innovative thinking through it (Yang, 2018). Thus, experts may think that Technology application and materialized ability are instrumental abilities, while Design thinking, Maker spirit, and responsibility belong to higher-level thinking. Thus, its importance is relatively lower.

The weights of these five components under this domain are as follows: Knowledge integration (0.290), Technical understanding (0.248), Making (0.171), Cooperation and communication (0.167), and Information collection (0.124). “Technical understanding” refers to having a good command of the characteristics of tools, materials, and equipment and the ability to select and utilize them effectively. “Knowledge integration” refers to comprehensively using the knowledge of mathematics, science, engineering, art, technology, and other disciplines; integrating the content and experience that point to problem-solving; and providing more options for prototype construction. “Making” refers to the use of technical tools and equipment to assemble hardware, process materials, and code programs. “Information gathering” is a process of searching and collecting information related to problem-solving through observation, text search, investigation, and so on, to evaluate the reliability of the information at hand. “Cooperation and communication” refer to sharing knowledge, achievements, and resources with an interactive and open mind to compete rationally and promote one another.

Among these components, experts give Knowledge integration the highest weight because it is a process of knowledge construction (Glaés-Coutts and Nilsson, 2021), basically an important part of higher-level cognitive abilities. In the era of a knowledge economy, which attaches great importance to the cultivation of thinking, the transfer and application of knowledge are essential abilities for excellent Makers. Meanwhile, Technical understanding is an important support for the completion of an artifact and belongs to instrumental knowledge, while Knowledge integration belongs to higher-order thinking. Thus, it is reasonable that the weight of the former is slightly lower than that of the latter.

Meanwhile, “Making” is the basic ability to complete Maker activities and entails the process of transforming creative ideas into reality. Technical understanding is an important support for hands-on practice, so its weight is slightly lower than knowledge integration. Compared with the other four indicators, experts may think that mutual assistance and collaboration belong to literacy from the social perspective, which is less important than literacy improvement from the individual perspective. Thus, given that information gathering is a kind of instrumental literacy, it is given the lowest weight.

Maker spirit and responsibility

“Maker spirit and responsibility” refers to perceiving and understanding technological phenomena and problems and then gradually forming positive attitudes, beliefs, and social responsibilities in the process of materialized practice. This dimension consists of two components: Maker spirit and Maker responsibility. The weight of the Maker’s spirit and responsibility is 0.348, which gives it the second most important position in the three domains. Many international scholars believe that literacy is a multi-dimensional unified system that includes knowledge, skills, and attitude, of which attitude matters most (Stoof et al., 2002). Jensen et al. (2001) proposed the formula, “literacy = (knowledge + skills) * attitude,” where attitude connects knowledge and skills by multiplication (Jensen et al., 2001). The core aspects of Maker education, such as spontaneity, innovation, creation, collaboration, interest, and inquiry, require learners to have strong vitality and a sustainable learning attitude (Connolly et al., 2020). How to be a creator characterized by ethical responsibility has become a necessary requirement for citizens. In this sense, it is reasonable for “Maker spirit and responsibility” to be given a higher weight than others (Cruz et al., 2021).

“Maker spirit” refers to having the qualities of being curious, imaginative, and inquisitive about technical experiences. A person with a “Maker spirit” has the qualities of patience, dedication, and meticulousness, always strives for perfection, and shows strong perseverance and courage in the face of technical challenges. Meanwhile, “Maker responsibility” refers to an ecological and environmental awareness of saving materials and turning waste into treasure, a sense of responsibility for safety practice, and a technical ethics awareness of respecting others’ achievements and preserving intellectual property rights. The weight of the Maker spirit is 0.571, which is higher than that of the Maker responsibility (0.429). This is because the Maker activity is characterized by emphasizing students’ enthusiasm. During the process of the Maker activity, students are required to invest a great deal of energy and self-consciousness, which requires the support of exploration, curiosity, courage, and strong perseverance. While the awareness of certain responsibilities, such as environmental protection, safety practices, and intellectual property rights, embedded in Maker responsibility is not unique to Maker education, it is a necessary quality for conducting for Maker activities. Therefore, experts have assigned a slightly lower value to Maker responsibility. The course’s focus is on fostering technical proficiency and innovative thinking in the context of making (Xia and Zhong, 2018). Thus, understanding the nature and building blocks of Maker literacy enables educators to deliver training and activities that will help their students reach their full potential as future Makers. Furthermore, breaking down Maker literacy into its constituent parts (domains) is crucial for elucidating its hierarchy and developing a unified model of evaluation.

In conclusion, the major Maker literacy subdomains recognize the foundation, path, and goal of Maker literacy development. Developing Maker literacy is a directed and purposeful endeavor, and the above three domains are intended to support and enrich one another. A primary school student with deal-Maker literacy (defined as having design thinking, comprehensive technical understanding, flexible making ability, awareness of technology exploration and responsibility, and interdisciplinary knowledge integration and cooperation and communication abilities) can produce more outstanding, imaginative, and creative outcomes upon performing Maker activities (Zhong et al., 2017).

Conclusion and implications

This study defined the foundation of Maker literacy and the variables that may be utilized to construct more complete evaluation methods for assessing students’ Maker literacy in Chinese primary schools. In addition, several indicators were chosen with Chinese students in mind, taking into account the distinct cultural environment in the country. Here, we gave precedence to strategies that have previously been implemented in China. The collection of specified variables indicates how the hierarchical structure of the creation of “Maker literacy” is reflected in Chinese primary school students.

In particular, the chosen variables are in alignment with the aims of present Maker education and meet the required catalog of higher-order opinions, skills, and talents, in accordance with the existing Chinese education program. The establishment of a systematic assessment method to determine the quality of Chinese primary school students’ Maker literacy should be the focus of future studies. For instance, future studies can construct an evaluation instrument and validate its relevance and accuracy to further investigate Maker literacy in Chinese primary schools.

Nevertheless, this research has led to the creation of a model for evaluating Maker literacy, which takes into account the whole range of factors that can affect the success of conducting projects in the context of Chinese primary education. The goal is to address the issue of insufficient imagination and skill in Chinese primary school students, which can be clarified through Maker education. Thus, the conditions in Chinese primary schools can be improved, and the significance of Maker education can be heightened.

The implications of this study extend further assessment approaches; it embraces a broader viewpoint on education that emphasizes experiential learning, ingenuity, and cooperative problem-solving. To successfully traverse an era marked by rapid technology breakthroughs, nurturing a generation of kids skilled in Maker literacy becomes a desired aim and a requirement to prepare students for the difficulties and possibilities of the future. The Maker Literacy Assessment Model is a practical tool for educators to customize their teaching methods, enlightens policymakers on the significance of including Maker education within the curriculum, and promotes continuous research to enhance and adjust the model to changing educational environments. In the long term, implementing this technology can substantially contribute to developing competent and flexible personnel, establishing the basis for a future generation of imaginative and inventive individuals in China and other regions.

Foremost, the Maker literacy assessment model comprehensively reflects the essential indicators of Maker literacy for Chinese primary school students and forms the systematic structure of Maker literacy. This is helpful in guiding Maker teachers in determining teaching objectives and selecting appropriate and useful content. Second, through the weight analysis of the items included in the model, this study is able to demonstrate that distinct abilities and traits have varying levels of importance to students. This finding enables teachers to develop suitable Maker literacy activities with goals and priorities according to students’ knowledge and ability base. Third, the Maker literacy assessment model provides a scientific framework for the development of measurement tools, which are helpful in grasping the current situation of the existing student Maker literacy. This also helps in the assessment of whether a certain teaching program has achieved the expected effect and in the identification of gaps between students.

In brief, creating a Maker Literacy Assessment Model for primary school kids in China enhances research, policymakers, and implementation in education by fostering creativity, fairness, and high standards in teaching and learning. The objective of our study is to design and apply a Maker Literacy Assessment Model specifically designed for students in primary schools in China. It provides several advantages and prospects for the educational environment. It improves educational results. For instance, educators can enhance their comprehension of students’ aptitude in critical thinking, problem-solving, creativity, and teamwork, fostering comprehensive educational results that fit with the capabilities required in the twenty-first century. It can have advantageous effects on promoting creativity and entrepreneurial spirit among primary school students in China. Our study findings encourage entrepreneurship and creative thinking among elementary school kids, cultivating a culture of creativity, exploration, and risk-taking crucial for tackling challenging real-world problems.

Limitations and future research

This research has a couple of drawbacks. First, it is important to note that the acceptance of the Delphi methodology will not generally indicate that the perfect conclusion has been observed. Rather, it only indicates that professionals are striving to uncover and generalize new ideas and techniques. Therefore, additional reviews, testing, validations, and amendments are needed to bring the final result nearer to worldwide standards. Second, the framework for measuring Maker literacy and the related variables for Chinese students was created within the current state of China’s Maker education curriculum and its outcomes. Therefore, international comparisons should be made with caution. Third, this study used the AHP approach, which is a subjective weighting approach that could possibly introduce bias into the framework of Maker literacy. Finally, given the geographical location of the study, the findings in this paper cannot be generalized. Thus, in future research, we aim to collect data from other provinces and from developing nations, such as India, Pakistan, Nepal, Bangladesh, Thailand, and Vietnam.