Introduction

During the past few years, tests for evaluating future performances have been increasingly used for the purpose of hiring employees and selecting university students. High School grade point average (GPA), aptitude tests, psychometric tests, interviews and manual dexterity tests were all studied in order to predict the students' future success. In the United States, the Dental Admission Test (DAT) was introduced at the beginning of the 1950s and is still in use, with a few changes, as a regular screening test for candidates applying to the dental school. Presently, while in the United States DAT is a well codified instrument, consisting of four major parts (Part I Survey of Natural Science, Part II Perceptual Ability, Part III Reading and Comprehension, Part IV Quantitative Reasoning), in Europe parameters taken into consideration to select students change markedly from one country to the next and even within each country, from one university to the next.1,2,3,4,5,6

One of the most debated issues among the parameters used for selecting dental students was and still is basic manual skill: the question is whether it is still valid for selecting students and whether it is linked to their professional success. Many tests, such as perceptual motor ability, chalk carving, paper and pencil, as well as those proposed by Wilson7 and by Suddick,8 have been used to correlate manual dexterity with success in dental schools. Although interesting, some such studies produced controversial results and none reached any clear conclusion. In any case, while many of the studies could fairly well correlate manual dexterity with academic and pre-clinical laboratory skills, few of them were set up to assess the correlation between manual dexterity and clinical success. Really, dental students should acquire an 'academic biomedical knowledge and gain the professional skills and attitudes of a broadly based general practitioner'.2,8,9,10,11,12,13,14,15,16,17

Although an admission test (based on a multiple choice test, including questions on biology, mathematics, physics, chemistry and general culture, on a psycho-attitudinal test and on the high school degree) is in use at the Catholic University of Rome for applicants for the CLOPD, during the past few years we were permitted to systematically investigate BMD, for research purposes, thanks to a specific examination of manual ability, even if it was not intended to have any effect on admission itself. The examiners' curiosity had been aroused by the fact that even university students who were distinguished both for their school performance and their achievements were rather clumsy at the beginning of practical training, both clinical and in the laboratory.

The purpose of this study is as follows: a) to study BMD in candidates applying for CLOPD and to determine if it is influenced by the high school they attended; b) to determine whether this is an important parameter to predict success in all types of skills – at academic, pre-clinical and clinical level – which may therefore represent an important parameter in selecting students for CLOPD; c) to assess whether initial manual dexterity in students admitted to CLOPD can improve with training in pre-clinical and clinical practice, required for CLOPD.

Materials and methods

Despite the existence of some manual ability tests (ie eye-hand coordination test, psycho-motor test, Vayer, cognitive test, analysis of requisites for writing capacity, 'Frostig perception test', computer simulated exercises, tremometer test, 2-hand sinusoid test, Thurstone's test, Embedded Figures test, O'Connor Tweezer test, Mac Quarrie test), only a few tests have been developed so far to globally evaluate basic manual capacity.18,19,20,21,22,23,24,25,26,27

The formulation of exercises measuring basic performance was inspired by a careful analysis of specific aspects of manual dexterity, aimed at breaking down complex behaviours into their simpler components. Therefore, exercises investigated both specific abilities and the capacity to effectively combine sub-abilities.

To determine BMD, we used 10 exercises that required the use of specific materials. With the exception of cards, which were used in the Frostig perception test and in the Vayer psycho-motor profile (exercises 8 and 9), all the other exercises used Montessori-type materials, that proved useful in facilitating manual experiments.23 The following materials were used (with the number of the corresponding exercise in parentheses): wooden tablets of identical size but different weight (1); wooden tablets of identical size but variable surface texture (2); pairs of pieces of fabric of equal size but different consistency (3); box of various geometric forms to be inserted into corresponding holes (4); needle and thread (5); wooden board with holes placed along different lines, wooden needle, string (6); wooden grate, anatomical forceps, string (7); sheet of paper with labyrinth pattern, black pencil (8); sheet of graph paper (9); pack of playing cards (10); box with double opening for the examiner to see the movement of students' hands during tests of tactile discrimination (1, 2, 3); chronometer.

Description of exercises (Table 1)

Table 1 Description of exercises
Full size table

The first three exercises are done blindly; students are asked to examine the basic characteristic of each object (weight, roughness and consistency), which in our opinion are the prerequisites to develop manipulative ability; candidates are then asked to execute them with their hands inside a box with a double opening, for the examiner to observe how they perform.

Another exercise is intended to develop the ability to quickly distinguish different small solid objects (exercise 4) and to place them into corresponding concave spaces, thus showing the skill to manipulate with dexterity in small spaces.

Next (exercises 5-9), specific exercises were formulated to study eye-hand coordination, highlighting the fundamental importance of the eye and the hand working together and modifying the action performed, according to their respective sensory information. Tasks were performed that required acting with just one hand and using the other as a support (exercises 5 and 6). The following exercise required operating with both hands in a co-ordinated action (exercise 7).

The last two exercises of this group (exercise 8 and 9) were the only ones taken from pre-existing materials; in particular, the first one, which was a test on rapidity, was taken from the Vayer psycho-motor profile,19 while the second one, a labyrinth, from the Frostig perception test.20

The final exercise (exercise 10) investigated laterality. This test did not focus so much on laterality already achieved, but rather on the possibility of too wide a discrepancy between using the dominant hand and the other, during the execution of the same task.

The test we used made possible an evaluation of BMD, while excluding the capacity acquired through experience or training; in other words, an appropriate test for young people fresh out of high school.

Evaluation criteria

Exactness in execution, time employed and strategies used were the essential parameters to evaluate each applicant for each exercise. For each exercise four points were given to the best performance and one point to the worst (partial scores). At the end of the test, each applicant had 10 partial scores; the average of partial scores represents the Basic Manual Dexterity score (BMDS). The Examiners' Board, consisting of three people (one CLOPD teacher and two psychologists who were experts in learning) was the same throughout the entire research project and examined all the applicants to the test, as well as all the subjects of the re-test (see later), to obtain reproducibility and homogeneity of the results.

Finally, we wish to point out that the examiner did not make any comments on the results of the exercises to the students, nor were the results ever revealed at a later stage.

Study population (Fig. 1)

Figure 1: Study design.
figure 1

During the admission exam of five consecutive years, a test for evaluating basic manual dexterity was carried out on all the applicants. A first and a second retest were performed three and five years respectively, after the beginning of the experimentation, only on admitted students. Thus, students admitted to the dental school at the beginning of the experimentation (Group A – 7M/8F) were retested 32 and 52 months after the initial test. On that occasion, students admitted to the school of dentistry during the second (Group B – 9M/6F) and the third (Group C – 6M/9F) years of the experimentation were also retested. The fourth and fifth year students were not retested

Full size image

A total cohort of 433 applicants to the CLOPD of the Catholic University of Rome (262 males and 171 females) were tested for BMD (Table 2) at the same time as the admission test. A manual ability test, that was not intended to have any effect on admission, was carried out at the beginning of five consecutive academic years. Before each test, full instructions were given to each candidate as to how to perform individual exercises (VERBAL ASSIGNMENT). At the time of the experimentation only 15 subjects per year could be admitted to the CLOPD.

Table 2 Study population stratified according to the high school attendance
Full size table

In addition to the parameters reported in the evaluation form (ie the partial scores of each exercise, an overall evaluation and some general observations on the candidate's behaviour during the exercise), the partial scores were averaged for each candidate, revealing the mean value of Basic Manual Dexterity score – BMDS.

It was possible to study the influence of high school of origin and of gender on the BMDS of subjects applying for admission to CLOPD and compare the BMDS of those admitted to the CLOPD with that of candidates who had been refused, in order to assess whether this parameter is in keeping with the results of the test adopted for the selection of students.

Subjects admitted to university courses during the five years of the experiment were also analysed in order to identify any relationship between their initial level of BMD and the average score of all CLOPD exams (of all the five years). In our opinion, this last score, which we call 'university success', is a very important parameter since it is the result of academic, pre-clinical and clinical skills.

Moreover, we wanted to perform a longitudinal study, whereby students were asked to repeat the same manual ability test during the years following their admission to the university (retest). This retest enabled us to identify any improvement, during the years following their admission, resulting from their practical training, and to verify whether the difficulties encountered at the beginning were still present or had been overcome.

Thus, a first and a second retest were performed three and five years respectively, after the beginning of the experimentation, only on admitted students. Therefore, students admitted to the dental school after the beginning of the experimentation (Group A) were retested 32 and 52 months after the initial test. On that occasion, students admitted to the school of dentistry during the second (Group B) and the third (Group C) year of the experimentation were also retested. Group B, first tested at the beginning of the second year of the experiment, was then retested 20 and 40 months later, while Group C, first tested at the beginning of the third year of the experiment, was retested eight and 28 months after the first test.

Statistical analysis

Quantitative variables were tested for normal distribution, compared using the two-tailed ANOVA test; a post-hoc Bonferroni test was used for identification to determine differences between groups. Frequency variables were assessed using the Chi-Squared test and eventually Yates' correction. Odds ratios (OR) were calculated to evaluate if high school and gender were potentially correlated (risk factors) for admission to CLOPD. To determine the statistical significance 95% test-based confidence intervals (95% CI) were used. A linear regression model was used to determine any possible relation between BMDS and university success.

Results

The results of the BMDS average for the entire population as well as of stratification based on high school attendance and gender are reported in Table 3. Comparing BMDS average and stratifying the population on the basis of high school attendance, it was possible to observe some differences among candidates, since those having a prior university degree or a scientific high school degree proved better than those who studied a classical or vocational high school curriculum.

Table 3 Average votes obtained in manual dexterity tests (BMDS) for the total population and stratified according to high school attendance, gender and admission to CLOPD or rejection
Full size table

In the same table, it is possible to appreciate that there are no statistically significant differences between males and females, whereas the BMDS average of subjects admitted to CLOPD (75 out of 433 candidates) was slightly higher than that of candidates who were not admitted.

Table 4 reports score frequencies obtained in each individual test for the entire population and for the population stratified on the basis of admission to CLOPD or rejection, in order to identify the most appropriate test to discriminate candidates. Generally speaking, the distribution of score frequency obtained in each exercise varies markedly and responses to most exercises appear fairly well distributed among at least three of the four levels of partial scoring; on the whole, this confirms the validity of criteria adopted for the subdivision of scores. Only a few exercises revealed abnormal distributions: in exercises 1, 4 and 6, 50% of responses – or little more – were mostly concentrated on one level. The 'eye-hand coordination: rapidity' test was found to be the most useful and statistically significant one (p = 0.03), with a higher distribution of best results (score 3-4) in the group of admitted candidates compared to non-admitted. All the other tests failed to show a statistically significant difference between admitted and non-admitted candidates.

Table 4 Score frequencies obtained by candidates in each test. Total population: 433 subjects; 356 not admitted and 75 admitted during the 5 years of the experiment
Full size table

Odds ratio was also performed to verify whether gender and high school attendance correlated with the ability to pass the admission test to CLOPD (Table 5). Classical and scientific high school degrees correlate significantly, while dental mechanical school attendance negatively correlates with the ability to pass the admission test. A prior university degree, vocational high school attendance and gender do not seem to be correlated.

Table 5 Odds ratio for high schools and for gender
Full size table

In 75 of 77 subjects admitted to CLOPD, the linear correlation between BMDS at the beginning of the course and the 'university success' was not statistically significant (β:-0.4428492, P = 0.672, R-squared = 0.0024).

Results obtained in the test are reported in Table 6. A statistically significant improvement of BMDS has been observed in Group A performances only, while in Group B a non-significant improvement was observed at 20 months from the first test, neither at 40, presumably because of the small number of subjects taking the second test. In Group C we failed to observe any improvement.

Table 6 Average performances for each year and in retests
Full size table

Discussion

Although this study was not intended to describe a new type of test, exercises were meant to investigate elementary skills, such as tactile discrimination, as well as more complex abilities, requiring eye-hand coordination and laterality. All these skills were not specific to dentistry, but were more general manual skills.

Since exercises had no effect on the final evaluation for admission to CLOPD, students did not always take the test at the top of their potential and their level of attention and personal commitment were not optimal. Nonetheless, the 'un-official' atmosphere around the test greatly helped keep anxiety and fear of failure at a low level.

The statistically significant difference found among groups of students coming from different high schools (Table 3) shows to what extent basic manual dexterity is affected by cognitive and logic abilities acquired during high school, rather than from actually exercising such skills. The scientific lyceum aims at developing logics and mathematics skills, while vocational high schools, including the dental mechanic course, seem to develop mainly practical and technical skills. When groups were compared by a post-hoc (Bonferroni) test, we failed to identify any statistically significant difference among groups, presumably because the small group of students with a prior university degree has falsified the post-hoc analysis. The diversity found among student groups confirms the assumption that basic manual ability is not simply practical and mechanical in essence, but strongly correlates with 'global mental capacities'. This confirms that manual ability is not only a matter of 'hand', but also of 'head' and that manual performance is supported by the capacity to plan, organise, analyse and verify.

Moreover, the non-homogeneous distribution of levels of manual ability among groups of candidates attending different high schools raises a number of questions as to the actual validity of BMD as a parameter to discriminate candidates, possibly being a bias in the selection of candidates, precisely because of the non-homogeneous distribution of groups of candidates coming from different educational backgrounds.

Regarding the slight superiority of BMDS average in subjects admitted to CLOPD, compared to those not admitted, the difference showed borderline statistical significance (p = 0.0648 – Table 3), but seems to indicate that the admission test to CLOPD, while mainly assessing academic and cognitive skills, indirectly considers manual abilities as well, since it selects students with a better manual dexterity.

A few important findings may therefore result from this study: 1) in our sample, students attending the scientific lyceum possess a high baseline manual dexterity; 2) attending the scientific lyceum is a variable that positively correlates with passing the current admission test; 3) manual dexterity in admitted subjects, albeit not fully statistically significant, is higher than in non-admitted subjects. Comparing the official results of the admission test (which was not influenced by the result of the manual dexterity test) with those from our study, we notice that the students admitted to the CLOPD (exclusively on the basis of scientific knowledge) had the better manual dexterity (Table 3), even if not completely statistically significant (p = 0.0648).

Our data seem to confirm the conclusions reached by Wanzel,28 according to whom, having assessed the correlations among visual, spatial and manual dexterity, a manual dexterity test for the selection of students does not seem to be necessary. This theory is in keeping with Ackelman's theory25,29 on the learning of skills, suggesting three phases for learning and training purposes: a) a cognitive phase, to understand the theory; b) a phase of association, to diminish the latency time between a given stimulus and the output and c) a phase of autonomy, where performance is given with a minimum of attention, a higher performance speed and a minimum of control on the information flow coming from motor and non motor control systems (ie control on results).

It was also possible to study individual exercises, to possibly find a better test to select students (Table 4). The only exercise that had a different distribution between admitted and non admitted candidates is exercise number 9 (eye-hand coordination: rapidity) that, while investigating fundamental characteristics in the clinical practice, such as performance rapidity, accuracy and the ability to work inside a structure delineating small spaces, seems to be the only one to be taken into consideration as a test to be added to the admission test.

Doubts and question marks may arise as to the validity of using tests investigating manual dexterity as a parameter to predict the global success of students, regarding the non statistically significant correlation between BMDS values at the beginning of the course and the university success (ie academic, pre-clinical and clinical) (Slope: 0.4428392, p = 0.672, R-squared = 0.0024).

It is also important to stress that verbal assignments were a sensitive and relevant part of nearly all exercises; actually, this five year experimentation with the test revealed the need to develop concise, linear and clear assignments.

We deem it appropriate to highlight two limitations of this study: in the analysis of data, we did not introduce an important variable, such as the age of candidates which, regardless of schooling, may have an impact on basic manual abilities. The second limitation was that, even if the examiners' board consisted of the same three people throughout the entire research project, the partial score assigned to each exercise could have suffered from a high grade of subjectiveness.

Samples studied in the retest were smaller, because only 15 students were admitted in each academic year; this could account for the incomplete statistical significance.

Retest results seemed particularly interesting in that they enabled us to identify some characteristics of the test, over a period of time, despite the limitation due to the small sample. First of all, when comparing BMDS averages in Group C, where students were retested after a short period of time, we realised that students had not grown accustomed to the test; also, students never clearly knew any of the reference evaluation parameters and were not fully informed of their errors and this was certainly a proof of the fact that they had not learned from the test.

Students who could follow training for the 52 months of the entire course improved significantly. This finding is very important and in agreement with those of Gansky,4 Luck26 and de Andres27 who feel that manual dexterity can be educated and improved by means of exercise and that tests to assess manual dexterity should be mainly used for identifying those subjects in need of a more careful, attentive and monitored training, rather than to select students right from the beginning. These reflections seem to confirm that manual dexterity can be trained, that it should not necessarily be taken as a parameter of exclusion from a study course like CLOPD which, in the end, thanks to its practical training, offers good manual levels even to subjects with initially poor manual ability. We may therefore infer that an initially poor manual ability does not seem to be a discriminating factor for the success of anyone who wishes to become a professional dentist. This is a particularly important consideration, not only for dentistry, but probably for any other branch of medicine, such as surgery, where great manual ability is required.