How are we Evaluating the Effectiveness of Simulation in Dental Education? Are the Skills Transferrable? A Review

Background: Providing fluent transition of clinical skills from preclinical level into dental clinics is a major concern amongst dental educators, for which various effective and valid tools have been developed. Previous research in dental education have not adequately addressed the effect of simulation events or environment on skill transferability from preclinical level into clinics. Objectives: To find effect of simulation events or environment on skill transferability from preclinical level into clinics. Methods: Out of 3297 searched articles, a total of 15 articles were included in this review searched through “MEDLINE, EMBASE, and CENTRAL” as the main search engines. This included primary studies that comprised original studies and reviews and secondary study in the form of report were included. Findings: Virtual reality simulators have proved their feasibility in diagnosis, obtaining objective and immediate evaluation and developing critical thinking. On the other hand, interactive and Internet-based simulations have proved their efficiency in remote learning, promoting information retention in students’ memories, providing new scopes in courseware development, and enhancing the understanding of some theoretical aspects of dentistry. Finally, conventional simulation is still mastering dental manual practicing process and their role in acquisition of clinical skills remain valid until new technology and business world could create alternative tools. Conclusion: Dental skills transferability is affected by interaction of several factors which affects one another.


Background
Providing fluent transition of clinical skills from preclinical level into dental clinics is a major concern amongst dental educators, [1] for which various effective and valid tools have been developed. The basic requirement for it is a flexible mannequin head with dentoform (plastic teeth) placed at bench type laboratory environment to perform simulation task [2]. However, with the advanced technology and involvement of corporate business in it have changed the dynamics in recent times, where the competence in students' performance necessitates more emphasis on teaching and evaluating methodologies than on simulation. Simulation based learning is well established in professions like nursing. [3] From the aspect of dental education there is need to consider the complex nature of dental surgical procedures and understanding the requirement for assimilating theoretical knowledge and acquisition of practical skills. Innovation like medical simulation, advanced Web development, virtual reality techniques, intelligent tutoring systems, social networking and collaborative authoring tools when combined with the ability to solve dental cases, indeed reflect on the construction and the applications of the chosen tool [4].

Aim
Previous research in dental education have not adequately addressed the effect of simulation events or environment on skill transferability from preclinical level into clinics. This could be attributed to the acceptable level of students' performance in their clinics or it might be resulted from the decreased dental public education for patients who visit educational clinics. However, in this review the focus will direct the attention toward the psychometric effects of simulators in developing transferrable practical skills that will reflect on students' performance and will also reflect on the degree of patients' satisfaction level towards dental treatment..

Data Sources
Out of 3297 searched articles, a total of 15 articles were included in this review. This included primary studies that comprised original studies and reviews and secondary study in the form of report were included. On final selection, this study included only one conference report, whereas all other remaining papers were original studies ( Figure 1). All the study papers involved in this review were chosen by the aid of the "MEDLINE, EMBASE, and CENTRAL" as the main search engines. The primary search terms were "simulation in dental education" then the search was refined by excluding and including key words. Including key words were teaching, evaluation, training, practicing, performance, virtual reality, augmented reality, conventional training, assessment, preclinical, perception, effectiveness, and skill transferability. All of these searched keywords contributed to the selection process to meet the research criteria. While the excluding key words involved all non-dentistry related articles like dental hygiene studies, nursing, dental nursing, medical studies, non-clinical and non-practical focused studies.

Inclusion and Exclusion Criteria
Overall, as prescribed in the finding section, inclusion and exclusion criteria focused on characteristics of the studies, educational intervention, evaluation measures, evidence of effectiveness, and examination of quality. In addition, inclusion criteria comprised publications in English Language only.

i.
Characteristics of the studies: Both primary and secondary study types were involved. Various types of participants from different educational and professional levels in dentistry were also targeted.

ii.
Educational intervention: Simulation devices and innovations used for teaching and assessment of preclinical and clinical dentistry were the focus of this review.

iii.
Evaluation measures: All possible means to evaluate simulation intervention in education were considered in the inclusion criteria of this current study.

iv.
Evidence of effectiveness: Studies that reported successful applications or comparison of simulation devices in accordance with the educational values were included.

v.
Examination of quality: Publications that reported results around the merit of educational tools especially for skill transferability comprised the inclusion criteria of this review.

Characteristics of the studies
The studies were published between 1998 and 2020 and all of them were published in peer-reviewed journal papers and one was conference report, [5] that had presented some abstracts that reviewed clinical skills' teaching quality assessment in dental schools in United Kingdom. Whereas the remaining studies were of experimental manners that either compared different types of teaching styles, teaching equipment, training procedures or studied students' and/or teachers' perceptions toward certain simulation. Sample sizes ranged from ten participants to 243 in one extended study. Participants include undergraduate and postgraduate dental students at different levels of their education period, general dentists, and dental teaching staff members. Furthermore, these research studies were conducted in various countries such as: Thailand, China, United States of America, United Kingdom, Israel, Australia, Brazil, and Canada.

Educational Intervention
All 15 studies were concerned about the quality of teaching preclinical skills using different types of simulations. Five studies [6][7][8][9][10] conducted comparison between conventional training and virtual reality simulation. While six study papers discussed regarding simulation trends in dentistry, [11][12][13][14][15][16] one studied augmented reality [17] and two on haptic simulation [18,19]. Generally, these trends embraced intervening with simulation prototypes in order to achieve effective education by discussing their advantages and disadvantages, and the usage of these tool in enhancement of educational aspects. Whereas the conference report paper [5] reviewed set of published and unpublished abstracts regarding the quality of dental clinical skills training.

Evaluation Measures
Various methods of result assessment were used in the included studies. One form of assessment was a questionnaire distributed among dental students to reflect their responses toward preclinical exercises they had been practiced [6]. Whereas, evaluating psychomotor and behavioral aspects of participating students was the prime measure in most studies. An example of such was multiple performance-based assessments that aimed to make comparison between conventional and virtual realitybased simulation [7]. Another study tested the performance of students in order to predict their performance on conventional simulation [8]. Faculty perceptions and expectations of students' abilities were evaluated by multiple sets surveys. One survey studied faculty perception of non-virtual reality simulation on students' skills and two surveys studied faculty perceptions and expectations of virtual reality simulation on students' skills [9]. Alternatively, combination of performance assessment tests and survey questionnaires were utilized in a multi-studies research set to reflect upon students' skills and attitudes toward virtual reality technology in operative dentistry laboratories [10]. In a Chinese study, dental 3D multimedia system efficiency was assessed through practically assessing students' performances in comparison with conventional simulation training methods [11]. In a similar study, the authors investigated students' perceptions of effectiveness when utilizing web-based interactive learning [12]. They chose students' questionnaire type assessment to evaluate the perceptions [12]. While in another study, simulated patients' scenarios used multimedia programs in interactive way, then the authors allocated a self-assessment computer-based evaluation to measure students' competence [13].
In another reviewed study, internet-based child behavior management simulation was evaluated in cognitive manner by testing students' knowledge using short and open-ended essay questions [14] Apart from web-based simulation, the periodontal training simulation study had used questionnaires distribution among both faculty and dental students for evaluating teaching quality of new simulation device [15]. Another study evaluated clinical performance of experts and dental students over virtual reality simulator to measure the effectiveness of such machine in teaching and assessing dental skills [16].
In the study conducted in Brazil [17], two questionnaires were distributed, first one to assess computational ability and computer knowledge and the second to determine their acceptance regarding new learning objects. Whereas it was conducted on dental students, post graduate students, professors and dentists participating in remedial course in prosthetics and/or dentistry. In the retrospective study [18], dental students who were in their first year were recruited in a preclinical course (Operative Dentistry) and completed a haptic exercise comprised of a manual dexterity test i.e. D-circle. During a single session at midterm, this was repeated eight times in succession. Whereas, for each trial a score was generated by based on accuracy and time taken which resulted in success or failure. Al-Saud et al. [19] allocated participants with no prior dental training into three groups which got different forms of feedback on the four tasks that they performed or were trained in. Here, one group got device only feedback i.e. visual display of simulator, second group received verbal feedback and the third group received the combination of the two. Later on, their skill retention was examined three times at different intervals i.e. immediately after training, at one week and at 1-month post-test. In the last included study, questionnairebased evaluation was used to discuss the perception of 14 UK dental schools on using simulation forms for clinical training purposes [5].

Evidence of effectiveness
All 15 studies reported the effectiveness of simulations in dental teaching and learning processes, and their different roles in improving trainees' dental skills and/or knowledge. On one hand, virtual reality simulation was reported as a useful objective assessment tool which had a positive impact on dental students, other than it decreased the time for faculty intervention during training, which reflected self-paced learning style [6][7][8][9][10]15,16]. On the other hand, students' results and/or faculty simulation perceptions and impressions had showed reasonable degrees of satisfactions toward simulation-based learning in dental skills acquisition as inferred from the included research. VRhaptic technology is highly effective in novice students [18,19]. Augmented reality simulation also had a wide acceptance among the students and practicing dentists [17].

Examination of quality
Evaluating skill transferability through assessing discussed research had been a sophisticated job. On one hand there was scarcity of the papers that had direct focus on the issue, on another hand there was difficulty in judging the strength and the weakness of the effect of simulation on specific skill with considering confounding factors. However, each included study represented some degree of evaluation. In the conference report, [5] the abstracts were focusing on implementing simulation as the main teaching strategy in British dental schools. However, this study is a decade old when compared with recent innovations and running studies in dental simulation field. Four-handed preclinical training [6] may improve students' communication skills and teamwork proficiency. Four-handed model of training is the one where the two hand of assistant help the two hands of dentists, hence reducing the time required for dental treatment. Though, proper individual judgment for dental cases plays major role in dental clinics where no but one dentist usually control the situations. Also, there was poor response rate (47%) in this study due to posting questionnaires through the mail after the end of the semester, which could be considered as a weakening stimulus for the study. However, the outcomes of the training were achieved as per the survey results.
Quinn et al. [7] used small groups of dental students to compare degree of effectiveness of two forms of simulations; the results showed no significance differences between them which meant either the study samples were not enough to robust the evidence or both simulations improved students skills equally. In predicting students' ability before they were being introduced into simulation, Imer et al. [8] used small group in their methodology, even though the results of this study presented high magnitude. Gottlieb et al. [9] successfully examined dental teachers' impression using three survey questionnaires distributed before and after students' training. Additionally, they considered time factor in questionnaires distribution, expectation of faculty members, and effect of simulation on students' performance. Buchanan [10] showed great enthusiasm toward virtual reality simulation through emphasizing on the potential of virtual reality in skill acquisition, although, the study sample she used was small.
However, her assumptions were built on multiple studies results (300 hours training) with single scope, testing the teaching quality of virtual reality simulation in dentistry.
Alternatively, the research members who studied the effects of 3D multimedia program on dental students' performance 11 recruited reasonable number of candidates for their study. By randomly selected 60 students, they achieved high response rate (90%) and checked multiple skills practices and assessments. However, the drawn conclusion did not correlate directly to the research aims. Salajan et al. study [12] had good population sample (193students) with a very good response rates and it was delivering and assessing the same level of knowledge even though some parts of the integrated skills were unclear or incomplete to be evaluated by the participants. Abbey [13] discussed the transferability of knowledge into practice using multimedia as a simulation event. The author successfully explained some newly discovered teaching techniques to enhance students' learning. However, testing these strategies experimentally with large sample size would result in better clarifications about their effectiveness.
Boynton et al. [14] on the other hand studied new trend in dental teaching that employ simulation. The high response rate of participating students was of good magnitude. However, the students' distribution within the study needed to be reconsidered again as there was a conflict raised about the significance of the training due to students' different learning backgrounds. Luciano, Banejee and De Fanti study [15] was directed to new teaching approaches in periodontology. This study also had a small sample size; however, majority of the faculty members participated and evaluated the program via questionnaires. They reflected upon their experience in training and assessment by this newly created simulation device that had been tested for validation before being introduced into this study. Another included study by Rhienmora et al. [16] had a small sample size and a prototype virtual reality simulator. After reviewing this study, it was not clear whether the device had proved its effectiveness in teaching or not. However, device ability to differentiate between novice and expert users might have good value as an objective assessment tool for dental training in the future.

Questionnaire
Based study abstracts

Dental schools
The modification of teaching method will shape dental learning tools.

Questionnaire based Dental students
The 'real life' clinical situation and teamwork increased reality of simulated environment.
Quinn [7] 2003 comparison between conventional training and virtual reality simulation.

performance-based Dental students
Effect of virtual reality simulator on skill acquisition and use of students' reflections on the training process is beneficial. Imber [8] 2003 comparison between conventional training and virtual reality simulation.

performance-based
Dental students Visual simulator is effective in predicting student's clinical potential.
Gottlieb et al. [9] 2011 comparison between conventional training and virtual reality simulation.

Questionnaire based Teaching faculty
According to faculty, student's performance increases by enhancing teaching effectiveness.
Buchanan [10] 2004 comparison between conventional training and virtual reality simulation.
Combination of questionnaire and performance Dental students Student's level of acceptance of new teaching strategy through virtual reality simulation is important for effective teaching.
Hu et al. [11] 2009 simulation trends in dentistry performance-based Dental students Lectures utilizing 3D images enhances student's knowledge comprehension.
Salajan [12] 2009 simulation trends in dentistry Questionnaire based Dental students web-based interactive simulation using self-directed learning expands students' conceptual understanding of theoretical contents in various dental activities.
Abbey [13] 2002 simulation trends in dentistry computer-based self-evaluation Dental students Dental multimedia has got acceptance among students in clinical learning and enhances critical thinking Boynton et al. [14] 2007 simulation trends in dentistry

Open ended essay questions Dental students
Digital text-based simulation increases student's knowledge but not necessarily advances practical skills in pediatric dentistry.

Questionnaire based
Both dental students and faculty Periodontal training simulation which uses digitalized 3D models is a good breakthrough Rhienmora [16] 2010 simulation trends in dentistry performance-based Both dental students and experts Training on virtual reality model can help in evaluation and teaching dental procedures. Even though all the studies assessed students' performance and showed varied degrees of improvement among the trainees, very few of them were concerned about the importance of teaching techniques to improve skills acquisition, or its implementation within the training outcomes. As stated earlier, five studies were compering traditional mannequin simulation with virtual reality technology in simulation [6][7][8][9][10]. Whereas six studies were discussing some prototypes or newly invented dental simulation devices [11][12][13][14][15][16] and the twelfth paper represented a conference report with multiple presentation abstracts [5]. The findings of the study are summarized in (Table 1).

Espejo-Trung
Suvinen et al. [6] invented simulation environment that included "real life" clinical situation with infection control measures and virtual reality head (the cruciform module) with remaining other auxiliaries. While the learning strategies used in this environment had involve motivation, enthusiasm, and teamwork collaboration, as students worked in pairs all along the training course, so they learned by practically experiencing it.
Furthermore, managing variety of dental pathologies increased the reality of the simulated environments. The conference report abstracts 5 explored teaching concepts by implementing and encouraging dental schools to allocated different forms of simulation as integral part of dental teaching curricula. They concluded that the modification of teaching method will shape dental learning tools. While, McConnell, who was one of the abstracts writers, reported that multi-disciplinary simulation environment would be ideal for assessing clinical skills and it would improve students' psychomotor skills. Marshall, another writer, had also emphasized on the effect of clinical environment design on the training outcomes. Lastly, Plasschaert focused on the importance of critical thinking in the form of problem-solving activities as a valuable teaching method.
Quinn at al. [7] studied the effect of virtual reality simulator on skill acquisition and used students' reflections on the training process as evidence to show the benefits from local adjustment of clinical scenarios. Imber et al 8 researched the effect of virtual

Advances in Dentistry & Oral Health
simulator to predict students' clinical potentials, aiming to improve and modify teaching strategies. However, to reach objective gold standard assessment they used students' records to measure the levels of practical improvements along the training period. Another two studies that compered conventional and virtual reality simulation were by Gottlieb et al9 that conducted faculty impressions of students' performances and by Buchanan [10] that reviewed virtual reality applications [10]. Both were concerned regarding enhancing teaching effectiveness as an ultimate goal. Bochanan [10] emphasized on the importance of the level of acceptance by students when introducing new teaching strategy. While another study by Rhienmora et al. [16] described training on virtual reality module that was able to assess and record the training procedures using hidden Markov models (HMMs).
New trends in dental teaching were discussed in the remaining five studies. First, was periodontal training simulator [15] that examined virtual device that mimicked periodontal hand instruments beside periodontal management measures that utilized variety of dental cases with periodontal problems, all showed as 3D models and shapes on the connected computer screen. New attempts in dental multimedia showed acceptance amongst dental educators as represented by both Abbey [13]; Jian Hu et al. [11]. Abbey [13] studied interactive multimedia patient simulation through interactive cases that simulated real clinical patient cases. This problem-solving strategy was able to generate critical thinking and allowed students to play exercises repeatedly, make mistakes, access feedback, and develop self-confidence. While 3D multimedia system study by Jian Hu et al. [11] introduced new computer program that allowed participants to review and to match actual lectures utilizing 3D images that played important role in enhancing students' knowledge comprehension of certain practical courses.
Another two studies [12,14] was implementing Internetbased learning methods. Boynton et al. [14] studied the effect of virtual child simulation on improving students' professional attitudes to describe the process, text-based description of child behaviors cases were displayed on the screen that prompted the student to select one proper action from the list provided for managing specific case. This simulation proved efficient in enhancing students' knowledge rather than their direct practical skills in pediatric dentistry. Salajan et al. [12] investigated web-based interactive simulation using self-directed learning which aimed to expand students' conceptual understanding of theoretical contents in various dental activities. In other study [17], the authors claimed to use this method in tandem with other course materials in order to achieve the highest advantages of knowledge acquisition. Using comparison between conventional and complex haptic simulator in preclinical operative dentistry, authors concluded that the later had significant predicting ability to examine students' performance in the preclinical setting [17]. In another study [18], the authors have examined and compared VR and conventional modalities. They found that there was a significant improvement of students' motor skills in both groups with no clear evidence on superiority of using VR technology. They recommend further investigating studies to know the long-term influence of applying VR in dental teaching. The only study that have prescribed AR technology was from Brazil [19], the authors have developed a new learning object can be used to teach Onlay preparation in operative dentistry. A thorough details of the process was presented and users' perception where then assessed. 2 questionnaires were applied to measure computational ability and the second to evaluate performance of the LO in the learning process. Results shows acceptance from all involved groups. Regardless the restrictions and limitations of this study, AR technology should be considered in future dental research. This study is an upgrade from previous review as AR-based in dental education was not mentioned in previous studies [20].

Conclusion
To sum, dental skills transferability is affected by interaction of several factors which affects one another. However, virtual reality simulators have proved their feasibility in diagnosis, obtaining objective and immediate evaluation and developing critical thinking. On the other hand, interactive and Internetbased simulations have proved their efficiency in remote learning, promoting information retention in students' memories, providing new scopes in courseware development, and enhancing the understanding of some theoretical aspects of dentistry. Finally, conventional simulation is still mastering dental manual practicing process and their role in acquisition of clinical skills remain valid until new technology and business world could create alternative tools.

Suggestions
In order to provide effective simulation:

i.
Dental students should learn the composition and the application of simulation devices, so they will be exposed to them prior to starting actual practice.

ii.
Integrating multidisciplinary interactive multimedia simulation in preclinical courses to enhance students' comprehension and understanding of the real nature of clinical work.
Teaching problem-based learning strategies by adding interactive multimedia programs that represent real patient cases simulation; this could be done by allocating these programs within the diagnosis part of each dental specialty e.g. endodontic, dental surgery, operative dentistry, etc. However, to determine degree of effectiveness of specific simulation, an age-based study is needed in which the study sample will be comprised of high school students, dental students of different levels and graduate dentists. They then will be trained on some skills using simulation with different difficulty levels and by comparing the post training results it will be possible to estimate the degree of difficulty needed for the training process on simulation.