Purposeful Use of Technology to Support Critical Thinking

A mixed methods research study was undertaken to examine undergraduate nursing students ‘perspectives one course redesigned to include purposeful integration of technology to support an inquiry-based approach to critical thinking instruction. The face-to-face and online technology-enhanced environments of learning were developed as settings to generate inquiry to achieve the desired student outcome of critical thinking. Critical thinking is necessary for graduates from higher education programs. With technology’s growth in use, diversity, and prevalence, technology has become a significant component in work and educational contexts. A course was planned to include inquiry to facilitate the development of critical thinking plat formed within two technology-enhanced environments. The findings of the study revealed student perspective of the use of technology with the instructional approach of inquiry, and their demonstration of the desired learning outcome of critical thinking. Thus, identifying factors important to the designing of a course to appropriately use technology to support student learning


Introduction
To have familiarity with technology and critical thinking are valuable assets for undergraduate students in higher education and upon graduation seeking employment Rochelle et al. [1] With the large extent of information easily accessible through technology use, being capable of intellectually and strategically acquiring and using information has become essential for students Benjamin et al. [2] Consequently, critical thinking is integral in education to develop discerning professionals capable of delivering quality decisions amidst complex work situations. Additionally, the increased presence of technology in workplaces and higher education leads to emphasis on integrating technology into education to enhance teaching and learning practices Kim et al. [3], Lavin et al. [4], Reeves et al. [5]. Inclusion of technology into education has potential to foster student capability of using technology to communicate, and manage and apply information.
Educational technology being the integration of appropriate technology to facilitate learning and development aligns with a constructivist approach to student learning and development, one that promotes participation in active learning, and critical thinking and inquiry De la Harpe & Peterson [6], Tedesco-Schneck [7]. Technology integration into education has been noted to be both innovative and disruptive Garrison [8]. Novel utilization of technology offering alternative means of teaching and learning can be beneficial but also necessitate change and thereby unsettle traditional practices. Educators unwilling to integrate technology into their courses cite obstacles including lack of support, availability and reliability of technology, and personal lack of knowledge and confidence leading to anxiety with technology use Wachira & Keengwe [9]. This study therefore aims to determine how to promote appropriate utilization of technology in teaching and learning to align with identified learner outcomes which for this course were to support student critical thinking development. The research question was therefore: To what extent can incorporating technology into the learning environment contribute to students' development of critical thinking? Attaining understanding of student perspective of beneficial instructional strategies with technology use can facilitate how best to design a course to support their learning. This can help educators more informatively include technology into courses they are teaching.

Technology Chosen
To familiarize students with technology, incorporating teaching strategies that use technology into higher education programs is advantageous. Course designs including the use of technology within the curriculum will create new teaching and learning strategies for educators and students Ainsley & Brown [10], Axley [11], Ezeonwu, Berkowitz & Vlasses [12], Patterson, Kilpatrick & Woebkenberg [13]. However, the technology used needs to be appropriate and the integration purposeful for the outcomes desired. Subsequently, the rationale for the technology selected was that it was suitable for the instructional Abstract A mixed methods research study was undertaken to examine undergraduate nursing students 'perspectives one course redesigned to include purposeful integration of technology to support an inquiry-based approach to critical thinking instruction. The face-to-face and online technology-enhanced environments of learning were developed as settings to generate inquiry to achieve the desired student outcome of critical thinking. Critical thinking is necessary for graduates from higher education programs. With technology's growth in use, diversity, and prevalence, technology has become a significant component in work and educational contexts. A course was planned to include inquiry to facilitate the development of critical thinking plat formed within two technology-enhanced environments. The findings of the study revealed student perspective of the use of technology with the instructional approach of inquiry, and their demonstration of the desired learning outcome of critical thinking. Thus, identifying factors important to the designing of a course to appropriately use technology to support student learning. approach chosen and the outcome sought for -critical thinking development. A classroom response system was decided upon for the in-class technology component, and the online discussion forum as the out-of-class element for inquiry-based instruction.

Classroom Response System
The classroom response system is a polling method that can be used in face-to-face learning environments to gather student responses. This software or hardware provides instructors with the opportunity to determine student understanding of the material that has been presented, to poll them as means of engagement and/or assessment of learning, and to collect summative assessment data or survey data Deal [14], DeBourgh [15] [21].
Instructors can focus the inquiry from recall to reasoning, by creating specific types of questions with applicable content, as they construct the classroom response system learning environment to promote critical thinking De Bourgh [15]. As such, questions can be created to generate enlightening discussions and further engage students in discussion and develop critical thinking Beatty, Gerace, Leonard, & Dufresne [22], Mollborn & Hoekstra [19], Shieh & Chang [23].
Anthis [24] noted that improving student awareness of selfmonitoring ability to comprehend the material being taught, as is accomplished through the classroom response system, can promote met cognitive ability, which can be beneficial to student learning and critical thinking. Integrating a classroom response system into teaching and learning can therefore have potential to engage students in active learning and critical thinking. However, its usefulness is dependent on "effective usage . . . which in turn depends on time and effort invested by the instructor in developing effective questions and in working these questions into the class flow in a seamless manner" Sevian & Robinson [25].

Online Discussion Forums
Online discussion forums are environments supported by technology to facilitate instructor and student interaction through asynchronous and synchronous dialogue. The online forums encompass a text-based, written technological communication strategy to "support information dissemination and interpersonal interaction across geographic and temporal boundaries" Dennen [26]. When utilized as the online learning component of a course, the discussion forum offers learners extension of their learning and interactions away from the classroom environment. The asynchronous nature of the online discussion forum cultivates depth and reflection in the learners, supporting its value in developing critical thinking Dennen [26]. The benefits of asynchronous online discussion forums include affording learners "anytime and anywhere" teaching and learning, the capability to store the written dialogue, time to process learning and provide reflected thoughts, and the online "unseen" forum to counter in-person shyness Arend [27], Cheong & Cheung [28], Pisutova-Gerber & Malovicova [29], Shieh [30]. Similar to the use of the classroom response system, the quality of the questions is significant, with more importance attributed to asking the right questions as opposed to seeking the right answers Richardson & Ice [31].

Use of Face-to-Face and Online Technology
While the use of the classroom response system promotes more immediate quick thinking, the online discussion forum allows for time to attain more reflective thinking. As an educational tool, classroom response systems have been acknowledged to encourage student engagement and participation, assessing student understanding of class content, and facilitating deeper thinking with extended discussion and inquiry stimulated by the responses and questions from student responses Dapremont & Lee, Shieh & Chang [23], Sternberger [32]. The online discussion forum encourages greater reflection, time to process and assimilate learning, and afford student demonstration of comprehension and application of learning. Participation in the online discussion is mediated through written communication which has demonstrated positive outcomes of reflection and critical thinking, eliciting higher cognitive functioning Dennen [26], Garrison [8], Szabo & Schwartz [33].
The use of the classroom response system and the online discussion forum as individual educational technologies has demonstrated some successes in engaging learners in critical thinking. Integrating these two technologies into a course would potentially have the capacity to engage students in critical thinking, by providing alternative learning environments to complement and supplement student learning. Significant to both is the need to develop quality questions to engage students in critical thinking. Questioning in the face-to-face setting with the classroom response system can initiate engagement in critical thinking, which can lead to discussion online to further stimulate inquiry, critical thinking, and reflection. Questioning has therefore been noted to be a strategy to facilitate classroom response systems and online discussion forums to engage students in critical thinking.

Using Inquiry as an Instructional Approach
From the literature reviewed, inquiry, using questioning, was identified as important to the development of critical thinking, with the types of questioning utilized to elicit the depth of thinking Richardson [36]. Modeling, scaffolding and coaching were cited to have the potential to provide initial support, guidance and encouragement for student development of own inquiry to achieve critical thinking Hosler & Arend [34], Stein & Wanstreet [37].
Inquiry, as such, has been considered significant to the development of critical thinking. Inquiry recognizes questions as important in education to promote discussion and reflection to facilitate meaningful thinking Garrison [8]. It has been contended that inquiry is the stimulus for thinking, with questioning driving thinking Paul & Elder [38]. In striving to encourage deep thinking, one would utilize deep questioning. Aligning with Bloom's Taxonomy of Educational Objectives which identify intended learning outcomes as instigated through the content of instruction Krathwohl [39], the type of questions asked would elicit the type of thinking desired from the students. Introducing remembering and understanding questions to stimulate recall and comprehension, with progression towards deeper thinking using application, analytical, evaluative, and creative questions to foster critical thinking. Using questions to promote analysis, interpretation, inference, explanation, evaluation, and selfregulation Facione [40] would evoke reflection beneficial to the development of critical thinking.

Implementation of Research Study
The study was implemented in a beginning nursing course with undergraduate students as the participants. The class was instructed by a faculty instructor. There were 127 students enrolled in the class. All students were involved in the study as the inclusion of the technology and the critical thinking instruction were incorporated into the course.
The course content was taught by the faculty instructor. The questions were created by the researcher for the classroom response system and the online discussion forum from the course material provided by the instructor. The questions were developed to stimulate thinking, and promote understanding and application of course content. The outcome of the responses to the classroom system was shared with the class to initiate discussion, inquiry, and thinking. Questions were also generated for the online discussion forum by the researcher to encourage further thinking and understanding of the course topics.

Research Sample
The identified population of novice nursing students in a higher educational setting were purposefully sampled. Data from this group of intentionally targeted students, in the early stages of their education, will help to better understand how to include technology and develop critical thinking in undergraduate students just beginning their studies in higher education. The majority of the students were 19 to 20 years of age, having just recently graduated from high school. Implementing the study with this class was an opportunity to instil the need for critical thinking and technology use within the students as they starting their programs of study. The implementation of technologyenhanced learning for this study was also an opportunity to further encourage students to develop their capability to learn with technology. Its use in education and future employment is ever present and will likely increase in the future.

Research Design
A mixed methods design collecting quantitative and qualitative data from the students was implemented. The quantitative data included the results from a critical thinking skills test and the Likert scale questions of the end-of-term survey. The end-of-term survey included 25 close-ended questions centred on five foci aligned with the research questions. These endeavoured to capture the student perspective of technology utilization to cultivate critical thinking: i.
the impact of a classroom response system on the development of student critical thinking; ii.
the development of student critical thinking within an online discussion forum; iii. the student perspective on technology integration into instruction; iv. the student perspective on the instructional approaches used to facilitate their critical thinking; and v. student satisfaction with the use of technology to enhance their learning environment.
The 25 close-ended Likert scale questions were based on a 5-choice scale, which ranged from strongly disagree, disagree, to agree and strongly agree, with a neutral choice in the middle. The qualitative data encompassed the open-ended questions in the end-of-term survey, and the online discussion postings. The open-ended survey questions were an opportunity for the students to respond using their own words. The four questions focused on the usefulness of the technology used and the type and content of the questions asked through the classroom response system and the online discussion forum. The responses provided a reflection of student perspectives and understanding of critical thinking which they demonstrated through their answers, and included expressions depicting characteristics of critical thinking. Data analysis was aligned with the type of data collected. The responses to the Likert scale survey questions yielded to descriptive analysis. While thematic analysis was more applicable for the responses to the open-ended questions. Content analysis was used with the online discussion postings.
Reflective journaling by the researcher was included to inform on the research process, including decisions made, course materials designed, and questions developed.

Schedule of the Study
The students were informed of the research study before the start of classes and then again on the first day of classes. To ensure informed consent, after being fully informed about the study they were asked to complete a consent form. The technologies were trialed in class with the students to ensure they were comfortable and knew how to use them before implementation for the course. A survey consisting of 25 Likert scale close-ended questions and four open-ended questions was implemented on the last class of the term (Appendix A).

Findings from the Study
Of the 127 registered students, 43 students (34%) completed the final end-of-term survey. In relation to the intentional use of technology in the course, the students expressed appreciation for having both the classroom response system and the online discussion forum as the technology used and aimed at supporting their achievement of their learning outcome. The student perspective was generally favourable towards the inclusion of technology to support learning and critical thinking. Of the responses, 63% (27 of 43 students) agreed that the use of the classroom response system and online discussions provided them with opportunity to actively participate in their learning, with 56% (24 of 43 students) experiencing engagement in their learning. One student stated that "It made class less boring and I was more alert. I was able to actively use the information I learn[ed] in class right away which helped me understand the lecture and information more." Another student wrote, "I was forced to engage in the content more actively." The students noted two themes for their endorsement of using the two forms of technology: (a) providing variety of technology to develop critical thinking, and (b) meeting students' diverse learning preferences and needs with different forms of technology.

Variety of Technology
From the survey responses, 65% (n=28) of the 43 students who responded noted that both formats of technology-enhanced learning were beneficial to their development of critical thinking. They valued having different ways to strengthen and support their learning, and opportunities to practice responding to different types of questions. Students noted that both forms of technology "helped me be aware of time constraints while teaching the importance of reflection," and "both technologies encouraged participation and critical thinking." Students appreciated the variety of different approaches, "both technologies were beneficial. They allowed for different methods of reinforcing my learning" and "they represent two ways of answering questions."

Diverse Student Learning Preferences
With the diversity in today's classrooms and different individual learning preferences, it is beneficial to provide variety in educational approaches and technology use to engage students and develop critical thinking Andreou, Papastavrou, & Merkouris [41], Spencer. From the end-of-term survey data, 54% (23 of 43) of the students agreed that the integration of the classroom response system and the online discussions addressed variety in learning needs. Of the total student responses to this end-of-term survey, 58% (25 of 43 students) preferred the classroom response system while 40% (17 of 43 students) indicated preference for the online discussions as tools to develop critical thinking. A student shared that "I really liked using Top Hat [the classroom response system]. It was a great way to test my knowledge." On the other hand, another student noted that "online discussions were more beneficial because I had to critically think about the topic and create a cohesive response. It required me to think about the topic and everything [that] influences the topic."

Preference for the Classroom Response System
From the perspective of the students, the classroom response technology was identified as beneficial to their development of critical thinking, with 70% (30 of 43) of respondents indicating an enhanced experience of participating in their learning with its use. Of the 43 end-of-term survey responses, 58% (25 students) expressed increased engagement and 75% (32 students) indicated they actively participated having the classroom response system in the classroom. The use of the classroom response system was deemed to reduce angst when responding to questions by 74% of the students (31 of 43 students), with 56% (24 of 43 students) indicating it increased their satisfaction with learning. In sum, the students considered that including the classroom system as in-class technology was beneficial in helping them develop critical thinking.
The reasoning expressed by the students in identifying a preference for the classroom response system was that the questions used would "better prepare for exams," or "I got to see the type and level of questions that I would see on my exams and it also was a great review for exams." Students also stated that "anonymity was more comforting" and "I wasn't afraid of answering wrong." Certain students appreciated that the classroom response system "allowed for immediate discussion among peers," rendered "immediate feedback and classroom participation," and "was done in class time and it covered what we just learned."

Preference for Online Discussion
From the end-of-term survey data, 50% of the students (22 of 43 responses) did indicate enhanced participation in their learning through the use of the online discussion forum. While 60% (26 of 43 students) considered online discussions helpful in promoting reflection on classroom learning, 40% (17 of 43 students) did not consider using it a pleasant addition to their learning. The online discussions were engaging for 42% (18 of 43 students), promoting active participation for 40% (17 of 43) of the surveyed students. Students noted that the online discussions facilitated connections and helped them apply their learning in such comments as [they] A. "allowed me to connect my learning and use critical thinking" B.
"allowed me to incorporate learning from other courses and what we have learned about in the class that day" C.
"helped me make greater connections between class concepts" D. "required us to apply our knowledge and our beliefs".
The online discussions were perceived to be beneficial, as "[they] made me think of an answer rather than discern which answer was right/wrong," or "discussion questions enforce critical thinking and help relate lecture material to practical applications." The students recognized the value of online discussions in developing critical thinking, writing that "the questions do prompt me to reflect on the course information" and "by thinking and reflecting together about the questions and connections [they] shaped my understanding and enhanced my critical thinking." Another student noted that "they allow me to pull from the knowledge I learned in class and apply them to questions in discussion format." The students viewed the online forum as opportunities for further discussion, thinking, and collaborative learning. They valued the additional time and a place to discuss issues with other students online. According to one student this "allowed me to see what others were thinking and allowed [me] to further develop my understanding of the topic." As another student responded: I like how the discussions allow us to speak with others in the class and see how opinions differ among our classmates. I also like how the discussion questions let us think about and review topics that we've discussed in class [42,43].

Questions Developed
To ensure alignment with course content, the questions were created based on the course content and relevant information. The purpose was to further student learning and application of material. The questions created for the classroom response system and the online discussions were founded on the Bloom's revised taxonomy Krathwohl [39], and focused on encouraging the applying, analyzing, and evaluating levels. Appendix B shows examples of these questions, which asked the students to apply their learning, and analyze and evaluate practice situations. The level of thinking desired promoted by the questions asked.
Of the 43 end-of-term survey respondents, 93% (40) regarded the types of questions and responses utilized within the in-class and online learning environments to be beneficial in facilitating their development of critical thinking. Additionally, 69% (30 of 43) of the students did ascertain that the questions implemented with the classroom response system and the online discussions required them to use critical thinking to respond.

Classroom Response System Questions
Of the questions used for the classroom response system, the students identified preferences for questions with multiple correct answers and questions which applied course content to real practice situations. Students appreciated "the variation between 'select all that apply'." From the 43 survey responses, 81% (35 students), noted that discussion following the classroom response system questions encouraged thinking and facilitated knowledge building. Students identified "applications questions," "ethical questions, problem-solving scenarios," "ones addressing current headlines on the news," and "questions about what factors caused a condition encouraged me to look holistically at the client" as beneficial to their development of critical thinking.

Online Discussion Forums
Online discussion questions that students deemed beneficial involved being asked to incorporate their personal perspective, apply course knowledge, and incorporate practice scenarios. The online discussions were thought helpful in promoting reflection on classroom learning by 60% (26 of 43) of the students. Additionally, 49% (21 of 43) considered the types of question used for the online discussions as helpful to their understanding of the course information. One student summarized how the online discussion questions were supporting her learning: I like how the discussion questions place us in different situations and in that way, forces us to combine both the lecture material to real life.
This helps cement what we learned beforehand and in our readings by making it more personal and thus, more meaningful. Additionally, the researcher did facilitate the online discussions and students did appreciate the responses which encouraged them to expand and deepen their thinking. A student shared that "I like how [the facilitator] replied to our discussion, asking Socratic questions that would encourage more discussion and thought." Other students acknowledged, "I like the inputs from [the facilitator] . . . makes me think harder about the question" and "allowing us to expand and how [the facilitator's] responses increase further insight or further thought." These comments indicate that these students valued the questions and responses as they encouraged deeper thinking.

Conclusion
To encourage appropriate use of technology to enhance student learning, the technology chosen needs to be suitable for the instructional approach undertaken and the learning outcomes desired. The technology needs to be suitable to meet the requirements of the course objectives to facilitate student learning towards the desired outcomes. There are benefits to having different instructional approaches to support the development of critical thinking, and meeting the different preferences and needs of students. To encourage student participation, the technology needs to be purposefully integrated into the course, and that the technology is conducive to student engagement and learning. When students understand the reasoning for using a particular technology and have clear instructions about how to do that, it can be established so that it enhances learning.
Incorporating variety in technology allows for different means of learning and thinking. Using a classroom response system in-class facilitates on-the-spot thinking, provides immediate feedback, and instigates classroom discussion. An online discussion learning environment provides for more reflective thinking and promotes discussion that requires clear written communication. The different approaches consequently enable student learning with different means of questioning and responding. With a classroom response system, the questions require that students select responses from pre-determined answers, while the online discussions require students to develop written responses that are informative and clearly convey their answers.
The students did note varied preferences for the use of the two forms of technology: the classroom response system and the online discussion forum. The classroom response system was favoured because (a) it could be completed in class time, (b) the questions could be answered by the student choosing a response and (c) they found the in-class discussion to be useful.
The online discussion forums were favoured because they offered the students (a) time for reflection, and (b) the opportunity for discussion outside of the classroom. With both the classroom response and the online discussion forum, the level and content of the questions influenced student learning. It was therefore important that the questions developed for use with the technology align with domain knowledge, promote inquiry and critical thinking. The questions needed to be aimed at stimulating the higher levels of thinking -to encourage students to apply, analyze, and evaluate course content and to make connections to real life practice situations. Students expressed the questions they deemed beneficial as those which connected and applied their learning to real life situations, personalized their learning, led to further discussion, and encouraged reflection, thinking, and understanding.

Summary
The findings from this research study have revealed student perspectives on the use of technology to support their learning and development of critical thinking. The students have identified what they find helpful in the types and content of questions useful to their learning as platformed on the two different forms of technology. Intentional integration of different technology into a course provides students with opportunity to experience different forms of technology and appeal to different student learning preferences.

2.
Was the integration of both technologies more or less beneficial to your development of critical thinking? Why or Why not?

3.
What questions used during Top Hat stood out for you in encouraging you to critically think?

4.
What questions or prompts used during the Online Discussions stood out for you in encouraging you to critically think?

Appendix B
Examples of questions used withthe classroom response system and online discussion forums: To promote critical thinking using Blooms' revised taxonomy cognitive levels

1.
Online Discussion Question: RNs work with many of the population who are at high risk for contracting Influenza and developing serious complications. Considering the infectivity of influenza, should influenza vaccination be mandatory for RNs? Why or why not?

2.
Online Discussion Question: How do SEDoHs influence the risk of TB in the Aboriginal, immigrant, and homeless populations? Why might there be an increased risk of TB in correctional facilities? What SEDoHs influence the risk of acquiring STIs? Explain.

3.
Top Hat Question: The public health nurse is preparing to target high priority populations for the yearly Influenza Clinic. Which groups should be considered? Select all that apply.