Assessment of Critical Thinking

The topic of critical thinking may have caught your attention because it is a frequently heard “buzz word” in academics or you may be interested in helping your students to become better critical thinkers. This section provides an introduction to critical thinking and discussion of how critical thinking can be applied to on-line courses. Although the topic of critical thinking can be fairly complex, there are many simple ways that instructors can begin to integrate critical thinking into their courses.

What is Critical Thinking?

Critical thinking, most simply defined, is thinking about your thinking using various standards so that your decisions and/or actions are sound. It can be used to evaluate and explore information in virtually all academic disciplines (e.g., science, management, history). Exploration and learning through discovery are what learning is all about. Exploration involves considering new ways of doing things, new answers and new perspectives. Critical thinking methods enhance learning through active involvement and reflection. Numerous definitions of critical thinking exist. Commonly cited definitions of critical thinking are given below.

Definitions of Critical Thinking

The "intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning or communication, as a guide to belief and action." (Scriven & Paul, 2004)

The "art of thinking about your thinking while you are thinking in order to make your thinking better: more clear, more accurate, or more defensible." (Paul, Binker, Adamson, and Martin, 1989 as cited in Longview Community College, 1996)

"Purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based." (American Philosophical Association, 1990)

"A process which stresses an attitude of suspended judgment, incorporates logical inquiry and problem solving, and leads to an evaluative decision or action." (NCTE Committee on Critical Thinking and the Language Arts, as cited in Tama, 1989)

"Reasonable, reflective thinking that is focused on deciding what to believe or do" (Ennis, 1985)

It might be nice to have a universal definition of critical thinking but there is not a consensus of opinion about the definition of critical thinking in the literature. In fact, there are many differences between the existing definitions of critical thinking. In a search of three on-line databases (CINAHL, MEDLINE, and EBSCO) from 1982-2002 using the keywords "critical thinking", Turner (2005) found over 150 key characteristics to define the term. Examples of these key attributes of critical thinking include: analysis, judgment, inference, interpretation, reflection, explanation, knowledge, openness, and self-regulatory. Most definitions of critical thinking address reasonable and reflective thinking and an active process in which the thinker manipulates the information in some way (e.g., evaluating).

 

Thinking question:
How can critical thinking, decision-making, and problem-solving be compared? Are they just different words for the same activity?

The process of problem solving and decision making are improved when critical thinking is used. Critical thinking is a process that can be used within these other processes to assure informed action choices and good judgments ( Brookfield , 1987). Critical thinking is broader and deeper than problem-solving or decision making (O Murtha & Muirhead, 2005).

Facione (1998) identified six core critical thinking skills which serve as the building blocks for critical thinking. The core skills are interpretation, analysis, evaluation, inference, explanation, and self-regulation. Interpretation deals with finding meaning or significance in situations, events, and data. Analysis is the process of sorting out "intended and actual relationships among statements, questions, concepts, descriptions or other forms of representation" (Facoine, 1998, p.4). Evaluation involves judging the value and credibility of situations and opinions as well as the logical soundness of the relationships. Inference involves activities such as drawing conclusions, forming hypotheses, and considering consequences. Explanation involves presenting one's reasoning in a clear and logical argument. Self-regulation, a skill that ensures continued improvement in thinking, involves monitoring one's cognitive process and correcting errors and biases as needed. Core critical thinking skills are listed below with example activities.

Core Critical Thinking Skills

Critical Thinking Skill
Example
Interpretation
  • Distinguishing a main idea in text from a subordinate idea
  • Organizing or categorizing a topic under study
  • Clarifying what a chart or graph means
  • Observing behavior and identifying meaning
Analysis
  • Identifying similarities and differences of various alternative solutions
  • Recognizing unstated assumptions
  • Determining the reasons offered to support a claim made in an article
Evaluation
  • Determining the credibility of a source of information
  • Judging the logical strength of an argument given hypothetical cases

Inference

  • Use information from multiple sources to formulate a synthesis
  • Develop a set of options for a current problem

Explanation

  • Describe the factors considered in making a complex decision
  • Give research results and describe the methods and criteria involved

Self-regulation

  • Change your conclusion because you had misjudged the importance of specific factors
  • Examine your views on a controversial issue and consider your own biases and personnel interest

Taken from: Facione, P.A. (1998). Critical thinking: What it is and Why it Counts. Retrieved from http://www.criticalthinking.org/aboutCT/definingCT.shtml

The critical thinking examples above illustrate the process used by good critical thinkers. Activities such as questioning, developing alternative solutions to a problem, and examining whether an assumption is true are important in critical thinking. Critical thinking reaches down to a deeper level of knowledge beyond mere factual knowledge. Critical thinking is based on the "ancient Greek ideal of living an examined life" (Foundation for Critical Thinking, 2004). Similarly, the ideal of learning is an examined process.

Thinking question:
What does thinking look like when it does not involve critical thinking?

Much of the thinking people do each day is not critical thinking in that it does not involve reasoning based on evidence. Examples include living life in an unexamined way, voting uncritically without learning the issues and giving thought to long-term implications, taking on a passive role in healthcare with little questioning of health providers, uncritically teaching as you were taught "giving assignments that students can mindlessly do", and missing opportunities to encourage critical analysis (Foundation of Critical Thinking, 2004).

Why is Critical Thinking Important to On-line Instructors?

Many university departments are encouraging instructors to teach and evaluate the critical thinking skills of students. This shift in learning focuses on teaching students how to think so that they can analyze and problem-solve situations in all areas of their lives. Proficiency in professional endeavors requires integration of sound critical thinking skills along with discipline-specific competencies. In contrast, traditional learning methods focus on what to think and acquiring knowledge (versus using knowledge) is the primary goal. It is assumed with traditional methods that students will learn how to think as they acquire more knowledge. A critical thinking approach to learning, which teaches thinking skills explicitly, requires new curriculums and strategies that incorporate necessary and sufficient conditions for critical thinking.

On-line learning is fertile ground for the development of critical thinking skills because it uses key modalities. Both discussion and writing are important strategies in developing critical thinking. Research in traditional classrooms by Goodlad (1984) found that less than 10% of classroom time was spent on discussion (as cited in Crow, 1989). The majority of time in the classroom was spent on lecture. In virtual classrooms, however, the lecture approach to teaching has largely been thrown out and discussion formats are usually the central pedagogical method. The asynchronous nature of many on-line courses gives ample opportunity for reflective thinking within discussions thereby enhancing the quality and depth of the discussion. Equally important, assignments in virtual classrooms are primarily centered on writing. Writing promotes greater reflection and ensures that every student participates. Wade (1995), after many years of trial and error approaches to teach critical thinking, concluded that "writing is an essential ingredient in critical thinking instruction" (p.24). She further described that when students write,

. they cannot remain passive players in the learning game. Even the simplest writing task, such as a summary of an article, requires that students make important critical choices: What information is most important to this argument? What might be left out? More complex writing assignments ask students to make more difficult choices about a topic (p.24).

Given that most on-line courses contain modalities optimal for teaching critical thinking, t he challenge becomes one of designing and structuring courses to teach and support critical thinking. Coursework needs to include instruction and coaching about critical thinking skills, assignments that require students to use critical thinking skills, and feedback to students about their thinking.

How Can Critical Thinking be Implemented in On-line Learning?

There are many ways to implement critical thinking in on-line courses. It is beyond the scope of this paper to fully cover this topic. Rather, this section covers where to begin when integrating critical thinking into on-line courses. Instructors can begin by:

  1. Developing an environment that supports critical thinking
  2. Using questioning as a central method in developing critical thinkers
  3. Incorporating critical thinking assessments into courses to monitor and encourage learning.

Creating an Environment that Encourages Critical Thinking

Integrating critical thinking into the classroom requires more than crafting several higher level thinking assignments. It starts with an instructor modeling a commitment to one's own engagement in critical thinking and using critical thinking in the classroom to solve problems ( Brookfield , 1997). A classroom environment that supports critical thinking includes structuring of interactions to foster critical thinking, feedback that focuses awareness on metacognition, and use of questioning to obtain and process information. It is also important to help students to understand the purpose and benefits of critical thinking so that they can be open to the opportunity to learn and use critical thinking.

The role of the instructor changes from "sage to guide, giver to collaborator, and instructor to investigator" (as cited in O Murchu & Muirhead, 2005). Instructors and students begin to view learning as a shared responsibility with students taking an active role in the learning process. Students participate in the critical processes of acquiring, interpreting, analyzing, evaluating, making inferences, and explaining information.

Students enter the classroom with a range of critical thinking skills and affective dispositions (e.g., open-minded, inquisitive). Critical thinking needs to be made the subject of instruction and expectations need to be clear (Crow, 1989). Many instructors focus primarily on subject matter and assume that students know how to think about the content. Explicit teaching of critical thinking is the goal. The statement "the critical mind is a questioning mind" highlights a key strategy used in critical thinking (Foundation for Critical Thinking, 2004). Research has shown that questions can be used to teach critical thinking. Learning is advanced when students are given example questions and coached with appropriate support and feedback (as cited in MacKnight, 2000). Discussion prompts, for example, that guide students to address specific critical questions about a required reading set more clear expectations for students and raise the level of thinking overall.

Thinking question:
Some authors have alluded to the idea that instruction in critical thinking should take priority over subject matter content? What issues are important to consider in taking on this perspective and what evidence can be used to support this position?

A few initial ideas to start answering this question . Although subject content has been the focus of traditional education, advancing the depth of understanding (versus breadth) results in greater interest and retention of the material covered (Muilenburg & Berge,2000). Schafersman (1991) opined that when the information content of a discipline increases, as we have seen in science for example, it becomes even more important to spend time on critical thinking skills (e.g., interpreting and evaluating information).

Using Questions to Encourage Higher Order Thinking

The "level of questions asked influences the depth of thinking that occurs" (MacKnight, 2000, p. 39). Questions that deal strictly with memorization of facts (e.g., who are the characters in the story) result in a surface knowledge of the topic. Questions that probe beyond the facts (e.g., How is the author thinking about the world?) generate more debate, discussion and further questioning. Every thought can be considered in more depth by considering at least four directions (Paul, 1991):

Questions such as these, used to cover a wider and more in-depth area of the subject matter, are referred to in the literature as Socratic prompts or full-spectrum questioning . These options are like tools in the builders tool belt, each "tool" selected is based on the purpose of the task. A more complete list of critical thinking questions with examples is given in Appendix A. Another well-known source is Bloom's Taxonomy which covers six cognitive levels of questioning (Allen, n.d.). Questions can be used as part of the structure of an assignment or as tools to deepen the dialogue when moderating discussion. An example assignment based on critical questioning asks students what questions they should ask about a self-help book before taking its advice seriously based on what they know about scientific methods and critical thinking" (Wade, 1995). Sample questions used when moderating a discussion to deepen the dialogue include "Why do you agree with that point?" and "How do you define the term you used?". As a moderator, caution is suggested against too strongly directing the flow of questioning so that the sense of ownership by participants is not lost (Collison et al., 2000).

Instructors can also encourage students to examine their own thinking processes (metacognition) with question prompts. Examples of metacognition reflection questions include "What have I learned?", "What's working well?", "Does this make sense?", "At what point in the class did I feel most engaged?". Metacognition questions can be incorporated into post-unit activities, learning logs, collaborative group assignments, journals and other course activities.

Assessment of Critical Thinking

Formative assessment refers to on-going evaluations to improve instructional methods and learning. Continuous monitoring and practice throughout the course is important in the development of critical thinking skills.

Classroom Assessment Techniques

Various classroom assessment techniques (CATs) can be used to evaluate critical thinking. Most CATs involve students "reflecting on or explaining their learning, usually by writing brief, anonymous responses to simple questions or prompts" (Angelo, 1995, p.6). CATs are formative assessments, intended to be used throughout the semester for the purpose of giving teachers and students information about learning. Sample CATs include (Angelo & Cross, 1993):

CATs make the learning process explicit and help students to develop self-assessment skills and metacognition. It is important to give feedback to students about Classroom Assessments. Often by discussing the results of the assessments, students become more interested in the process of learning and develop a stronger sense of their classroom as a learning community (Angelo & Cross, 1993).

Rubrics Used to Assess Critical Thinking

A variety of rubrics and rating scales can be used to evaluate critical thinking. Rubrics (or "scoring tools") are a set of scoring criteria that provide objective standards with which to evaluate student performance. Rubrics clarify faculty expectations for students and assist instructors in prioritizing critical thinking. For the purposes of assessment, critical thinking can either be broken into its essential parts or viewed as a whole.

Washington State University (WSU) developed The Critical Thinking Rubric which describes developing skills in seven areas essential to good critical thinking (e.g., problem identification, multiple perspective taking, examination of evidence) (Washington State University, n.d.). A two-level rating is used for each skill area (i.e., emerging or mastering). The rubric has been viewed favorably by students and faculty, and critical thinking abilities of students at WSU have improved with the use of the rubric. Instructors have adapted the rubric in various content areas (e.g., history, physics, crops and soil).

Facione and Facione (1994) developed the Holistic Critical Thinking Scoring Rubric which is composed of four levels of critical thinking. An overall critical thinking rating is assigned to students, reflecting the level of critical thinking and presence of personal dispositions relevant to critical thinking. The highest level of critical thinking is used for an individual who "consistently does all or almost all" of the activities involved in critical thinking (e.g., accurately interprets evidence, evaluates major alternative points of view).

Sinclair Community College in Dayton , Ohio (2004) uses the Thinking Checklist as a guide for faculty in evaluating student work and as a description for students of the characteristics of good thinking. It contains 10 simply stated traits that are part of good thinking (e.g., provide support, recognize assumptions, respect alternative approaches).

Hofmeister & Thomas (2005) developed a rating scale to assess the cognitive complexity of on-line discussion board postings. Scoring categories advance from a simplistic text- dependent response (level 1) to a more complex and insightful text -independent response (level 4). More complex discussion, for example, involves considering alternative viewpoints or addressing the "so what?" question.

Exam Questions to Assess Critical Thinking

Summative evaluation can be used to evaluate critical thinking in a variety of ways. Often instructors believe that a multiple choice format is not conducive to measuring critical thinking. This is incorrect; well-designed multiple choice questions can evaluate thinking skills. Multiple choice questions can be developed using any of the critical thinking categories (e.g., assumptions, inference). An inference question, for example, can have a description of data in the body of the question and students must evaluate several interpretations of the data as choices (Donovan, 1989).Similarly, an assumption question can describe both the data and interpretation of the data in the body of the question and students need to identify the assumption necessary to reach that conclusion. Additionally, short case studies can be used in the body of the question with subsequent questions asking for inferential or evaluative information.

Short answer or essay questions that ask students to explain and justify their position or draw conclusions from data presented, for example, serve as good measures of critical thinking skills. A sample short answer question is, "Contrast the relative advantages and disadvantages of the light and electron microscope" (Schafersman, 1991, p.11).

Problem-solving assignments and case studies, which require students to apply information learned, are especially valuable learning opportunities for students and provide strong evidence of critical thinking to instructors. For example, a common short answer test question could ask for a list of procedural steps to collect a specimen for microbiological studies; a task which requires memorization of the steps. A more challenging question using the same content could provide a scenario in which a lab technician collects a specimen; Students are asked to list mistakes made by the technician (Schafersman, 1991). This adjustment of the original question requires memorization as well as reasoning (e.g., evaluation).

Best Practice Examples of Critical Thinking in Online Courses

The following course examples demonstrate innovative learning activities that foster critical thinking. These examples represent a small sample of possible options available to instructors.

Wade (1995) used a very straight forward approach to integrating critical thinking into her courses. She developed her own model of critical thinking based on activities that critical thinkers perform including:

  1. Asking questions and wondering
  2. Defining problems clearly
  3. Examining evidence
  4. Analyzing assumptions and biases
  5. Avoiding emotional reasoning
  6. Avoiding oversimplification
  7. Considering alternative interpretations
  8. Tolerate uncertainty.

She integrated the above activities into lectures and assignments in her Introductory Psychology course. Sample assignments include:

Lewis Kleinsmith, in the Biology Department at the University of Michigan , developed a new course, entitled the "Biology of Cancer" and built the syllabus (and course activities) around three questions. The questions were: 1) What is cancer, 2) What causes cancer? and 3) Can cancer be cured or prevented? (Kleinsmith,1989).

Mock interview exercises are used at Pennsylvania State University to promote critical thinking ( Walker , 2005). Students read about a person who is an expert in a related content area and are required to develop questions to ask the expert and cite reasons for choosing the questions. Questions and justifications are then discussed and critiqued by the class.

Collaborative groups are used often in virtual classrooms to increase the interest level among students and promote critical thinking (MacKnight, 2000). Collaborative learning allows students to build on ideas of others, think in novel ways, ask good questions, and hear other view points. Brookfield (1997) views learning to think critically as a "social process" (p.19). Instructors and other students become "important critical mirrors" that enlighten thinking about assumptions and perspectives which individuals may hold (p.19). Hearing the perspective of peers adds clarity to one's own positions and reasoning. Collaborative groups can be structured in many ways, including using a focus question (e.g., "how do good readers read") or case study to guide the discussion. Jigsaw groups are another option in which subgroups are formed to discuss various parts of a topic and then the information is presented to the full group.

Hypothetical scenarios and case studies can be used as a means to simulate real-life problems. Scenarios involve, for example, situations where the main character is making some kind of decision or initiating a plan of action. Students are instructed to put themselves in position of the main character and consider the assumption underneath the character's actions, ways to check the validity of the assumptions, and other perspectives and options (Brookfield, 1997).

References

Allen, T. (n.d.) Taxonomy of educational objectives. Retrieved from http://www.humboldt.edu/~tha1/bloomtax.html

 

Angelo, T.A. & Cross, K.P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco : Jossey-Bass Publishers.

 

American Philosophical Association, Committee on Pre-College Philosophy (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction - The APA Delphi Report (ERIC Doc. No. ED 315 423).

 

Brookfield , S. (1987). Developing critical thinkers . San Francisco : Jossey-Bass Publishers.

 

Brookfield , S. (1997). Assessing critical thinking. New directions for adult and continuing education, Fall, 17-29.

 

Collison, G., Elbaum, B., Haavind, S., & Tinker, R. (2000). Critical-thinking strategies. In Facilitating online learning: Effective strategies for moderators (pp. 127- 164). Madison , WI : Atwood Publishing.

 

Crow, L.W. (1989). The why, what, how, when and who of critical thinking and other higher order thinking skills. In L.W Crow (Ed.), Enhancing critical thinking in the sciences (pp. 9-12). Washington D.C. : Society for College Science Teachers.

 

Donovan, M.P. (1989). Critical thinking questions for examination and exercises. In L.W. Crow (Ed.), Enhancing critical thinking in the sciences . Washington , D.C. : Society for College Teachers.

 

Ennis, R.H (1985). Goals for the critical thinking curriculum. In A.L. Costa (ed.), Developing minds (pp. 68 - 76), ASCD, Virginia.

 

Facione, P.A. (1998). Critical thinking: What it is and why it counts (pp. 1-16). Retrieved from
http://www.criticalthinking.org/aboutCT/definingCT.shtml

 

Facione, P.A., & Facione , N.C. (1994). Holistic critical thinking scoring rubric. Millbrae , California : The California Academic Press

 

Foundation for Critical Thinking (2004). Our concept of critical thinking. In The Critical Thinking Community . Retrieved September 10, 2005 at http//www.cct@criticalthinking.org

 

Hofmeister, D, & Thomas, M. (2005). Using virtual learning modules to enhance and assess students critical thinking and writing skills. In P. Comeaux (Ed.), Assessing on-line learning (pp.72-87). Bolton , MA : Anker Publishing Co. Inc

 

Jones, D. (1996). Critical thinking in an online world. Retrieved from http://www.library.ucsb.edu/untangle/jones.html

 

King, A. (1995). Designing the instructional process to enhance critical thinking across the curriculum: Inquiring minds really do want to know: Using questioning to teach critical thinking. Teaching of Psychology, 22 (1) , 13-17.

 

Kleinsmith, L.J. (1989). Critical thinking about the biology of cancer. In L.W Crow (Ed.), Enhancing critical thinking in the sciences (pp. 49-58). Washington D.C. : Society for College Science Teachers.

 

Longview Community College (1996). Critical thinking across the curriculum project. Retrieved from
http://www.kcmetro.cc.mo.us/longview/ctac/definitions.htm

 

MacKnight, C.B. (2000). Teaching critical thinking through online discussions. Educause Quarterly , 4, 38-41

 

Muilenburg, L., & Berge, Z.L. (2000). A framework for designing questions for online learning. Retrieved at
http://www.emoderators.com/moderators/muilenburg.html

 

O'Murchu, D. & Muirhead, B. (2005). Insights into promoting critical thinking in online classes. International Journal of Instructional Technology and Distance Learning, June. Retrieved from www.itdl.org/Journal/Jun_05/article01.htm

 

Paul, R.W. (1991). Dialogical and dialectical thinking. In A.L. Costa (Ed.), Developing minds: A resource book for teaching thinking (pp. 280 - 289). Alexandria , Virginia : Association for Supervision and Curriculum Development.

 

Center for Instructional Technology at Northern Essex Community College (n.d.) Some techniques for writing multiple-choice items that demand critical thinking. Retrieved from http://cit.necc.mass.edu/atlt/TestCritThink.htm

 

Scriven, M. & Paul, R. (2004). Defining critical thinking: A draft statement for the National Council for Excellence in Critical Thinking. Listed on The Critical Thinking Community at http://www.criticalthinking.org/aboutCT/definingCT.shtml

 

Sinclair Community College (2004). Thinking checklist. Retrieved from http://www,sinclair.edu/about/gened/assessment/tools/thinking/index.cfm

 

Schafersman , S.D. (1991). An introduction to critical thinking. Retrieved from http://www.freeinquiry.com/critical-thinking.html

 

Tama, M.C. (1989). Critical thinking: Promoting it in the classroom. ERIC : ED 306554. Retrieved from http://www.ericdigests.org/pre-9211/critical.htm

 

Turner, P. (2005). Critical thinking in nursing education and practice as defined in the literature. Nursing education perspectives , 26(5), 272-277.

 

Washington State University (n.d.). The critical thinking rubric . Retrieved 9-12-05 from http://wsuctproject.wsu.edu

 

Wade, C. (1995). Using writing to develop and assess critical thinking. Teaching of psychology , 22(1), 24-28.

 

Walker, G. (2005). Critical thinking in asynchronous discussions. International Journal of Instructional Technology and Distance Learning, June. Retrieved from www.itdl.org/Journal/Jun_05/article02.htm


Guest Lecturer

Susan Gallagher-Lepak, RN, PhD, is an Assistant Professor at the University of Wisconsin - Green Bay in the Professional Program in Nursing.