Critical Appraisal and Critical Thinking

Critical Appraisal & Critical Thinking

What is the difference between critical thinking and critical appraisal? Essentially both are intellectual disciplines that seek to examine the basis of claims made and their veracity. In my view, they are complimentary skills.

What unites critical appraisal and critical thinking is a concern for the adequacy of the justification of conclusions. The oft-cited phrase in medicine: "What is your evidence" is a request for a justification of the choice of diagnostic test or therapeutic intervention. The point of this course is to demonstrate that not all justifications are equal and that some are sounder than others. This is what we mean by reasoning skills.

Critical Appraisal

Sackett et al. have defined critical appraisal as "the application of rules of evidence to clinical (signs and symptoms), paraclinical (laboratory and other diagnostic tests) and published data (advocating specific treatment maneuvers or purporting to establish the etiology or prognosis of disease in order to determine their validity (closeness to the truth) and applicability (usefulness in one's clinical practice). Critical appraisal then is focused primarily on "evidence" derived from empirical research studies published in the scientific literature.

Unfortunately, as the authors of this influential textbook point out, you cannot critically appraise editorials or arguments for that matter. Critical appraisal is indispensable as a technique for evaluating the strength of factual claims in health care. However, it is addressed only to the facts and not to the arguments.

Critical Thinking

Critical thinking looks at arguments that use both facts and value claims and seeks to assess the soundness of such arguments. It can be defined as follows:

Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness. It entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-issue, assumptions, concepts, empirical grounding; reasoning leading to conclusions, implications and consequences, objections from alternative viewpoints, and frame of reference. Critical thinking -in being responsive to variable subject matter, issues, and purposes - is incorporated in a family of interwoven modes of thinking, among them: scientific thinking, mathematical thinking, historical thinking, anthropological thinking, economic thinking, moral thinking, and philosophical thinking.

Critical thinking can be seen as having two components:

1. a set of skills to process and generate information and beliefs, and
2. the habit, based on intellectual commitment, of using those skills to guide behavior.

(A draft statement by Michael Scriven and Richard Paul for the National Council for Excellence in Critical Thinking)

How Do Argument Analysis and Critical Appraisal Fit Together?

Argument analysis and critical appraisal should be viewed as complimentary skills. Critical appraisal consists of a set of questions to be posed of empirical studies in health care. These questions are designed to assist readers in determining how well the study has been designed and carried out in order to determine whether the results are credible. The critical appraisal frameworks should not be seen as guarantors of the truth of a study. They are best regarded as heuristic tools.

If you are familiar with critical appraisal, you can move on to the next section of the course. If you are not familiar with critical appraisal, we recommend you visit the following sites and familiarize yourself with critical appraisal and evidence based medicine.

Centre for Evidence-Based Medicine - Oxford University

Centre for Evidence Based Medicine - Mt. Sinai Hospital

Fallacies

An important component of argument analysis is the detection of fallacies. Fallacies are defined as arguments that appear to be valid, but are in fact invalid. We must remember that in logic, validity refers to the inferences from premises to conclusions. This is very different from the way the term is used in health care and epidemiology where validity often is used synonymously with truth. An invalid argument is one that cannot be credible, that is, there is no reason to accept the truth of the conclusion as it does not follow from the premises.

The closest analogue to a fallacy in the critical appraisal of studies is bias. Bias occurs when faults in study design lead to a misrepresentation or distortion of the relationship between an exposure (such as a drug) and an outcome (such as cancer or reduction of blood pressure). A biased estimate is incorrect, and like an invalid argument, lacks credibility.

There are many fallacies that have been identified in informal logic. We provide here a set of links to sites that have catalogues of fallacies:

Logical Fallacies: The Fallacy Files

Logical Fallacies and the Art of Debate

The Fallacy Zoo

Of particular interest is the following site that catalogues fallacies according to where they can occur in the framework of the Toulmin diagram.

We are interested in starting a collection of examples of fallacious reasoning around health care topics. If you find examples in the media or in the published medical literature, we encourage you to submit them to us. We will be posting them here on the Exercises section of our website.