Note: This was developed lead a journal
club lasting about 50 minutes. Feel free to adapt it for your own purposes.
I. Select a provocative article
A. If possible, the article should be one you pulled as a result of an encounter with a particular patient. If not, it should deal with a clinical problem we commonly encounter.
B. It should report original research. Reviews are out -- the article needs to have a methods section. Meta-analyses, decision analyses and cost-effectiveness analyses are OK, but they are harder to assess critically because the results often depend on whether you can trust the authors and their underlying assumptions. I usually don't bother with them if they are industry sponsored.
C. If the methods are valid, it would change the way we diagnose, treat, or conceptualize the clinical problem or would clarify management of something currently controversial. (It's hard to excite people about critically reading an article which, if valid, would mean that we shouldn't do anything differently.)
D. Sometimes a pair of articles with opposite conclusions, as long as they are not too long or difficult. (For example, back to back articles on coin ingestions with opposite conclusions and recommendations made a good journal club.)
II. Prepare yourself
A. Read the article critically. Write out what the authors did, what results they got, and what they concluded according to the outline below.
B. Think about each of the decisions they made in designing the study, and what they concluded from the results. Were these good design decisions? Were the conclusions reasonable? What are possible problems with the design, sampling, measurements, and so on? How likely are these problems? How would they impact on the results and conclusions?
C. Pick out a few MAIN POINTS OR CONCEPTS that you think are most important in reading this study critically. Examples of these sorts of concepts are: sample size, bias in measurement of outcome, loss to follow-up, unrepresentative subjects, etc.
D. Meet with me (or other designated preceptor). (Give me a copy of the article in advance). I will go over the study with you, and help you identify and understand key concepts.
III. Prepare the participants. A journal club is always better if people show up having read the article!
A. Distribute the article about 7 days in advance.
B. Make sure everyone has received the article and knows that you are looking forward to their participation
C. Bring a couple of extra copies of the article to the session. Several people usually forget to bring it with them, even if they have read it, and it helps if everyone has a copy in front of them.
D. MAKE SURE EVERYONE KNOWS DATE, TIME, AND PLACE!
E. The best way to do this is to
1. Look at the schedule and get the names of everyone who is supposed to attend
2. Either on a separate cover page or on the first page of the article, write the date, time and place of journal club.
3. Write the name of each person who is supposed to attend on the copy of the article you are going to give to them.
IV. Leading the discussion
A. Basic rules and tips
1. Start and end on time! (Of these, ending on time is most important!)
2. The more key points the participants make themselves (rather than you pointing them out) the better. Avoid lecturing and answering your own questions!
3. Try to make sure everyone is involved and interested. It is OK to call on people, including faculty members, if you do it in a nice way. If people fall asleep, it is perfectly acceptable to wake them up; it is distracting to others to have anyone clearly not participating. Similarly, if one or two people are dominating the discussion, say, "I want to hear from some other people now" and try to get others into the discussion
4. Use the board. Many of us have much better visual than auditory memories--it really helps us to see things written down. Also, putting stuff on the board helps keep the people who come late from slowing things down by asking stuff that has already been covered. It also helps remind you to proceed in a systematic fashion. It is helpful to think in advance about what you will put where on the board. Otherwise you end up erasing stuff you wish was still up there. Write small and only put down key things. One good way to do this is to put the RQ, Design, Subjects, etc. in order starting at the top left of the board, and leave the right side for issues that get brought up during the discussion
B. Format for discussion: Just as with a clinical case presentation, it is helpful to review the factual information before proceeding to discussion of judgement and interpretation. Plan on spending the first few minutes reviewing the clinical case that led to your selecting the article, and maybe a minute or two on how you found this particular article. Then take about 20 minutes going through what the authors of the study did, what results they got, and what they think the implications for clinical practice are. Then the second half of the discussion can center on whether the design and results justify their conclusions, and what to do with the patient that led to your pulling the article.
C. Tips on timing: A common problem is to run out of time just as the discussion is getting interesting. This can result from spending too much time on the more boring stuff at the beginning. You don't want the discussion of what is in the article (as opposed to the interpretation part) to go on for more than half the session. You can speed up by reducing the number of things you ask the participants or the number of chances or amount of time you give them to answer. For example, if you are short on time, you can just write the inclusion and exclusion criteria on the board yourself, rather than asking participants to find them and read them to you. Similarly, if you are afraid things are going too fast, you can slow things down by involving the participants more.
V. Outline of the content of the article: The same sort of learning that allows one to get better at obtaining relevant information from a patient, organizing it, and presenting it to others applies to reading journal articles as well. After using the structure below to review the article yourself, lead the journal club participants through it. Write the main headings one at a time on the board, explain what they mean, and get the participants to fill in the data from the paper. The elements of a study, analogous to the Chief Complaint, HPI, and so on are:
A. Authors and funding source: This is analogous to the "identifying information and source of history" you're taught to put at the very beginning of your H & P. It's a good idea to start with these items so you don't forget them later. Who are the authors? Do you know of any of their previous work, and has it been reliable? Who paid for it? It's not that research sponsored by industry (or, for that matter by the NIH) is necessarily untrustworthy, but knowing who sponsored it, just like knowing the study design, gives you a head start at knowing what sorts of biases to look for. For example, if a study sponsored by a drug company finds that their drug is unsafe or inferior to others, you can probably assume that the results have been carefully scrutinized, and any possible threats to their validity have been evaluated!
B. Research Question: What is the question this study was designed to answer? Sometimes it helps to picture a clinical situation you'll be better able to handle if the study is valid. Examples of research questions are: "Does oral amoxicillin reduce morbidity in infants 6-24 mos old with fever > 39 degrees and no source?" or "Does passive smoking increase hospital admissions for respiratory disease in children?" Often the last line of the abstract gives the author's answer to the research question.
C. Study Design: What type of study is this? Randomized blinded trial? Cohort study? Case-control study? Cross-sectional study? Case series? If you are having trouble remembering the differences between these, you might want to review a book called "Interpreting the Medical Literature" by Stephen Gehlbach, or any basic epidemiology or clinical epidemiology text. I can also help you with this.
D. Study subjects: Who was in the study? How were they selected? Who was excluded? How many subjects were there? Knowing how they selected the subjects is important in order to know whether the study results are valid (sometimes called "internal validity") and whether they are generalizable to the sort of patients you are likely to see ("external validity").
E. Predictor variable(s):
1. What they are: Sometimes called "independent variables," predictor variables are what the authors think might cause or predict changes in the outcome variable. For example, in a randomized trial, the main predictor variable is group assignment: i.e., whether the subjects were randomized to get the test drug or the placebo. In a study of passive smoking and respiratory tract disease, it would be some measure of passive smoking. In the studies of coin ingestions, predictor variables were all the things the authors thought might predict whether the coin would pass spontaneously into the stomach--things like what size of coin, how long ago the ingestion occurred, and whether it was causing difficulty swallowing.
2. How they are measured: For example, passive smoking may be measured crudely by asking the number of adults in the house who smoke or the amount the mother smokes. Sometimes problems with how the variables are measured invalidate the study.
F. Outcome variables:
1. What they are: the clinically significant phenomena the investigators are trying to predict, prevent, or treat. Examples are presence or absence of disease, measures of symptom burden, survival time, etc. Watch for studies that show an effect on an outcome variable that is only marginally interesting. For example, moderate jaundice affects neonatal BAER's but has no effect on their hearing. Some studies of cough suppressants compare cough latency (the amount of time it takes a dog to cough when his trachea is irritated); these studies have little clinical relevance.
2. How they are measured: If it's a disease, what are criteria for diagnosis? Are those determining clinical improvement blinded to the treatment group of the subjects?
Sometimes it takes some effort to figure out exactly what the authors were measuring. For example, in studies of drugs for asthma, a common outcome measure is the percent improvement in peak flow or FEV1. If a child comes in with FEV1 = 40% of predicted, and improves to 70%, that could be considered a 30% improvement (70%-40%). Or, it could be considered a 75% improvement ((70%-40%)/40%). It's important to clarify this, e.g., to know whether a 30% improvement is clinically significant.
G. Results: What did they find? Usually the key results are summarized in tables or figures--it may be helpful to walk the group through the most important tables to make sure everyone can see what results were obtained. If there's a lot and you don't want to put it on the board, you can make a couple of transparencies.
Make sure you consider not just statistical significance, but the effect size -- that is, the magnitude of the difference between groups. Relative effect size is measured using the risk ratio (RR), odds ratio (OR) or relative risk reduction (RRR). It is important for assessing causation but less relevant clinically than the absolute effect size, measured by the risk difference, or absolute risk reduction (ARR) and the number needed to treat (NNT). It's probably easiest to illustrate these with a simple example. If the rate of a bad outcome in the treated group is 15% and in the control group is 20%, then the RR is 15%/20% = .75 and the RRR = 1-RR = 25% and the ARR = 20%-15%=5% and the NNT=1/ARR=20.
H. Conclusions: What do the authors think the results mean? At this point don't discuss yet whether you agree with them.
VI. Discussing the validity of the study. The first part of the discussion dealt with facts, all of which were in the paper. The second half of the discussion deals with interpretation. There are no longer clear right and wrong answers--judgement comes into play.
A. Identify possible biases or flaws in the study. Was the sampling scheme reasonable? Were the measurements valid? Is the study design appropriate to answer the research question? Listing possible biases is akin to listing the differential diagnosis.
B. For each one, estimate how likely it is to have affected the validity of the results, and figure out in what direction it would affect the results. This step is crucial. In a case presentation, it's not that helpful just to throw out a lot of obscure possible diagnoses. You need to see whether the features of the case make these diagnoses likely enough that it's worth doing a test for them. Similarly, no study is perfect. When someone suggests a possible problem, you need to discuss whether this is something that is really important, and how it would affect the results. A COMMON error is to dismiss a study because of "flaws" that are unlikely to account for the results or would have biased the study in the opposite direction from what was found. (This is particularly true for randomized double-blind trials, in which most errors will bias the results towards finding no difference between groups.)
VII. Wrapping up: The most important part of the discussion is the "bottom line." MAKE SURE you leave enough time for this! If the journal club started with an actual case, go around the room and see whether the article has changed how people would manage that case. If you don't have a specific case in mind, make one up. For example, at the end of the discussion on coin ingestions you could ask: "OK. You get a telephone call from the mother of a two-year old who has swallowed a quarter 10 minutes ago, but is asymptomatic. How many would have them come in? [Show of hands.] How many would do an X-ray?" Then you can spend the last 5 minutes or so having people justify their answers.
Authors and funding source:
Possible Biases or flaws:
-Biases towards null hypothesis:
-Biases away from null hypothesis:
Vote on who would change practice based on this article: