Pocket Cards

Use the arrows or swipe to the left or right to switch between the pocket cards.

CARD 1A - Prognosis

Is this evidence about prognosis valid?

  1. Was a defined, representative sample of patients assembled at a common point in the course of their disease?
  2. Was follow-up of study patients sufficiently long and complete?
  3. Were objective outcome criteria applied in a ‘blind’ fashion?
    4. If subgroups with different prognoses are identified:
  4. Was there adjustment for important prognostic factors?
  5. Was there validation in an independent group of ‘test-set’ patients?

Is this valid evidence about prognosis important?

  1. How likely are the outcomes over time?
  2. How precise are the prognostic estimates?

Can we apply this valid, important evidence about prognosis to our patient?

  1. Is our patient so different from those in the study that its results cannot apply?
  2. Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

CARD 1B - Qualitative Research

Are the results of this qualitative study valid?

  1. Was the selection of participants explicit and appropriate?
  2. Were the methods for data collection and analysis explicit and appropriate?

Are the results of this valid qualitative study important?

  1. Are the results impressive?

Are the valid and important results of this qualitative study applicable to my patient?

  1. Do these same phenomena apply to my patient?

CARD 2A – Diagnosis

Is this evidence about diagnosis valid?

  1. Was there an independent, blind comparison with a reference (“gold”) standard of diagnosis?
  2. Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)?
  3. Was the reference standard applied regardless of the diagnostic test result?
  4. Was the cluster of tests validated in a second, independent group of patients? Is this valid evidence about diagnosis important?

Is this valid evidence about diagnosis important?

Target Disorder (iron deficiency anaemia) Totals
Present Absent
Diagnostic test result (serum ferritin) Positive
(≤ 65 mmol/l)
731
a
270
b
1001
a + b
Negative
(>65 mmol/l)
78
c
1500
d
1578
c + d
Totals
a + c
809
b + d
1770
a + b + c + d
2579

Can we apply this valid, important evidence about a diagnostic test in caring for our patient?

  1. Is the diagnostic test available, affordable, accurate, and precise in our setting?
  2. Can we generate a clinically sensible estimate of our patient’s pre-test probability?
    • From personal experience, prevalence statistics, practice databases, or primary studies
    • Are the study patients similar to our own?
    • Is it unlikely that the disease possibilities or probabilities have changed since this evidence was gathered?
  3. Will the resulting post-test probabilities affect our management and help our patient?
    • Could it move us across a test-treatment threshold?
    • Would our patient be a willing partner in carrying it out?
    • Would the consequences of the test help our patient reach his or her goals in all this?

CARD 2B – Diagnosis

Diagnostic usefulness of five levels of a test result

Diagnostic test result Serum ferritin (mmol/L) Target disorder (iron deficiency) present Target disorder absent Likelihood ratio Diagnostic impact
n % n %
Very positive <15 474 59% (474/809) 20 1.1% (20/1770) 52 Rule in “SpPin”
Moderately positive 15–34 175 22% (175/809) 79 4.5% (79/1770) 4.8 Intermediate high
Neutral 35–64 82 10% (82/809) 171 10% (171/1770) 1 Intermediate
Moderately negative 65–94 30 3.7% (30/809) 168 9.5% (168/1770) 0.39 Intermediate low
Extremely negative 48 5.9% (48/809 1332 75% (1332/1770) 0.08 Rule out “SnNout”
Totals 809 100% (809/809) 1770 100% (1770/1770)

CARD 3A – Diagnosis

Likelihood ratio nomogram.

CARD 3B – Diagnosis

Pretest Probability

Is this evidence about pre-test probability valid?

  1. Did the study patients represent the full spectrum of those who present with this clinical problem?
  2. Were the criteria for each final diagnosis explicit and credible?
  3. Was the diagnostic work-up comprehensive and consistently applied?
  4. For initially undiagnosed patients, was follow-up sufficiently long and complete?

Is this valid evidence about pre-test probability important?

  1. What were the diagnoses and their probabilities?
  2. How precise were these estimates of disease probability?

Screening and case-finding

Guides for deciding whether a screening or case-finding maneuver does more good than harm

  1. Is there RCT evidence that early diagnosis really leads to improved survival, or quality of life, or both?
  2. Are the early-diagnosed patients willing partners in the treatment strategy?
  3. How do benefits and harms compare in different people and with different screening strategies?
  4. Do the frequency and severity of the target disorder warrant the degree of effort and expenditure?

CARD 4A – Therapy (single trials)

Is this evidence about therapy (from an individual randomized trial) valid?

Was there a fair start?

  1. Was the assignment of patients to treatment randomized?
  2. Was the randomization concealed?
  3. Were the groups similar at the start of the trial?

Was there a fair race?

  1. Was follow-up of patients sufficiently long and complete?
  2. Were all patients analyzed in the groups to which they were randomized?

Some finer points:

  • Who was blinded: Were patients, clinicians and study personnel kept blind to treatment? Were groups treated equally, apart from the experimental therapy?

Is this valid evidence about therapy (from an individual randomized trial) important?

  1. What is the magnitude of the treatment effect?
  2. How precise is the estimate of the treatment effect?

  Event rate = Stroke (mean follow-up 5 years) Relative risk reduction (RRR) Absolute risk reduction (ARR) Number needed to treat (NNT)
Control event rate (CER) Experimental event rate (EER) |CER − EER| / CER |CER - EER| 1/ARR
MRC trial 5.7% 4.3% |5.7 − 4.3%| / 5.7% = 25% |5.7 − 4.3%| = 0.014 or 1.4% 1 / 1.4% = 72
Hypothetical, trivial case 0.000057% 0.000043% |0.000057 − 0.000043%| / 0.000057% = 25% |0.000057 − 0.000043%| = 0.000014% 1 / 0.000014% = 7142857

Can we apply this valid, important evidence about therapy in caring for our patient?

  1. Is our patient so different from those in the study that its results cannot apply?
  2. Is the treatment feasible in our setting?
  3. What are our patient’s potential benefits and harms from the therapy?
  4. What are our patient’s values and expectations for both the outcome we are trying to prevent and the treatment we are offering?

CARD 4B – Therapy

Guides for whether to believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients


A qualitative difference in treatment efficacy among subgroups is likely only when ALL the following questions can be answered “yes”:
  1. Does it really make biological and clinical sense?
  2. Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant?
  3. Was it hypothesized before the study began (rather than the product of dredging the data)?
  4. Was it one of just a few subgroup analyses carried out in the study?
  5. Is this subgroup difference suggested by comparisons within rather than between studies?
  6. Has the result been confirmed in other independent studies?

The likelihood of help vs harm (LHH)

In applying a systematic review or RCT to an individual patient, we need to consider:

  • Our patient’s risk, relative to patients in the trial, of the event we hope to prevent with the treatment: ft.
  • Our patient’s risk, relative to patients in the trial, of the side-effect we might cause from the treatment: fh.
  • Our patient’s perception of the severity of the event we’re trying to prevent relative to the side-effect we might cause: s.

The likelihood of help vs harm is (1/NNT)×ft×s vs (1/NNH) ×fh

For example, suppose we’re applying a trial with an NNT of 9 and an NNH of 12 and we think our patient is at just half the risk of the event but at twice the risk of the side-effect, then the “raw” LHH before we adjust it for our patient’s per- ception of relative severity is 1/9×0.5 vs 1/12×2 = 1/18 vs. 1/6, or three times as likely to harm vs help the patient. However, if our patient regards the severity of the event that the treatment might prevent to be six times worse than the side-effect it might cause, then the final LHH = 1/18 × 6 vs 1/6, or two times as likely to help vs harm.

CARD 5A – Therapy - Systematic Reviews

Is the evidence from this systematic review valid?

  1. Is this a systematic review of randomized trials?
  2. Does it describe a comprehensive and detailed search for relevant trials?
  3. Were the individual studies assessed for validity?
A less frequent point:
  • Were individual patient data (or aggregate data) used in the analysis?

Is the valid evidence from this systematic review important?

  1. Are the results consistent across studies?
  2. What is the magnitude of the treatment effect?
  3. How precise is the treatment effect?

Translating odds ratios (ORs) to NNTs

1. When the odds ratio (OR) < 1. The numbers in the body of the table are the NNTs for the corresponding odds ratios at that particular patient’s expected event rate (PEER). This table applies when a bad outcome is prevented by therapy.

Patient expected event rate (PEER) For odds ratio LESS than 1
0.9 0.8 0.7 0.6 0.5
0.05 209 104 69 52 41
0.10 110 54 36 27 21
0.20 61 30 20 14 11
0.30 46 22 14 10 8
0.40 40 19 12 9 7
0.50 38 18 11 8 6
0.70 44 20 13 9 6
0.90 101 46 27 18 12

2. When the odds ratio (OR) >1. The numbers in the body of the table are the NNTs are for the corresponding odds ratios at that particular patient’s expected event rate (PEER). This table applies both when a good outcome is increased by ther- apy and when a side-effect is caused by therapy.

Patient expected event rate (PEER) For odds ratio GREATER than 1
1.1 1.2 1.3 1.4 1.5
0.05 212 106 71 54 43
0.10 112 57 38 29 23
0.20 64 33 22 17 14
0.30 49 25 17 13 11
0.40 43 23 16 12 10
0.50 42 22 15 12 10

CARD 5B – Systematic reviews

Formulae to convert odds ratios (ORs) and relative risks (RRs) to NNTs
For RR < 1:
NNT = 1/(1 – RR) X PEER
For RR > 1:
NNT = 1/(RR – 1) X PEER
For OR < 1:
NNT = 1 – [PEER X (1 – OR)]/(1 – PEER) X (PEER) X (1 – OR)
For OR > 1:
NNT = 1 + [PEER X(OR – 1)]/(1 – PEER) X (PEER) X (OR – 1)

Can we apply this valid, important evidence about therapy in caring for our patient?

  1. Is our patient so different from those in the study that its results cannot apply?
  2. Is the treatment feasible in our setting?
  3. What are our patient’s potential benefits and harms from the therapy?
  4. What are our patient’s values and expectations for both the outcome we are trying to prevent and the adverse effects we may cause?

CARD 6A – Harm/Etiology

Is this evidence about harm valid?

  1. Were there clearly defined groups of patients, similar in all important ways other than exposure to the treatment or other cause?
  2. Were treatments/exposures and clinical outcomes measured in the same ways in both groups? (Was the assessment of outcomes either objective or blinded to exposure?)
  3. Was the follow-up of the study patients sufficiently long (for the outcome to occur) and complete?
  4. Do the results of the harm study fulfill some of the diagnostic tests for causation?
    • Is it clear that the exposure preceded the onset of the outcome?
    • Is there a dose–response gradient?
    • Is there any positive evidence from a “dechallenge–rechallenge” study?
    • Is the association consistent from study to study?
    • Does the association make biological sense?

Is this valid evidence about harm important?

  Adverse outcome Totals
Present (Case) Absent (Controls)
Exposed to treatment (RCT or cohort) a b a + b
Not exposed to treatment (RCT or cohort) c d c + d
Totals a + c b + d a + b + c + d

In a randomised trial or cohort study:


Relative risk (RR) = [a/(a+b)/c/(c+d)]

In a case-control study:


Relative odds = ad/bc
  1. What is the magnitude of the association between the exposure and outcome?
  2. What is the precision of the estimate of the association between the exposure and the outcome?
To convert odds ratio (or relative odds) to an NNH:
NNH = 1 + [PEER X(OR – 1)]/(1 – PEER) X (PEER) X (OR – 1)

CARD 6B – Harm/Etiology

Patient expected event rate (PEER) For odds ratio LESS than 1
0.9 0.8 0.7 0.6 0.5 0.4 0.3
0.05 209 104 69 52 41 34 29
0.10 110 54 36 27 21 18 15
0.20 61 30 20 14 11 10 8
0.30 46 22 14 10 8 7 5
0.40 40 19 12 9 7 6 4
0.50 38 18 11 8 6 5 4
0.70 44 20 13 9 6 5 4
0.90 101 46 27 18 12 9 4

Can we apply the valid, important results of this harm study to our patient?

  1. Is our patient so different from those included in the study that its results cannot apply?
  2. What is our patient’s risk of benefit and harm from the agent?
  3. What are our patient’s preferences, concerns and expectations from this treatment?
  4. What alternative treatments are available?

Patient expected event rate (PEER) For odds ratio GREATER than 1
1.1 1.25 1.5 1.75 2 2.25 2.5
0.05 212 86 44 30 23 18 16
0.10 113 46 24 16 13 10 9
0.20 64 27 14 10 8 7 6
0.30 50 21 11 8 7 6 5
0.40 44 19 10 8 6 5 5
0.50 42 18 10 8 6 6 5
0.70 51 23 13 10 9 8 7
0.90 121 55 33 25 22 19 18

© 2017 Knowledge Translation Program