Therapy - Single Trials Clinical scenario

Man with stroke, moderate carotid stenosis

Returning to our clinical scenario from the question formulation tutorial:

You admit a 65 year old man with a stroke. On examination you find that he has mild weakness of the right arm and right leg and bilateral carotid bruits. You send the patient for carotid doppler ultrasonography and subsequently receive the report that he has moderate stenosis (50-69% by NASCET criteria) of the ipsilateral carotid artery. You've noticed in the pile of journals that is accumulating in your office that there has been some recent literature addressing surgical versus medical therapy for patients with symptomatic carotid stenosis but you are unsure of what the results of these studies indicate.

In the tutorial on clinical questions, we formulated the following question: In a 65 year old man with stroke and moderate carotid stenosis, can carotid endarterectomy decrease the risk of stroke compared with medical therapy?

Our search of the literature found article from the Best Evidence (1999;130:33).

How do we critically appraise this therapy paper? We'll start off by considering validity first and the following list outlines the questions that we need to consider when deciding if a therapy paper is valid.

Was the assignment of patients to treatment randomized? And, was the randomization list concealed?
Randomisation helps ensure that patients in treatment groups are identical at the study onset in their risk of the event we are hoping to prevent. It balances groups for prognostic factors (good or bad) that if they were unequally distributed amongst the groups, could increase, decrease or nullify the effect of the therapy.
We need to check if the randomisation list has been concealed from the clinicians who entered patients into the trial. This is done so that the clinicians won't be aware of which treatment the next patient would receive.
The study that we found was randomised (which is one of the inclusion criteria for a therapy article in Best Evidence). From the original article we can see that the randomisation list was concealed and details on the randomisation process were also provided.
Was follow-up of patients sufficiently long and complete?
We'd want to see that the duration of follow-up was sufficiently long to see the outcomes of interest. It is also important that the investigators provide details on the number of patients followed up and if possible, on the outcomes of patients who dropped out of the study. If we are unsure of what effect the dropouts may have on the study result, we can perform a 'sensitivity analysis' for a 'worst case scenario'. For the group that did better, assume that all the people who were lost to follow-up did poorly. For the group that did worse, assume all the people who were lost to follow-up fared well. If the result still supports the original conclusion, than the follow-up was sufficiently complete. It would be unusual for a study to be able to withstand more than a 20% loss of follow-up and therefore most journals of secondary publication (including ACP Journal Club and EBM) use this as an exclusion criteria for article selection.
From the abstract we identified in Best Evidence, 99.7%!! of patients were followed up for 5 years.
Were all patients analyzed in the groups to which they were randomized?
Anything that happens after randomisation can affect the chance that a study patient has an outcome event. Therefore, we need to see if the investigators analysed the patients in the groups to which they were randomised, even if they crossed over to the other treatment group. This 'intention to treat' analysis preserves the value of randomisation.
An intention to treat analysis was done in the study that we identified. (This information was provided in the abstract available on Best Evidence.)

And some less important points:

Were patients and clinicians kept blind to treatment?
Were groups treated equally, apart from the experimental therapy?
Blinding of clinicians and patients helps to prevent additional treatment. The provision of treatment (received in addition to the experimental treatment) to just one of the groups is called cointervention. If either the patients or the clinicians weren't blinded it could lead to the reporting of symptoms or the interpretation of these symptoms to be affected by suspicion about the effectiveness of the treatment under investigation.
In the NASCET study, all patients received antiplatelet therapy (this was usually ASA and the dose was left to the discretion of the neurologist at each study centre), and when indicated they received antihypertensive and or antilipidemic medications.
Blinding is not always possible (such as in surgery trials) and in these situations we should check to see if outcome events were assessed by blinded investigators. For example in NASCET, outcome events were assessed by 4 groups: the participating neurologist and surgeon; the neurologist at the study centre; by 'blinded' members of the steering committee; and by 'blinded' external adjudicators.
Were the groups similar at the start of the trial?
This is usually reported in the 'Table 1' of the article. If the groups aren't similar, we need to see if there was an adjustment made for the potentially important prognostic factors.
The medical and surgical groups were similar in NASCET. For example, the percentages of patients who were prescribed antihypertensive or antilipidemic medications were similar.

If the study fails any of the above criteria, we need to decide if the flaw is significant and threatens the validity of the study. If this is the case, we'll need to look for another study. Returning to our clinical scenario, the paper we found satisfies all the above criteria and we will proceed to assessing it for importance.

Are the results of this study important?

What is the magnitude of the treatment effect?

There are several ways that information about treatment effects can be presented. This discussion will be illustrated using the results of NASCET (for any stroke at 5 years) as shown in the first row of numbers in the table below.

Control Event Rate	Experimental Event Rate	Relative Risk Reduction	Absolute Risk Reduction	Number Needed to Treat
0.264	0.198	25%	0.066	15
0.000000264	0.000000198	25%	0.000000066	15,000,000

The control event rate (CER) is the proportion of patients in the control group (in this study, the group that received medical care) that had the outcome event of interest (in our scenario, this would be any stroke). The experimental event rate (EER) is the proportion of patients in the experimental group (patients in the carotid endarterectomy group) that had the outcome of interest.

The relative risk reduction (RRR) is one way of describing the treatment effects and is calculated as:

RRR = |EER-CER|/CER
= |0.198-0.264|/0.264
= 25%

Applying this, we can say that if we treat people who have moderate carotid stenosis with carotid endarterectomy we can decrease their risk of future stroke by 25% compared to those people who receive medical therapy only.

If the experimental treatment increases the risk of a good event, we can use this same equation to calculate the relative benefit increase (RBI). Similarly, if the experimental treatment increases the risk of an adverse event we can use the equation to calculate the relative risk increase (RRI).

The RRR has limitations. Consider the second row of numbers in the table above - when the CER was incredibly small (0.000000264) the RRR remains at 25%. The RRR is unable to discriminate between small treatment effects and large ones and doesn't reflect the baseline risk of the event.

One measure that overcomes this is the absolute difference between the CER and EER or the absolute risk reduction (ARR). It is calculated as:

ARR = |EER-CER|
= |0.198-0.264|
= 0.066

If the experimental treatment increased the risk of a good event, we can use this same equation to calculate the absolute benefit increase (ABI). Or, if the experimental treatment increases the risk of an adverse event, we can use the equation to calculate the absolute risk increase (ARI).

Returning to the data in the table, we can see that the ARR reflects the baseline risk of the event and that it discriminates between small and large treatment effects. However, because it is not a whole number, it is often difficult to remember and to translate to patients.

To overcome these difficulties, we can take the inverse of the ARR which tells us the number of patients that we'd need to treat with the experimental therapy in order to prevent one additional bad event. This is called the number needed to treat (NNT) and in our example, the NNT is 15. We can see from the table that the NNT (like the ARR) is able to differentiate between small and large treatment effects - in the second row of the table, when the CER and EER are very small, the NNT is over 15 million!

When the treatment increases the risk of adverse events, we can calculate the number of patients that we'd need to treat with this therapy to cause one additional bad event and this term is called the number needed to harm (NNH). The NNH is calculated as 1/ARI.

How big should an NNT be for us to be impressed? Consider some examples. We'd need to treat 40 people who have suspected MI with aspirin to prevent 1 additional death. And, we'd only need to treat 20 people who have suspected MI with aspirin and thrombolysis to prevent 1 additional death. If you want to see more examples of NNTs, please click here.

What is the precision of the treatment effect?

The confidence interval around the NNT can be calculated as the inverse of the confidence interval for the ARR. The smaller the number of patients who have the event of interest, the wider the confidence interval.
Calculate the confidence level for the NNT online.

Where to go from here?

Now that we've decided our article is both valid and important, we need to decide if we can apply it to our patient.

Other options:

Do you want to consider the validity of a therapy paper?
Do you want to see a 'CAT' for this paper? (not yet online)
Do you want to learn about critically appraising:
- Diagnosis articles
- Prognosis articles
- Systematic reviews of therapy articles
- Harm articles
Do you want some practice critically appraising therapy articles from other clinical specialties?
Do you want more reading about critically appraising therapy articles?

Therapy articles from other clinical specialties

Child Health In children with mild to moderate croup, does nebulised budesonide decrease the risk of hospital admission compared with placebo? Klassen, T.P. Feldman, M.E. Watters, L.K. et al. Nebulized budesonide for children with mild to moderate croup. NEJM 1994; 331(5):285-289.
Complementary Medicine In children with hyperactivity, does any form of dietary modification improve behaviour? Schmidt MH, Mocks P, Lay B, Eisert HG, Fojkar R, Fritz Sigmund D, Marcus A, Musaeus B. Does oligoantigenic diet influence hyperactive/conduct-disordered children-a controlled trial. Eur Child Adolesc Psychiatry 1997;6:88-95.
Critical Care Medicine In a critically ill patient, will restrictive blood transfusion practices be equivalent to liberal transfusion practices? A multicenter, randomized, controlled clinical of transfusion requirements in Critical Care. NEJM 1999;340:409-17. EBM in Developing Countries Among Filipino patients with suspected MI, will administration of streptokinase decrease in-hospital mortality? ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988; ii:349-360.
Gastroenterology and Hepatology In a patient with nonulcer dyspepsia and helicobacter pylori infection, will helicobacter eradication therapy result in a reduction in symptoms of dyspepsia? McColl K et al. Symptomatic benefit from eradicating helicobacter pylori infection in patients with nonulcer dyspepsia. N Engl J Med 1998;339:1869-74. In 35 year old man who is HCV RNA positive does treatment with interferon + ribavirin, compared to interferon alone, or no treatment, offer a significant chance of viral clearance? Poynard T, Marcellin P, Lee SS, et al. Interferon a2b and ribavirin increased the loss of HCV RNA in chronic hepatitis C. Lancet 1998;352:1426-32.
General Practice In patients with frequent migraines, is riboflavin effective in the reduction of migraine frequency or severity? Schoenen J, Jacquy J, and Lenaerts M. Effectiveness of high-dose riboflavin in migraine prophylaxis. A randomised controlled trial. Neurology, 1998; 50: 466-470.
General Surgery In patients with acute cholecystectomy, what is the complication rate of laparascopic cholecystectomy versus open cholecystectomy? Kiviluoto T et al. Randomised trial of laparoscopic versus open cholecystectomy for acute and gangrenous cholecystitis. Lancet 1998;351:321-5.
Geriatric Medicine In patients with isolated systolic hypertension, do diuretics decrease the risk of stroke and death? SHEP co-operative research group. Prevention of stroke by antihypertensive drug treatment in older persons with isolated systolic hypertension. Final results of the systolic hypertension in the elderly program. (SHEP). JAMA 1991;265:3255-64. In an elderly patient who lives at home, does a comprehensive geriatric assessment decrease the risk of nursing home admission and improve functional status? Stuck AE, Aronow HU, Steiner A et al. A trial of annual in-home comprehensive geriatric assessments for elderly people living in the community. NEJM 1995;333:1184-9.
Mental Health In a mentally ill homeless person, does a "critical time" intervention prevent extended homelessness? Susser E, Valencia E, Conover S, et al. Preventing recurrent homelessness among mentally ill men: a 'critical time' intervention after discharge from a shelter. Am J Public Health 1997 Feb; 87: 256-62.
Neonatal Medicine In a term infant with hypoxic respiratory failure, does the use of inhaled nitric oxide decrease the need for ECMO? Ehrenkranz R.A. The Neonatal Inhaled Nitric Oxide Study Group. Inhaled nitric oxide in full-term and nearly full-term infants with hypoxic respiratory failure. N Engl J Med 1997; 336(9): 597-604.
Nursing In school age children with colds, are zinc lozenges safe and effective for relief of cold symptoms? Macknin ML, Piedmonte M, Calendine C, Janosky J, Wald, E. Zinc gluconate lozenges for treating the common cold in children. A randomized controlled trial. JAMA 1998;279:1962-7.
Physiotherapy Practice Is prophylactic physiotherapy for patients undergoing upper abdominal surgery effective in preventing post-operative pulmonary complications? Fagevik Olsen M, Hahn I, Nordgren S, Lonroth H, Lundholm K. Randomized controlled trial of prophylactic chest physiotherapy in major abdominal surgery. British Journal of Surgery 1997;84:1535-1538.
Purchasing Should rivastigmine be considered in patients with Alzheimer's for improving quality of life and reducing carers' costs? Rosler M, Anand R, Cicin-Sain A, Gauthier S et al. Efficacy and safety of rivastigmine in patients with Alzheimer's Disease: international randomised controlled trial. BMJ 1999;318:633-40.