Sham Controls in Medical Device Trials

08 Sep Sham Controls in Medical Device Trials

When a drug is found, after being approved by the Food and Drug Administration (FDA), to have unacceptably dangerous side effects or insufficient therapeutic benefits to outweigh its risks, patients can discontinue its use. But what if the approved therapy that is later discovered to be ineffective or unsafe is an invasive procedure or an implanted medical device? Patients who have already undergone the procedure were put at unwarranted risk, and those with an ineffective or dangerous implanted device must decide whether to leave it in their body or incur the risk associated with another procedure in order to remove it. In this sense, medical procedures and devices pose potentially greater harm to patients than drugs do.

Approval standards for high-risk medical devices, however, are generally less rigorous than those for pharmaceuticals. Only 1% of all medical devices reach the market through the premarket-approval route — the only pathway that requires the submission of clinical data. Research has shown that premarket approvals are often based on data from one small trial that used surrogate end points and included only short-term follow-up.1 Blinded, randomized, controlled trials (RCTs), in which the proposed therapy is compared with a placebo or a “sham” (nontherapeutic) intervention, are common for drugs but rare for medical devices. The lack of such trials for devices is due, in part, to the understandable reluctance to conduct a trial in which some patients are subjected to a sham procedure or implant. However, it has long been established that benefits do not have to accrue to all patients in a clinical trial.

In light of mounting evidence that medical procedures can produce a strong placebo effect that can be mistaken for actual effectiveness, I believe it is time for more frequent use of interventional trials in which patients are unaware of their randomized assignment.

For example, on the basis of unblinded trials, a catheter-based radiofrequency ablation of the renal arteries, known as renal-artery denervation, was thought to lower blood pressure. The recent SYMPLICITY trial, however, found renal-artery denervation had no beneficial effect on blood pressure beyond that achieved with a sham procedure.2 Although the reasons for this lack of benefit will be debated for some time, this important result would not have been discovered without use of a nontherapeutic intervention.

Debate over the ethics of performing a sham procedure or surgery dates back more than 15 years to double-blind trials of fetal-tissue transplantation for Parkinson’s disease, discussed by Freeman et al. (1999). The sham procedure involved making twist-drill holes in the patient’s forehead and was considered necessary and ethical for determining whether there was an effect of treatment beyond the placebo effect (there was not). The institutional review board believed that the risks of sham surgery had to be weighed against the greater risks of mistakenly believing an invasive procedure to be useful because of its placebo effect. Indeed, had there been no trial including sham surgery, many Americans with Parkinson’s disease might be receiving craniotomies for only a placebo benefit.

Another important lesson on the value of sham controls came from vertebroplasty, a procedure in which bone cement is injected into a fractured vertebra for treatment of a compression fracture. Vertebroplasty became popular in the early 2000s, on the basis of observational studies and a nonrandomized trial. Fueled by position statements from various U.S. radiologic and neurologic surgical societies arguing the benefits of these procedures, the number of vertebroplasties performed in Medicare patients nearly doubled between 2001 and 2005, increasing from 45.0 to 86.8 per 100,000 enrollees.3 In 2009, however, RCTs that included a group assigned to receive a nontherapeutic procedure found that pain relief in the sham-procedure group was no different from that in the group that received the actual procedure.4 These examples establish not just ethical precedent but also the importance of comparing device-based interventions and surgeries with an equivalent sham control.

It’s important to understand the power of the placebo effect. Researchers at the Institute of Medical Psychology in Munich recently quantified that power for various types of placebo treatments in studies of migraine prophylaxis. They found that 58% of patients had a positive response to sham surgery and 38% had a positive response to sham acupuncture, while only 22% had a positive response to oral pharmacologic placebos.5 This research shows not only an astonishingly high response rate for sham procedures, but also a significantly higher response rate for placebo physical interventions than for placebo drugs. These results highlight the importance of devising a control that will sufficiently distinguish the specific effect attributable to the placebo.

Not all device trials necessarily require nontherapeutic controls. For example, after a therapeutic benefit beyond the placebo effect was established, subsequent iterations of a device would not need to be compared with a sham control. In addition, trials with only objective end points, such as mortality, do not need a nontherapeutic control. Interventional studies that would most appropriately be conducted as blinded RCTs include early studies of a new technology and studies whose primary outcome measure is susceptible to a placebo effect, such as pain.

For example, percutaneous coronary intervention (PCI), a widely used procedure for treating stable coronary artery disease, has never been investigated in a blinded trial. Some nonblinded RCTs have shown that PCI has a beneficial effect on anginal symptoms, but there appears to be no difference between PCI and medical therapy in rates of the objective end points of nonfatal myocardial infarction and death due to cardiac causes. It is possible, therefore, that the perceived symptomatic benefit is actually a placebo effect and not attributable to PCI. Although a blinded trial would be relatively straightforward if two groups of patients were randomly assigned to a cardiac catheterization procedure, as was done for renal-artery denervation, such a study has yet to be performed, and the important question of PCI’s actual clinical benefit therefore remains unanswered.

Subjecting patients to sham procedures is not without risk, and it gives rise to ethical concerns about “unnecessary” invasive procedures that will have no actual therapeutic effect. I believe, however, that the examples above show that sham interventions are ethical when the benefits of information from a sham-control trial exceed the risks of using an intervention not shown to be more therapeutic than a sham. Moreover, the risk associated with performing unnecessary procedures should be weighed against the risk of mistaking a placebo effect for therapeutic benefit and therefore subjecting thousands or millions of patients to a procedure that actually does them no good. In a controlled trial, patients are informed of and consent to the risks; when an ineffective procedure is accepted into practice, however, patients who subsequently undergo it most certainly have not knowingly consented to an ineffectual procedure. Without careful use of nontherapeutic controls, we may be subjecting millions of Americans to harm from risky, invasive procedures without benefit. Ethical concerns regarding a placebo group should, of course, be acknowledged and addressed by institutional review boards and through informed consent, as they are in drug trials and have been in the examples above.

In the SYMPLICITY trial, the risks were weighed and managed. There were risks associated with the sham control, which included a femoral-artery puncture and renal angiography. However, the finding that the procedure lacks any apparent benefit will spare many patients from undergoing a risky procedure that apparently has only placebo value. There are clear benefits of preventing ineffective procedures and devices from gaining widespread use, which means that true therapeutic benefit should be established before FDA approval.

The SYMPLICITY trial thus adds to mounting evidence that medical procedures can have a substantial placebo effect. This knowledge may require Congress to articulate a clear standard for establishing true therapeutic benefit for FDA approval, to ensure that all devices we provide to our patients are safe and effective.