Cross My Heart and Hope to Die
August 29, 2013
by Jennifer Phillips, Ph.D.
Sometimes being the resident scientist on this blog is a challenging job. There are promising new developments to write up, but also stories that are more complex, controversial, or just plain worrisome . I’ve put off writing this post for a while because it’s been a tough one to process, but I sincerely believe our readers need to make informed choices about any new treatment options that might come their way, and I hope this write-up will help make that possible.
The topic I’m going to discuss is acupuncture—specifically, acupuncture used as a treatment for Retinitis Pigmentosa, as described at the ARVO meeting last May. Remember that Valproic Acid talk I blogged about? As luck would have it, the very next speaker in that talk session presented on a Clinical Trial conducted at Johns Hopkins testing the effects of acupuncture on RP. My ears pricked up right away, because the rising popularity of acupuncture as a treatment for nearly every ailment you can name has been a significant interest of mine for a while now.
I assume most readers have heard of acupuncture before, and probably know in general what it is. Some of you might even have sought acupuncture treatment for one thing or another—lots of people do. But before we proceed with the study, I want to lay out a few definitions about the general practice of acupuncture so that we’re all on the same page.
Classic acupuncture is the practice of placing thin needles into the skin at specific points on the body, for the treatment of a wide variety of ailments. Acupuncture is considered a part of ‘Traditional Chinese Medicine’ and is an ancient practice. It operates on the premise that an energetic life force (called ‘chi’ or ‘qi’) flows within living things along pathways called meridians. The practice is based on the idea is that when the ‘chi’ is blocked from flowing freely along these pathways, illness results, and thus introducing acupuncture needles at certain points within the body where meridians cross is thought to unblock the chi and resolve the condition.
I freely admit that I raised my eyebrows a bit when I first heard this explanation because it sounds, well, not very science-based. Substitute ‘humors’ for ‘chi’ and ‘bloodletting’ for ‘acupuncture’ and you’ll see the trend of a philosophical approach to medicine, harkening back to a time where very little was known about human anatomy or physiology and practitioners were making the best guesses that they could under the circumstances. Modern science and medicine have since provided an evidence-based model for investigating how the human body works, and using this model, one would think that if there really is an energetic life force that moves along predictable pathways within the body and can be acted upon by physical means (i.e. needles), it should be detectible. Energy is a physical thing, after all, and as such can be measured, quantified, and analyzed by physical means. And yet, no evidence for the existence of ‘chi’ has been detected to date. Thus, from the perspective of the knowledge imparted by science and modern medicine, one must conclude that the mechanism by which acupuncture works, as envisioned thousands of years ago, is increasingly unlikely to exist.
But couldn’t acupuncture work by some other means? Even if ‘chi’ isn’t a real thing, couldn’t these acupuncture points map onto some demonstrable and well-understood nerve pathways that might respond to being stimulated with a needle? If reliable, repeatable results were obtained from putting needles in those particular places, I might buy that. But…they don’t. Time after time, studies pitting true acupuncture against a control ‘sham’ acupuncture show no significant difference between the two. ‘Sham’ acupuncture techniques range from putting the needles in random places (i.e. those not established by traditional acupuncture), to the use of retractable needles that don’t actually penetrate the skin, to simply twirling toothpicks in random places on the skin. In every case, when patients were unaware of whether they were receiving true acupuncture or sham, their responses to the treatment were equivalent.
This strongly suggests that the main effect of traditional acupuncture is that of a placebo, and this conclusion is further supported by the wide range of medical conditions that acupuncturists assert to be able to treat. Acupuncture proponents claim that some combination of acupuncture points can treat nearly every possible disease or injury, and when one method can be applied so broadly, a specific mechanism seems far less likely than a generalized placebo effect.
Despite this rather underwhelming performance record, acupuncture has positioned itself pretty firmly into the mainstream offerings of complementary and alternative medicine (CAM). This rising popularity is due in large part to good old- fashioned marketing by testimonial. However, another way in which acupuncture has gained credibility is through the inclusion of extra treatments along with the traditional needle placement. One very popular co-treatment is known as ‘electroacupuncture’ or EA. In EA, an electrical current is passed between two acupuncture needles placed in adjacent spots in the skin. Delivering a current through the skin to stimulate nerves is actually an established, conventional medical treatment for neuromuscular pain. However, it’s not acupuncture. It’s called Transcutaneous Electrical Nerve Stimulation, or TENS, and while the efficacy of TENS on the conditions it’s routinely used to treat is not set in stone, it’s pretty clear that whatever effect it might have has nothing to do with acupuncture points, or the presence of needles for anything other than conduction.
Based on my prior knowledge of acupuncture, I admit that I was a bit skeptical going in to the ‘Acupuncture for RP’ presentation. However, the trait that distinguishes a skeptic from a cynic is the ability to change one’s mind in the face of convincing evidence. Unfortunately, this study fell quite a bit short of presenting a compelling case for acupuncture.
The study (which as of this writing, isn’t published yet, but apparently will be soon) was explained thusly:
- Based largely on the testimonials of success in a New Jersey Naturopath’s private practice, this study was conducted at Johns Hopkins’ Wilmer Eye Institute, under the direction of an Optometrist (an OD who also has a PhD in Clinical Investigation) on the faculty there. Some 30 year old studies were also cited to establish the prior plausibility
- 12 RP patients between 18 and 78 years of age participated, 5 of whom had previously tried some form of acupuncture and reported positive outcomes. No information was given on what types of RP were represented in the study, and only limited information was presented on baseline visual function. (In fairness, this was a short format talk, and I expect to find these data in the peer-reviewed publication.)
- Pre- and post-treatment vision testing was administered, and patients were subjected to two 30-minute treatments per day (Monday-Friday) for two consecutive weeks. During these treatment sessions, patients received EA at acupuncture points around the eye and traditional acupuncture elsewhere on the body.
- Visual function testing consisted of an assortment of visual acuity and visual field tests, including testing specifically for night vision. OCT (scanning the topography of the retina) and testing for ocular blood flow were also performed. Notably, none of the patients appear to have undergone the same series of tests.
The presenting author also explained that because the researchers weren’t sure which controls they should incorporate, they decided not to have a control group at all. Now, not every study, especially clinical trials with small numbers of people, gets to have a control group. The thing to do in such a case is to make the study just as rigorous as it can be otherwise and limit one’s conclusions, neither of which these authors did, in my opinion. Moreover, having limited numbers of patients to test is, I think, a valid reason for not having a control group in a small clinical study. Not being able or willing to make decisions about what controls to use? Not so much.
I further made note of the bias that nearly half of the participants came in with (having tried acupuncture before), and the obvious bias of the researchers (being strong proponents of CAM, including but not limited to acupuncture). Having patients who expect to receive beneficial treatment treated by researchers who have expectations of positive results can definitely influence the results, especially on the subjective vision tests given to the participants. Mind you, I’m not suggesting conscious, deliberate fudging of the results by any means, but subconscious bias is a known confounder of research studies. That’s why it’s so critical to have good controls. When patients are effectively shielded from knowing whether they are getting the experimental treatment or a sham treatment, it’s far more likely that the placebo effect will be consistent across both groups and thus any difference seen between the groups can be more confidently attributed to the experimental treatment. When the researchers are also shielded from the knowledge of who is receiving real vs. sham treatment, the results get even more robust.
Finally, the fact that EA was used at points around the eyes and on the forehead, while traditional non-electrified acupuncture needles were placed in other unspecified areas on the body (i.e. not the head or the eyes), is also noteworthy. At this point, whatever results were presented have a massive caveat attached to them, because the modality being tested is not “acupuncture” but TENS.
The speaker reported that 8 of the 12 subjects had “significant improvement” in their visual function. What was somewhat underplayed was the fact that not every patient had the same testing. Two results slides with complex graphs were shown during the talk, but the graphs were far too small to see from the audience, and were not explained in any detail. Both graphs are, however, included in the meeting abstract. Apparently, nine of the twelve patients were tested for dark-adapted visual field. Looking at the graph it seems that one patient showed a relatively large improvement, and this patient happened to be the one with the best dark-adapted vision going before starting the treatment. Other patient data on this graph show an increase when comparing pre- and post-treatment test performance, but as no statistics are shown and the error bars are rather large, it’s very difficult to see whether these changes are in fact significant, and if so to what extent.
Other test results were shown for two patients, and results of two additional tests were discussed for two patients each (whether the same two subjects underwent all three tests wasn’t stated) but no data were shown.
Based on this somewhat erratic collection of data from all patients, the authors derived their stated success rate of 8/12, but frankly, with a study this small and variable, it’s really hard to tell what’s going on. Even if you take away the questionable efficacy of acupuncture, positive effects in small studies like these are often overturned by larger and/or more rigorous studies—just look at the VPA story for a recent, relevant example.
Regarding acupuncture specifically: Like many other forms of CAM, acupuncture has a long track record of producing positive outcomes in small and/or uncontrolled trials that then fade into insignificance when the same research question is readdressed in larger trials more methodologically rigorous trials. While studies of acupuncture on retinal health are rare, this inconsistency has been demonstrated many times over with acupuncture studies for a number of other ailments (see links above). With all this in mind, I’m very skeptical of the positive outcomes reported in this study.
Finally, the issue of claiming to test for acupuncture when actually incorporating electrical stimulation is troubling. As stated above, I am unconvinced by the ‘positive results’ reported in this study. But what if there really is some benefit to RP symptoms when the eye area is subjected to electrical stimulation? I don’t think this is likely, but it is an easily testable question, in principle. Alas, a study like the one I’ve just described could never illuminate such a specific effect, with all the trappings of extraneous needles in other parts of the body and murky explanations for their influence on human physiology. That’s one of the glaring shortcomings of this and virtually every other CAM study I’ve read: It’s nearly impossible to tease out a plausible mechanism of treatment from the deep drifts of supposition and magical thinking. In short, it’s just bad science.
Since I’ve been blogging here, I’ve tried to delineate between hope and false hope. To me, genuine hope for a treatment has to have some basis in reality, some chance of long-term positive outcome. There are just too many similarities between this study and many other CAM-positive studies I’ve read over the years that promise far more than they deliver and in doing so raise false hope. I think that’s harmful in and of itself. In terms of actual, physical harm, while it’s true that well-trained acupuncturists have lower incidences of injury than others, one must acknowledge that there are inherent risks associated with being stuck with needles. There is a risk of infection, bleeding, bruising, nerve damage, and collapsed lungs, just to name a few. In addition to the physical risks, there is also the consideration of becoming a lot poorer during the course of treatments like these.
That’s certainly not to say that conventional medical treatment is risk-free OR cheap. For most every medical procedure, one is briefed on and required to acknowledge the risks, in writing, before the fact. It’s worth noting that many CAM practitioners do not disclose risks quite so candidly. As for cost, well all our US readers, at least, have seen the billing statements from the Ophthalmologist’s office. The time and expertise of medical doctors is expensive, no question. However, cost/benefit analysis comes into play here. Accepting the risk and paying the bills for a medical treatment with known, demonstrable outcomes is one thing. Accepting the risk (assuming it is explained up front) and paying for acupuncture treatment which, from a scientific perspective, is unlikely to be of benefit is quite another.
I’m a scientist. My bread and butter is examining the validity of conclusions based on experimental results, and in this regard, acupuncture falls far short of the mark. It may seem a bit harsh to see this stated so frankly, but I believe that the hard, prickly truth is always better than a sweet illusion, and thus I am committed to full accuracy in what I report here.
Cross my heart, and hope to die.
