US regulation of food ingredients and supplements employs a concept known as, “generally recognized as safe,” or GRAS. Under FDA regulation a food additive or substance is GRAS if qualified experts believe it is safe based upon scientific evidence. However, for substances in use prior to 1958 GRAS does not require any scientific evidence; “a substantial history of consumption” is sufficient.
The premise of this part of the GRAS rule is that if a large number of people use a substance over a long period of time, any safety issues would emerge and would be known. This premise, however, is naïve, and is contradicted by historical evidence.
Marketers often rely on the naturalistic fallacy to sell the safety of their supplements. “All natural,” a term without unambiguous definition or legal regulation, is almost ubiquitous on supplement advertising. This is little more than the naturalistic fallacy, however. Being “natural” (whatever that actually means) is no guarantee of safety. Plants and animals evolved a wide variety of toxins and poisons for their own purposes. I would not recommend eating a random plant unless you know exactly what it is – most “natural” things will kill you or at least make you sick.
If alternative medicine wants to be taken more seriously, the studies must be better designed and be put in the proper context.
UK’s The Telegraphreported last month that a study published in the journal Complimentary Therapies in Clinical Practice showed that reflexology was “as effective as pain killers.” It’s a bold claim.
However, this claim is backed up by nothing in the study. In fact, all the methodological flaws encourage a reflexive rejection of the study’s conclusions.
No Control, No Power
You don’t have to be a scientist to know what questions to ask about a study. Some of the most basic are “What was the sample size?” and “Was it double-blinded?” Even these basic questions can tell you a lot about what researchers find.
The reflexology study had a sample of 15 participants, most of them women, and each received both experimental conditions (we will come back to this point later on). If 15 sounds like a small number to you, that’s because it is. In fact, because the statistical analyses they were using looked at group averages, this small number gets broken down even further. With so few participants, this study does not have the power to comment on very much. In larger studies, vexing variations between individuals “cancel out” to hit on some average value. Whether this study hit on something interesting or not, we wouldn’t be able to tell—values are lost in the large variations between so few people.
Amanda Berry, Gina DeJesus and Michele Knight – three women who went missing over a decade ago and were presumed dead – have been found to be alive. This item especially interested me because in 2004, on the totally irresponsible Montel Williams Show, “psychic” Sylvia Browne told Amanda Berry’s mom that her daughter was dead, causing her great shock and sorrow. And the mother, Jouwana Miller, went to her grave believing – only on Browne’s word – that her daughter had been murdered. She died in March of 2006, and friends were of the opinion that her passing was hastened by Browne’s totally uninformed and callous guess…
The gravel-voiced talk-to-the-dead woman (born Sylvia Celeste Shoemaker, 1936) who says that she sees and hears ghosts, used to be “Brown.” She added the “e” in an attempt to distance herself from her former husband, Kensil Dalzell Brown, known as “Dal,” with whom she was involved back in 1988 in a $1.3 million loan scheme involving a gold mining operation using the “psychic’s” magical powers, a plot that sent Dal to jail and Sylvia to probation.
Literally for decades now, I have been directing attention to the fact that Browne has been a continual, proven, failure. This is only the most recent example.
Early in 2010, an article in Skeptical Inquirer provided an extensive study of Sylvia Browne's predictions about missing persons and murder cases, along with her messages and visions “from beyond the grave.” It examined every episode of the Montel Williams TV show after 2002, when she began to be featured there regularly, and explored older cases in newspapers, finding 115 examples of these appearances and articles, and comparing them with the actual facts and Browne’s oft-repeated claim that her accuracy rate, to quote her exactly:
…is somewhere between 87 and 90 percent, if I'm recalling correctly.
Sylvia’s recollection – strangely – is very, very, poor. It was shown that in not one of those 115 records – some of which had to be recovered from data that had been deleted from video records and/or published documents – was she correct!
If you missed The Amaz!ng Meeting 2012, you can still catch great talks, panels, and workshops on science and skepticism given live at TAM 2012 on our YouTube page. Today, we are pleased to share one of those remarkable panels.
The Future of Skepticism
JREF president D.J. Grothe moderates this panel discussion with Jamy Ian Swiss, Barbara Drescher, Tim Farley, and Reed Esau about the future of skepticism as an approach to exploring claims and as a movement to advance that approach.