*for some definitions of "evidence."

Swift reader Jeff (Yes, there is more than one of us) sent this article from the American Journal of Infectious Disease along in response to my Big Homeopathy piece from a few days ago. Take a look at the report, which is downloadable as a PDF.

I am not an epidemioligist, histologist, or a scientist of any type. I'm just a lay person with an interest in science, and at a glance, that article looks like science to me. It's in a professional science journal, on a site that warehouses scientific content, and it's in a format that matches my vague understanding of how scientific papers are presented.

And, in that brief glance, I can see their conclusion: "Ultradiluted belladonna could inhibit JE virus infection." In other words, this study backs up the claim that homeopathy works.

Now hold up, hula dancers. As a lay person, it's very easy for me to accept the fact that I don't have the training or knowledge necessary to interpret such a document. However, I am allowed to have questions, and reading over the article has raised many.

Here are several of them:

Apparently, the virus that causes Japanese Encephalitis will also infect chicken eggs. This doesn't surprise me as vaccines are made in a similar way. But it is important to note that this study was not done on human subjects, but rather on a vascular membrane of chicken embryos. I'm pretty sure that if I injected pure alcohol into a chicken embryo, it would kill MANY viruses, as well as the chicken embryo. I'm also pretty sure that injecting enough alcohol into a human bloodstream would be enough to kill the human. So even if they claim to have discovered something that kills the JE virus in chicken eggs, that doesn't translate into a useful medicine for us.

However, we're talking about homeopathy here, specifically belladonna. Belladonna, also known as "Deadly Nightshade" looks very much like a tiny tomato, but contains poisonous alkyloids such as atropine. As its name suggests, the plant is extremely toxic. Apparently the symptoms of belladonna poisoining resemble the effects of Japanese Encephalitis, and that's why it was chosen in homeopathy to treat that illness.

Take a look at the concentrations used. According to the study, they used dillutions of 3C, 6C, 30C, and 200C, or 10⁶, 10¹², 10⁶⁰ and 10⁴⁰⁰. That's not a typo... in the strongest, that is the most concentrated dosage, there was one part of belladonna extract per every 1,000,000 parts of water. I won't fill the page typing the other concentrations, and it doesn't matter, because there was no belladonna in any of them. But that's fine for the homeopath, because the water retains the "memory" of the substance. In fact, the more dillute the substance the more powerful it is.

Ok, fine. So without looking at the results, we can imagine what a positive result would look like: The 3C dosage would have some effect greater than the control, the 6C dosage would have more of an effect, the 30C would have an even stronger effect, and the 200C would have the strongest effect of all. And yet, that's not what we see.

The "t" value is what we want. For the first two, there is an effect, and the "P" value suggests it's a strong one. But then in the 30C trial we see less of an effect than the 6C trial, and the 200C trial has an even lower effect. So, if there was an effect, how can it be attributed to homeopathic priniciples when the results don't match what homeopathic principles would predict?

There are a few other anomalies. Why does the "N" or number of trial subjects change? And why do the pock counts on the controls vary so much? And where is the data on the "Eggs that were dead or yielded deformed or absent CAM" which is mentioned at the bottom?

Now, look at this study this way: injecting the same substance into eggs should produce the same results time and time again if every other variable is controlled for. As none of these preparations had any belladonna in them, what could explain the different results? I can only hypothesize, but my guess is that the experiment was poorly controlled and that there were many unaccounted for variables affecting the results.

Another interesting note: I can't find any reference to this being a double-blinded study. If it wasn't, that opens the door to many forms of bias, and the resutls are automatically suspect.

But, as I said, I'm a layman – I may have gotten all this wrong. This institution, the School of Tropical Medicine in Calcutta, part of a school that has programs in homeopathy and ayurveda, may be on to something, and perhaps this will shake up what we know about science and the world around us. But something has to happen first: the study has to be replicated. And somehow, I suspect that's not going to happen.

If a homeopath points to this study and says "See? Homeopathy works!" they're going to be challenged on several points:

1) Why didn't the more dilute substances produce a better result?

2) What mechanism explains what happened?

3) Where are the replicated studies?

and 4) What happens when this treatment, which should be harmless according to both homeopathic and scientific principles (note the distinction), is applied to human subjects?

I'll say it a third time: I'm a layman. I'll happily accept correction if I missed something or misinterpreted something. My point is that just because it quacks like a duck (It was found in a legitimate scientific journal) and it looks like a duck (it resembles scientific papers) doesn't mean that it is a duck (real science). Duck or not, my lay opinion is that this study is quack medicine.

TrackBack URI for this entry

Comments (19)

Subscribe to this comment's feed

...
written by kenhamer, July 29, 2010

I won't fill the page typing the other concentrations...

I will, but only because I wanted to see what 200C really looks like. It looks like this:
1 part in
10,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000.

report abuse

vote down

vote up

Votes: +49

...
written by kenhamer, July 29, 2010

I must admit that I have found homepathic remedies (particularly those in liquid form) useful for treating several conditions, including thrist, dirt, fire, and wilting plants.

report abuse

vote down

vote up

Votes: +65

No belladonna?
written by nickjohnson, July 29, 2010

I hate to point out the obvious, but 1 PPM is not 'none'. If you doubt that, I'm happy to give you 2 litres of 1 PPM ricin toxin and see how you get on.

report abuse

vote down

vote up

Votes: -1

...
written by Criticalist, July 29, 2010

The most obvious problem with the study that I can see from a brief read was that it was not blinded. Unless I am the one who is blind, there is not a mention of it in the clinical methods,so we have to assume the researchers knew which specimens were which. The primary endpoint was counting the number of "pockmarks" made in the membrane - something easily open to bias. Add the fact of a non peer reviewed open access journal, and you have a worthless study.

report abuse

vote down

vote up

Votes: +9

...
written by Mark P, July 29, 2010

Nick: I read the "no belladonna in any of them" to apply only to the 6C onwards. Jeff gave the dilution for the 3C.

It's slightly hyperbolic. There will be a small amount of belladonna in a 6C solution: even at one part in 1,000,000,000,000 there will be a few billion atoms in a small swig. And they are doubly effective for being shaken remember

I think a standard homeopathic dose of 6C ricin would be easily survived though, as poisonous as it is. (They don't give 2L doses, unsurprisingly.)

From 30C onwards we are literally talking about no atoms in a dose.

report abuse

vote down

vote up

Votes: +5

...
written by nickjohnson, July 29, 2010

I think a standard homeopathic dose of 3C ricin would also be easily survived: I calculated the 2 litres based on the LD50 dose of Ricin being 1.7mg for an average adult.

report abuse

vote down

vote up

Votes: +2

...
written by MadScientist, July 30, 2010

You can fool anyone if you really want to - after all, no one can be an expert at everything. That's why it's important to get information from sources which are properly vetted (but finding such sources is hard - even peer-reviewed medical journals are full of hocus-pocus, so when it comes to medicine who do you trust).

This particular pseudo-medicine article is a good example of what lengths con artists would go to fool people. It pretends to reference other articles (I don't have time to check the references and see if they are relevant) and even pretends to have conducted an experiment. I can write a loooong list of what's wrong with the article, but if you have a pet statistician then show them the numbers and ask them to comment on it; I think the numbers will be a good test of whether your statistician knows basic statistics ... (I don't mean to insult statisticians - I'm a chemist, but there are many who claim to be chemists who don't know basic chemistry - in other words, not really chemists). Pay attention to the pock count in the control. Even if all "dead" eggs were thrown out, the stated sample sizes are conveniently a multiple of 100. Gee - of, say, 620 eggs innoculated in each batch, just the right number died off to create sample sizes of 500, 300, and 200? Suuuure - not impossible but an incredible coincidence. (The number of dead eggs and why they are excluded is extremely important experimental data which should be reported - you can not rule out any data whatsoever just because you feel like it.) Next look at the number of pocks in the control groups - they vary wildly despite the large claimed sample size. Even if the experiment were genuine (which is definitely not the case here), such variation in the control is an extremely obvious sign that the experiment and its results is no damned good. The snakeoil treatment pock count suffers a similar credibility problem. Now a good statistician can look at the numbers and actually tell us if the numbers themselves are faked (no experiment was conducted to obtain this 'data') - any statisticians out there care to do an analysis?

report abuse

vote down

vote up

Votes: +13

...
written by MadScientist, July 30, 2010

@criticalist: Proper experimental blinding eliminates many possible sources of bias; however, it is not necessary for an experiment to be blinded in order for it to be valid. However, reproducibility is still very important and even if the reported results were genuine (which they are not), I could immediately conclude that the experiment is not reproducible.

report abuse

vote down

vote up

Votes: +1

Danger, Will Robinson!
written by MadScientist, July 30, 2010

If you follow that link to scipub and download the PDF, a window pops up offering your a 'patch' to install. Anyone care for an infected computer? I wonder if it will be infected by the JE virus? Hehehe. You might need to sprinkle infinitely diluted belladona extract onto the computer to reduce pocking of the monitor.

report abuse

vote down

vote up

Votes: +7

...
written by MadScientist, July 30, 2010

Just adding: this is nothing new. For example, I was visiting that excellent website quackwatch.org and reading through some pages (I do so now and then; I tend to forget things that I have read) and found this:

http://quackwatch.org/01QuackeryRelatedTopics/ploys.html

The pseudo-medicine article cited here falls neatly in the "Backed by scientific studies" category of ploy used by quacks. Snakeoil salesmen from over 100 years ago used that ploy - "recommended by Dr. Bombay" or such. It's basically the same as the well-known advertising gimmick "trusted by doctors", "recommended by dentists", etc - a simple appeal to (a nonexistent) authority.

report abuse

vote down

vote up

Votes: +4

Hey homeopaths, tell that to the lobby groups!
written by Yakaru, July 30, 2010

The researchers should tell that to all those lobby groups for the homeopathic industry who are currently lobbying the EU for an exemption from normal testing. If there really are positive tests, then countries like Germany, France and the UK can strike all those laws giving homeopathy special exemptions from properly controlled testing.

report abuse

vote down

vote up

Votes: +4

What happened to the mice!
written by Ben (UK), July 30, 2010

The discussion section says "In this study JE virus infections on CAM and in
mice were challenged...". What happend to the mouse data?

There is some evidence of an agenda in the rather non-techincal decscritpion of fig 2: "Remarkable decreased pock count on JE virus infected CAM with Belladonna". Also, a figure like this should really show the extrema (or maybe the quartiles) and the SD of the data rather than just the mean.

The procedure for excluding samples is not clear. At the very least the numbers rejected in each group should be given.

report abuse

vote down

vote up

Votes: +3

...
written by FrankH, July 30, 2010

KenHamer wrote "I must admit that I have found homepathic remedies (particularly those in liquid form) useful for treating several conditions, including thrist, dirt, fire, and wilting plants."

I've been making my own homeopathic thirst remedy for years. Salt makes you thirsty. I have found a 400C solution of salt works well. You've no idea how delighted I was to discover that I could use alcohol instead of water to dilute the active ingredient. I find it particularly effective especially if the alcohol comes from one of the Rhone valley vineyards.

report abuse

vote down

vote up

Votes: +8

...
written by Leszek, July 30, 2010

From the study PDF:

CONCLUSION
In conclusion, we may claim that ultradiluted
Belladonna 3,6,30,200 have potential role in
diminishing JE virus infection on CAM. The probable
mechanism of action of these ultradilued preparations
appeared to be due to glycosidase inhibitor action of
calystegines present in Belladonna.

Wait, what? So if we dilute the solution until there isn't any belladonna left, how can there by any calystegines present? What happened to water memory?

Now I didn't have time to read the article in depth here at work, but I wonder what part of this study can support this conclusion. The simple fact of whether or not it had an affect doesn't provide any information of the mechanism. Did anyone catch how they are supporting this conclusion?

report abuse

vote down

vote up

Votes: +7

CEO
written by randi, July 31, 2010

There seems to be some wonderment expressed here that the homeopaths fail to provide any "support" for why the experiment(s) appear to have worked. Folks, homeopaths live in Wonderland, not here on Earth; their "science" is magic, their assumptions are legion, and they prefer the "just accept" approach.

To me, the comment that Jeff Wagg made in this most excellent account - that he sees no indication of the experiment(s) having been performed double-blind - makes the whole process highly questionable, though of course this team is eminently eligible for the JREF million-dollar prize if they can provide evidence of success... I wonder when we can expect an application form to arrive...?

In that regard, I should mention that the JREF prize is no longer being held by Goldman Sachs, but by the Evercore Wealth Management group, who provide us with a better rate of investment return. It's in Account #1007550, presently standing at $1,137,082.32, though only one million of that is the actual prize amount.

Randi.

report abuse

vote down

vote up

Votes: +4

Homeopathic Medicine
written by HarryBG, July 31, 2010

Hi Jeff,

I don't know if you know. Wikipedia says (yes, I know, scientists don't care about Wikipedia) that homepathic "medicine" does not need to be created in a aseptic environment. So whatever caused the effect in a 3C or higher delution may be just an impureness.

Kind regards
Harry

report abuse

vote down

vote up

Votes: +3

Weird statistics
written by Ancyent, August 01, 2010

MadScientist wrote "if you have a pet statistician then show them the numbers and ask them to comment on it; I think the numbers will be a good test of whether your statistician knows basic statistics". Well, I'm a card-carrying biostatistician working on clinical trials (and observational studies) and also a statistical reviewer for several medical journals, including JAMA. Of course, that doesn't necessarily mean I'm any good at statistics. But at least so far I haven't been caught out too badly.

MadScientist also mentioned the possibility of a virus if one downloads the PDF of the article so I haven't done that and restrict my comments to the table in Jeff's article. Some of the numbers there don't make sense.

MadScientist has pointed out that the sample sizes are convenient multiples of 100. That's not a problem if the numbers are before exclusion of "Eggs that were dead or yielded deformed or absent CAM". Fortunately - and strangely - the authors give us both the SD and the SEM in each case. (Why it is strange is that usually just one of the two is given because one can be calculated from the other. The SEM is just the SD divided by the square root of the sample size in that treatment group.) In this table, in every group SEM = SD/6. So, it seems that after excluding the "Eggs that were dead or yielded deformed or absent CAM" there were exactly 36 eggs left in each group. That is an amazing coincidence, especially considering that for some dilutions they started with N=500 eggs and others with N=200 or N=300.

MadScientist also pointed out the substantial variation across the means in the control groups. Using the same statistical test that the authors used to compare "Virus control" and "Virus Medicine" within each section, there are highly statistically significant differences between some pairs of control groups, such as for "Belladonna 3" versus "Belladonna 200".

report abuse

vote down

vote up

Votes: +5

I like to masquerade as a biostastitician
written by jockmcdock, August 01, 2010

and so far none of my employers or journals have caught me out. Maybe it was the major in mathematical statistics I did at Uni. Who knows?

I'd like to back up the comments of madscientist and Ancyent. Everything they say about variation across controls, lack of reported blindness, the SD/SEM ratio of 6, etc makes this article stink. I tried reproducing the t-test for the first comparison and got a much more significant result. So I suspect they got that wrong.

report abuse

vote down

vote up

Votes: +4

...
written by Puddinhead, August 03, 2010

Jeff,

Thought I might point out that double blinding isn't necessary when your patient is a chicken egg Having someone other than the guy who inoculated the eggs do the counting of pocks would have been a sufficient single blind, I would think. Doesn't look like they did this though.

Mad Scientist,

i suspect those sample sizes reflect the number of eggs they started with in each experiment. It would be nice to know, however how many eggs were discarded in each experiment and how many eggs were counted in the test groups and control groups of each study.

The bit that I find most curious is in the methods section. In the test eggs, 50uL of the belledona preparation in water was dropped on the membrane, followed 5 to 10 minutes later by 50uL of the virus suspension. In the control, they mixed the virus suspension with the water and inoculated the eggs with this combined volume. After inoculation, the eggs were rotated to distribute the droplets. This is not a very precise way of ensuring even mixing of the two drops in the test eggs. My main concern is that cutting the buffered viral suspension with an equal volume of water will lead to a hypotonic solution. I would fear that this solution might lyse, or otherwise compromise, the cells of the CAM and lead to more efficient viral infiltration. If the belladona droplet is dispersed (or partially evaporated) before the viral suspension is added in the test eggs, the CAM won't necessarily experience this hypotonic situation, leading to less effective viral infiltration. I've performed this CAM assay with some of my own drugs and seem to remember the droplets running down the membrane into the corners of the windowed shell. Would certainly affect mixing of the two droplets. Would be nice to see this repeated with a proper blinding, more thorough reporting, and a more appropriate control.

report abuse

vote down

vote up

Votes: +1