letter to Nature
April 7, 1998
Nature
Book Reviews
Dear Sirs--
A week ago I sent you a letter regarding a review published in Nature a
few months ago. Do you acknowledge such correspondence? I realize you must
receive hundreds of letters so I am prepared to be patient. In case it has
been misplaced I am resending my review:
Sincerely yours
Nick Herbert
BAD GOOD
Recently (Oct 23 1997 p 807) Nature published a review
by I.J. Good of Dean Radin's "Conscious Universe". Radin's
book is a critical analysis of past experiments in parapsychology and new
experiments of his own.
In more than 30 years as a scientist and science writer I have never encountered
a more intellectually dishonest treatment of another man's careful work.
After citing occasions of fraud and self-deception in parapsychology, the
reviewer pretends to show that Radin's demonstration of persistant, robust,
psi effects over many decades is spurious.The reviewer focuses on "the
file drawer effect", which is a way of biasing results in a positive
direction by intentionally or otherwise suppressing negative or null results.
The magnitude of the file drawer effect is expressed as the number N of
such suppressed experiments for each published experiment that would be
necessary to bring the results below a particular level of significance,
conventionally .05 (or odds of twenty to one against chance).
Groups of experiments with a small file drawer factor are vulnerable to
the charge that a full report of all data would cause the alleged "effect"
to vanish. But what is a "small" file drawer factor? Harvard psychologist
Robert Rosenthal has suggested that a ratio of 5 unpublished studies to
one published study is sufficient to call the observed effect "robust".
To reject data with a file drawer factor (FDF) this large amounts to positing
a conspiracy on the part of parapsychologists to conceal or suppress 5 experiments
out of every 6. Unless one has independent reasons to suspect fraud or self-deception,
rejecting a meta-analysis with an FDF greater than 5 suggests a pathological
need to deny the data's validity at all costs.
Seizing on a particular card-guessing study analyzed by Radin, the reviewer
(using a level of significance of 0.01 rather than than the conventional
0.05 -- a choice which acts to deflate the calculated FDF) obtains an FDF
of 16. The reviewer then triumphantly declares: "This conclusion largely
undermines Radin's meta-analysis which is central for his case for ESP."
The reviewer's logic is surely flawed: "A small FDF is sufficient to
reject psi. I have calculated a large FDF. Therefore I reject psi."
For this class of logical fallacy I suggest the name "Proof by Loud
Shouting".
The reviewer recommends a book by Hansel as a skeptical alternative to Radin's
mostly positive arguments. Discovering that Radin quotes Hansel, and finding
a statistical flaw in Hansel's quote, the reviewer attacks Radin for the
other man's flawed argument. This is certainly an original if fallacious
rhetorical device--holding a researcher responsible for his critic's mistakes.
The claims of parapsychologists are important because they have the potential
to radically change our view of self and world. Radin argues persuasively
that massive experimental evidence favors the view that mind is not localized
in the brain. Radin's evidence deserves to be critically examined not "explained
away" by patently foolish logical stunts. You do the cause of science
a great disservice and shame your wonderful journal by publishing such a
hamfisted hatchet job under the guise of a "fair review".
Sincerely yours
Nick Herbert
quanta@cruzio.com
Boulder Creek, CA
USA
dear colleagues-- April 12, 1998
Dean Radin recently published a book "The Conscious Universe"
that presents persuasive experimental evidence that appears to refute any
conceivable materialistic model of mind. This book has the potential just
in time for the next millenium to launch a revolutionary new science of
consciousness.
Radin's book was recently reviewed in Nature--science's most prestigious
international journal--and seemingly debunked by the most astonishing assortment
of rhetorical tricks and statistical shenanigans I have ever run across.
For the serious student of rhetorical devices I have taken the liberty to
reproduce the Nature review at:
http://mail.cruzio.com/~quanta/review.html
Here one will find Proof by anecdote, Guilt by association, Red herring,
Straw man and some original rhetorical fallacies apparently invented solely
for this review--everything but argumentum ad hominem. Such restraint!
For those more interested in truth than rhetoric I have provided links to
corrective responses by Dean Radin, Brian Josephson and myself.
cheers
Nick Herbert
http://members.cruzio.com/~quanta
Dear Dr Herbert, April 14,1998
Thank you for your Correspondence about Professor Good's review of Radin's
book. We have already printed a correction of the (editorial) error you
mention in your penultimate paragraph; and I will be back in touch with
you about your other points as soon as I have had a chance to discuss them
with Professor Good.
Yours sincerely,
Peter Tallack
Book Review Editor
Nature
nature@nature.com
Dear Mr Tallack-- May 8, 1998
I am pleased that Nature has seen fit to correct a minor error in Good's
blatantly dishonest review of Radin's "Conscious Universe", I
am hopeful that you will swiftly correct Good's other (and more serious)
erroneous claims or at least publish my letter pointing out Professor Good's
flagrant misuse of statistics to discredit Radin's important book.
More than three weeks have gone by since your last letter. Have you and
Professor Good conferred on this matter? When will Nature respond to my
charges of serious misconduct on the part of a reviewer?
sincerely yours
Nick Herbert
Dear Mr Tallack-- May 18,1998
In the pages of Nature you allowed I.J. Good to abuse his authority as an
professional statistician to discredit a fellow scientist's research. It
is entirely appropriate that Nature consult with Good before publicly correcting
his mistaken and utterly dishonest review of Dean Radin's "Conscious
Universe".. But why is this consultation taking so long?
It is now more than a month since you promised to "discuss these points"
with Professor Good. Is dishonesty, fraud and scientific misconduct of such
little importance to the editors of Nature that they are in no hurry to
remedy a wrong they continue to perpetrate on Dean Radin and his work?
sincerely yours
Nick Herbert
http://members.cruzio.com/~quanta/badgood.html
On July 30 1998 Nature magazine published the following two letters
in an attempt to resolve this controversy:
Sir--
In his review (1) of my book "The Conscious Universe" I. J. Good
suggests that something "must be" wrong with my statistical arguments
in favour of the reality of psychic phenomena. In fact, his inability to
reproduce my estimate that 3,300 unpublished, unsuccessful experiments would
be required for each published extrasensory perception (ESP) card experiment,
in order to nullify the outcome, follows from a misunderstanding.
Good assumed incorrectly that my words "more than" (in the phrase
"more than a billion trillion to one", referring to the cumulative
odds against chance for the 186 experiments listed in ref.2) could safely
be replaced by "approximately equal to" implying a P-value of
about 10^-21. Instead the real P-value is about 10^-2000, which, following
the application of standard methods (3), gives my reported figure.
I am encouraged that a statistician of Good's repute did not discover any
genuine flaws in my comprehensive analysis of the empirical evidence for
parapsychological phenomena.
DEAN RADIN
I.J. Good replies--
Radin's claim of my "misunderstanding" relates to four million
guesses of cards made up to the year 1939. In his book (p. 97) he says that
the combined result amounts to "odds against chance" of more than
a billion trillion (10^21). Most readers would take this as implying that
the real odds were not more than a trillion trillion (10^24) and would be
surprised to hear that by "more than a billion trillion" Radin
meant more than 10^100--increased to 10^2000 after the book was published
(information from Radin; Brian Josephson, personal communication)
Readers should bear in mind that Robert Rosenthal's analysis of the "file-drawer
effect" (4) makes no allowance for intellectual, observational nor
ethical lapses; nor does Radin make any such allowances in his calculation.
S. G. Soal, regarded by many as the leader of ESP research for many years,
was showen by Betty Markwick (5) to have almost certainly been a cheat,
thereby confirming previous suspicions. Radin "abolished" Soal,
in the sense of George Orwell's "1984". The proportion of lapses
in parapsychology studies is difficult to estimate, and is made more difficult
by "abolishments". An abolishment of an important lapse is another
lapse.
I.J. GOOD
(1) Good, I.J. Nature 389, 806-807 (1997)
(2) Pratt, Rhine, Smith et al "Extrasensory Perception After 60 Years"
Bruce Humphries, Boston 1966)
(3) Rosenthal, R. "Meta-analytic Procedures for Social Research"
Sage, Newbury Park (1991)
(4) Rosenthal R. Psychol. Bull 86, 638-6412 (1979) Markwick, B Proc Soc
Psychical Res 56, 250-280 (1978)
My opinion (Nick Herbert) concerning this controversy is that in his
professional role as an eminent statistician Good has failed to find a single
statistical flaw in Radin's work--a work which argues strongly that ESP
is real and deserves further scientific scrutiny. Instead Good merely repeats
the charge made in his original review that, since Soal cheated, everybody
else's data is suspect too, an argument that makes as much sense as the
suggestion that since physicist Rene
Blondlot in 1903 fraudulently claimed to have observed "N-Rays",
then we should dismiss all claims by physicists to have discovered any new
particles since.
To suggest that Radin's painstaking work be dismissed as possibly fraudulent
without producing any outright evidence of fraud in the data that Radin
presents is in itself a fraudulent argument on the part of a famous statistician.
Good's flawed review does not do justice to Radin's careful work. Read "The
Conscious Universe" and decide for yourself whether or not his evidence
favors the existence of ESP.
See also Brian
Josephson's page for additional letters to Nature criticizing Good's
review.