How Do I Know This Isn’t Garbage?
I’ve said elsewhere that science can be distilled down to two questions: “What is the world like?” and “How do I know this isn’t garbage?” Richard Feynman stated the second question as:
The first principle is that you must not fool yourself — and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.
(Emphasis added.)
Over the years, scientists have discovered a great many ways to fool yourself and others. So It’s nice to read by Peter Norvig, listing some that even professionals get tripped up on. It gives the distinct impression that the hardest part of doing an experiment is not the business with the test tubes or telescopes or particle accelerators or what have you, but simply avoiding all of the mistakes that others have made before you, that could invalidate your results.
He has an equally good companion piece that analyzes a bunch of studies on the effect of intercessory prayer. (Summary: the experiments can be divided into two main groups: those that show no effect, and those that are flawed.) Most interesting for believers is the way that he points out exactly what the flaws in the papers are. Well worth reading.
(HT PZ for the link.)