He stood up in front of the screen and said “Support for the theory is weak.”
That’s not an unusual thing to hear at a scientific conference. But, it was surprising in context because he had just spent 15 minutes systematically demolishing a theory (or so it seemed to me). The theory was by a couple of guys named Wilson and Wilson (2005). Their theory made detailed predictions about when people would jump in on one another’s conversation in a dialogue. I haven’t read their paper myself, but it seems to involve a model of two oscillators, coupled together. The idea is that the phase of each oscillator tells whether or not each person is ready to speak. “Coupled oscillators” means that the phase of one affects the oscillation of the other.
The speaker explained the predictions that the theory made, and, one by one, compared his data to what it predicted. <<…and the prediction is for two peaks in the histogram, symmetrically before and after the end of the sentence…>>[that’s a semi-quote] while the data he displayed showed a single peak, well after the end of the sentence.
I sat there, nodding, thinking “Good job that.” as he showed that one prediction after another failed to match his data. Personally, I think it should be the goal in life for experimenters to shoot down a theory. Theorists, after all have an easy life in a field like Cognitive Science or Linguistics. Everyone likes theories because they are much easier to remember than a lot of experimental detail, and unlike us experimenters, theorists never have to walk in through a blizzard to meet an enthusiastic volunteer who has a day off from classes and nothing better to do than your experiment.
So, I wrote off that theory. He had tested a bunch of clear predictions; they didn’t work; game over. I thought, “Good try, theorists, you just lost one to the experimenters. Come back with another theory some time and we’ll work that one over, too.”
But, he hadn’t written it off. On his conclusions slide, I saw this: << • Support for Wilson & Wilson is weak.>> Well, yes. That could have been a proper British understatement if accompanied with a raised eyebrow and that special intonation that says “You and I know what I really mean.” Except that it wasn’t, and he wasn’t British.
Except that I asked him. I stood up and said that science sometimes proceeds by disproving things. That he had presented an excellent paper and that he had done the a good job of disproving a theory, the best of all the papers that I’d seen at the conference. That this would let us all focus on other theories, ones that might work.
But he didn’t believe his own work. Despite having pointed out drastic differences between his work and the theory’s predictions, he didn’t want to conclude that there was something wrong with the theory. <<There are some hints of the predicted effects>>, he said. He flicked back a few slides and pointed to an insignificant bump on a histogram. I can only guess that he was dedicated to proving the theory, not testing it or disproving it.
There is nothing too much wrong with hoping to confirm a theory, but if you’re not going to listen to your own data, why bother to collect it?