Depressing Thoughts for the Day

From statistician William Briggs, who puts a statistical program to work on a series of random numbers, and then comes up with a statistical model that has a p value of under .05:

I am, of course, a statistician. So perhaps it will seem unusual to you when I say I wish there were fewer statistics done. And by that I mean that I’d like to see less statistical modeling done. I am happy to have more data collected, but am far less sanguine about the proliferation of studies based on statistical methods.

There are lots of reasons for this, which I will detail from time to time, but one of the main ones is how easy it is to mislead yourself, particularly if you use statistical procedures in a cookbook fashion. It takes more than a recipe to make an eatable cake.

* * *

[After describing the invented model:]

Nobody, or very very few, would notice that this model is completely made up. The reason is that, in real life, each of these x’s would have a name attached to it. If, for example, y was the amount spent on travel in a year, then some x’s might be x7=”married or not”, x21=”number of kids”, and so on. It is just too easy to concoct a reasonable story after the fact to say, “Of course, x7 should be in the model: after all, married people take vacations differently than do single people.” You might even then go on to publish a paper in the Journal of Hospitality Trends showing “statistically significant” relationships between being married and travel model spent.

And you would be believed.

I wouldn’t believe you, however, until you showed me how your model performed on a set of new data, say from next year’s travel figures. But this is so rarely done that I have yet to run across an example of it. When was the last time anybody read an article in a sociological, psychological, etc., journal in which truly independent data is used to show how a previously built model performed well or failed? If any of my readers have seen this, please drop me a note: you will have made the equivalent of a cryptozoological find.

From an interview with Timothy Keller, author of "The Reason for God":

Even [C.S.] Lewis, in his Weight of Glory series, Lewis said that, before World War One, the average educational experience was twelve or thirteen people sitting in a room listening to a paper by one person then tearing it apart till 2 a.m. in the morning. And he says, now, the quintessential educational experience is listening to a celebrity lecturer, with a hundred or two hundred other people taking notes and then taking an exam. Even he said, between the wars, he saw a diminishment in people’s ability to really think hard and long about issues.

People want you to get to the point quickly. And they want you to tell them what’s going on quickly. And they just don’t have the attention span. You can look at television, you can look at the Internet, you can look at the so-called rise of narrative and loss of trust in logic—I think it’s cumulative . . . I don’t want to say it’s all relativism or all the Internet because people don’t read long articles anymore. But I just know that it’s very hard to find people who can wade through—unless you’re a professional academic, you’re not going to wade through these books anymore.