Posts Tagged 'regressions'

Confidence in astrophysics

[by JSC5]

This paragraph from an essay by Dennis Overbye on discoveries in astrophysics really blew me away:

“Call it the two-sigma blues. Two-sigma is mathematical jargon for a measurement or discovery of some kind that sticks up high enough above the random noise¬† to be interesting but not high enough to really mean anything conclusive. For the record, the criterion for a genuine discovery is known as five-sigma, suggesting there is less than one chance in roughly 3 million that it is wrong. Two sigma, leaving a 2.5 percent chance of being wrong, is just high enough to jangle the nerves.”

If you think back to your first statistics class, you’ll remember a bunch of ways for testing an observation for significance, and I’ll bet you $10 you used the 95% confidence level for just about everything you did in that class. It’s become the default level for significance in most social sciences. Run the regression, and if p<.05, bam, you’re done. Call it significant and move on.

Then along comes a hard science like astrophysics that puts everyone else to shame. These guys run right past .05¬† without looking back, at .025 their nerves start “jangling”, but it’s not until 3.33 x 10^-7 that they’re ready to go ahead and say, “Excuse me, sir, but I think I have a genuine discovery on my hands.” Meanwhile the economics/poly sci grad student next door has run 1000 new regressions and ‘discovered’ a bunch of things that turn out not to be quite right but still get published with frightening frequency.

And that’s why people trust astrophysicists.


Subscribe by email, feedburner

Subscribe by e-mail

or subscribe with feedburner

This is a group blog. JSC5 currently writes from the US. JSC7 writes from behind the Great Firewall of China.

wordpress statistics

Categories and tags

Archives