Thursday, August 07, 2008

Bush Administration counts to one and statistical stupidity

Recently saw a story in the paper about a salmon return. After having a hatchery release millions of salmon, the scientists prepared to count the returning numbers. As the spawning season arrived only ten fish returned. They were big and healthy fish, but the numbers were much smaller than expected. The Bush administration published a report, "Returning Salmon, Larger Than Normal." I made this up, to show that when you start talking about large systems, you can't take the few successful instances and claim success. You have to look at the larger system and the failures as well.

The reason I bring this up is that I got a comment from a person who claimed to be a recent SIU graduate. They claimed to have a degree in some kind of science, and further claimed that they got a great education at SIU. They went further and told me that SIU was a great place and any criticism I have of it, must be misplaced. There are a couple of sampling problems with this, for example, how does a 22 year old, who has a sample size of one, judge how good their education is? Of course, there are successful grads from SIU, maybe they are one? If you have an undergraduate in science, shouldn't you understand the statistical problems from using such a small sample? What I know is that graduates of better schools, are better able to be successful in their first five years out of college. A whole lot of that post graduation success is because the better schools attract better students.

If you use me as a one person sample for SIU grads, everyone would have a liberal arts degree, everyone would work in computers, everyone would have a blog, everyone would start companies, everyone would live in Oregon, etc, etc. You can easily draw conclusions, based on small sample sizes. What is difficult, is to draw conclusions (or the truth), without a bigger picture. This was the power of this blog, access to a bigger picture about SIU and Carbondale, by accessing the information that is easily available in town, assuming you talk to both the business people and the university people (which no one really does).

My father doesn't understand how you can be an engineer, without being able to do math. I don't understand how you can be a science grad and not understand statistics. I do understand how you can draw conclusions based on a very small sample and have no idea if you are right.

No comments: