The Deep Structure of the Universe? (Some Friday Fun)

I went to an interesting talk last week on why individuals don't choose to buy long-term care insurance. While the speaker did not have an answer to that question just yet (it's a work in progress), she did have a lot to say on the use of qualitative methods as a set up for and complement to statistical methods. In the past I've made fun of qualitative work as a "be all, end all" strategy in attacking a given research question ("its not representative," "it takes forever to do," "I don't believe things if there is no math or hard data," etc). But I do agree that qualitative work can help give direction to statistical analysis and aid in the interpretation of sempirical results.

Anyway, in explaining the mechanics behind the qualitative methodology, the speaker pointed out how sampling in qualitative methodology essentially obeys the law of diminishing returns (called the saturation principle, here). Basically, you keep adding to the sample until you stop learning anything new - when the marginal returns to interviewing approach zero. Apparently the majority of qualitative studies saturate somewhere between 25 and 30 subjects/interviewees.

I had a sudden flash of insight when I heard these numbers. The Central Limit Theorem (the big one) posits that the sum of a vector of random variables distributed with finite variance approaches a normal distribution for large values of n. Imagine a really screwed up probability distribution. Draw n numbers from it, and take the sum. Do it over and over and look at the distributions of the sums. Boom! Its looks normal! Its a beautiful result, and the proof is quite elegant, too.

The Central Limit Theorem appears to kick in around n of 30 or so. Thats whyI got so excited about the saturation numbers: its just interesting that two apparently distinct phenomena attain right around the same sample size. Obviously, I could be exhibiting a classical behavioral economics bias and attributing patterns to what are basically unrelated phenomena that just so happen to agree with one another. On the other hand, maybe there is more to it - something fundamental and deep.

Some more: If you like these kind of oddities and puzzles, you should read Fermat's Last Theorem by Simon Singh. It's one of the best books I've read in a long time. Basically it covers the 300+ year history of this annoying and outstanding math problem that kids can understand but adults (including Hall of Famers like Euler) could not solve for several centuries. The beast was laid to rest in 1993 by a Princeton mathematician who spent something like a decade of his life working on this problem and this problem alone. While the problem for a long time looked like some trivial curiousity, the Last Theorem ultimately speaks to some deep connections between branches of mathematics thought to be distinct.

The book is chock full of short biographies of all the mathemeticians who made contributions to solving the problem, interesting nuggets about number theory and the fundamental importance of prime numbers, irrational numbers, etc, and insights into how seemingly trivial mathematics could have mighty big things to say about the natural and physical world. It's a great read, and allows us normal folk to catch a glimpse into the beauty of mathematics.