Assorted Afflatuses

March 2009

Random Annoyance

By Joseph Kibe on 21 March 2009 2:23 PM

Yesterday I took the second of three exams in my statistics course. While I felt reasonably well prepared for the exam when I sat down to take it, I realized—about three minutes after handing in my work—that I had incorrectly answered the first part of the first question on the exam. Unfortunately, this probably means I also answered the second, third and fourth parts of the question incorrectly.

Of course I have no one to blame but myself. Perhaps I should have triple checked my work. Perhaps I should have done more studying. Perhaps I shouldn't have eaten that cookie before the exam. Still, I feel a certain indignation.

For whatever reasons, I find my professor's lectures do virtually nothing to improve my understanding of the subject matter. (Anecdotal evidence suggests I'm not alone.) Worse still, I find the assigned text for the course (which I blogged about last week) difficult to understand and lacking in depth.

Not one to just give up, I did some research. Professors Charles M. Grinstead (Dartmouth) and J. Laurie Snell (Swarthmore) provide a fantastic introductory probability text online as a free download. I also bought myself a copy of Professor Morris H. DeGroot's (CMU) excellent introductory statistics textbook, Probability and Statistics from Amazon.com. I also owe a big thank you to the folks at MIT's Economics Department, who have published a set of lecture notes for their economics statistics course, 14.30, on OpenCourseWare. Armed with those tools, I have essentially been teaching myself basic statistics and probability, including a raft of material that my professor and the assigned textbook don't so much as allude to.

On the one hand, this approach has provided me with a great deal more material to digest and enjoy than I would have seen otherwise. At the same time, these two—arguably superior—texts do not put the same weight on certain subjects as my professor or the assigned text, which, as I have now discovered, leads me to over study some subjects and under study others.

Which leads me back to my indignation.

It drives me crazy that I will receive a lower grade than other students because I have difficulty following the style of teaching my professor uses, and have taken steps to go beyond the course's purview to master the subject as a consequence. In some sense, I'm being punished for learning too much.

More than that, though, this problem underscores a more fundamental problem with the Bates Economics Department's quantitative course requirements. To their credit, unlike many other schools, our economics department requires students take two quantitative courses—probability and statistics, and econometrics—rather than one course that mixes together probability, statistics and econometrics in a single semester.

But while many people speak highly of the econometrics half of the sequence, I have yet to find a student with the same amount of praise for the statistics half. At least from my perspective, the statistics course would be much, much better if the course had a math prerequisite—single variable calculus is a must for the study of continuous probability distributions—and were a more in-depth study of the material.

To his credit, I suspect my professor chose the awful textbook he did because, unlike many other, more modern texts, it include a fair number of "proofs" of various mathematical properties. Unfortunately, most of these "proofs" aren't really proofs, in a mathematical sense. That, and the text omits some standard mathematical conventions for dealing with probability and statistics. The first few chapters, for instance, don't even mention sets (in the mathematical sense) when treating probability theory.

That and both the exams tested my ability to memorize formulas, not my ability to use the formulas as tools to reach meaningful conclusions, which should be the goal of an economics and statistics course. (By comparison, the exams I studied with on MIT's OpenCourseWare, taken from a 2006 section of 14.30, provided a formula sheet that could easily have represented the answer key for the exam I just sat.)

Writing the course evaluation for this class will be a real treat come April.

When Lotteries Fail

By Joseph Kibe on 16 March 2009 4:44 PM

Earlier this evening I participated in the annual Bates College housing lottery. The supposedly efficient process was—and, assuming they don't have some brilliant, massive overhaul planned, is—far from efficient. I would also argue the process is also incredibly inequitable. But that's a more subjective question.

Ideally the housing office would simply charge students higher prices to live in the dorms with higher demand and lower prices to live in the dorms with lower demand. I like the idea of some kind of auction. Of course, this would price some students out of certain buildings, creating the possibility of a chasm between students with more money and those with less money. It seems understandable, from the perspective of harmony and equality, that surpluses and shortages are not cleared out with an auction-style mechanism.

Still, the method the school has chosen to mitigate the potential haves-versus-have nots issue is far from satisfactory. Students who value certain rooms more than others can't express their preferences in a meaningful way.

It seems to me that the College should give every student some number of virtual "Bates Points," a sort of virtual currency. The College could easily restrict the exchange and resale of the Points, eliminating or at worst mitigating the possibility of wealthier students exploiting other students without the same financial resources. These points could be redeemable for a variety of scarce resources, housing included.

The College uses completely random lotteries to allocate a number of scarce goods. Students don't receive a parking space automatically: they put themselves into a parking lottery. Seats in popular classes don't go to students who value the seat the most: they're distributed at random to those who put themselves into the pool for the seats. (Granted, the class selection process isn't quite as unjust as some of the other lotteries: student can appeal the outcomes.)

Presumably, some students value having an on-campus parking space more than having first pick in the housing selection process. And I suspect other students value priority in class choices more than having an on-campus parking spot.

We should simply give every student a fixed, equal number of Points at the beginning of each school year. I have my doubts about giving more senior students more perks purely on the basis of seniority—more senior employees in the "real world" generally make more than their more junior colleagues because they have more experience and are thus more valuable—but it wouldn't be too unreasonable to give more senior students more points.

Then, students could use their points in an auction to pick their rooms, parking spaces, classes and other scarce resources. This could all be done online, eliminating the need for an multi-day, multi-hour process that requires compensating an untold number of employees and volunteers. Students could simply log on to the Internet, put in bids on rooms they wanted and repeat. As soon as no one outbids a given student after some sort of waiting period, say 24 or 48 hours, the student receives the room and the number of points they bid is deduced from their accounts. No mad rush, no huge staff to coordinate and pay, and students have a more meaningful way of expressing their preferences. More students receive rooms they want, or rooms closer in quality to the rooms they want.

This system also helps mitigate the problem created by students who choose a room and don't return following semester, depriving others students from a room one of them might have wanted. In this system, if a student decides to take the semester off or goes abroad unexpectedly, the room would just go back on the market. Other students interested in the room could put in bids on the newly available room, and, presumably, it would go to the person who valued it the most.

My system would probably work best if students have a wide variety of ways to dispense their points. If there aren't enough places to spend them, it's easy to conceive of a situation where the market fails to clear because too many students decide to allocate the same number of points to a given room or parking spot. A simple proof of concept software implementation of the idea could be a very good spring break project.

Of course, at least on the housing side, this is really just a symptom of a deeper problem. As one College executive—who shall remain unnamed—put it, many of the residence halls on campus don't meet "modern standards." The brand new residence hall I'm living in this year boasts, among other amenities, network infrastructure that copes with modern Internet use patterns. The heated competition for certain rooms would not be quite as heated if there weren't such a huge chasm between some rooms and others.

Even if every room had casement windows and hand carved walnut furniture, though, this system would still reduce the stress students experience and the number of people required to assign students to one room or another. I hope someone with power reads this.

30 Years

By Joseph Kibe on 15 March 2009 11:14 AM

It has been quite awhile since my last post. I can't say why. I'm no busier this semester than the last, aside from my discouraging hunt for summer employment. It goes without saying that now is not a particularly good time to be in the market for a job in finance.

But I digress. This semester I'm taking economics 250, a course in probability and statistics with economic applications. It's the first of two classes in quantitative methods for economists.

In an effort to calm students' outrage at textbook prices, my professor for economics 250 assigned "Statistics for Economists" (1972) by Ralph E Beals as the textbook for the course. On face, this is an absolutely brilliant idea. As my professor put it, the rules of probability haven't changed in the last 30 years. Thus, it makes no sense to spend $150 on a modern text when an older text, which is ostensibly the same, can be had for less than $20.

(As an aside, the Beals text was made available to students at a price of $17 in the form of a bound photocopy. I'm not sure who in the College's legal department approved this idea. Ralph Beals is still a living, breathing professor at Amherst, which means the textbook is still under copyright and will remain so for 70 years after his death.)

Of course something rather dramatic has changed over the last 30 years that pertains directly to statistics for economists: the price of computing power. According to data from the Dallas Fed, the price of 1 MHz of computing power in the 1970s was somewhere around $368,000 in 1970s dollars. Today, even looking at a relatively expensive computer, such as the new unibody MacBook Pro, the price of 1 MHz of computing power is only $0.41. Not to mention, it's much easier to program using an IDE and C than punchcards.

As such, most modern probability and statistics textbooks (and thus most probability and statistics courses) make extensive use of computers to run simulations and do data analysis.

This aspect of the text is not a problem per se. A professor could certainly augment the text with other material to make sure students leave the class with a working understanding of modern statistics software.

My professor, however, has chosen to subject the class to a hodgepodge of "simulations" in Microsoft Excel. As much as I love Microsoft's spreadsheet application, it's really no substitute for a real statistical analysis environment, such as R, Stata or SAS. And, given the limited scope of the simulations in Excel, students without any experience using Microsoft Excel learn how to use only a small set of relatively less useful tools in the software.

Running simulations can be extremely instructive if they're done well. But when it takes a half hour to setup a simulation with an n=100—a task that might take a few minutes even for the most inexperienced R programming—the simulation loses its pedagogical power. It makes it difficult, for instance, to illustrate the Central Limit Theorem.

In just a few minutes with Mathematica, for instance, I managed to whip up a dynamic, interactive simulator that ran anywhere from 1 to 100,000 Bernoulli trials and plotted the results in real time. I could then drag the n slider from 1 to 100,000 and watch as the PDF of the binomial distribution looked more and more like a normal distribution. And it only took a dozen or so lines of code! It's not rocket science!

I also take issue with the text at a more intrinsic level. Unlike most college-level probability and statistics courses, it does not require that students have even a little exposure to calculus. This confines the text to speaking about continuous probability distributions in vague generalities. Integrals are referred to as "shaded areas under the curve." Important subjects, like the Poisson, Beta and Gamma distributions are, as far as I can tell, not even mentioned in the text.

Once again, the best intentions do not necessarily produce the best results. I might be more forgiving of the text's lack of applications using modern software if my professor were more tech savvy. Unfortunately, he is not. This is not a problem per se. But it is annoying when I have to go out of my way to teach myself how to use R and Stata rather than consulting my professor or the assigned textbook.