Thursday, August 21, 2008

Ha ha ha, very Funny!

You know, we here at Saurian Sagacity have taken a fair amount of ribbing from Brian at MgoBlog for Mergz' blogpoll methods. It strikes me as odd that someone who goes through the trouble of creating a poll of bloggers, presumably because bloggers can do it better, would belittle thinking that attempts to approach the idea of a pre-season poll with some logic and homework.

It's worth repeating that Mergz doesn't put any stock whatsoever into preseason polls but that since we are participating in the blogpoll we are agreeing to submit a preseason ballot. Rather than submit a totally opinion-based ballot (what voters in the A.P. and coaches polls do - and apparently the other voters in the blogpoll) Mergz has tried to bring some objectivity to the exercise.

The basic premise of Mergz' preseason blogpoll methodology is that schools that have acquired superior talent over the last four years should be the most successful. Simple enough. Now of course there's huge complications. Who determines the talent level that a school has acquired? Mergz uses data from rivals.com. We can argue the merits of recruiting information services and which one is better but the truth is that these services have dedicated themselves to exactly the kind of evaluation that neither Mergz or myself could ever hope to engage in. And their reputation depends on them being right. But still there are issues. Even if we accept that Rivals (or Scout.com) have done their homework properly there's a lot of variables to account for. What if you recruit a 5-star player who never pans out because of academics or off-field antics? Also, a particular concern I've had with Mergz' model is that it gives equal weight to each of the last four recruiting classes. But logic dictates that upperclassmen usually have a greater impact on a team's success or failure than underclassmen. Of course there are huge exceptions (Tim Tebow) but the fact is that it's generally considered good to have a team loaded with senior talent.

All of this is to say that we recognize that there is no magic bullet in predicting success before a single down has been played. We are only attempting to make sense of a random mess that is college football in week zero.

I decided to look back at the preseason ballot we submitted for the blogpoll last year and compare it to not only the final A.P. poll for the year but also the preseason blogpoll.


As it turns out, 14 of the teams we said were top 25 ended the season in the A.P.'s top 25. In contrast, 15 of the blogpoll's top 25 ended up in the A.P. top 25.

Five of the teams we picked for our top 10 ended up in the A.P. top ten. The blogpoll had six top ten teams make into the A.P.'s final top ten.

Our model provided some comical results. For example we had Florida State at number 9 and of course they finished off of the chart. But the blogpoll collective had Louisville at 10 and they likewise finished out of the top 25.

For the teams we correctly placed in the top 25, I found that we missed slotting teams in the right place by an average of 6 slots. A similar analysis showed that the blogpoll missed by an average of 5.4 slots.

In sum, the collective wisdom of blogpollers was marginally more predictive than Mergz' recruiting based model which he has tweaked this year. By no stretch of the imagination was our exercise an embarrassment. We'll take the ribbing for now, but I can assure you that I'll be here to check whether their guffaws were warranted or not.

No comments: