Monday, October 06, 2008

Traditional Polling Doesn’t Work

At least when it comes to deciding who’s “best”

Every comment thread in response to my resume ranking for the BlogPoll contains some comment along the lines of “How can you rank so-and-so ahead of such-and-such. Such-and-such is BETTER”.

To which I say -Bullshit.

You don’t know whose better, and you make an ass of yourself pretending you do.

Let’s go to the record. For our purposes here we will use Week 2 of the USA Today Poll from this season, so we can avoid the total situation of uncertainty that is a preseason poll.
.

The Week 2 Coaches Poll –

.


This was the consensus of the razor-sharp minds of the voters after week 2 (not altogether different than the BlogPoll at that point).

Let’s take each in turn –

1. USC – Were they “better” than Oregon State (unranked), who they would be dominated by 3 weeks later?
2. Georgia – “Better” than Alabama (ranked 16th), who they would get destroyed by a month later?
3. Oklahoma – Looking good for now.
4. Florida – “Better” than Ole Miss (unranked), who they (we) would lose to at home?
5. Ohio State – Well, they were ranked worse than USC. But is 4 slots 32 points worse?
6. Missouri – Continues to look good.
7. LSU – Ditto.
8. Texas – Still hasn’t played anybody, but unbeaten.
9. Auburn – They were ranked lower than LSU, who proved “better”, but not Vanderbilt, who was in the “Others Receiving Votes” category at this point.
10. Wisconsin – A month later, 2 losses, including to Michigan (unranked).
11. Kansas – Not “better” than USF, ranked 18th here.
12. Texas Tech – Another Big 12 team hanging on.
13. Arizona State – Dropped 3 since this poll, to UNLV (unranked), Georgia and Cal (25th).
14. Oregon – Dropped two since this poll, to “better” team USC and Boise State, which was unranked.
15. BYU – Still going strong.
16. Alabama – Vastly under-ranked here, if ranking is to predict who is “better”
17. Penn State – Under-ranked
18. South Florida – Lost to Pitt, who is unranked here.
19. Wake Forest – Were they “better” than Navy (unranked)?
20. East Carolina – Not “better” than either unranked NC State and Houston…
21. Fresno State – “Better” than unranked Hawaii?
22. Utah – Still rolling
23. Clemson – Not “better” than unranked Maryland.
24. West Virginia – Proved not to be “better” than Colorado (of Others Getting Votes here).
25. Cal – Were they better than “Maryland” (unranked), who they would lose to a week after this poll?

If we look at the Week 2 poll from a standpoint of teams that have lost to teams they were supposedly “better” than (teams that were lower ranked), the poll was WRONG for 15 of the 25 teams, or 17 total times (accounting for the two losses of lower ranked teams of Arizona State and East Carolina). 15 or 25 teams, or 60% of the rankings, were wrong from a prediction standpoint. And remember, this was only 4 weeks ago!

The strength of resume voting is you can’t be wrong in the sense you predicted one team was better than another, because that isn’t the point. You are merely evaluating the teams based on what has already happened. You are a grader, not a predictor.

Now you can argue with my grades, and I welcome that. But for the last time, don’t tell me so-and-so is better.

20 comments:

Anonymous said...
This comment has been removed by a blog administrator.
Mergz said...

You sir didn't read a single thing I wrote.

Or, you didn't understand it.

Mergz said...

On second thought, troll post deleted.

Anonymous said...

I'm really starting to come around to your mode of thinking. There are posters on ZFZ complaining that Vandy is not as good as its ranking. I pointed out that, if anything, it should be way up there: 5-0 with wins over Ole Miss, South Carolina, and Auburn--a resume that we (Florida) would be thrilled to have.

Florida, also, would be in the top 5 with that resume. So would any other Big Name Team. If I were doing the poll, I'd place Vandy only beneath Alabama based on quality wins. Do I think they'd beat Oklahoma? Not really. But the teams will prove their mettle by season's end; to handicap them this early, as if the big bowls were tomorrow, does them a grave disservice.

So kudos to your rankings!

-bimigator

Anonymous said...

I agree with the comment about where Florida would be if they had done exactly what Vandy has done so far. They'd be top 5, absolutely no question.

That's what coaches and AP voters, and Harris voters should do. Take away the names, and what do you get?

Trader Rick said...

Florida is better than Mississippi.

Henry Louis Gomez said...

Trader Rick,

That may be so but losing to Ole Miss gets you knocked out of the top 25 for now.

We can argue that Florida is "better" than Ole Miss but the only empirical data we have on that is 60 minutes of football played at Florida's home stadium in which Florida had no major injuries.

jj gator said...

Trader Rick, had Florida brought their "A" game to The Swamp that day we would have won. Yes we are a better team than Ole Miss and Vandy for that matter too (I wasn't impressed with the way they played Aubutn on Saturday night); if we could have avoided all the foolish mistakes we made in the past two games our ranking would be higher. Yet we shot ourselves in the foot and have nobody to blame but ourselves for the subpar play - and it shows in the polls.

Mergz said...

Whether a team is “better” than another is little more than a hunch.

All we really have to compare is performances, and even that becomes convoluted. You can make a strict performance based argument that Stanford is better than USC. Why? Well Stanford beat Oregon State, who soundly beat USC. Now do I really think Stanford is “better” than USC? No, my hunch tells me they are not. But I thought that last year too…

I’ll say it once again – the best part of resume ranking is that I am absolutely unconcerned with who is “better”. Thus, I can’t be wrong when a surprise team jumps up and beats some stronger team.

Which leads us to a whole other discussion – what is the purpose of ranking? To determine who’s better (it fails miserably if so)? To determine who has done better? To determine who gets to play in the BCS game?

I feel a whole post on the topic coming on…

Anonymous said...

The ultimate purpose of college football rankings is to justify putting "The Greatest Team of All Time" and UF's Columbus campus (AN ohio state university) in the MNC game...


...or at least that's what the voices tell me.

yankeegator

Anonymous said...

My definition of better is: Who would win the most games in a best of 5 series?

Missouri - 4 games
Vanderbilt - 1 game

That's my opinion. Thus, referring to my definition, Missouri is better.

Anonymous said...

.. That is why in baseball they play a "best of" series because any team can potentially knock any other team on any given day. (It is that way across physics (and sports, ie College Football)) So, in order to make it clear who should be the winner they play a series and noone complains about who is better.

In summation - If they wanted to cut the crap out of college football, let them play a series to decide the national champion. (Understand, I think that idea is quite ludicrous, but it would certainly put the issue to rest.)

Anonymous said...

"Florida is better than Mississippi."

That's NOT what the SCOREBOARD said...

:(

yankeegator

Henry Louis Gomez said...

This ain't baseball. In baseball they play 162 games. You could argue that the short series in the baseball playoffs (particularly the first round best of five) are complete horseshit because of how they don't resemble the regular season. You could be the best team in your league by 10 games in the regular season and lose 3 games in the Division series and be out.

Anonymous said...

It's cute to delete a post and call it a troll because it exposes a flaw in your argument.

You cherry-picked a week (early in the season) where there were lots of upsets. Good work. But even a rudimentary understanding of statistics should suggest to you that this is a flawed defense of your methodology.

If you were actually interested in figuring out if you're right about the predictive power of "better than" polls, you could track the predictive value of the polls through-out a number of seasons, looking at preseason polls impact on week 1 games, week 1 polls on week 2 games, and so on... what I suspect you'd find is that the predictive value of the polls increases as more data becomes available (shocking theory, I know) and that by the end of the season there aren't nearly as many upsets as your cherry-picked week.

Of course, surely you also realize that there isn't a methodology enforced on poll voters, so some of them are bound to use different models, thus reducing the predictive value of the polls in general.

In short: you're clearly entitled to rank the teams using whatever methodology you want, but it would serve you well to be a little less self-righteous about it. Either that, or go get the data to actually support your point.

Anonymous said...

Mergz,

I empathize with your frustration.

I got fed up with the standard "My conference is better than yours" banter on Rivals' CFB message board, so I started a thread on why people don't like computer rankings. I think a lot of the responses I got reflected the same mindset that's apparently driving you crazy.

Anonymous said...

cant we just wait to rank til about this point in the season (i didnt come up with this idea)?

Henry Louis Gomez said...

Anon,

It's cute that you come back here and act like the offended party after your initial jackass posts. Whatever douchebag. If you don't like it, change the channel.

Mergz said...

Anon -

Your first post was deleted because of its insulting nature. If the most recent post had been your first it would still be there.

My guess is that later polls are more predictive, as will mine be. Even then they often fail to get it right (Ohio State was ranked higher than Florida in 2006, for instance). But once again, I'm not sure "prediction" is the purpose overall, and it certainly isn't my purpose.

As for your suggested study, do it on your own if it interests you. I chose the 2nd week because it was after some evidence had started, - the preseaon poll was even more wrong.

Anonymous said...

Resume ranking seems to be just as good as any system out there. At least there's justification for your ranking based on evidence.

Keep up the good work.