Also, warning: it's long. For those who like their baseball games in ESPN highlights, and their Melville in Cliff Notes, I put bullet points under each heading.
I wanted to generate a discussion on different polling strategies, and come to a consensus on what we expect from NCAA polls.
- Polls are not and will never be exact, even at the end of the season. There is no "right answer." Comparing over 100 teams with hideously unbalanced schedules with absolute accuracy is nigh impossible.
- We want polls anyway.
- A higher-ranked team is considered better than one ranked below it.
- Even if we produced that theoretical "perfect poll" there would be plenty of people who disagree on it.
- To a degree, there is an unstated general consensus that some teams are better than others, i.e. the masses can agree on certain things, like Florida is in the Top 3, and Michigan isn't.
- We will know more as the season progresses.
- The perfect poll would be the exact same in the preseason and at the end of the season, and still be entirely justifiable.
- Consensus is the ultimate goal -- corollary: fewer polls is better.
- Best At: Being a ranking on this year's performance that actually has its basis in this year's performance
- Worst At: Providing a non-laughable poll before November
- Primary Gripe: Small sample = useless
I have respect for resume voters because they have the same standard throughout the season. The downside is their polls take awhile to come together. Resumes grow more demonstrative only after there's experience on there. If I showed you the resumes of two 16-year-olds and you had to pick which one will end up making the most money by the time they are 50, we would be clueless.
At least it's a metric that makes some sense. But the wild variance defeats the purpose of having these polls in the first place: it's not to generate discussion, it's to provide a frame of reference for assessing the difficulty of beating one team or another. If Cincy loses next week, nobody's going to believe it if you say "oh wow, they beat the No. 1 team in the country."
It also, when it's used in concert with other voting metrics, has the unintended effect of compounding things like an overrated conference. A great example is the Big East a few years ago, when South Florida, Rutgers, Louisville and West Virginia took advantage of some early season flukes and an incredibly soft middle of the schedule to leap-frog each other to the top of the polls. This was the primary culprit in the short-lived appearance of USF at No. 2 in the BCS poll -- any ranking that has South Florida second in the nation in anything beside STDs is a travesty.
The upside of resume voting is that every week it gets more and more feasible. The BCS poll has been, in many of its incarnations, essentially a resume poll, which had the good sense to begin releasing data late in the season. Ultimately, resume voting is a justifiable system so long as it remains pure, but isn't very useful early in the year at providing a poll's primary objective: to provide a plausible ranking of NCAA's best teams.
Suggestion for improvement: Stay out of it until near the end. I want resume to determine who plays for the National Championship, but I'd rather not half-finished resumes affecting the mid-season polls. Other words: I'm with you if you wanna put '03 LSU and '03 Oklahoma in the Championship, but let's call '03 USC No. 1 right up until the end of the Rose Bowl, just so we're clear that Michigan is facing the hardest team in the country. Make sense?
- Best At: Pre-Season Poll that passes credulity test, Mid-season difficulty rankings
- Worst At: End-of-Season Poll that passes credulity test
- Primary Gripe: Not enough data, plays down this year's performance, which, like, isn't that what the poll is about?
Early in the season, this is most polls, including the AP and Coaches. Since no games have been played, it's a vote based primarily on how good the team was last year, with plusses for returning players, minuses for returning players.
This does a much better job of placating the masses in the pre-season. As the season progresses, however, as opposed to resume voting, this metric tends to disappear almost entirely, which I think is a major disservice to these polls.
Essentially, they fall victim early on to resume voting, rather than stick to their guns. This means big drops for teams as they lose. The downside, of course, is that if there's a consensus No. 1 team that loses its only two games early in the year, you'll see a major shift in that team's ranking -- big drop, steady incline, etc. This hurts the usefulness of the poll, since it changes its base metric mid-way through, essentially calling out its own initial justification.
A roster-based poll shouldn't be oblivious to the unfolding season, but it also shouldn't abandon its basis. Updates would be based on roster shifts, such as Oregon losing Dixon, Pat White losing a finger, or Michigan discovering one of its 4-star freshman recruits is already a more-than-serviceable and perhaps awesome college QB. This does not seem to generate much shift, but revelations abound in college football -- if someone pays close attention, we could end up with a fairly decent poll insofar as showing how much of a challenge each team should present.
Like resume polling, a roster poll is justifiable -- last year's performance, injuries, player statistics: these are all available metrics.
However, as the year progresses, such a poll would require A TON of input to remain accurate. Barring a UFR for every team, a roster poll seems unfeasible.
I can't think of a poll that keeps this metric throughout the season. I'd like to see one in the blog poll. It would wrack up a lot of Mr. Stubborns, and a few other outliers as other voters respond to season upsets, etc. And more importantly, while it's very useful at showing which team is the hardest to beat talent-wise early in the year, the more the season progresses, the more you'll have major incongruities, like a highly talented 4-loss team in the Top 5 while a lucky, scrappy, undefeated mid-Major team lingers at the bottom of the Top 25.
After about 8 weeks, a roster-voted poll would get lapped by the resume voters in placating the general populace, and take a lot of flack along the way. And at the end of the year, it would be totally useless.
Suggestion for improvement: This needs statistics, or it's as bupkis as pre-season polls. One day (I'm already looking into it) there will be UFR-like statistics kept for every player on every team. This will facilitate player and position rankings. And coaching ratings, too. And team rankings (offensive/defensive efficiency, etc.) The more info compiled and thrown in, the more this type of polling becomes feasible. Never going to be useful for who belongs in a championship, but I, for one, would find such a stat very interesting when having one team go up against another.
- Best At: Pre-Season Polling
- Worst At: BCS Selection, Precision
- Primary Gripe: Factors are compounded
This is a straight-up attempt to get the final poll right in Week 1. A lot of AP voters fall into this trap, as evidenced by the justification they give for their preseason ballots.
"I ranked Ohio State 1st because the lolBigTen is so weak the Buckeyes can knock off a freshman-quarterbacked USC, then tapdance to the BCS championship again."In this example, does this hypothetical
Predictive voting does have a strategy for keeping itself in line, which makes it somewhat useful, if still inaccurate, for mid-season and late-season polling. Essentially, teams are not down-rated at all when they lose something they were expected to lose in the fashion in which they were expected to lose it. They play against their expectations.
Predictive voting is often used in concert with another metric, most often as a correction to Roster Voting ballots that generally have mid-Majors and giants in weak BCS conferences underrated. It generally has a lot of opportunity to look stupid as the season progresses, since the swings after unexpected wins and losses, in practice, are never truly in line with expectations. It also doesn't account for surprises, like Notre Dame losing to Michigan (not expected) but demonstrating that its offense is for real (i.e. they're not worthy of a major fall).
Predictive voting is, however, not a bad way, conceptually, to achieve the goal of a preseason ballot that bears some resemblance to the end of the season. Of course, it's hideous at providing an accurate ranking of teams' actual ability. But it does a fair job of passing the eyeball test, and remains a well-used tool for college polling.
Suggestion for improvement: Accuracy is the problem, because all changes are totally subjective. So use computers. Run 10,000 simulations of every game left in the season. This becomes the base prediction for each team, and should provide a solid framework for an initial season. Derivation from expectation down-ranks them or up-ranks them as the season progresses. Easier way: use the spread -- gamblers know what they're doing.
- Best At: Wooooo!!! Tate Forcier is a god!!! I'm gonna go online now and see if the national consensus agrees! Woooo!!! They agree! We Rock!!!!
- Worst At: NCAA Polling
- Primary Gripe: Loose grip on reality
This metric is among the least justifiable of the non-biased metrics, but is also rampant. Except it's also the easiest way to create a poll that readers generally agree with mid-season. It's basically rearranging teams each week based on carrots like "so-and-so deserves a 10-slot bump" or "Team X defeated Team Y so team X should go above Team Y."
It passes the eyeball test, which is the whole point of hype voting. But it also generates a goodly chunk of the eyeball rolling from other pollsters who want something more concrete behind their polls.
Suggestion for improvement: This basically comes down to faking it to get the results you wanted when solid metrics fail. I'm of a mind to either improve metrics or believe them before turning to pre-conceived notions out of convenience.
- Best At: No. 3 Notre Dame @ No. 1 USC. TONIGHT on NBC!!!
- Worst At: Honesty
- Primary Gripe: Subversion of polling for selfish gain
Brian uses the Coulter/Kos Award to keep the bloggers honest about their own teams, but I don't know how much he's watching what they do to their rivals and opponents. Just because you wear your bias on your sleeve, that doesn't mean you're immune from it (e.g. Coulter, Kos).
Suggestion for improvement: Not that Brian hasn't said it 1,000 times, but this bears repetition upon repetition: MAKE ALL VOTES PUBLIC AND HOLD VOTERS ACCOUNTABLE.
What's Best? Obviously, aside from a few resume polls, most polls are a combination of many of these metrics, all of which have major holes in them that strain credulity, over/under-reward scheduling and biases and notoriety, etc. At any given point during the season, and depending on the function a poll is meant to serve at that point in the season, there are better metrics than others.
So let's go back to our suppositions, and pick out what it is we want from a poll at any given time:
- Preseason: Closest as possible to the final poll, plus something that passes the eye test, i.e. readers can generally agree with it. For this, I suggest a combination of Roster and Predictive polling. Both are in dire need of better statistics, but the stats are out there already, and currently being employed to good effect by oddsmakers, who have a stake in getting it right (although they move their bets based on hype). We know who's on what team, and who will most likely be playing X amount of time at each position. We have a record of play for every year prior for every player on every team. We know the recruiting value of incoming freshmen, and we know the base value of freshmen to keep the recruiting value in perspective. As the season progresses, we have more records of play, which should make us more accurate. Transcribing this to a statistical value is not impossible, just very time-consuming.
- Early Season: Still, I would stick to exclusively Roster and Predictive polls, for reasons shown above. I think one consensus poll would be best for this period.
- Week 8 to Bowls: Start publishing a second poll, sort of like the BCS numbers, but not really, because it would be entirely Resume based (note: would also be used to determine playoff spots). This poll would show teams ranked by their resume If they were to win every game left on their schedule. It seems counter-intuitive, since, yeah, a lot of them play each other. But actually, that keeps it cleaner -- those that play each other get credit for doing so based on where each is at before the inevitable down-ranking of each other.*
- End of Season: Publish a final Resume-based poll.
It would be awesome for fans, as major programs try to schedule each other early to build a high resume before Week 8. Then, as injuries deplete rosters and cold sets in, each team is in do-or-die mode every week, or else risk losing their place in line.
Okay, I've said my piece. As with everything else I write, I ask you to please find as many holes in it as you can (except typos, which I plan to go back and fix when time allots).
Back in the 80's and 90's poker had 3 types of players basically. The conservative math guys that new statistically what the right play was. The Doyle Brunson devotees that always applied pressure with aggressive betting and you had donkeys that gave these types of players money. College football was much the same you had your conservative programs like Michigan, Penn St, Notre Dame and much of the SEC consistently winning . You had the super aggressive teams taking over like Florida St , Miami and Nebraska with aggressive blitzing defenses and high powered offenses. These cultures clashed in spectacular fashion with merits to both successful styles. The donkeys like Indiana and Mississippi tried different things and continually got their teeth kicked in because they were not smart nor good.
Then the internet age came in with ESPN televising poker and we we're introduced to a new style of poker. This kid was scoffed at by the top pros with his small ball style of poker with small raises and playing 34 offsuit, he was labeled a donkey by the establishment. When he had success it was considered luck that could not be sustained. Eventually Daniel Negrenau won people over and a new style of poker emerged where you play a lot of hands and since it went against the grain of the popular styles it was wildly successful. This reminds me of a football coach that I like a lot named Rich Rodriguez. The spread was thought of as a gimmick and now it is wildly accepted. The thing I like the best is that while Rodriguez and Negreanu are thought of as innovators and wacky they really believe in old school beliefs and values but the fancy dressing throws people off. Rodriguez offense is founded in being physical and running the ball and you rarely see Negreanu making wild bluffs or calling bad bets that don't have value.
The interesting thing is that now we are past both these stages and now that both styles have been accepted as a credible strategy they are no longer just successful because they are contrarian, they are now part of the establishment. To me this age we are in is not about styles, but more about who is the smartest and the toughest and who is willing to adapt. Now the best offenses are no longer just spreads, or option or passing but blends of styles. Oklahoma, Florida and hopefully soon Michigan have a multitude of looks that can quickly attack a defenses weakness instead of pounding your style relentlessly hoping to outwill your opponent. Much like the best poker players are the ones that have adapted to the internet maniacs and have a style that they can adjust to the table or setting they are in. I have full confidence Rod is one of these poker players that will win for Michigan. He's not a lunatic like Weis or mathematician like Carr. I think he is a master play caller much like Holtz in the the 80's or Spurrier in the 90's that can and has already adapted to his teams strengths and other teams weaknesses while not straying from his core beliefs. Rod's a good poker player and now that he has added Forcier and Robinson he's getting some pocket Aces to play with making it a little bit easier to win with.
UPDATE: Part Two is here. Includes an alrernative to Passing Efficiency.
Today's Stat: Passing Efficiency(Full NCAA Rankings)
Players of note: Ryan Mallett, Arkansas (1st, 210.25); Jimmy Clausen, ND (3rd, 196.31); Kirk Cousins, MSU (6th, 186.71); Tate Forcier, Michigan (21st, 161.69); Terrelle Pryor, OSU (79th, 116.92)
Why it's important:It's pretty much the golden standard for measuring the (wait for it) efficiency of a quarterback. It's not flawless by any means, but overall is a pretty good indication of how good a quarterback is. Once there's a good sample size (at least 100 attempts), it's pretty safe to say that a player in the top 20 of the efficiency ratings is a good quarterback, and a player outside the top 50 isn't quite as high-caliber.
Why it's flawed:Passing Efficiency measures just that -- efficiency. How efficient something or someone is usually boils down to how much of 'x' they can do in 'y' amount of tries. It's no different in the world of college football. The equation for Passing Efficiency in College Football is as follows:
(Completions x 100) + (Yards x 8.4) + (Touchdowns x 330) - (Interceptions x 200)
So while all of that stuff on top is really important, it really boils down to how many passes the quarterback has attempted. For example:
Quarterback A plays basically the whole game and racks up some pretty good numbers, but in the red zone gets bruised up and comes out for a play.
Quarterback B comes in for that one play and throws an eight yard touchdown pass, and is right back on the bench, and remains there for the rest of the game.
Quarterback A's stats: 28/35, 310yds, 3 TDs, 1 INT
Quarterback B's stats: 1/1, 8yds, 1 TD
Go ahead and take a stab at each quarterback's rating. Or just scroll down a bit and look at the actual answers, you cheater.
Quarterback A's Efficiency Rating: 246.2
Quarterback B's Efficiency Rating: 497.2
Quarterback B, the backup who came in for one play, isn't necessarily a better quarterback than Quarterback A.. there's actually a good chance that he's a good deal worse. His efficiency rating, however, is more than twice that of Quarterback A, who had a damn good day throwing the ball. However because that one pass attempt that he did have was a successful one, his Efficiency Rating is about 287 points higher than the current highest rating in Division 1.
Applying this to current statistics:Ryan Mallett: 17/22, 309yds, 1 TD (210.25)
In the one game he's appeared in so far, Mallett has only attempted 22 passes (remember, the smaller the sample size the more skewed the rating), and completed 17 of them. A 77% completion percentage is second only to Sean Canfield (OSU, NTOSU), who has the 14th highest efficiency rating. He only has the one touchdown and has yet to throw a pick (not as important as you'd think, as you'll see later). Not stellar numbers by any means, but he did pretty well against Missouri State.
Jimmy Clausen: 40/60, 651yds, 7 TDs (196.31)
Not too much to say here, the efficiency rating is pretty well deserved so far. Quite the interesting comparison to Mallett's numbers, however. Clausen's numbers are obviously superior in every way but completion percentage. Clausen is clearly the superior quarterback here, yet because of the small sample size in Mallett's case, he has the higher rating.
Tate Forcier: 36/53, 419yds, 5 TDs, 1 INT (161.69)
Tate's numbers compared to his rating are also pretty interesting. He actually has a higher completion percentage (67.9) than Clausen (66.7), has a respectable touchdown percentage (9.43% of his passes are touchdowns, compared to Clausen's 11.7%), and only has the one interception. However even if we take that interception away (it wasn't even his fault!), Forcier's rating doesn't improve too dramatically. If the pass fell harmlessly to the ground, his rating would be a 165.5, good for 17th. If the pass was completed for a 15 yard gain his rating would be a 169.7, putting him in 16th.
The TakeawayQuarterback Efficiency Rating is an effective way to rank the overall efficiency of quarterbacks, especially later in the season once there is a decent sample size of attempts to go by. Until then, however, it's a stat that's easily skewed by a few attempts going for big yards and touchdowns. We all know Quarterback A in the example above had a better game than Quarterback B, but the formula for efficiency rating doesn't. Quarterback B did complete 100% of his passes, and 100% of his attempts went for touchdowns.. the thing is there was just the one attempt. Therein lies the flaw.
Just for fun, try to guess which stat line would garner the higher efficiency rating. Answers are at the bottom of the post.
Situation 1 Situation 2
A. 25/30, 250yds, 2 TDs, 2 INTs l A. 30/40, 300yds, 1 TD
B. 15/17, 140yds, 1 TD l B. 10/12, 100yds, 2 TDs
A. 20/24, 200yds, 2 TDs, 4 INTs
B. 20/40, 250yds, 5 TDs
Behind the Numbers will be back soon with another look at a stat from the world of College Football. Any stats you want to be examined a little closer? Or even just a stat you've been interested in for a long time? Let me know in the comments and I'll do my best to get to it in the next few installments of BtN. Thanks for reading!
Situation 1- A: 162.0 B: 176.8; Situation 2- A: 146.2 B: 208.3;
Situation 3- A: 147.5 B:143.75
My main webpage is http://webpages.charter.net/ultimakhan/ and Brian was gracious enough to post it in one of his posts a few years back -- I just wanted to add a diary entry for those looking for an easier link to the content.
The full topic list on the site is as follows:
Current Poll Summaries (conference -- includes 10-year lookback)
All-time AP Poll Information (several entries)
Average 1-A (FBS) scores for FBS vs. FBS games only (current year)
Bowl Information (10-year records¤t streaks)
Win-Loss Information (current year and 10-year lookback)
Also included at the end are additional stats on the Big Ten records and some Michigan specific records.
I welcome any feedback to improve the content and/or correct mistakes (with your information showing me what was wrong).
During the ND game on Saturday, ESPN used a graphic that showed ND had slipped to #3 in all-time wins, behind U-M and Texas. I had not realized this happened at the end of last season.
Here are the current (up-to-date following 2009 Week 2) rankings for wins and winning percentage. Comments/observations below.
1. MICHIGAN - 874
2. Texas - 834
3. Notre Dame - 832
4. Nebraska - 819
5. Ohio State - 809
6. Penn State - 802
7. Alabama - 801
8. Oklahoma - 792
9. Tennessee - 777
10. Southern Cal - 768
ALL-TIME WINNING PERCENTAGE:
1. MICHIGAN - .740 (874-295-36, 1205 GP)
2. Notre Dame - .736 (832-285-42, 1159 GP)
3. Texas - .718 (834-317-33, 1184 GP)
4. Oklahoma - .716 (792-298-53, 1143 GP)
5. Ohio State - .715 (809-307-53, 1169 GP)
6. Alabama - .709 (801-316-43, 1160 GP)
7. Southern Cal - .707 (768-303-54, 1125 GP)
8. Nebraska - .702 (819-337-40, 1196 GP)
9. Tennessee - .694 (777-328-53, 1158 GP)
10. Penn State - .690 (802-349-42, 1193 GP)
1. In terms of wins, MICHIGAN's got a huge lead over Texas and Notre Dame, followed by another drop-off to schools that have pretty recently cracked 800 wins.
2. In terms of percentage, MICHIGAN and Notre Dame have a tremendous lead.
3. Around the 1200 GP point, a win raises MICHIGAN's percentage by about .0002 (1/5 of a point). A loss drops MICHIGAN's percentage by about .0006 (just over 1/2 of a point); so 2008 was pretty tough on the all-time stats.
4. MICHIGAN's substantial leads in each category I think can be attributed to MICHIGAN's two highly dominant eras as far as number of wins: Yost and Bo. The other schools on those lists have had dominant stretches here and there, but generally only one truly dominant era each.
5. Also, since 1970, MICHIGAN has generally avoided (thus far, fingers crossed) a multi-season dead era of a bad coach or a few bad coaching searches in a row, such as has occurred with every other team on those lists, save for Penn State (though one could argue the late 1990s and early 2000s had the same effect there). 4-5 lousy seasons in a row, or a full decade of mediocrity, really takes a toll on winning percentage.
6. The top ten on each list are the same teams, in slightly different order. So number of wins is generally analogous to winning percentage. Duh. BUT:
7. Just outside of the top ten in percentage, a few precocious upstarts pop up. Florida State sits at #11 with .670 and only 460 wins. Miami (Fla.) is #14 with .634 and only 546 wins. Other than those two notables, the list roughly holds true: more all-time wins roughly equals greater winning percentage. Since the top ten traditional power schools racked up most of their wins and drove their percentages higher in an era with nowhere near the parity we have today or in the past 30 years, I think what Florida State and Miami did in the 1980s and 1990s was pretty darned impressive.
8. Until the late 1990s, MICHIGAN and Notre Dame had each hung around the .745 mark for quite a while, then Notre Dame slipped off, and Michigan followed in 2005-2008.
QUESTIONS AND INVITATION FOR PREDICTIONS:
When will MICHIGAN get to 900 wins?
When will MICHIGAN get back to .745?