Tuesday, November 25, 2014

In Praise of Expert Panels (in Football, at Least)

-- Posted by Neil H. Buchanan

It is a short holiday week, and even if I had something to say about the big national news story from Ferguson, Missouri, I would rather write about something relatively frivolous.  And, as Professor Dorf's recent personal remembrance of his days as a jock demonstrated, nothing quite matches sports for its marriage of complete unimportance with intense interest.  When it comes to American college football, I have written some rather serious things (see most recently here) about the institutions of higher learning that serve as host/sponsors to the lucrative games, and I hold what appears to be a minority view about how the lucre should be spent.

Today, however, none of that matters, for I want to talk about the College Football Playoff (tm).  More accurately, I want to complain about the consistent inanity of the talk among college football commentators about the playoff.  Not that this should surprise anyone.  This is still a world in which people seriously argue that a coach in 2014 is historically great, because he has just set the record for most consecutive 9-win seasons.  This is ridiculous, of course, because it conveniently ignores that it is only in the last decade or so that teams have played 13 or 14 games in a season, whereas within my lifetime teams would play ten games and, even if they won almost all of them, would not play in a bowl.  Nine-and-five is just not the same as 9-1, but you would never know that from listening to anyone on ESPN.

Even so, there are two remarkable things about people's reactions to the new playoff system.  First, as I predicted some time back, the new four-team system (replacing the BCS two-team championship that lasted for almost two decades) satisfies no one.  Just last weekend, one of the sports shows displayed the results of a poll of college football coaches regarding the optimal size of a playoff system.  A majority favored either 8 or 16 teams.  To hear the talk last year, the only problem was that it was all terribly unfair when a third team was left out of the BCS championship in the occasional year when there were three undefeated teams.  Now, the problem is that we're not sure whether a team with a bad loss on its record (e.g., Ohio State, which lost at home to a terrible Virginia Tech team in September) will be able to show that it really deserves to be #4.

I promised not to be serious here, but it is at least worth mentioning that the extra games are physically debilitating for the players.  The players themselves, of course, will always look into a camera and say, "Let us decide it on the field," but it is notable that the NFL players' union has consistently opposed increasing the length of the regular season schedule, as well as expanding the playoffs.  This is still one of the most brutal and dangerous sports that people are willing to watch, and adding rounds of playoffs would guarantee many more serious injuries, many of them career-ending.

It is not, therefore, as if it is "free" to add more games.  Nonetheless, it is at least amusing to see how earnestly people on the sports shows will talk about the injustice of having only four teams make the playoff.  I have always liked the pre-BCS system, in which there was no attempt to coordinate an on-the-field champion.  Last Saturday, because nearly all the big-time teams were playing "cupcake games," I ended up watching the Harvard-Yale game.  The second-highest division of the NCAA in which the Ivies play has a playoff system, but the Ivy League prohibits its teams from playing in it.  One of the commentators said that he had asked the players whether they felt cheated by not being able to compete for a national championship, and apparently they said that they like knowing that the last game of the year will be their big rivalry game.

In pre-BCS days, Oklahoma and Nebraska would play nearly every year for the Big 8 championship, and the winner would play in the Orange Bowl.  Meanwhile, Ohio State or Michigan (or, occasionally, Wisconsin or Iowa) would play USC or UCLA in the Rose Bowl.  The best years were those where, say, Oklahoma would beat Notre Dame in the Orange Bowl, and USC would beat Ohio State in the Rose Bowl, and then no one was happy.  Except that everyone was happy, because they had something to argue about for years.  Ask any Michigan player if beating Ohio State at the end of the season, and then playing (most likely losing) in the Rose Bowl was a successful season.

So, nothing has been gained in terms of making the game more exciting, fulfilling, or anything else by dumping the old system.  "Deciding it on the field" sounds like it means something, but unless we are really going to go to a 64-team playoff, the arguments will never end.  Even in men's basketball, where the NCAA does run a 64-team playoff, there are arguments about "bubble teams," and about bad calls and all that.  How about best-of-three championship series?  Make that best-of-seven!  Oh brother.

The second remarkable thing that has emerged in this year's new playoff system is the expert panel that will anoint the four semifinalists.  That panel includes a collection of respected college athletic directors, former coaches, and so on -- and, for reasons that mystify me, Condoleeza Rice.  (I know, she was Provost at Stanford before becoming a neocon princess, but so what?)  Starting at mid-season, that committee has been releasing weekly rankings of teams, with members of the committee offering cryptic comments about what they are taking into account, all of which leads to mad speculation about what they really care about.

And it turns out that this committee is actually pretty savvy.  To hear the commentators on the sports shows (former coaches and players, mostly) talk about it, the primary thing that should matter is win-loss record.  Except, you know, when it doesn't.  So, this year, everyone is in a tizzy because the committee has said that Florida State is #3, even though it is the only undefeated team among the "Power 5 conferences."  The thing is, Florida State does not at all look like the best team in the country.  They have played no one of any consequence, and they keep barely winning.  Now, of course, the new narrative is that winning close is their "signature."

Meanwhile, take a look at the Western Division of the SEC.  Those seven schools beat up on each other every week, but the evidence is that they are the best in the country.  The teams in the division have lost exactly three games combined outside of the division, and all three were to the two best teams in the SEC East (twice to Georgia, once to Missouri).  I am no fan of the SEC, having grown up in Big 10 country, but quality is quality.  If a team plays tough games every week, sometimes they will lose.  Meanwhile, Florida State barely wins against mediocre opponents.  Everyone seems comfortable with the idea that undefeated Marshall cannot be taken seriously, because they "haven't played anybody"; but at least Marshall wins its games convincingly.  Florida State has been blessed with weak opponents, too, and it barely manages to win.

The traditional polls (the sportswriters for the AP, and the coaches for USA Today) are impressed by the win-loss mark.  True, one-loss Alabama tops the coaches' poll this week, but Florida State received more first place votes than 'Bama.  And the commentary is all about "proving it on the field."  One of my favorite lines is: "Florida State has won every game.  What more could they do?"  Marshall?  "Well, they don't play anybody."

Which is why I write to praise this particular expert panel.  They are actually trying to figure out which teams are the four best in the country, based on a combination of factors that seem to make sense.  (There are still some oddities, like having TCU ahead of Baylor, even though TCU lost to Baylor.)  In a system that should not exist, and where the matter being decided really could not be less important, this committee has decided to ignore the encrusted conventional wisdom and actually think clearly about the issues.  If only the buffoons on the Bowles-Simpson committee had done the same!


Paul Scott said...

I was a fan of BCS, though I would have preferred a BCS system when the computer just churned out the ratings as final (either before or after the Bowl games). I don't care for the use of expert panels but I like even less a playoff system.

In the NFL, the better team wins about 2/3rds of the time in a single game. Now, parity in the NFL is a lot better than parity in the NCAA, but I am not so sure that is true "at the top."

I find it hard to accept that either the former AP/Coaches poll system or BCS computer rankings "got it wrong" 1/3rd of the time.

Playoffs are more exciting than a poll or experts or an algorithm, but they will show you the better team far less often. And the more rounds, the more likely you are to get the wrong result.

Looking at just 4 teams and assuming that the judges always get the best team in the mix. If we assume these teams are closely matched - at least as close on average as the NFL on average then we get only a 44% chance that the playoff will give us the best team. Extend this a round and the probability goes down to 30%.

To me, what this all comes down to is $$. A playoff system is going to bring in more T.V. dollars. This is especially true since the playoff system is going to exist independent of the bowl system. The entire idea is a way for the NCAA to extract more cash from the lives of people it purportedly is there to protect.

Michael C. Dorf said...

Obviously the best approach would be an 11-team playoff featuring all undefeated teams (if held today, FSU, Marshall, and Harvard) plus the remainder to get to 11 (if today, the top 8 teams with one or more loss). There would need to be some bye rounds, but it would all be worth it because 11 is one louder.

Paul Scott said...

If you use Swiss, you don't even need buys!

egarber said...

Although it's not perfect, I like the new playoff format. I would rather be arguing about a 5th-place bubble team than a 3. And if they expand it, the attenuation is even better. Sure, there will always be arguing; however, prior to the playoff format, too much was decided that way, imo. So while I understand that "settling it on the field" sounds easier than it actually is, I still like the spirit of that goal. As for the number of games, I would be fine getting rid of a regular-season game to square the balance.

You're right about the SEC. As a Georgia fan / grad, the cannibalism can be pretty brutal. As an aside, given how sick I am of Florida State, I am pulling for an in-state rival to beat them. Go Georgia Tech! :-)

Paul Scott said...

egarber (or anyone else who likes a playoff system),

What are you trying to "settle" "on the field?"

What question(s) does(will) a playoff answer for you?

I ask because the only correct/legitimate answer I can see is "It won't tell me a thing. But I really like watching college football at its highest level and the four games I will watch will be great!"

That is a sentiment I understand. And since this is a completely trivial matter it is as good a reason as any. We don't need the "National Champion" to actually be the best team. We don't demand that of any other sport (though the NBA seems to come closest to that mark).

My only real interest in this subject is knowing that people who want playoffs don't want them because they think it is a better lens to finding the best team each year. The playoffs are a significant step in the wrong direction if that is what you care about.

egarber said...

Paul, a few thoughts in random order.

>>My only real interest in this subject is knowing that people who want playoffs don't want them because they think it is a better lens to finding the best team each year.

I’m not sure what “best team” even means. To me, it’s about watching a team come together when the stakes are high. Whatever the credentials going in, the biggest test is how teams perform when it matters most. Indeed, this is precisely why I’m a sports fan in the first place: the “best” team doesn’t always win.

My Braves are a good example. Without a playoff system – in a world where you just make the regular-season winner the champion – Atlanta would have won a bunch of titles in the 90’s. But the playoff round exposed something, the reality that we couldn’t come together when it arguably meant the most. That has to mean something in the larger assessment of who might be the "best."

>>What are you trying to "settle" "on the field?" What question(s) does(will) a playoff answer for you?

I want to know how top-ranked teams perform against each other when everything is on the line, regardless of who “should win” on paper. It’s how a team handles the pressure of the journey, which encompasses both a long (regular season) and short (playoffs) view. Or put another way, why can’t “best” be defined this way – i.e., this team did the “best” job of navigating the terrain? And if the format for qualifying is imperfect, so be it. It is still the right KIND of solution, imo.

egarber said...

I should qualify by pointing out that the regular season is a very crucial component -- because it is the test for continuing on.

One of the constant debates I have with friends centers on this question:

Across a 10-year span, would you rather be in the playoffs every year and win only one title? Or is it better to make the playoffs only twice, with each of those runs resulting in championships? I'm in the former camp, while many of my friends think all that matters is championships. They therefore say the latter team is "better". That strikes me as wrong.

But the debate reflects the difficulty of even defining "best."

Michael C. Dorf said...


Channeling Paul (and Michael Lewis), I think the answer is not that the 90s Braves were lacking in any way but that playoffs are a small sample size. There are, to be sure, some circumstances in which a baseball team is structurally better suited to the playoffs. This year, for example, I think the Royals were probably a better team overall than the Giants, but in the regular season, the Giants' Ace would not get to pitch 2.5 out of 7 games. Still, the premium on top-two pitchers you see in the playoffs was not a problem for the 90s Braves because they had three great starters during that period. They really did just have bad luck.

Which is not to say that I don't enjoy playoff games too.

Paul Scott said...

Best is actually easy to define, even if impossible to know. Best is defined as the team to would win all the time given a sufficiently long series of games. If there is any difference at all in the talent level of two teams you can find some number of games X where a best (X/2+1) of X series will always produce the same result. At least in theory. Since X is likely to be in the millions, it is not a practical solution.

When you say things like "the biggest test is how teams perform when it matters most" you are completely discounting luck - which is a large category and completely overwhelms skill/talent in determining the outcome of a single game of football.

The closer in skill the teams are, the more luck matters. In a game where two teams are picked *because* they are believed to be the two best teams, luck is almost 100% of the result.

egarber said...

stupid Jim Leyritz in 96 :)


egarber said...

Hmmm... I'm wondering if your definition is a little too mechanical -- good for six sigma and the assembly line, but maybe not so much for human behavior.

I mean, suppose a baseball team consistently wins 100 games a year, but loses in the first round every season in the playoffs. Sure, the sample size is small for the playoffs, but what about the human factors like mental blocks, locking up, etc.? Is it not possible that pressure creates a different "game," whereby playoff scenarios and the regular season can't simply be combined into one sample? And if so, don't you have to factor in playoff success separately as a component? I guess what I'm saying is that I think it's possible to be a bad post-season team, because of human considerations. (It of course is no comparison, but I know that I personally suck in the tennis playoffs compared to the regular season, because my head gets in the way. Of course, I'm not that good in the regular season either, but that's a different problem :) )

This is a stretch, but it sort of reminds me of economics. For the longest time, people thought markets were rational and predictable, like a machine. But humans and their associated herd behavior are often irrational and unpredictable. You can't simply deduce a result mathematically when conditions change.

egarber said...

And I don't think it's "luck" if someone with lesser physical skill wins nonetheless because of heightened mental toughness in a pressure situation. That mental toughness is more required in the playoffs -- hence the sample maybe being a different kind of thing.

egarber said...

>> whereby playoff scenarios and the regular season can't simply be combined into one sample

better wording is "whereby playoff scenarios and the regular season are different types of samples."

egarber said...

Or let me make the point another way.

If the Braves win one world series in 15 years of making the playoffs, and the Yankees tend to win every 3 playoff years (just making that up), does that not say something about the Yankees being "better" in some meaningful way? I guess I'm bothered by the notion that these distinctions should just get washed out by statistics, that somehow we don't learn anything additional from these results.

Michael C. Dorf said...

Nearly all of the conventional wisdom about things like confidence, being great under pressure, etc., turns out to be false when tested statistically. E.g., there's no such thing as a "hot hand." It's true that some people wilt under pressure, but professional athletes are not those people or they wouldn't have made it to the pros. So a great deal really does come down to luck.

David Ricardo said...

The fact that this post has generated 16 comments (now 17 with this one)is a depressing reflection on that state of political, economic and legal discourse in this nation. I know of no recent post by either of the three regulars on this Forum that has generated that many responses.

More importantly though can anyone get me two tickets to the Auburn Alabama game, preferably between the 30's?