Monday, April 13, 2015

The Evidentiary Threshold for Instant Replay Reversal is Too High

by Michael Dorf

Late in last week's NCAA Division I men's basketball championship game between Duke and Wisconsin the ball went out of bounds as several players were grasping for it. In real time, it was difficult to tell who had touched it last but the referee judged that it was a Wisconsin player and thus awarded possession to Duke. Because the play occurred within the last two minutes of the game, the referees were able to review it via instant replay. They did so and were unable to tell who touched the ball last. Thus, the original call of Duke ball was left standing because the extraordinarily high standard of Rule 11, Sec. 1, Art. 1 was not satisfied. That rule requires "indisputable evidence that the call on the floor was incorrect" in order for instant replay to change the call.

As it happens, the referees were wrong. There was indisputable evidence that the ball had been last touched by Duke player Justise Winslow--as millions of viewers watching the replay on television saw--but the replay that the referees reviewed did not include the recording from the angle at which the ball's trajectory off of Winslow's finger was evident. That blown call was one of several questionable decisions for Duke by the referees down the stretch of a tight game--they also failed to notice when a Duke player stepped out of bounds while in possession of the basketball and awarded a Duke player a basket and a blocking foul call on a Wisconsin player, when it appeared that the Duke player had committed a charge.

Perhaps Duke would have won anyway even absent the help from the refs. And it's tempting to dismiss the complaints as sour grapes from supporters of a Wisconsin team that would have won if they had hit their free throws and executed better. But looking beyond this one high-stakes game, the blown instant replay on the out-of-bounds play may prompt calls for rules changes designed to enable the referees to use all available replays. That would be sensible.

Here, however, I want to suggest that the use of instant replay is flawed in a more fundamental way. The requirement of "indisputable evidence" strikes me as too stringent, for reasons I shall elaborate. Before doing so, however, I would note that this sort of extremely high threshold--which, on its face, appears to be more than the beyond-a-reasonable-doubt standard for a criminal conviction--can be found in other sports as well. In the NFL, for example, a referee needs "indisputable visual evidence" to overturn the call on the field.

Are these extraordinarily high evidentiary thresholds justifiable? To answer that question, we need to understand the benefits and costs of instant replay.

The chief benefit is obvious: The outcome of a game should turn on the performance of the players rather than the limitations of the referees; even well-trained referees may not be in the right position to make an accurate call and occasionally, even if they are, the limitations of human perception in real time sometimes lead to errors. With instant replay available to viewers at home (and via jumbotrons at the game), it seems downright unfair to permit such fortuities to decide a close contest.

The chief cost of instant replay is that it disrupts the flow of the action. A football team that was driving down the field in a two-minute drill can lose the advantage of tiring the defense. And even when the players themselves are unaffected, fans may become bored waiting for replay officials to rule. Already, American basketball and football take about three times as much time to play the game as the time that officially elapses. Frequent resort to instant replay can exacerbate the problem.

But--and this is my crucial point, indeed my only point--an extremely high evidentiary threshold for reversal is not well calibrated to balance the costs and benefits of instant replay.

Let's begin with football. In the NFL, each team is permitted to request an instant replay (via a coach "challenge") twice during the game, with a third challenge awarded if the first two are successful. A coach who makes an unsuccessful challenge loses a timeout. The numerical limits mean that there is no possibility that coach-initiated replays will interrupt a game more than six times, and they rarely interrupt that many times. In the last two minutes of each half in football, there is the opportunity for "booth review" initiated by an instant replay official rather than by the coaches. Because these reviews only occur in a limited period, they are unlikely to seriously disrupt the flow of the game. And when they do occur, they are likely to be in just those situations in which instant replay is most valuable: on close calls in the crucial moments of close games.

Instant replay in basketball is even more limited. There are no coach-initiated replays, although coaches sometimes attempt to work the refs by screaming for a replay. But as in football, so in basketball, the kinds of plays that can be reviewed are limited, with those limits being somewhat relaxed in the last two minutes of regulation and overtime.

The upshot of all of this is that basketball  coaches have no opportunity to call for instant replays and football coaches have no incentive to call for instant replays when they have little chance of prevailing. So far as coaches are concerned, there is no need for a very high standard of proof to prevent instant replay from interrupting the flow of the game.

Accordingly, if the very high evidentiary threshold serves a purpose, it must be to discourage officials from too willingly resorting to instant replay. But how exactly is it supposed to do that? An official will only be tempted to go to instant replay when he has reason to think his call (or the call of a fellow official) may be wrong. Until he actually looks at the replay, he won't have a good sense of whether there is indisputable evidence or any evidence that the call was wrong. So the standard of proof doesn't really bear on whether the official will want to look at the replay.

Meanwhile, the need to find indisputable evidence has two harmful effects. One is obvious: There will be some substantial number of cases in which the referee makes a call at an important juncture of the game, the replay makes pretty clear that the call was wrong, but the evidence is not so overwhelming as to justify changing the call. In these circumstances, the referees will have looked carefully at the replay, typically in slow motion, and yet decided to affirm a call that is likely wrong.

Second, the need for indisputable evidence will sometimes prolong the time necessary for reviewing instant replay. Suppose the officials look at a replay and quickly come to the conclusion that the call on the field or court was wrong. But they are not 100% confident in that conclusion. They now must continue to review the replay from various angles to assure themselves that the evidence of the original call's wrongness is indisputable.

I suspect that the reasoning behind the extraordinarily high evidentiary threshold for instant replay reversal even combined with the other safeguards against instant replay unduly disrupting the flow of the game is a kind of belt-and-suspenders thinking. But because the high threshold itself has costs, that thinking is flawed. Sports that utilize instant replay with the other safeguards of the sort one sees in football and basketball would do better to allow reversal based on a somewhat weaker threshold, perhaps clear-and-convincing evidence or--heretical though it may sound--even a mere preponderance of the evidence.


Shag from Brookline said...

Is this a belated April Fool's Day post or a parable on CJ Roberts' view of the judge as an umpire calling balls and strikes? We can concede that some decisions of the Court are not indisputable. If oral arguments before the Court were televised, perhaps instant replay of the Justices might disclose via their body language a better guide of their views on a case than mere audio.

Unknown said...

@Shag: The body language of the Justices would not make a difference in analyzing whether or not a ruling's rationale is sound, making the televising pointless unless One is a partisan Hack looking to manipulate the video of the court in order to politicize it.

I also admit I was expecting the article to tie into something Supreme Court related only to find nothing. :-(

Michael C. Dorf said...

In response to Shag and Unknown: There is a long and noble tradition of using the tools of legal analysis to study games and sports, which are governed by rules and standards in much the way that other domains of life are. For example, HLA Hart devotes a good deal of attention to how, if at all, the nature of a game changes when an official scorer (referee) is added. Ronald Dworkin famously opined on the NBA rule forbidding players from leaving the bench during an altercation. Etc. You may treat these as metaphors or examples for more "worthy" controversies if you like, and that is how Hart (but not so much Dworkin) intended their examples; but there is nothing wrong with looking at games and sports as a domain of rule and standard governed conduct in their own right.

Nonetheless, if you want a tie-in to "real" law, I would note that on the FB fan page for this blog, History Professor (and my college debate partner) Benjamin Alpers writes:

"I think the indisputable evidence standard serves another, more intangible, function: if too many calls were overturned it might make the officials look less competent. No, I don't think this is a good argument for the standard, but I do think that building a strong bias in favor of the call on the floor is designed, in part, to bolster the legitimacy of the refs and assure that they not lose control of the game."

I agree with Prof. Alpers and here's your tie-in: The very high standard that a petitioner must meet to obtain federal habeas corpus relief likewise serves to bolster the legitimacy of state criminal courts, thus obscuring their flaws.

Shag from Brookline said...

Mike, I might take an example from golf, if golf is really a sport, and suggest that you might take a mulligan on this: "Shoemaker, Stick to Thy Last." Maybe Dersh has a thing or two or three ... to add on the subject. March Madness is only a game, perhaps to distract us from political dysfunction. By the way, Mike, how did your picks work out?

Perhaps Unknown lives in a cocoon and doesn't recall Justice Alito's body language (and lip movement) during a fairly recent State of the Union address. What might videos of Court orals indicate in showing a possible reaction by a Justice or two to a question/comment/observation by another Justice? Or do the robes hide body language? Frankly, the Court often politicizes itself via the Justices. Often the soundness of a ruling by the Court is in the bias of the beholder.

Howard Wasserman said...

If the standard of review were lower, wouldn't more football coaches challenge, since they would be more likely to prevail on the challenge?

There is absolutely no correlation between NCAA pools and knowledge of the sport.

Michael C. Dorf said...

The answer to Howard's question is almost certainly yes, but the numerical limit on challenges prevents serious abuse.

andy grewal said...

"this sort of extremely high threshold--which, on its face, appears to be more than the beyond-a-reasonable-doubt standard for a criminal conviction"

Do you think the "indisputable evidence" standard really is an indisputable evidence standard? I don't watch NCAA hoops but I do watch tons of NFL games. In practice, it seems like the NFL almost always follows a more likely than not standard.
Circumstances where they say "Well, it looks like this should probably be reversed, but I'm only 70% sure so I'll let the call stand" seem rare, especially because fans get very pissed off when the instant replay call is "wrong" (and I think those of us on our couches go by the 51% standard).

The "indisputable evidence" standard strikes me as mostly a tie-breaker. If the replay is inconclusive, they let the call stand, but if it looks like it probably should be reversed, then it will be.

James Longfellow said...

There is another reason to support the refs and that is the principle of subsidiarity. Professor Dorf claims that the main purpose of this is to cover up "flaws" but that is simply untrue. The better way to think about the issue is that the person who is closest to the action gets the benefit of the doubt.

Now, one can think more or less of this principle as a principle and I take it from this post and others that the Professor thinks very little of the principle. Fair enough. But for some people the principle is worth defending /as a principle/ which is the reason I suspect that they evaluate the cost and benefits different than the professor.

Michael C. Dorf said...

I do not think very little of the principle of subsidiarity, which, in any event, has no application here. Deciding the issue by a replay official is not replacing local authority by central authority (as the principle of subsidiarity counsels against). A replay official is at the same level of authority as the official on the field or court. The principle that James Longfellow apparently intends to invoke is the one that forbids appellate courts from reviewing trial courts de novo on matters as to which a worm's eye view ostensibly affords an advantage, such as eyewitness credibility. But being close to the action in real time without the benefit of replay angles and slow motion often puts on-field officials at a disadvantage.

Perhaps Mr. Longfellow is only reacting to my comment about habeas, but there too he is mistaken, because I was (tacitly) criticizing the high threshold for reversing state court determinations of federal law, as to which the principle of subsidiarity is also inapplicable (although it is and should be applicable to questions of fact).

Now, as to Andy Grewal's comment, I agree that in practice the standard is less strict than it is in writing, but I think there is still some deference given to the original call, even in the NFL. I shall consult with my colleague Valerie Hans--who studies juries empirically--to find out whether juries in reality disregard the standard of proof required.

Paul Scott said...

I think standards of review in instant replay are silly altogether. Why should the opinion at the time of the official who made the call matter at all? Review should be de novo.

andy grewal said...

The official who made the call may have seen things that weren't caught on camera.

In theory, anyway:

Shag from Brookline said...

Paul Scott's:

"Review should be de novo."

might then lead to "appellate" review to overcome any possible biases of the officials (especially the one who's call is being reviewed) on the floor. [Isn't something like this done in the NFL?]

But keep in mind, it's only a game.

Patrick S. O'Donnell said...

I may be the only person in the world who is not a fan of instant replay of any kind, in any sport. Let them, on occasion, make mistakes and let the players and fans live with the results. I'm absolutely horrified to find it now being used in baseball.

Shag from Brookline said...

I'm with Patrick especially if law profs were to interpret instant replay.

t jones said...

I think Mr. Alpers has come the closest to the mark. He does not mention that the leagues were dragged more or less kicking and screaming to the adoption of instant reply review by the increasing availability of instant replay to fans and the resulting body of indisputable evidence that their officials were making critical, game changing mistakes. When they got there, they passively-aggressively, made replay review as unlikely to result in actual change as they could.
The leagues (clubs of multi-millionaires, including the Universities which make up the NCAA, after all) don't like to be told what to do, even by their fans. And, I suspect that the officials were against replay review as bruising to their egos.
Personally, I doubt that game flow or game prolonging have much to do with the leagues' dislike of instant replay. As others have been pointed out, reducing the time consumed by commercial breaks would have a far greater game shortening effect, and there's no movement in that direction.

summermobile said...

Circumstances where they say "Well, it looks like this should probably be reversed, but I'm only 70% sure so I'll let the call stand" seem rare, especially because fans get very pissed off when the instant replay call is "wrong" (and I think those of us on our couches go by the 51% standard).

Guo Guo said...

guowenhao20150430calvin klein outlet
swarovski crystal
links of london
michael kors outlet
burberry outlet online
lacoste outlet
ray ban sunglasses
nba jerseys
ray ban wayfarer
michael kors handbags
mac makeup
kobe bryants shoes
michael kors outlet
tods outlet
toms shoes
ray ban uk
ray ban sunglasses
coach factory outlet
celine outlet
adidas wings
marc jacobs outlet
abercrombie and fitch
timberland boots
true religion outlet
michael kors factory outlet
coach outlet store online
louboutin shoes
beats solo
nike roshe run
ray ban sunglasses
air force one shoes
ray ban sale
herve leger dresses
michael kors outlet
cheap oakley sunglasses
salvatore ferragamo
michael kors handbags
cheap toms
mcm bags

Shijun Lin said...

shijun 5.20
kids lebron james shoes
jordan shoes
abercrombie fitch
jordan retro 5
louis vuitton handbags
michael kors handbags
coach outlet store online
ray ban wayfarer
replica watches
michael kors
cheap lebron james shoes
pandora rings
tory burch outlet online
timberland outlet
ralph lauren outlet
coach factory outlet
louis vuitton outlet
louis vuitton outlet
coach outlet store online
chaenl bags
coach outlet online
michael kors handbags
louis vuitton handbags
louis vuitton handbags
ray ban aviators
michael kors
soccer jerseys
coach outlet
michael kors uk
coach outlet store online
michael kors outlet
lebron 12
oakley store
michael kors handbags
louis vuitton outlet
kate spade
concord 11s
tods sale

chenlina said...

michael kors handbags
knockoff watches
louis vuitton outlet
michael kors
retro 11
toms wedges
fake watches
hollister jeans
mulberry uk
true religion outlet
ray bans
air jordan retro
nike air max
jordan concord 11s
michael kors outlet
ray ban sunglass
lebron 11
polo outlet
coach factory outlet
true religion sale
oakley sunglasses wholesale
burberry sale
christian louboutin sale
new jordans
louis vuitton handbags
celine bags
ray ban wayfarer
louis vuitton
kevin durant basketball shoes
adidas wings
oakley eyeglasses
oakley sunglasses wholesale
concords 11
jordan 11 columbia
christian louboutin shoes
louis vuitton handbags
coach outlet store online
abercrombie store
oakley sunglasses cheap