Thursday, October 19, 2017

What to Do about SCOTUS Mistakes: A Not-So Modest Proposal

By Eric Segall

This week ProPublica issued a report with the headline "It's a Fact, Supreme Court Errors aren't Hard to Find." The group reviewed "dozens of cases" and said that it found a "number of false or wholly unsupported claims." Examples included erroneous voter registration rates in Chief Justice Robert's Shelby County decision striking down a key section of the Voting Rights Act; an unsupported conclusion by Justice Alito about the percentage of American companies that require background checks for its workers similar to the ones used for federal employees; and Justice Kagan's statements about the reliability of drug sniffing dogs in an important Fourth Amendment case. ProPublica claimed in the report to review 83 randomly selected opinions over a five year period and found assertions of "legislative facts" in 24 of those cases. The group alleged that seven of those opinions, more than 25%, contained false or unsupported statements of fact.

This report, which should trouble all Court watchers, commentators, and the American people, caught the attention of Fordham Law Professor John Pfaff, who wrote an op-ed on the problem in the New York Times. Pfaff observed that "policy is a major part of the Court's docket now" but neither the Justices nor their clerks have "any serious training" in empirical methods such as statistics. He argued that the Court has traditionally relied on amicus briefs "to provide it with that broad empirical background" but unfortunately "many amicus briefs include false or unsubstantiated empirical assertions, at least some of which make it into the Justices' opinions." Pfaff also noted that many, if not most Supreme Court cases, get to the Court after years of lower court litigation "providing interested groups plenty of time to gin favorable findings." So, in Pfaff's words, "what to do?"


Pfaff makes several suggestions. One is to "establish a group of technical advisers to the Court." These social scientists and statisticians could help the Justices analyze the empirical evidence relevant to their decisions. As Pfaff  correctly observes, "there is no shortage of scholars with Ph.D's who would be eager to do that work for the court." He also suggests that perhaps the Justices could hire law clerks with better empirical training or even hire outside experts to help the Justices handle specific issues.

Pfaff recognizes that these proposals run the risk of the Justices hiring "partisan experts arguably making things worse." But he says there are ways around that such as requiring 6 out of 9 Justices to agree on the experts or having the clerk's office make the hiring decisions. He concludes that, "in the end, the question is a comparative one. It's not, is there a perfect solution ... but ... can the Court make its policy decisions better? The answer to the latter question is clear."

The ProPublica report and Pfaff's op-ed come on the heels of Chief Justice Robert's quip during the recent redistricting oral arguments that empirical evidence in that case amounted to "sociological gobbledygook." Of course, as many have reported since that argument, the Chief regularly uses such evidence to support his opinions, such as the erroneous voter registration rates he highlighted in Shelby County v. Holder.

There is an unwritten premise in both the important ProPublica report and Pfaff's fine op-ed that I would like to challenge. There is an assumption that the Justices' use of empirical evidence plays a significant role in their decision-making. That may be true in some of the Court's less controversial cases, but does anyone really believe that the Justices' views on same-sex marriage, gun control, abortion, campaign finance reform, affirmative action, or even redistricting are impacted by "facts" about the real world? I am not arguing that this kind of evidence is never result-generating but I think the occasions when it really matters are rare.

Chief Justice Roberts and the other conservatives would have struck down the key section of the Voting Rights Act regardless of the rate of voter registration in the South, and Justice Alito (along with the other eight Justices who joined his opinion) would have likely allowed background checks for employees working for government contractors (a change of policy after 9/11) no matter the percentage of private employers who also used such checks. These kinds of cases don't turn on facts but on values. As Dean Erwin Chemerinsky has written, "constitutional law is about defining and protecting society's most cherished values .... Ultimately the [Court's] decisions must be criticized or defended for the value choices the Court made. There is nothing else."

This is not to say of course that the Court should use erroneous data when explaining "policy" decisions made on the basis of values. It is to say that in most cases the Justices should leave the data behind and more transparently highlight the real value choices driving the decisions. What the Court does not need are more technical advisers to add post-hoc justifications for their decisions based on social science data and statistics which inevitably can be manipulated by experts and mathematicians in most hard cases.

My proposal is that in most of the Court's cases the Justices first clearly announce the values choices that are driving their decisions. For example, in Shelby County, Chief Justice Roberts could have cited his own 1982 legal memo that he wrote when he worked for the Justice Department announcing that parts of the VRA amounted to "the most intrusive interference imaginable by federal courts into state and local processes." Only after the clear articulation of such value preferences, and the tying of those judgments directly to the Court's legal holdings (with a "therefore" or a "thus"), should the Justices discuss the social science data or other facts which may support their holdings. My guess is if the Court committed itself to that kind of approach, the Justices would employ fewer contestable facts, make fewer errors, and write more transparent and honest decisions. Win, win and win.

4 comments:

Joe said...

Judges do like to speak in ex cathedra terms (including the royal "we") but such opinions probably do tend to express the values of the justices, perhaps couched in some legal reasoning. Facts are inserted to back this up. See, e.g., the infamous footnote in Brown v. Board of Education.

It's probably fairly unlikely that facts aren't going to be in legal opinions, and they will be imperfectly used, so being open about how they are used is a good policy. Likewise, especially on the appellate level, they should be avoided as much as possible. For instance, I think it would have been best to set forth the rules in D.C. v. Heller & send it back to have a hearing to determine how the facts apply.

The approach provided in the main discussion is helpful at any rate -- e.g., facts used to determine executions for those offenders under 18 or in cases of rape will be seen in context and seen as imperfect aspects of a wider whole. Our values are guided to some degree by facts, but with a lot of gray.

Bob Moss said...

In the last paragraph, may I suppose that "parts of the decision upholding the VRA amounted to 'the most intrusive interference imaginable by federal courts....'" is intended?

John Barron said...

A remedy already exists: the petition for rehearing.

greg rubin said...

In the cases you mention you are probably right that the decision drove the facts, but there is a better example... In McKune v. Lile the USSC accepted that the recidivism rate for sex crimes offenders was up to 88%. Which was a whole made up nonsense statute, but the acceptance of that statistic justified all of the sexual registry nonsense we are dealing with now. The quote “The rate of recidivism of untreated offenders has been estimated to be as high as 80 percent.” is from a 1986 Psycology Today article that has absolutely zero scientific basis, has never been supported except by sheer exaggeration, and even at the time of publishing had no basis in fact. Yet it was the lynchpin to the entire case that justified lifelong restrictions on all sex offenders.

Would it have mattered to the court if the real recidivism rate for sexual offenders was used instead? I tend to think so.