Thursday, June 13, 2013

Bad Journalism, Again: Fact Checkers Think That Facts Are a Matter of Opinion

-- Posted by Neil H. Buchanan

Two weeks ago, in a Dorf on Law post, I discussed "a small example, with larger implications" of some truly sloppy work by a highly regarded journalist (the White House correspondent for NBC News), who mindlessly took a quote out of context, regarding the ACA supposedly being a "train wreck" waiting to happen.  It was a shameful moment, but hardly the worst that we have seen from our free press recently.

I am by no means, of course. the only person who has been disappointed, and sometimes stunned, by the disastrously low standards of the current media culture in this country.  It is even worse when one sees such incompetence first hand.  Last summer, for example, I described the completely predictable media circus that surrounded the release of the Supreme Court's decision in the ACA case.  I was asked by a media outlet to be one of the idiots standing on the Court's steps, reading the decision in real time and shouting at the camera as I was reading.  I declined.

Another such example, perhaps even more disturbing, occurred a few weeks ago.  During the first day of my "Basics of U.S. Federal Income Taxation" course, which I was teaching to a group of Austrian students, I explained the legal standards for the burden of proof in tax cases under the Internal Revenue Code.

Not ten minutes after the class ended, I received an email from a reporter for PolitiFact, one of the self-appointed fact-checking groups of reporters in the U.S.  It seemed that a 7th-term Republican Congressman from Georgia, Randy Forbes, had gone on Fox News and said, "The IRS doesn't have to prove something against you. They can walk in and you've got the burden of proof."  The reporter wanted to know if this was true or false.  I replied, in part:

"Rep. Forbes is completely wrong.  Sec. 7491(a)(1) of the Internal Revenue Code clearly places the burden of proof on the government, not the taxpayer, when the taxpayer produces evidence that would allow a court to make a legal determination: 'If, in any court proceeding, a taxpayer introduces credible evidence with respect to any factual issue relevant to ascertaining the liability of the taxpayer for any tax imposed by subtitle A or B, the Secretary [of the Treasury] shall have the burden of proof with respect to such issue.'"

Now, PolitiFact has a spotty record, at best, in its assumed role as the nation's arbiter of truth and lies.  Almost every one of their analyses that has crossed into areas with which I have any familiarity has been wrong.  The group even declared a true statement by President Obama in 2012 not only false, but they awarded it their "lie of the year" award, which strikes me as a jump-the-shark moment for any journalistic operation, much less one that is specifically trying to separate facts from falsehoods.

Be that as it may, the burden-of-proof issue is a crystal-clear example of a legal question for which there is simply a right answer.  The congressman says the taxpayer has the burden of proof.  The law says the government has the burden of proof.  The congressman's statement is false.

To its credit, PolitiFact does recognize that some statements have nuances that make it more difficult to call them simply true or false.  Context matters, words can have multiple meanings, and so on.  PolitiFact's reports, therefore, do not merely label every statement as "true" or "false," but sometimes "mostly false," "pants on fire," and so on.

For reasons that I will explain in a moment, I would have been unsurprised if the report on Forbes's false claim had concluded that he was "Perhaps Understandably Wrong," even though it is absolutely correct to call his statement simply "False."  How did PolitiFact actually describe his statement?  "Mostly true"!  As the kids say: wtf?!  This conclusion is actually even worse, because the only reason they added "mostly" to their verdict is that they discovered that criminal tax cases place the usual "guilty beyond a reasonable doubt" burden on the prosecution, which is true but irrelevant to the inquiry.

In other words, for non-criminal tax cases, PolitiFact declared that Forbes's false statement was true.  How could they reach that completely incorrect conclusion?  This is where I would have been willing to say that the congressman's statement was "perhaps understandably wrong."  In the statutory language that I quoted, the (extremely clear) declaration that the government bears the burden of proof is preceded by the requirement that "a taxpayer introduce[] credible evidence with respect to any factual issue relevant to ascertaining the liability of the taxpayer for any tax imposed."

What does that mean?  In my email to the reporter, I explained: "This means that the 'burden of production' remains on the taxpayer to produce evidence, but that's exactly as any sensible person should want it: If the IRS had the burden to produce evidence, it would have to be given access to individuals' private files, in order to find that evidence.  Is that what we really would want the IRS to be doing?  The taxpayer usually possesses evidence of donations, expenditures, and so on.  If they want to claim that, say, they donated $25,000 to a charity, they can surely produce a canceled check to support that assertion."

I was happy to see that the PolitiFact piece quoted my rhetorical question from that email ("Is that what ... ?"), but they still completely missed the point.  They seem to have been thrown off by a statement from an IRS webpage (from the small business and self-employed taxpayers section of its website), which incorrectly uses the term "burden of proof" to describe both the burden to produce evidence and the burden to prove the legal issue in court.  As I explained in my email to the reporter:

"[W]hen the IRS says (from your email), 'The responsibility to prove entries, deductions, and statements made on your tax returns is known as the burden of proof. You must be able to prove (substantiate) certain elements of expenses to deduct them,' they're apparently trying not to confuse non-lawyer citizens with the difference between 'burden of proof' and 'burden of production.'  Having the 'responsibility to prove entries, deductions, and statements' is simply to say that you, the taxpayer, have to show up in court and show your evidence that backs up what you say."

Think about the reaction if the IRS included on its public pages a legal distinction between production of evidence and proving points of law.  Congressman Forbes would surely say something like, "How is a regular person supposed to understand that?!  This is why people hate the IRS!"

Even so, it is true that one of the IRS's web pages does conflate the two concepts, which is why I think one could call Forbes's statement "understandably wrong."  (Why a statement on the "Small Business and Self-Employed" section of the website should be viewed as definitive for every taxpayer is another matter, but I will let that go for now.)

Nevertheless, PolitiFact then completely garbles the analysis by saying: "In other words, the IRS is presumed to be correct unless the taxpayer 'produces "credible evidence"' to counter the agency’s finding. ... So, for the most common interactions with the IRS, the burden of proof is indeed on the taxpayer."

No, that is not what it says "in other words."  The production of evidence is a matter of supporting an affirmative statement on the part of the taxpayer, not a matter of "counter[ing] the agency's finding."  The agency has not "found" anything that the taxpayer must disprove.  The taxpayer is making an affirmative claim about a fact of the world that (she hopes) will allow her to reduce her tax bill.

For example, suppose that there are two taxpayers whose employers have correctly reported that their respective salaries are both $80,000.  One taxpayer then says, "But I don't have to pay taxes on that income, because I gave it all away to a charity.  So the other guy should pay taxes, and I shouldn't."  At that point, the taxpayer would be legally obligated to produce credible evidence demonstrating that he has, indeed, given $80,000 to a charity.  Doing so would involve demonstrating that the charity is a legitimate charity, donations to which are tax deductible.  (There will also be legal analyses regarding annual limits on the total charitable deductions that Congress allows under the tax laws, and other issues.)

The point is that the taxpayer's requirement to produce evidence is not a "presumption of guilt," or "forcing the taxpayer to counter the agency's finding," or anything like that.  It is a requirement that, when a taxpayer claims that her tax computation should be based on certain facts, she must be able to produce evidence to support those factual claims.  The IRS is not presumed to be correct, because it is not making any claim other than, "Unless the facts are otherwise, this is your tax liability."  And the taxpayer has the ability to bring those additional facts to light.

PolitiFact dismisses as a mere technicality the production/proof distinction, and then claims that, for "the most common interactions with the IRS," Forbes's statement is correct.  But how do they get around the clear language from Section 7491, that "the Secretary [of the Treasury] shall have the burden of proof," even in (as PolitiFact describes it) "the most common interactions with the IRS"?  The article quotes a tax litigator as saying: "Nothing the congressman said on the burden of proof would strike me as outside of the norm or the general rule if I or any other tax litigator heard it in everyday conversation."

So, even though the law is absolutely clear that the burden of proof is on the government, PolitiFact is satisfied that this is not true, because one lawyer told them that tax litigators often hear people make the same mistake?

This, it seems to me, perfectly captures -- and then takes to a surreal level -- the modern media's obsession with a perverse notion of "balance": If you can find two people who disagree about something, simply report it as he-said-he-said.  Anything else is "biased reporting."  Critics of the media have joked that this will surely lead one day to absurdities like this: "Scientists say that the sun rises in the East.  Others say no.  The debate has not been resolved."  We now have something even worse: "The law clearly says X.  One lawyer tells us that some people say not-X.  So those people are right."

Here, as I stated above, one could acknowledge that it is easy to become confused about the meaning of the burden of proof, and conclude that this is not a "pants on fire" lie from a Congressman.  It is surely interesting that even some tax lawyers are sloppy in their language (or, more accurately, hear such sloppiness often enough that it is no longer remarkable).  But, we actually have a fact staring us in the face -- and PolitiFact is a fact-checking organization!  (Note, among other things, the organization's name.)

And just as clearly, we know for sure that taxpayers do not have to disprove any assertions by the government, but must only support their own assertions of fact.  That is not just a technical statement of the law, but a description of a system that respects taxpayer/citizens' presumption of innocence.

This, like the "train wreck" comment about the ACA, is not an error by the media that will alone change history.  Still, it is especially depressing to see that a fact-checking organization -- even when directly supplied with the most relevant fact -- does not even bother to report the facts as they stand, and instead invents its own notion of what happens "most commonly" on the ground.  A statement cannot be "true" -- mostly or otherwise -- if it is false.  And Forbes's statement is false.  Perhaps understandably false, but false.  And our media watchdogs (seemingly deliberately) missed another one.


Michael C. Dorf said...

Neil: Thanks for this very interesting post.

When I used to teach civil procedure, I found that there were many concepts that students found confusing, but the distinction between a burden of production and a burden of proof was not one of them. It's the sort of thing that can be illustrated clearly with a good example or two and then students get it. It's true, as you say, that in non-technical-legal discourse, people often use the term "burden of proof" to encompass burden of production, which is one reason why we usually talk about a "burden of persuasion" rather than a burden of proof. But again, the underlying distinction the IRC and you are drawing is not an inherently difficult one, just unfamiliar.

This raises the question of why the PolitiFact reporter didn't try to explain the distinction. One possibility is that, despite your best efforts, the reporter didn't understand the distinction. If so, that itself could be partly a product of shrinking journalism budgets. In the old days, reporters covering legal issues had some legal training; some were even lawyers. Now, with staff stretched thin, each reporter has to range more widely, and thus has less expertise to bring to bear.

A second possibility is that the reporter did understand the distinction but, as you say, thought it was just too technical for the public. That strikes me as potentially more worrisome. Assuming that the general public are too dim-witted to follow a careful, nuanced analysis is self-fulfilling.

Bob Moss said...

We're all familiar with this principle in the criminal context. If the police tell me, "Joe Blow died in his car from a blow to the head at 10:00 yesterday evening, and your DNA is in the car," it's up to me to volunteer that I was at a popular bar from 6:00 p.m. yesterday to 1:00 this morning, where I was seen by hundreds of people who can so testify, and I was in Joe's car at 8:00 yesterday morning.

Howard Wasserman said...

In Civ Pro and Evidence, I use "Burden of Persuasion" and "Burden of Production," further speaking of an "initial" burden of production to indicate when that burden has shifted. So I would say that the taxpayer bears the initial burden of production, then the IRS must produce evidence showing the taxpayer is liable and it bears the burden of persuasion of that liability.

But I really doubt the reporter would have the time, or inclination, to try to understand that.

David Ricardo said...

Two points

The situation with the IRS is very similar in law to that of the "rebuttable presumption". In certain instances, bankruptcy law for example, a situation is presumed to be x, and if the opposing side presents evidence that x is not the case then the burden of proof switches.

The point here is that we have self-appointed fact checkers who do not understand the issues or concepts, yet still opine. They are not stupid, they are just ignorant.

The second point is that fact checkers long ago abandoned their mission. A fact is a binary situation, it is either true or false. There is no such thing as a fact being 'mainly true' or 'mainly false'. Politifact admitted that they changed their criteria to bring their subjective judgments into play. This is rediculous.

If they want to add after a discussion that a fact was technically true but used out of context, fine, but let's just rate facts as True or False.

Need further convincing, next time any of the academics here like Mr. Dorf and Mr. Buchanan give a test, put some True/False questions on it, and provide a selection of True, Mostly True, Mostly False or False. See how fast the students rebel and how toally inoperative such a test would be.

Fact checkers are not fact checkers, they are opinion writers. They should be ignored (but unfortunately are not) until they return to the role of checking the facts. Today, as this post has amply illustrated, they are just another corruption of modern journalism.

tjchiang said...

I think you are being a bit unfair on the journalist here. "Burden of proof" is a phrase that is used to denote both the burden of persuasion and the burden of production. See James B. Thayer, The Burden of Proof, 4 Harv. L. Rev. 45 (1890). Forbe's statement is accurate if one understands "burden of proof" to refer to the burden of production and not the burden of persuasion. You might think nobody sensible would advocate placing the burden of production on the IRS; you might think that, since the statute itself uses "burden of proof" to denote the burden of persuasion, it is misleading to use "burden of proof" to denote the burden of production in a tax context; you might even think that the burden of production is not what Forbes actually has in mind (though I think the charitable read is that he is in fact thinking of the burden of production concept); you might think Polifact could have done a better job explaining the conceptual confusion; but it is not "clearly" wrong in the way that you make it seem. From a certain point of view, Forbes statement is in fact "mostly true."

David Ricardo said...


I am sorry but a fact cannot be "mostly true". A fact can either be true or not true. To call a fact 'mostly true' is to render an opinion and rendering an opinion is not fact checking.

About the best that can be said for Mr. Forbes is that because of ambiguity in the terminology 'burden of proof' it cannot be determined if his statement is true or false. And that's a fact.

When so called 'fact checkers' return to actually fact checking then we can all start to pay attentin to them again. Until then they should be ignored and disregard. Here is what I wrote over 18 months ago on the blog The Dismal Political Economist.

"At issue was a statement in the President’s State of the Union Address,

'In the last 22 months, businesses have created more than 3 million jobs. Last year, they created the most jobs since 2005."

On the surface this would seem like an easy fact to check. It is an objective statement involving data, either it is true or it is not true. PolitiFact rated it Half True, a ridiculous award, then changed their mind.

Here is their explanation.

But this morning we reviewed our work and decided to change it to Mostly True because we concluded Obama was not crediting his policies as directly as we originally thought.

The process that got us to Half True illustrates the challenge we face in rating the many claims of blame and credit in our political discourse.

Notice what the statement is saying, PolitiFact has the challenge in their minds of rating claims of blame and credit. No, that is not what they are supposed to do, in fact it was not what they originally were doing.

In our first couple of years, we treated many of those claims very literally. If someone said jobs had gone up since a governor was in office, and we found the numbers backed it up, the statement earned a True.

So what changed, well PolitiFact decided that their job was to make a value judgement on the claim, a subjective judgment involving nuance and intent.

'we began rating those types of claims as compound statements. We not only checked whether the numbers were accurate, we checked whether economists believed an office holder's policies were much of a factor in the increase or decrease.

We give a lot of Half True ratings because the numbers are often right, but experts repeatedly tell us that the policies of a single executive have a relatively small impact in a big and complex economy.'

Look, there is no body of “expert” economists. There is no consensus. You can find prominent economists to support any position. Nobel prise winners disagree. There is no way to interpret subjectively what the person is saying or to run it by a panel of experts and get their opinion on the subliminal meaning. That is not fact checking, that is garbage."

It was garbage then, it is garbage now. A fact can be mostly true only if a woman can be mostly pregnant.