Admittedly Confusing Click Bait: Is AI Better than the CIA?
[Update: A reader who has reason to know wrote to tell me that the CIA was/is not as reliable re gathering facts as I indicate in the column below. Or, to paraphrase myself: “Actually, the CIA ain’t no CIA, either!” I’ll accept that as a friendly amendment.]
I must begin with a confession: It was only after I wrote the headline for this column that I noticed that the "I" in both acronyms stand for "intelligence" (an oversight that caused me to question mine). Be that as it may, there is in fact a potentially useful comparison between Artificial Intelligence (AI, which should in fact be referred to as LLM's, for large-language models, but I am not going to fight that battle here) and the Central Intelligence Agency. Trust me that this column eventually includes some very amusing content, but it does begin with seriously bad news.
Back in the 1970's and '80's, the CIA was rightly under political pressure for its many foreign policy disasters. It had, for example, been responsible for decades of coups d'etat in Iran, Chile, and elsewhere. Its nefarious activities went far beyond those relatively famous examples, however, to the point where there is a very long entry on Wikipedia titled "List of CIA controversies." It is a depressing read.
Even so, one of the most useful data sources available throughout that time was called the CIA World Factbook. It provided, according to another Wikipedia page, "almanac-style information about the countries of the world." Even though I am the last person to defend the CIA, I recall that the Factbook was very useful when I wrote a short law review article comparing US health outcomes with those in other countries. I also discovered that it was also a very good source for apples-to-apples comparisons of public debt and other economic data in countries around the world.
Why bring that up here? I will answer that question momentarily, but first it is important to share news that most definitely was buried under the avalanche of other atrocious news in 2026. The Factbook's Wikipedia page now includes this information in the introductory section: "On February 4, 2026, following the restriction and closure since 2024 of other government-funded information websites, the CIA announced, with no warning and no explanation, that The World Factbook was discontinued." The CIA's website on that date published a short page with this title: "Spotlighting The World Factbook as We Bid a Fond Farewell." Again, no explanation is given, only saying that the publication "has sunset."
So as a threshold matter, the axing of the CIA World Factbook is very bad news indeed, especially as it is part of the Trump Administration's broader attack on reality -- as in, facts. It is the kind of thing that makes the Project 2025 people who are working with Trump so dangerous, because it is yet another example of how many ways in which the federal government is being destroyed and warped to serve a larger, anti-democratic agenda. Facts be damned, so they are making those damned facts disappear. And I would not have even noticed this particular move if I had not thought about the Factbook in the context of this column.
Which brings me back to that question of why I even thought to mention the Factbook today at all. During the years of the CIA's most extreme notoriety, I heard a commentator describe the CIA's fact-gathering operations in a surprising way. That agency, he said, was absolutely unreliable whenever it came to analysis of facts, advocating policy choices, or anything that involved judgment and good-faith reasoning. Even so, it was absolutely the gold standard when it came to collecting data. The best way to think about it, this long-forgotten commentator put it, was to say that we could trust the CIA whenever it made a factual statement, but we should never trust it when it offered an intelligence assessment or -- heaven forbid -- an action plan.
So that is/was the CIA, or at least a usefully simplified way to think about its two-pronged work. For those readers who are lawyers, think of this as the fact-law distinction, where the CIA absolutely nails the first part of every legal brief but then goes completely off the rails in the second part with a Samuel Alito-like rant that no one should take seriously.
How is that like, or not at all like, AI? This is where things become amusing, because AI's boosters claim that this new technology can do both of those things amazingly (literally super-humanly) well, but it cannot even get the facts right. And that is a monumental problem, because the one thing that even a skeptic might have thought AI could do well is gather facts. Indeed, the technology is built on an ability to "scrape" facts from the internet quickly and supposedly accurately, which means that even though it is so often used to create Uncanny Valley-like slop or offers bad arguments, at least the facts should be correct.
As an aside in a column this past October, I noted one example of an obvious failure on AI's part when it comes to basic facts. I had entered an internet search regarding college football, where a coach had been fired because of a disastrous record of 4 wins and 21 losses against Top 10-ranked opponents, but I could not remember those four victories. I did not want an AI answer, but Google had installed an impossible-to-delete AI feature at the top of its searches, and the answer that it gave included one that was simply wrong. The claim was that that coach's team had beaten Michigan by a score of 21-17 in 2023, which just happens to be the year in which Michigan went undefeated and won the national championship. Oops.
I should add that there might have been more such errors, but I had no reason to check them. And that is the core of the problem. As one podcaster put it recently: "If you presented a calculator to me and said, 'This'll do any calculation you want, but it'll get a bunch wrong, and you don't know how often, and you don't know which ones will be wrong, so you have to manually check them all anyway,' isn't that useless?"
I happened to know that that Michigan score had to be wrong, but most people would never notice. So it is important to continue to remind ourselves that a randomly-wrong source is truly useless, even -- or especially -- when it is unpredictably wrong about facts.
AI ain't no CIA.
What is even worse is when an AI-generated answer is completely confusing. I recently needed to work through some time zone adjustments when a colleague in Europe sent me some proposed meeting times in CET. I knew that CET is Central European Time, and it is one time zone to the east of GMT. But this is summer time, and even the home of GMT (the UK) uses daylight savings time, or BST -- British Standard Time -- which is one hour later than GMT. My colleague specified CET, not CEST or CEDT (which seemed like the two candidate for summer: Central European Summer Time or Central European Daylight Time).
This should, I thought, be as simple as saying that, for example, Chicago in the summertime is on what amounts to Eastern Standard Time. I thus entered this question into the search engine: "Is CET the same as BST?" The AI answer was: "No, CET (Central European Time) and BST (British Summer Time) are not the same." But it went on: "BST is 1 hour ahead of GMT (UTC +1)," which is true, and then added: "CET is UTC +1." So wait, does that not mean that CET is the same as BST? But the answer continued: "Because of this, BST and CET are usually the exact same time (e.g., 12:00 PM in London is 12:00 in Paris. However, the UK switches to GMT in the winter, and Central Europe switches to CEST (UTC +2) in the summer. The differences depend on the time of year."
Got it? No, you are not insane. This is self-contradictory gibberish. And as an even better (and genuinely hilarious) example of nonsense, a friend recently sent me the results of a search based on this question: "How many days of the week have the word 'fish' inside their spelling?" Part of the AI-generated answer:
Exactly three days of the week have the word "fish" hidden inside them when spelled out:
• Tuesday
• Wednesday
• Thursday
The other four days (Monday, Friday, Saturday, and Sunday) do not contain the word "fish" at all.
Another part:
Two days of the week literally contain the word "fish": Tuesday and Saturday (which both contain the letters f-i-s-h).
And another:
Only one day of the week has a "fish" in it: Sunday. If you look at the spelling of the days, only Sunday contains the word "day" (which is a type of fish)!
Again, AI ain't no CIA. Luckily, financial markets are not in the midst of an AI-fueled bubble. This will definitely end well.
- Neil H. Buchanan