In “Caliphate and the Social Epistemology of Podcasts”, Josh Habgood-Coote (2021) writes that the New York Times’ (referred to as the Times throughout) podcast Caliphate’s central flaw is that it “failed to accurately represent the credibility of someone whose speech [was being presented]” (27). Habgood-Coote argues that this is not a failure of reporting, and to grasp what went wrong with Caliphate, we need to understand the epistemology of podcasting … [please read below the rest of the article].

Image credit: Kristin Wolff via Flickr / Creative Commons

Hagbood-Coote’s introduction to the epistemology of podcasting is insightful and much-needed. I was persuaded that we should pay more attention to the logic of this new genre as we unpack the scope of what went wrong with Caliphate. I was less convinced by the suggestion that in order to make epistemic assessments about Caliphate, we ought to focus primarily on the conventions of podcasting. I argue that given the specific professional norms to which the Times subscribes and which listeners expect the Times to respect, Caliphate’s failures are most appropriately characterised as reporting failures.

A Snapshot of Reporting Failures

In “Caliphate and the Problem of Testimony” (2020), I argued that we need to understand the failures of Caliphate in light of past reporting failures concerning weapons of mass destruction (WMD). (For a thorough examination of what those reporting failures were, see When the Press Fails by W. Lance Bennett, Regina G. Lawrence, and Steve Livingston). While the WMD case and Caliphate case are different, they have similarities that are epistemically relevant.

In both cases, journalists built stories on the testimony of certain sources and conveyed those stories to the public, leading to the public believing the stories to be true and accurate. And in both cases, the testimony the journalists received was false and the stories that were conveyed to the public were misleading, leading to the uptake of false beliefs. In ‘Caliphate and the Problem of Testimony’, I tried to articulate the messiness of this kind of testimonial exchange: journalists trust certain sources and we trust journalists. Usually, mistakes happen, corrections are made, and we all move on. But in the WMD case and the Caliphate case, journalists failed to adequately do their jobs and meet professional standards by not verifying sources and corroborating information. In doing so, they put stories into the public space that should not have been aired. The public’s trust in journalists is grounded in our belief that journalists will respect professional standards and norms.

In journalism, there are professional standards that govern the conduct of journalists and media organisations at every stage of the process. There is some variation in the articulation and scope of those standards, and certain media organisations may choose to emphasise particular norms over others (see here, here, and here for examples). Nevertheless, there are a core set of standards that are the salient norms for the profession; two of the most prominent norms concern truth and accuracy. Journalists are expected to seek truth and strive to be as accurate as possible.

Practically speaking, journalists do this by trying to find reliable sources, corroborating information they hear from sources with other eyewitness accounts and evidence, avoiding distortion of the facts or contents (including visual and audio information), and promptly correcting errors of fact and errors of omission. When a journalists media organisation talks of a “reporting failure”, they have in mind a mistake that occurs when the reporter(s) or media organisation do not meet one of the set standards for reporting, e.g. the failure to corroborate.

Crucially, these standards apply across media and platforms, from news articles to editorials to radio to podcasts. As the Society of Professional Journalists states, “neither speed nor format excuses inaccuracy”. Journalists whose stories appear primarily online do not follow different norms from journalists whose stories appear in print.

It’s worth taking a moment to remember the nature and scope of the reporting failures in the creation of Caliphate, especially the failure to corroborate. Thanks to the excellent work of a number of media critics, we have a good sense of what, specifically, went wrong with corroboration. Here’s a rundown:

In March of 2018, the Rukmini Callimachi, Andy Mills, and the producers of Caliphate sent draft scripts of the series to the Times’ international editor. That editor, Michael Slackman, warned the podcast team that the whole podcast depended on the credibility and testimony of a single character, Mr. Shehroze Chaudhry, whose story was uncorroborated. This is when the urgent dash to try to corroborate the story began. A few days after those meetings in March, the Times hired Derek Henry Flood, based in Syria at the time, to try and corroborate the story. He couldn’t. Other Times reporters across the Middle East were called on to try and confirm Mr. Chaudhry’s ties to ISIL/Daesh. They could not.

In Washington, more reporters were asked to try and corroborate Mr. Chaudhry’s story. Finally, someone found gold (or what then seemed like gold): Eric Schmitt, who appears on an episode of Caliphate, is told by two different officials in the U.S. government that Mr. Chaudhry was a member of ISIL/Daesh. But as Ben Smith reports, Mr. Schmitt and his colleagues “never determined why those government officials viewed him as part of ISIS, or if indeed they had any evidence of his [ISIL/Daesh] connections other than the professed terrorist’s own social media pronouncements.”

With this fake gold in hand, the podcast gets the green light from the editors and goes to air. Some of the messiness that went into reporting the story is put into an episode (the now-infamous “chapter 6”) explaining the ‘behind-the-scenes’: How reliable is this source? Should we trust him? How do we make sense of the inconsistencies in the story? But as Erik Wemple writes, chapter 6 is flawed for many reasons (including the failure to credit other sources and journalists). Most notably, however, Callimachi fails to convey the scope of the inconsistencies in Mr. Chaudhry’s account, leading listeners to believe in his story. In episode 6, Callimachi says:

Look, it makes sense to me that somebody that has been in the caliphate, that if he’s trying to exaggerate a little, you know, that if he’s trying to — “Oh, yeah, yeah, yeah, I was there when Baghdadi, you know, announced it, oh, my God!” Whatever. That makes sense to me. But not going there at all and making up all of those details about the Albu Nimr tribesmen, about this execution, about what it’s like to hold the gun, about what it’s like to actually whip somebody, about the fact that the blood splashes back up on you, that would — I mean, that’s a level of invention? It’s too much! I mean, it’s — he’s providing details that nobody knows, you know?

We know what happens next. Caliphate’s central story is revealed to be haux, Callimachi’s reporting history is put under the microscope (and while there are good reasons for doing that, she takes a bigger reputational hit than any other culpable actors involved). Fast forward to now: the NYT has returned its Peabody, one of the most prestigious awards in journalism, and handled the fallout over Caliphate very clumsily, leading to more apologies.

Caliphate is a standout example of what-not-to-do for journalists and media organisations: the Times did not meet professional standards, broke journalistic norms, and unintentionally caused the uptake of false beliefs. Crucially, the Times also characterises their failings in this way. Speaking on The Daily in the wake of the Caliphate fallout, Times executive editor Dean Baquet, going so far as to recognise the shortcomings as an “institutional failures”, expressed:

When the New York Times does deep, big, ambitious journalism in any format, we put it to a tremendous amount of scrutiny at the upper levels of the newsroom … We did not do that in this case. And I think that I or somebody else should have provided that same kind of scrutiny, because it was a big, ambitious piece of journalism. And I did not provide that kind of scrutiny, nor did my top deputies with deep experience in examining investigative reporting.

The Salient Norms

There is a mismatch between what Hagbood-Coote takes the epistemic norms of podcasting to be, and what the Times takes the epistemic norms for its own content and products to be. For Hagbood-Coote, we should interpret Caliphate in light of the conventions of the “inquiry-driven narrative non-fiction podcast” (29). But as Hagbood-Coote himself recognises, this is not the way that the Times understood the shortcomings of the podcast: in the Times’  post-mortem of what went wrong, the failure is characterised as failure of reporting (34). Hagbood-Coote notes that this kind of response is “puzzling” (34).

So what’s going on? Has the Times just made a mistake in articulating the relevant epistemic standards? Has the Times misunderstood the norms that pertain to their own products and content?

Before we can answer these questions, a word on norms. One of the best accounts on how epistemic norms function is provided by Peter Graham in ‘Epistemic Normativity and Social Norms’. In the piece, Graham offers a conceptualisation of ‘social norms’ (which he takes epistemic norms to be) that are relevant for our purposes. Graham explains that a social norm is both descriptive and prescriptive; what connects the two is:

…the role the second part—the patterns of approval and disapproval, our evaluations of ourselves and each other—plays in producing or sustaining the first part—the actual regularities in behavior. A social norm is genuinely a norm when genuinely normative, when the general belief that it ought to be done motivates compliance, in one way or another” (251).

To break it down, there are three parts to a social norm. First, the rules must be followed with some regularity in a community; that’s what established “what’s normal, what’s usually done”. Second, the rules are prescriptive. When someone confirms to the rule, members of her community will evaluate her positively; when someone breaks the rule, she is evaluated negatively. Third, these rules are followed with regularity by members of the community because they are prescriptive. Members of the community might learn to follow the norms because they face social criticism and sanction if they don’t; they may also learn to follow the norm through “internalisation”, meaning that they come to endorse the rule and desire to follow it.

I suggest this account of norms can help us understand why the Times and many listeners take the central failing of Caliphate to be a reporting failure: the epistemic norms that are set for Times podcasting content are the same norms that govern all other reporting. The Times’ acknowledgement of its failures in reporting makes perfect sense given the kind of institution the Times is, the journalistic standards that journalists, editors, and producers commit to, and the expectations for rigour in foreign-policy reporting.

In Hagbood-Coote’s diagnosis of the Caliphate failure, it is primarily the presentation of Mr. Chaudhry’s credibility that we should be concerned about; the Times “unjustifiably and irresponsibly inflated his credibility” (32). I agree that there is a significant problem with the way in which Mr. Chaudry’s credibility is presented in the podcast but I do not think this point fully captures the heart of the problem. What went wrong with Caliphate is a breakdown of the relevant epistemic norms; the problem is deeper than the presentation of credibility. Media organisations like the Times and journalists who subscribe to certain professional standards do not have the obligation to entertain us; they do have an obligation to report the facts truthfully and accurately. When the facts cannot be confirmed, there are strong reasons for the story not to be aired.

To be sure, there are significant differences in style, tone, and structure between an article and a podcast. The distinction between “old media” vs. “new media” is important, but it is somewhat overstated. As the Times and Callimachi’s apologies indicate, they hold Caliphate to the same epistemic standards as other content. So long as listeners tune in to the Times’ podcasts because of the Times’ reputation for journalistic excellence, the Times will seek to apply those same journalistic standards to all its content.

These norms may change in the future. The expectations of listeners may change, and the norms for reporting through podcasting may look different to the norms for print media. The pushback against sensationalist reporting might die out, and consumers of the news may not critique media institutions for privileging entertainment value over truth. Comedians reporting the news on their shows (for example, the Daily Show and the Colbert Report) do not have to follow journalistic norms; perhaps in the future, journalists who are podcasting might also be off the hook.

At this moment, however, there is no significant difference in the kinds of standards journalists need to meet across media, and a journalist’s mistake to corroborate evidence is just as serious when it’s made on a podcast as when it’s made in print. Hagbood-Coote’s piece illustrates the complexity of adapting norms and standards to new forms of media. While he is right to bring the epistemology of testimony into the conversation about Caliphate, the general epistemic standards that pertain to the genre of podcasting are not the most relevant ones for Caliphate. The appropriate standards, as stodgy as they seem, are journalistic standards.

Acknowledgments

Thanks to Joshua Habgood-Coote.

References

Bennett, W. Lance, Regina G. Lawrence, Steven Livingston, 2007. When The Press Fails: Political Power And The News Media From Iraq To Katrina. University of Chicago Press.

Callimachi, Rukmini and Andy Mills. Caliphate. 12-part audio documentary. New York Times. Produced by Andy Mills, Larissa Anderson, Wendy Dorr, Asthaa Chaturvedi. Podcast. https://www.nytimes.com/column/caliphate.

Cibralic, Beba. 2020. “Caliphate and the Problem of Testimony.” Social Epistemology Review and Reply Collective 9 (12): 33-36

Graham, Peter J. 2015. “Epistemic Normativity and Social Norms.” In Epistemic Evaluation: Purposeful Epistemology edited by David Henderson and John Greco, 247-273. Oxford University Press.

Habgood-Coote, Joshua. 2021. “Caliphate and the Social Epistemology of Podcasts.” Social Epistemology Review and Reply Collective 10 (2): 27-35.

Tracy, Marc, Katie Robertson, Tiffany Hsu. 2020. “New York Times Says ‘Caliphate’ Podcast Fell Short of Standards.” New York Times. 18 December. https://www.nytimes.com/2020/12/18/business/media/new-york-times-caliphate-podcast.html.

Wemple, Erik. 2020. “The New York Times has its hands full with review of ‘Caliphate’.” The Washington Post 9 October. https://www.washingtonpost.com/opinions/2020/10/09/new-york-times-has-its-hands-full-with-review-caliphate/.

Categories: Critical Replies

Tags: Beba Cibralic, Caliphate podcast, deception, fiduciary basis of knowledge, Institutional Responsibility, Joshua Habgood-Coote, journalism, media, New York Times, Rukmini Callimachi, Testimony, trust