Report: Phone polls aren’t dead yet

Inquiry

Quantitative

Surveys


Fig. 1. Graph showing the deterioration of response rates by the Pew Research Center, February 27, 2019, fair use.[7]

When the response rate, which should be ninety percent of a representative sample or more,[8] is instead in the single digits,[9] pollsters have a self-selecting sample that cannot represent non-respondents, particularly when, due to non-response, we have no idea, none whatsoever, how members of that self-selecting sample differ from all those non-respondents. This completely invalidates the methodology and I don’t care what rhetoric pollsters deploy to excuse themselves—the claim that polling works regardless[10] is belied by all the instances in which it doesn’t[11]—or what statistical magic they think they can employ to get around this problem—an absence of data remains an absence of data. This is a non-methodology that persists because we are desperate for an imaginary certainty of numbers,[12] and because people have built entire careers around this non-methodology.

Steven Shepard, “Report: Phone polls aren’t dead yet,” Politico, May 15, 2017, https://www.politico.com/story/2017/05/15/pollsters-phone-polls-238409

Courtney Kennedy and Hannah Hartig, “Response rates in telephone surveys have resumed their decline,” Pew Research Center, February 27, 2019, https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/

David A. Graham, “The Polling Crisis Is a Catastrophe for American Democracy,” Atlantic, November 4, 2020, https://www.theatlantic.com/ideas/archive/2020/11/polling-catastrophe/616986/

Mona Chalabi, “The pollsters were wrong – again. Here’s what we know so far,” Guardian, November 4, 2020, https://www.theguardian.com/us-news/commentisfree/2020/nov/04/the-pollsters-were-wrong-again-heres-what-we-know-so-far

Steven Shepard, “Dem pollsters acknowledge ‘major errors’ in 2020 polling,” Politico, April 13, 2021, https://www.politico.com/news/2021/04/13/dems-polling-failure-481044

Dan Balz, “2020 presidential polls suffered worst performance in decades, report says,” Washington Post, July 18, 2021, https://www.washingtonpost.com/politics/2020-poll-errors/2021/07/18/8d6a9838-e7df-11eb-ba5d-55d3b5ffcaf1_story.html

David Byler, “Polling is broken. No one knows how to fix it,” Washington Post, July 22, 2021, https://www.washingtonpost.com/opinions/2021/07/22/polling-is-broken-no-one-knows-how-fix-it/

Joshua Yaffa, “Why Do So Many Russians Say They Support the War in Ukraine?” New Yorker, March 29, 2022, https://www.newyorker.com/news/news-desk/why-do-so-many-russians-say-they-support-the-war-in-ukraine

Nate Cohn, “Yes, the Polling Warning Signs Are Flashing Again,” New York Times, September 12, 2022, https://www.nytimes.com/2022/09/12/upshot/polling-midterms-warning.html

Perry Bacon, Jr., “Polls are useful. They just can’t predict elections in swing states,” Washington Post, September 27, 2022, https://www.washingtonpost.com/opinions/2022/09/27/polls-accuracy-midterm-elections-perry-bacon/

  1. [7]Courtney Kennedy and Hannah Hartig, “Response rates in telephone surveys have resumed their decline,” Pew Research Center, February 27, 2019, https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/
  2. [8]This according to Valerie Sue, the professor in my first research methods class, at California State University, Hayward (now East Bay), Fall 2003.
  3. [9]Steven Shepard, “Report: Phone polls aren’t dead yet,” Politico, May 15, 2017, https://www.politico.com/story/2017/05/15/pollsters-phone-polls-238409; Courtney Kennedy and Hannah Hartig, “Response rates in telephone surveys have resumed their decline,” Pew Research Center, February 27, 2019, https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/
  4. [10]Steven Shepard, “Report: Phone polls aren’t dead yet,” Politico, May 15, 2017, https://www.politico.com/story/2017/05/15/pollsters-phone-polls-238409; Courtney Kennedy and Hannah Hartig, “Response rates in telephone surveys have resumed their decline,” Pew Research Center, February 27, 2019, https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/
  5. [11]Dan Balz, “2020 presidential polls suffered worst performance in decades, report says,” Washington Post, July 18, 2021, https://www.washingtonpost.com/politics/2020-poll-errors/2021/07/18/8d6a9838-e7df-11eb-ba5d-55d3b5ffcaf1_story.html; David Byler, “Polling is broken. No one knows how to fix it,” Washington Post, July 22, 2021, https://www.washingtonpost.com/opinions/2021/07/22/polling-is-broken-no-one-knows-how-fix-it/; Mona Chalabi, “The pollsters were wrong – again. Here’s what we know so far,” Guardian, November 4, 2020, https://www.theguardian.com/us-news/commentisfree/2020/nov/04/the-pollsters-were-wrong-again-heres-what-we-know-so-far; David A. Graham, “The Polling Crisis Is a Catastrophe for American Democracy,” Atlantic, November 4, 2020, https://www.theatlantic.com/ideas/archive/2020/11/polling-catastrophe/616986/; Steven Shepard, “Dem pollsters acknowledge ‘major errors’ in 2020 polling,” Politico, April 13, 2021, https://www.politico.com/news/2021/04/13/dems-polling-failure-481044
  6. [12]Jacques Ellul, The Technological Society, John Wilkinson, trans. (New York: Vintage, 1964).; Neil Postman, Technopoly: The Surrender of Culture to Technology (New York: Vintage, 1993).

8 thoughts on “Report: Phone polls aren’t dead yet

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.