Michael Shermer is something of a hero of mine. He often does excellent and important work defending science and reason, criticizing pseudoscience and claims about the paranormal, criticizing the view that religious faith is a source of knowledge, supporting freedom of speech, and defending a broadly liberal order in which government acts to protect people’s rights.
It is therefore especially disappointing to me that Shermer misstates the results of a study—itself deeply flawed—about defensive gun use to promote his cultural agenda regarding guns.
At issue is a claim Shermer makes in an October 6 op-ed in the Los Angeles Times:
[S]hould you, dear reader, choose to own a gun?
Consider this finding from a 1998 study published in the Journal of Trauma and Acute Care Surgery: “Every time a gun in the home was used in a self-defense or legally justifiable shooting, there were four unintentional shootings, seven criminal assaults or homicides, and 11 attempted or completed suicides.”
In other words, the fantasy many of us have of facing down an intruder with a firearm is belied by the fact that a gun is 22 times more likely to be used in a criminal assault, an accidental death or injury, a suicide attempt or a homicide than it is for self-defense.1
But Shermer’s “22 times” claim is complete nonsense. Perhaps the most important problem with the study in question is that it ignores the large majority of defensive gun uses. The study counts only defensive gun uses that result in a criminal being shot and killed or treated in a nearby clinic, but most defensive gun uses do not end with that result (details below). So, by Shermer’s “logic,” if a criminal breaks into my home, and I pull a gun and say “leave or I’ll shoot,” and he leaves without me firing a shot, that does not count as a defensive gun use—which is ridiculous. Another important problem is that Shermer draws unwarranted conclusions about the typical gun owner (again, details below).
Shermer’s misstatement of the findings of the 1998 study—and his refusal to recognize the study’s deep methodological flaws—go beyond casual carelessness on his part. How so?
For one thing, Shermer has made a career out of exposing unscientific claims, so, more than most people, he should know better. If anyone deserves the title of “Mr. Skeptic,” it is Michael Shermer, founding publisher of Skeptic magazine. If someone (mis)cited a study of similar poor quality to support a claim with which Shermer disagreed, he would very quickly see through the ruse. But in this case he is self-blinded.
For another thing, Shermer has willfully ignored evidence that disproves his claims. He made an almost identical claim in 2013, again in the Los Angeles Times:
According to a 1998 study published in the Journal of Trauma and Acute Care Surgery, for “every time a gun in the home was used in a self-defense or legally justifiable shooting, there were four unintentional shootings, seven criminal assaults or homicides, and 11 attempted or completed suicides.” In other words, a gun is 22 times more likely to be used in a criminal assault, an accidental death or injury, a suicide attempt or a homicide than it is for self-defense.2
To this, I replied via Twitter, “Incredible that @michaelshermer [Shermer’s Twitter handle] is peddling the pseudo-scientific, often-disproved ‘22x’ factoid re defensive gun use.” Shermer replied, “Do you have link to debunking of the 22x defensive gun factoid? It looks solid to me but am willing to look at other studies.” I replied, “Gary Kleck responds to other iterations of Kellermann’s work in ‘Armed’ and ‘Targeting Guns.’”3 (Physician Arthur Kellermann is the lead author of the 1998 study.)
The reason I cited Kleck is that he is an academic criminologist who has done extensive original research on guns in the United States. In the 1997 book Targeting Guns, Kleck replies to Kellermann’s previous and similar 1986 study, which makes many of the same types of errors that the 1998 study makes. In 1986, the ratio Kellermann claimed was not twenty-two-to-one, but forty-three-to-one. Both ratios are nonsense.
Kleck has this to say about Kellermann’s 1986 study:
The implied cost-benefit ratio is so meaningless that it can fairly be dubbed the “nonsense ratio.” [Barry] Bruce-Briggs described [a previously published version of the claim] as “ingeniously specious” . . . and quickly dismissed it, and most serious gun scholars have ignored [Kellermann’s study and comparable] studies. . . . Nevertheless, the nonsense ratio is a favorite of procontrol propagandists . . . and medical writers.4
Unfortunately, that remains the case even today.
Kleck estimates that a criminal is “wounded, even nonfatally,” only in around three percent of defensive gun uses.5 If that estimate is accurate, then Shermer and the 1998 study he credulously cites ignore around 97 percent of all defensive gun uses. Of course, crime reporting is notoriously plagued by difficulties. Survey results on gun ownership and defensive gun use are plagued by even greater difficulties. But even if the three-percent estimate is substantially off, it remains the case that the majority—probably the overwhelming majority—of defensive gun uses do not involve shooting the criminal.
At any rate, obviously Shermer did not take seriously his stated commitment to look into evidence that ran against his preestablished viewpoint.
Incidentally, on October 5, one day before his recent Los Angeles Times op-ed popped, Shermer repeated his “22 times” claim on Twitter. I replied, “It’s hard to believe you’re still peddling this easily discredited pseudoscience about defensive gun use.”6 (Shermer’s claim is “easily discredited,” as a Google search of “22 times Kellermann” reveals. Of course, Twitter is not a good format for making complex claims.) Shermer did not reply to this.
Before I move on to a more-detailed critique of the 1998 study and of Shermer’s statements about it, I want to point out that I find Shermer’s piece more broadly, and in general his claims about guns, to be riddled with specious arguments. However, here I focus on the “22 times” statistic.
As I’m sure Shermer recognizes in other contexts, often one party can spin nonsense much faster than another party can unravel it—and he has spun a lot of nonsense. Shermer spent almost no effort making his bogus claims about the “22 times” stat, either in 2013 or again this year, yet I will have spent many hours rebutting just that one claim. I hope to discuss other aspects of guns and gun-related crime in greater detail in the future, but, if I do so, the project will consume an enormous amount of my time. (Of course, some of Shermer’s claims are correct or at least on much stronger footing.)
Now, for those interested (and hopefully this includes Shermer), I’ll offer a more-detailed critique of the 1998 study and of Shermer’s statements about it. My remarks here indirectly shed light on certain other claims often made in opposition to civilian gun ownership or in favor of stricter gun laws.
Other Problems with Kellermann’s 1998 Study and Shermer’s Take on It
Above I discuss a major problem with the 1998 study: It claims to discuss defensive gun use but in fact ignores most defensive gun uses. I’ll call this the measurement problem—the study’s estimate of the number of defensive gun uses is radically off.
Here I want to discuss other significant problems with the study and with Shermer’s interpretation of its results.
First I need to summarize the study and its main findings, published in 1998 in the Journal of Trauma: Injury, Infection, and Critical Care.7
Notably, the first line of the study states that its objective is to “[d]etermine the relative frequency with which guns in the home are used to injure or kill in self-defense, compared with the number of times these weapons are involved in an unintentional injury, suicide attempt, or criminal assault or homicide.” The study doesn’t actually claim to measure defensive gun use; it just pretends that defensive gun use is the same as defensive gun use resulting in the injury or death of the criminal. In reality the category of defensive gun use is much larger than the subset in which the criminal is shot.
The study’s abstract summarizes the main parameters and findings of the study:
We reviewed the police, medical examiner, emergency medical service, emergency department, and hospital records of all fatal and nonfatal shootings in three U.S. cities: Memphis, Tennessee; Seattle, Washington; and Galveston, Texas. . . .
During the study interval (12 months in Memphis, 18 months in Seattle, and Galveston) 626 shootings occurred in or around a residence. This total included 54 unintentional shootings, 118 attempted or completed suicides, and 438 assaults/homicides. Thirteen shootings were legally justifiable or an act of self-defense, including three that involved law enforcement officers acting in the line of duty. For every time a gun in the home was used in a self-defense or legally justifiable shooting, there were four unintentional shootings, seven criminal assaults or homicides, and 11 attempted or completed suicides.
To add a detail regarding the measurement problem: The study probably understates the number of criminals who were shot at by home owners firing a gun in self-defense, as it relies on medical records in the relevant areas. But what if a criminal was grazed and did not seek medical care, or what if he traveled to a different city to seek medical care? (I use “he” recognizing that most, but not all, violent criminals are male.) Or what if the homeowner shot but missed, intentionally or not, and the criminal ran away uninjured? The study ignores these cases, which contributes to its measurement problem regarding defensive gun uses. (Again, a larger concern is cases in which a gun is brandished but not fired.)
Obviously the study totally ignores the deterrent effect of gun ownership—i.e., a criminal is less likely to break into any home in an area if he fears any given home owner may be armed.
Another major problem with the study, which I’ll call the causation problem, is that it attributes to gun ownership problems that are actually caused by other things.
Recall that Shermer begins his discussion of the 1998 study with the question, “Should you, dear reader, choose to own a gun?” Shermer therefore implies that the study in question applies to the typical reader of the Los Angeles Times or of Shermer’s articles, or, more broadly, to the typical person who might seek to buy a gun for self-defense. The study does no such thing. The study itself never asserts that its findings apply to the typical person—it just insinuates that they do. But obviously the vast majority of homicides (and many unintentional shootings) are concentrated in a small subset of the population that is not typical.
So, by Shermer’s “logic,” the fact that a gangster or a meth-head (or the like) who owns a gun is more likely to be involved in a gun-related homicide or unintentional shooting counts as a reason for the typical person, who is not a gangster or a meth-head (or the like), to avoid owning a gun for self-defense. Here too Shermer’s claims are ridiculous.
Notably, the study does not clarify which legally indefensible assaults and homicides that occurred with a gun in the home involved someone living at that home as the victim. Presumably, in some unknown number of cases, the home owner (or renter) was the perpetrator, and the victim was an outside party. So, in other words, if Gangster A visits Gangster B at Ganster B’s house, and Gangster B shoots Gangster A, that, for Shermer, is supposed to count as a reason for the typical, non-gangster person not to own a gun for self-defense.
Suicide involves a type of causation problem as well as a substitution problem. The substitution issue is that some (probably most) people who buy a gun to commit suicide easily could substitute another method of suicide (drugs, hanging, jumping off a cliff) if a gun were not available. Thus, to blame the gun for the suicide is, at least in some cases, dubious.
The causal problem at issue is that not everyone is equally suicidal. Obviously, most suicides occur among the small subset of the population that is suicidal. But the typical person is not suicidal, so Shermer is wrong to claim that gun-related suicides counts as a reason for a non-suicidal person to refrain from buying a gun for self-defense. Of course, if a person is suicidal or lives with someone else who is suicidal, that’s probably a good reason to not have a gun around or to keep a gun carefully locked up.
The extrapolation problem (or the problem of nonrepresentative samples) pertains to various aspects of the study. As we’ve noticed, it is unwarranted to extrapolate the study’s findings to the typical person, as Shermer does. It is also unwarranted to extrapolate results from the study’s small study zones—cities, where gang-related crime is higher—to the country as a whole, as Shermer also does.
To wrap up, “Mr. Skeptic” does not live up to his reputation in discussing the “22 times” factoid. Unfortunately, his muddled thinking typifies many of the claims made in today’s debates about guns, gun-related crime, and defensive gun use. I doubt that Shermer and I will ever fully agree on the appropriate policies regarding guns, but hopefully in the future we can both work toward a fair and context-rich assessment of the available evidence regarding guns and their use.
Related:
- Unarmed Civilian Response to Active Shooters: A Crucial Part of Reducing Casualties of Mass Attacks
- History of Colorado’s Gun Magazine Ban
Endnotes
1. Michael Shermer, “Guns in the U.S.: We’re Better at Killing Americans than our Enemies Are,” Los Angeles Times, October 6, 2015, http://www.latimes.com/opinion/op-ed/la-oe-1008-shermer-gun-data-20151006-story.html.
2. Michael Shermer, “A Rational Response to Sandy Hook,” Los Angeles Times, January 15, 2013, http://articles.latimes.com/2013/jan/15/opinion/la-oe-shermer-gun-control-20130115.
3. See the relevant Tweets by Michael Shermer and me of January 15, 2013, at https://twitter.com/ariarmstrong/status/291092694556614656.
4. Gary Kleck, Targeting Guns: Firearms and Their Control (New York: Aldine de Gruyter, 1997), p. 178.
5. Gary Kleck, Targeting Guns: Firearms and Their Control (New York: Aldine de Gruyter, 1997), p. 162.
6. See the relevant Tweets by Michael Shermer and me of October 5, 2015, at https://twitter.com/michaelshermer/status/651053455616819200.
7. Arthur Kellermann, Grant Somes, et al., “Injuries and Deaths Due to Firearms in the Home,” Journal of Trauma: Injury, Infection, and Critical Care, August 1998, vol. 45, no. 2, August 1998, pp. 263–267. Some of my criticisms of this study are inspired by those of Gary Kleck and others whose work I’ve read over the years.