Statistical significance in court
JESSICA BREGANT, JD, AND JENNIFER K. ROBBENNOLT, JD, PHD • UNIVERSITY OF ILLINOIS
In March 2011, the U.S. Supreme Court took up the question of how much emphasis courts should place on statistical
significance in a legal context. The case, Matrixx Initiatives, Inc.
v. Siracusano, presented the question in the somewhat unlikely
context of a securities fraud class action lawsuit. The plaintiffs
alleged that Matrixx, a pharmaceutical company, had failed to
disclose reports of a possible link between Zicam Cold Remedy,
its leading product, and anosmia, or loss of smell. Matrixx
argued that because the number of reported instances of
anosmia “was not statistically significant,” it was not obligated
to disclose these reports to investors.
According to the plaintiffs’ complaint, Matrixx first became
aware of potential problems with Zicam in 1999, when a
physician reported to Matrixx that a “cluster” of patients
had developed anosmia after using Zicam. In 2002, another
researcher discussed similar complaints with Matrixx’s vice
president for research and development. In 2003, Bruce Jafek,
MD, a researcher at the University of Colorado, presented
a poster at the American Rhinologic Society describing 11
patients who had lost their senses of smell after using Zicam.
Shortly after Jafek’s presentation, two lawsuits were filed
against Matrixx by Zicam users who claimed that they had
suffered anosmia after using the product. Matrixx did not
disclose these suits to investors. That same month, however,
Matrixx told investors that Zicam was “poised for growth in the
upcoming cough and cold season.” Over the next several months,
an additional seven plaintiffs filed suit. Notwithstanding these
lawsuits, Matrixx increased its public profit estimates for Zicam.
On Jan. 30, 2004, Dow Jones Newswire reported that the Food
and Drug Administration was looking into claims that Zicam
caused some users to develop anosmia. Following this report,
Matrixx’s stock price fell approximately 11 percent. Matrixx
issued a press release calling the claims against Zicam “completely
unfounded and misleading” — and its share price rebounded.
However, on Feb. 6, 2004, “Good Morning America” presented a
story featuring Jafek’s findings, and the stock sank again.
The plaintiffs — Matrixx investors — claimed that Matrixx
had failed to disclose “material information” to investors in
an attempt to keep the company’s stock price high and that its
statements about predicted revenues and product safety were
misleading. Matrixx filed a motion to dismiss, arguing that
even if all of the plaintiff’s allegations were true, the lack of “a
statistically significant number of adverse events” among Zicam
users meant that the patients’ complaints would not have been
“material” to a “reasonable investor.”
The court rejected Matrixx’s argument that statistical
significance should serve as a bright line rule. Although the court
did not carefully distinguish between statistical significance and
causal inference (Kaye, 2011), the court rightly acknowledged
that the question before it was what would have been material
to a “reasonable investor.” Justice Sonia Sotomayor, writing for
a unanimous court, found that the reports about a possible link
between Zicam and anosmia presented an important threat to
the “commercial viability of Matrixx’s leading product” and held
that this information was likely to have changed the “total mix”
of data available in the eyes of a reasonable person considering
investing in the company.
It is important for social scientists to recognize the role that
statistical significance plays in the law. The court noted that
whether a statistically significant association had been detected
was relevant in assessing the degree to which the adverse event
information would be important to investors, but found that
statistical significance could not be legally determinative in this
context. Other considerations are inevitably at play. Indeed, it
might be that even a statistically significant association does
not have practical significance for a particular legal question.
In this case, whether or not a statistically significant association
between Zicam and anosmia could have been ascertained and
whether or not Zicam and anosmia could be causally linked, the
adverse event reports and pending lawsuits would likely have
been material to investment decisions. n
Judicial Notebook is a project of APA’s Div. 9 (Society for the
Psychological Study of Social Issues).
• Joseph Gastwirth, Statistical Considerations Support the
Supreme Court’s Decision in Matrixx Initiatives v. Siracusano,
• David H. Kaye, Trapped in the Matrixx: The U.S. Supreme
Court and the Need for Statistical Significance, Product Safety &
Liability Reporter (2011).