Not all Scientific Studies are created equally. How and why improperly designed Lie Detection
Studies provide invalid results. Image source: Flickr user Kendra Renee ©.
Scientific Study Summary Information in this article derived from: O’Sullivan, Maureen, et al. Police Lie Detection Accuracy: The Effect of Lie Scenarios.” Law and Human Behavior, Dec. 2009.
Over the past several decades, a number of studies have been conducted that used methods designed to lead to the discrediting voice stress analysis (VSA) technology. However, it is important to keep in mind that most of these studies were funded and/or conducted by members of the polygraph community. As voice stress analysis technology proved itself and gained popularity with law enforcement worldwide, polygraph advocates became concerned about this competition and attempted to “prove” polygraph technology was superior to VSA. However, a meta-analysis led by Maureen O’Sullivan and Jaspeet Tiwana at the University of San Francisco, conducted in conjunction with Mark G. Frank and Carolyn M. Hurley at SUNY-Buffalo, indicates these “polygraph-backed” studies of VSA are not reliable, and suffer from significant flaws due to poor design and failure to observe established protocols.
Such misinformation presents a problem for law enforcement professionals seeking science-backed evaluations of voice stress analysis equipment such as the Computer Voice Stress Analyzer (CVSA ®). The results of the O’Sullivan meta-analysis, demonstrated conclusively VSA studies must be designed to include high-stakes lie scenarios similar to what law enforcement professionals would experience while conducting investigations in the field.
Impetus for the Study
The researchers decided to conduct the meta-analysis after observing that in some research, law enforcement professionals appeared to be much better at detecting deception than in other studies. The O’Sullivan team hypothesized it was not because the officers selected for those studies had some particular skill that made them better at detecting deception. Rather, they believed it was a result of how the studies were designed. They predicted that officers would be better at identifying deception in studies where the subjects were telling high-stakes lies. After all, those studies more accurately approximate the officers’ field experiences with criminal investigations, employment interviews, and other situations where interview subjects may be highly motivated to be deceptive.
If this hypothesis was correct, it would have major implications for many studies evaluating and comparing different types of truth verification equipment. In particular, it would mean studies where experiments only included low-stakes lies would be far less reliable than those that analyzed data from high-stakes scenarios. If a law enforcement officer is not able to identify deception under low-stakes conditions, it is not because of a problem with his or her ability to analyze data or with the truth verification technology itself. Rather, it is the failure with the study’s design to adequately approximate real-life circumstances. In fact, virtually all polygraph-backed studies of VSA are based on mock scenarios or conducted under highly controlled laboratory settings which are categorized as “low stake”.
Conducting the Meta-Analysis
To test their hypothesis, the O’Sullivan team conducted a meta-analysis of 23 studies that involved 31 different police groups from eight different countries. They divided the testing scenarios into two categories: high-stakes lying scenarios and low-stakes lying scenarios.
- High-stakes lying scenarios, according to the researchers, were scenarios in which the subjects were lying about something important to them and in which there was a significant positive or negative consequence. For example, the subjects may be trying to hide information regarding a strongly held personal opinion, a stressful event, a real criminal act, or an emotional situation. They may also have been promised a high reward for successful deception, such as a large sum of money, or a serious punishment, i.e. prison time.
- Low-stakes lie scenarios were defined by the researchers as situations in which the subjects were telling inconsequential lies, such as those about a loosely-held opinion or artificially simulated crimes the researcher told them to commit (i.e. acting out a fake crime or stealing phony money in a computer game). In low stakes lie scenarios, positive consequences are minimal, such as a free movie ticket, and negative consequences are usually nonexistent.
Based on their analysis, the researchers were able to confirm their hypothesis. Across these studies, it was clear that law enforcement professionals were much better at identifying deception in high-stakes scenarios than in low-stakes scenarios.
The Implications of the Study’s Results
Given the theoretical underpinnings of truth verification technologies such as the CVSA, the results of the O’Sullivan meta-analysis confirm the assertion of CVSA professionals that real-world consequence and jeopardy are a requirement for the technology to provide valid results. The CVSA detects changes in the oscillations of muscles in the vocal tract, which are caused by stress due to real-world jeopardy and consequence. In low-stakes lie scenarios, the subject is unlikely to experience any stress associated with real jeopardy and consequence, as they are aware there are no true penalties involved in artificial, game-playing scenarios. Therefore, it has been effectively demonstrated artificial laboratory studies cannot accurately replicate the circumstances under which the CVSA was designed to be used—that is, real investigations involving consequence and jeopardy.
Almost all of the contrived experiments and studies the polygraph community attempted to use to discredit voice stress analysis technology involved artificially manufactured low-stakes lie scenarios. As such, the results of the O’Sullivan meta-analysis discredit the underlying design and validity of those studies. Further, independent review of these studies has led to numerous fatal flaws being discovered in their design and execution. Put simply – these polygraph-funded studies of the CVSA are unreliable and their results are invalid. In contrast, there is ample evidence in support of the effectiveness of the CVSA for deception detection based upon data from studies that incorporate high-stakes lie scenarios, most notably the Chapman Study. The CVSA has also consistently proven itself in real-world settings—including criminal interrogations, employee screenings, and immigration investigations—making it a valuable and reliable tool for law enforcement agencies.
Please reach out to us at NITV Federal Services to learn more about our CVSA systems and training programs.