Every anonymous survey is an opportunity for some responders to express their hilarity through mischievous and intentionally inaccurate answers, but a new study say these responses can be found and allowed for.

“Mischievous responders” are the topic of new research from the US, which shows that their wayward responses can often severely skew the data on often important surveys.

Mischievous responders are youths who provide extreme and usually untruthful answers to multiple questions on self-administered questionnaires.

By providing misleading responses that they believe to be funny, these responders, even in small numbers, can lead researchers to wildly incorrect conclusions. If they are carried through research, the conclusions can lead to ineffective policymaking and may perpetuate negative stereotypes about certain groups.

“Inaccurate Estimation of Disparities Due to Mischievous Responders: Several Suggestions to Assess Conclusions,” by Joseph Robinson-Cimpian of the University of Illinois, suggests that mischievous responders can negatively alter readings of relative risk between groups of adolescents. His article introduces new sensitivity-analysis procedures for removing the bias that mischievous responders often introduce.

To identify mischievous responders, Robinson-Cimpian’s method relies on detecting patterns of unusual answers provided by survey respondents.

“If we find that youths reporting to be gay are more likely than those reporting to be straight to say that they are blind and deaf and extremely tall and parenting multiple children all at the same time, then we might question whether the data are valid,” said Robinson-Cimpian.

“Just like these jokester youths think it’s funny to say that they are gay and blind, they also think it’s funny to say that they are suicidal, engage in sexually risky behaviour, and take drugs. And this can dramatically affect our estimates of risk.”

“Within the past decade, much research has been criticized for failure to replicate and for exaggerated results,” wrote Robinson-Cimpian in his article.

“If we want sound research and policy, we need to have sound data. The procedures introduced here have broad relevance to research and can be widely, and easily, implemented.”

The four-step sensitivity analysis proposed requires first identifying youths who provide high numbers of low-frequency responses, and then comparing estimated disparities when including and excluding these youths.

While it is possible that some respondents will be mistakenly identified as mischievous, or that some mischievous respondents will not provide enough low-frequency responses to be identified as mischievous, there is great potential for both scientific and practical damage when no analysis is performed to assess data validity and ensure robust results, according to Robinson-Cimpian.

More details are available in the following video.