“Impact of COVID-19 on lifestyle habits and mental health symptoms in children with attention-deficit/hyperactivity disorder in Canada”: Can we trust the numbers from this Internet survey?

Abstract

The authors of this article (pxab030) address the important issue of the impact of COVID-19 on children with attentiondeficit hyperactivity disorder (ADHD) but use an Internet survey nonsampling strategy to do so. They do this by constructing a 113-item questionnaire and making it available online for several weeks until they accumulate 587 surveys. The authors state that their response of “587 exceeded the minimum sample size (N = 384) required to represent the pediatric ADHD population of Canada.” They also claim that they have a representative sample based on ethnicity, income and other factors but then list limitations that suggest this is not the case, i.e., the self-selecting nature of recruitment” and “families who were struggling less were likely more able to complete a 30 to 45 minute survey.” These limitations have consequences. Many parents of children with ADHD also have ADHD and it is likely that those parents who have the time and patience to complete a 113-item online questionnaire are substantially different from those who cannot. Note that this nonrepresentativeness bias cannot be eliminated by sample size adjustment. The authors must go beyond acknowledging these limitations in the discussion, address how those limitations are threats to validity and take reasonable steps to assess whether or not the bias might significantly affect the results. Rather than presenting the data as if it comes from a representative sample, they should warn the reader that the reported results may be substantially different from the population of ADHD families that they are trying to describe. There is a rich literature about non-representativeness and bias inherent in Internet surveys. Investigators those who wish to use such a potentially biased survey method should build into their study methodology some means of quantifying and minimizing those threats to validity.