Research methods

Advanced practice nurses and program evaluation: Can solicitation of an e-mail address lead to longitudinal selection bias?

Justin B Dickerson
Department of Health Policy & Management, School of Rural Public Health, Texas A&M Health Science Center, College Station, TX, USA

Matthew Lee Smith
Assistant Professor, Department of Health Promotion & Behavior, College of Public Health, The University of Georgia, Athens, GA, USA

Ashley McKinley
Research Associate, Department of Social & Behavioral Health, School of Rural Public Health Texas A&M Health Science Center, College Station, TX, USA

Marcia G Ory
Regents Professor, Department of Social & Behavioral Health, School of Rural Public Health Texas A&M Health Science Center, College Station, TX, USA

PP: 169 - 176

Abstract

Objectives - To identify if survey respondents providing an e-mail address for program evaluation represent a risk of longitudinal selection bias.

Methods - A survey was administered to advanced practice nurses after a chronic disease self-management presentation. Chi-square statistics and logistic regression were used to identify variables associated with successful solicitation of an e-mail address.

Results - Relative to those 'not at all likely' to suggest someone in their practice train to become a certified chronic disease self-management facilitator, those stating they were 'very likely' or 'quite likely' to take this action were 10.20 and 13.60 times more likely, respectively,  to provide an e-mail address. These differences were statistically significant (OR = 10.20, CI = 2.91 - 35.77, p < 0.001 and OR = 13.60, CI = 2.14 - 86.40, p = 0.006, respectively).

Conclusion - Soliciting an e-mail address could pose a risk of selection bias when developing a longitudinal sample for further analysis.

Keywords

longitudinal research; advanced practice nursing; program evaluation; selection bias

Article Text

Over the last twenty-five years, health survey research has become an important component in addressing critical issues in the current health care system (Berk, Schur, & Feldman, 2007). Yet, much of this focus has resulted in more data collection, not necessarily improved survey instrumentation (Berk et al., 2007). Internet-based surveys best exemplify the paradox of being able to reach more participants but not achieve a substantial increase in response rate (Sax, Gilmartin, & Bryant, 2003), leading to non-response bias.

In addition to using new technologies like the Internet to increase data collection, researchers are also beginning to take advantage of data-mining technologies facilitated by electronic management of medical records (Lowrance, 2003). This level of data collection often involves obtaining data that is personally identifiable. As a result, privacy and data security are of paramount importance when working with such datasets (Boulos, Curtis, & Abdelmalik, 2009).Yet, because of the information they contain, these datasets allow researchers to conduct important studies that greatly contribute to the field of health care research. Black (2003) notes several examples of the value of identifiable data such as the ability to: 'understand the natural history and development of disease; to identify causes of disease; to evaluate health care interventions; to assess equity of care, to describe trends in health care utilisation; and to ensure the methodological rigour of research'  (Black, 2003) (p. S1:36).

From this research, it appears four things are occurring in the conduct of health survey research. First, the sophistication of technology enables researchers to contact large numbers of people in a very economical fashion. Second, not only do health survey researchers want to reach large numbers of potential survey respondents, they also want more personal information from respondents to facilitate the activities noted above by Black (2003). Third, privacy remains a challenge when asking survey participants for personal information.  And fourth, new technologies are being challenged for their validity because of low response rates and potential for non-response and selection biases.

We believe all these issues are important in the study of health survey research. However, we contend that while much study has been devoted to new technologies, the value of the data they yield, and the associated privacy concerns; there has been less focus on selection bias and validity given the proliferation of technology. Specifically, we believe a substantial portion of the literature (Braithwaite, Emery, De Lusignan, & Sutton, 2003; Eysenbach & Wyatt, 2002; Guise, Chambers, Valimaki, & Makkonen, 2010) has focused on understanding bias associated with the Internet or e-mail as survey instruments versus traditional survey collection methods; whereas, less focus has been placed on understanding the bias associated with questions that can now be asked because of these technologies (e.g., 'what is your e-mail address?' or 'how often do you use the Internet?'). Further, the literature's (Beebe, Locke, Barnes, Davern, & Anderson, 2007; Menachemi, 2011) emphasis on bias associated with Internet or e-mail survey instruments has been about non-response bias; the study of how the results are influenced by characteristics of those who do not respond to an instrument. This is logical given typically low response rates to these survey methodologies. However, we believe greater attention is needed to learn more about those who respond to the survey instrument, particularly as it relates to defining a sample that will form the basis of a longitudinal study. Thus, we believe selection bias warrants more examination than it has been given. A potentially helpful way to evaluate such potential bias is through post-hoc analyses (i.e., retrospective analysis of a study instead of stating a priori hypotheses to be tested (Thomas, 1997)) of a survey conducted for the purpose of developing a longitudinal sample to be solicited through a technological medium (i.e., e-mail). 

This post-hoc analytical study examines the results of a short paper-based survey administered to advanced practice nurses at a continuing education conference. The aim of this post-hoc investigation is to examine whether or not asking participants for an e-mail address (in an effort to build a longitudinal sample for tertiary program evaluation) creates selection bias in the survey results. The purpose of this research question is to assess whether new questions brought about by emergent technologies described above, like the Internet, can produce selection bias when utilized in a traditional (i.e., paper) survey instrument.

View references

Navigate: Archives | Contents | Top of page

Search

Translate

Related Publications from eContent Management