page13
S
URVEY
G
UIDE
13
Even if you’ve used the same questions before or copied them from a
governmen
t repository, you’ll want others to check your survey for spelling and
grammatical errors, readability and flow, and consistency with the current
survey’s goals.
If the topic is sensitive or your results will be used to make high
stakes decisions, consulting an expert (e.g., the UW Survey Center) to review
your survey is a wise investment. You may also need to get final approval from
others in your unit or beyond.
Once your survey has passed muster internally, it’s time to “field test” it with a
sample of potential respondents to verify that your process is smooth and
completely understandable to your target population. Do people understand the
terms? Or are adjustments needed? Do people complete the survey as
intended? Or do they drop out before completing it? Are certain questions
regularly skipped? If the survey is electronic, does it launch properly and work
as expected with different browsers? The purpose of the field test is to get
estimates of the survey’s reliability and validity and to identify an
y final changes
that might be needed to ensure success.
Those involved in contacting potential respondents, conducting interviews, and
analyzing survey responses need to be briefed on the purpose of the study and
provided with training tailored to their role in the project. The goal is to have
everyone following a consistent process so that as little variation as possible is
introduced into the process.
If interviewers are administering the survey in person or by phone, develop a
script for them to follow so that the survey is presented in exactly the same way
to every respondent. Interviewers will also need to be trained to behave in
neutral ways to control their influence on the answer, and they need actual
practice reading the questions and script. For mail surveys, the mail and data
entry staff needs instruction in how the study will be mailed out and how the data
will be coded and entered.
A common question is whether those who participate in the field test can later be
respondents. The answer depends on how the pretest respondents were drawn
and whether the instrument has changed, and how much time has passed
between the field test and the main study. A good way to identify pretest
respondents is to draw a miniature sample like that to be used in the main study.
This approach allows field procedures to be tested as well as the instrument.
When this method is used, pretest respondents can sometimes be combined
with respondents from the main study.