Survey Methods Matter

PopulationA Community Engagement Survey was designed with our partner, University of Nebraska Omaha (UNO), College of Public Affairs and Community Services, Center for Public Affairs Research, to provide outcomes development and evaluation components.  After running a baseline survey in October 2013, we used the same instrument for all three pilot projects (2013-2014) and a concluding survey in October 2014. 

After reviewing the data for online and paper surveys, we suspected that the online surveys showed a bias.  Jerry Diechert, a UNO evaluation partner, ran a separate analysis on online vs. paper surveys for the baseline survey, Pilot Project 1, and Pilot Project 2 in August 2014.  His analysis confirmed that using only an online survey resulted in the data being skewed towards white respondents with high education levels and high incomes.  Asians and Hispanics were underrepresented regardless of survey method.  Using both online and paper responses resulted in a more balanced demographic picture (see the table below) –

Omaha Demographics

The total number of respondents for the Community Engagement Survey was 984 (N=984).  318 completed the baseline survey (32.3%), 226 completed the survey during the 3 pilot projects (22.9%), and 440 completed the concluding survey (44.8%).  758 (77.1%) completed the public survey while 226 (22.9%) completed the survey during their pilot project meetings.  298 (30.3%) completed the online survey while 686 (69.7%) completed paper surveys.  Only paper surveys were available to the participants in the pilot project groups.

The largest percentage of respondents (28.2%) completed the concluding survey in paper.  We conjecture that once we identified the bias with online only respondents, our branch staff more heavily promoted the paper survey during the conclusion period.  It was a very successful effort at our South Omaha Library and Charles B. Washington Branch which serve a more diverse, lower-income population.

Posted by Theresa Jehlik, Project Director

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s