MLA2014: Research Survey Design for Librarians

Planning for Effective Survey Design

  • Steps in effective survey design
    • purpose
    • problem statement
    • research questions
    • survey design
  • link between your purpose and problem statement
  • appropriate research questions — dictates type of data you need to collect and inform the methodology that you use
  • Sample
    • Is there a sampling frame (existing list)?
    • Probabilistic (greater chance to generalize results to larger population) vs non-probabilistic (limited generalizability – ex: snowball)
    • Stratification of the population
  • Instrument – your own? modified with permission? developed by someone else?  Pilot testing is essential!
  • Survey items – match to research questions
    • closed vs open questions
  • Common question pitfalls
    • double-barreled – don’t ask 2 things in 1 question
    • leading/biased
    • leaving out a choice


  • reliability (consistent and stable over time) and validity (is it measuring what its supposed to measure)

Managing Bias in Survey Research

  • dog whistle politics to get certain reactions from people using keywords or names – names of key individuals (ex: Sotomayor, Scalia), verb choice (ex: follow vs interpret), priming
  • common forms of bias on surveys (these are the top 5, but there are probably 80-100 examples out in the literature)
    • purpose of study
      • possible solutions
        • inductive analysis
        • peer review
        • lit review
        • focus groups
        • community-based participatory research
    • study population (largest bias area)
      • convenience sample bias
      • selection bias
      • Grove et al 2004 offers diagnostic statistical methods to detect
      • possible solutions
        • broaden scope of recruitment
        • intensive recruitment of underrepresented population strata
        • alternative communication methods
    • reactive effects
      • pre-conditioned due to recruitment process or consciousness of study participation (“Hawthorne Effect“)
      • possible solutions
        • seems to be diminishing due to growing habit of being observed via social media
        • decreases over time
    • self-report bias
      • stigmatized topics (ex: obesity, substance abuse, etc)
      • possible solutions
        • assume third-person perspective
        • observable behavior
    • data handling
      • see a statistician early in the process
      • can avoid by concealing the identities before statistician does the analyses

The Value Study: An Example of CBCR Survey Design

  • Is survey the best method for our research?  Is the most common method in medical library field
  • Community based collaborative research is research conducted by and for those most directly affected by the issue, condition, situation or intervention being studied or evaluated
    • form of participatory action research
  • Advantages of CBCR
    • yields better data
    • buy-in
    • local jargon
    • meet community needs


Interesting/Entertaining Tweets From Other Sessions

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s