{"id":1204,"date":"2018-01-25T09:21:07","date_gmt":"2018-01-25T17:21:07","guid":{"rendered":"https:\/\/www.hmc.edu\/institutional-research\/?page_id=1204"},"modified":"2018-01-25T09:21:07","modified_gmt":"2018-01-25T17:21:07","slug":"advice-for-running-a-web-survey","status":"publish","type":"page","link":"https:\/\/www.hmc.edu\/institutional-research\/survey-and-data-collection\/advice-for-running-a-web-survey\/","title":{"rendered":"Advice for Running a Web Survey"},"content":{"rendered":"\n<p>Test all the components of your survey. Your survey is more than just the questions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Take the time to design your survey<\/h2>\n\n\n\n<p>Constructing a survey instrument is an art in itself. There are numerous small decisions that must be made &#8212; about content, wording, format, placement &#8212; that can have important consequences for your results. While there&#8217;s no one perfect way to accomplish this, OIRE is available ([email]&#100;a&#116;a&#64;&#104;&#109;c&#46;&#101;d&#117;[\/email]) to offer advice and guidance to help develop your survey. Additionally, the <a href=\"https:\/\/www.socialresearchmethods.net\/kb\/survey.php\">Social Research Methods<\/a> site is a good resource.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Check the HMC Survey Calendar<\/h2>\n\n\n\n<p>OIRE maintains a <a href=\"https:\/\/calendar.google.com\/calendar\/embed?src=g.hmc.edu_655to6ub10q4m288ihstad1qa0%40group.calendar.google.com&amp;ctz=America%2FLos_Angeles\">survey calendar<\/a>, which shows what surveys are active on campus at what times for students, faculty and staff. Please contact us ([email]&#100;&#97;&#116;a&#64;&#104;&#109;c&#46;&#101;&#100;u[\/email]) and let us know about your survey and we will add it to the calendar. This helps OIRE coordinate survey administration, make sure that HMC is not overburdening students, faculty and\/or staff with survey requests and helps ensure that your survey has a better chance of getting a good response.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Think about the length of your survey instrument<\/h2>\n\n\n\n<p>Think about it again. Every additional question reduces the likelihood respondents will complete the survey. Don\u2019t ask items you don\u2019t know how you will use. It is good practice to provide survey respondents with an estimate of how many items are on the survey and\/or how long it takes on average to complete the survey in the survey invitation\/instructions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Don\u2019t ask if you know the answer<\/h2>\n\n\n\n<p>Basic demographic information (gender, race\/ethnicity, class year, major, etc.) is available through OIRE. When appropriate, suppress these questions and OIRE can provide that data for you. OIRE uses HMC ID as the link between demographic data and survey responses in creating a datafile. HMC also participates in many <a href=\"https:\/\/www.hmc.edu\/institutional-research\/assessment-of-student-learning\/institutional-results-by-instrument\/\">standard surveys<\/a>. If the results of these surveys can answer the questions you have, consider using these results rather than running your own independent survey. Not only will you get results faster and avoid asking HMC students questions they have already answered, you will know that the items have been independently tested and validated and you will be assured of a good response rate. A list of surveys that HMC participates in and the types of information available can be found at <a href=\"https:\/\/www.hmc.edu\/institutional-research\/assessment-of-student-learning\/institutional-results-by-instrument\/\">Institutional Results by Instrument<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Do not \u201cforce\u201d a response to a question<\/h2>\n\n\n\n<p>By doing this, you may annoy the respondent, causing them to abandon the entire survey, or encourage them to speak out in ways that negatively impact your survey. More importantly, voluntary participation refers to the exercise of free will in deciding whether to participate in any research activity, including answering a question on a survey. International law, national law, and the codes of conduct of scientific communities protect this right. Sample language from CGU\u2019s Institutional Review Board to include in your instructions: \u201cThis survey is entirely voluntary, and individuals may answer as few or as many questions as they wish.\u201d OIRE does not force or request a response to any survey or question. Allow the respondents to decide how much information they want to share.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Get feedback on your survey<\/h2>\n\n\n\n<p>It\u2019s a good idea to pilot test your survey with a few volunteers in advance of sending it out. Sample language to invite pilot-testers: \u201cIn advance of this survey going out to a wider audience, we are asking a select group of students\/faculty\/staff to respond to the survey early in order to get feedback on the questions, and to understand how long the survey takes to answer.\u201d If you make no changes from pilot test to launching the full version, this allows you to retain and use pilot test data. Many of us are concerned with getting as many respondents as possible. While there is no single method that will increase response rates for your survey, research (Dillman, 2007) does indicate several factors that contribute to higher response rates:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Perceived importance of the survey (value to the student, the college, perceived legitimacy)<\/li>\n\n\n\n<li>Level of interest students\/faculty\/staff have in the topic<\/li>\n\n\n\n<li>Trust that the data will be used and maintained properly<\/li>\n\n\n\n<li>Perception of reward for participation<\/li>\n\n\n\n<li>Minimizing respondent burden<\/li>\n<\/ul>\n\n\n\n<p>With these in mind, here are some practical considerations that may serve to help you get the best response to your survey:<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Who the survey is \u201cfrom\u201d matters<\/h2>\n\n\n\n<p>Who will your intended audience respond to? It may be a department chair, a Dean, or OIRE. \u201cOffice of the Dean of Students\u201d is less effective than \u201cDean Leslie\u201d. Framing who the survey is from can help you acquire more participation and better data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The \u201cSubject\u201d line of your invitation email matters<\/h2>\n\n\n\n<p>Think of your subject line as a headline. It should be:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Short (because people don\u2019t read)<\/li>\n\n\n\n<li>Rich in information (clearly summarize the email)<\/li>\n\n\n\n<li>Frontloaded with keywords (because people scan only the beginning of a list of items, and an email inbox is a list)<\/li>\n\n\n\n<li>Understandable out of context (your email is competing for attention with other emails)<\/li>\n<\/ol>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Provide the reader with a reason to explore your message further.<\/li>\n\n\n\n<li>The words \u201cHelp\u201d and \u201cReminder\u201d negatively affect survey participation rates.<\/li>\n\n\n\n<li>General Rule of Thumb: keep the subject line to 50 characters.<\/li>\n\n\n\n<li>Provide localization and context. \u201cHMC Core Curriculum Alumni Survey\u201d or \u201cAlumni Attitudes towards Core Curriculum\u201d is more effective than \u201cAlumni Survey.\u201d<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">The proper \u201cGreeting\u201d in the invitation will increase the likelihood of response<\/h2>\n\n\n\n<p>\u201cDear Alex\u201d is better than \u201cDear member of the Class of 2021\u201d is better than \u201cDear HMC student\u201d. \u201cDear Professor Xu\u201d is better than \u201cDear Faculty\u201d.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">If you are offering incentives, prizes or gifts for participation, check the rules<\/h2>\n\n\n\n<p>Any incentive program may require IRB approval. We recommend that you check with the <a href=\"http:\/\/mycampus.cgu.edu\/web\/sponsored-research-and-programs\/irb\">CGU IRB<\/a> as soon as you have made a decision regarding incentives. Other helpful hints around the use of incentives are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Incentives should be designed in a manner that maintains the voluntary nature of the survey. Please note, if you offer an incentive, your survey is no longer anonymous, as researchers will need to know the identity of those who participated in order to get them their incentive.<\/li>\n\n\n\n<li>Survey promotion (emails, flyers, etc.) should not emphasize the incentives to a degree that minimizes the requirement of survey participation.<\/li>\n\n\n\n<li>The amount or value of the incentive should not be so large as to appear coercive, interfere with a student\u2019s financial aid or violate NCAA eligibility rules.<\/li>\n\n\n\n<li>If your incentive involves a drawing (i.e., raffle or lottery), you should consult CA state law to determine whether your lottery is legal.<\/li>\n\n\n\n<li>If you plan to utilize an incentive for your survey, you must add that information about incentives to the instructions all participants receive before deciding to participate in the survey. This section would describe, in detail, the incentive program including the amount that could be won, an estimate of the odds (if you are utilizing a drawing), and how any drawing or other incentive program would be conducted. If you are conducting a web administration, this information should be included on the \u201cWelcome\u201d page.<\/li>\n\n\n\n<li>Alternative considerations include allowing the respondents to select their \u201cincentive\u201d (e.g. please select your \u201cthank you\u201d gift: a) $10 Claremont Cash, b) $10 Amazon Gift Card; c) $10 donation to HMC\u2019s Mary S Binder Prize or d) I do not wish to receive a gift) or making a donation for every valid survey response (e.g. donating $1.00 per survey response to a local charity or cause).<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Set up your reporting before you launch the survey<\/h2>\n\n\n\n<p>You want the best survey possible. In order to obtain quality results, that means doing some work upfront.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enter variable names and values into your datafile.<\/li>\n\n\n\n<li>Generate test responses (pilot test the survey).<\/li>\n\n\n\n<li>Check and see that reporting is capturing data as you intend.<\/li>\n\n\n\n<li>Create an analysis strategy (e.g. list of analyses you will do).<\/li>\n<\/ul>\n\n\n\n<p>For more helpful information, please consult:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.hmc.edu\/institutional-research\/survey-and-data-collection\/survey-guidelines\/\">Survey Guidelines<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.hmc.edu\/institutional-research\/survey-and-data-collection\/survey-checklist\/\">Survey Checklist<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Test all the components of your survey. Your survey is more than just the questions. Take the time to design [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":1202,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-1204","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/pages\/1204","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/comments?post=1204"}],"version-history":[{"count":0,"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/pages\/1204\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/pages\/1202"}],"wp:attachment":[{"href":"https:\/\/www.hmc.edu\/institutional-research\/wp-json\/wp\/v2\/media?parent=1204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}