How to Ask Questions Using Survey Software
The way you ask your questions can affect the answers you receive. Most experienced researchers know this. Both how you phrase the question and how you ask for responses can both affect your results. Sometimes researchers knowingly choose the words they use to ask a question to get their desired response. This is most common in political surveys. As one example, the American public has always given higher levels of support when asked about the “Affordable Care Act “than when asked about “Obamacare.” There are people out there who like the concept of affordable care, but don’t like Obama.
Another example in the political field is support for the death penalty. Researchers consistently find more support for the death penalty when the question is phrased “Do you support the death penalty?” than when the question is phrased “Do you support the death penalty, life in prison without the possibility of parole, both or neither?” Logically, support for the death penalty shouldn’t depend on whether the researcher mentions life in prison without parole, but reminding people of this alternative does affect the answers given.
Another difference in the responses you will get is how you ask a question that lets people pick more than one answer. For example, “Which of the following magazines do you read?” You will get more people saying they read the magazines mentioned if you ask them for each magazine “Do you magazine X?” than when you ask “Which of the following magazines do you read?” This is especially true of online or paper surveys. Your survey software should give you the choice. In this case, what you should do is not completely clear. While you would get more responses asking the series of yes or no questions then you would asking people to pick items from a list, it’s not completely clear of which is more accurate. People who do research on research have looked at both methods and their conclusions differ. It probably depends on the particular items you are asking about.
While sometimes researchers may try to get the answer they want by wording questions in a particular way, it is probably much more common for people who are less experienced at creating surveys to do so unconsciously. A poorly worded question will likely give you unreliable results. The proliferation of limited, but easy-to-use survey software has encouraged people who don’t know how to write questions to get opinions. Writing good questions is a skill, like many other job skills. People without training often do it poorly.