Weighting Your Sample with Survey Software I

Posted on : May 8, 2017 - by :

In an ideal world you would get exactly the same ratios of different types of people completing your survey as exist in the population you are studying. In the real world that doesn’t always happen. If the discrepancy is small, it is probably best to ignore it. If the discrepancy is larger, you may want to consider some kind of weighting to adjust your sample to match the population.

Since it is not ideal to make these kinds of adjustments, it is always better to make an effort to get the right ratios of relevant groups. By “relevant” I mean different groups who may respond differently to your survey questions. For example, on a number of political issues, product concept evaluations and other areas of research, men and women tend to give different responses. In order to present research results that reflect the population you are studying as closely as possible, you may need to make sure the ratios of men and women in your sample represent their ratios in that population.

The best way to do that is probably to use quotas, which control how many people of different kinds you interview or let take your online survey. But sometimes you cannot do that or you may only realize after you have collected your data that some groups in your sample are giving significantly different answers. In these cases you can use weighting to adjust how much each group contributes to the overall results. Your survey software should let you establish quotas or use weighting as needed.

As a weighting example, you may be studying a population which has equal numbers of men and women, but have 60% women and 40% men in your sample. This difference between the sample and the population can skew the overall results on questions in which men and women answered differently. You can use weighting to multiply each woman’s answer by .833 and each man’s answer by 1.25. That way women in aggregate and men in aggregate will each contribute 50% to the overall answers. Sophisticated survey software lets you make these kinds of adjustments. It also helps you calculate the weighting numbers you should use, given either the number of people you have in your sample or the percentage of people in your sample represented by a particular group compared with their number or percentage in the population you are studying.