Q: The 2010 Caucasus Barometer includes about 2,000 completed interviews in each country: Armenia, Azerbaijan, and Georgia. However, the three countries vary in size; the population of Armenia is just under 3 million, Georgia has a population of about 4.6 million, and the population of Azerbaijan is about 8.4 million (according to the CIA World Factbook). How can the same or a similar sample size be appropriate for each country?
A: Great question! Contrary to popular belief, the total population size has little effect on the necessary sample size. Necessary sample size is more dependent on the amount of variability between members of a population. Only one person would need to be sampled if there were no variability in a population and every member would give identical answers.
Let’s use a physical example to make this more clear:
The two populations above have the same average height, but the members of Population B have much more variability in height than the members of Population A. Thus, if you were sampling Population B you would need a much larger sample size in order to reach the same level of certainty about the population’s average height than you were if you were sampling Population A. In short, the greater the amount of variability in the population, the larger your sample size needs to be in order to capture that variability.
Other issues that affect sample size include how accurate you want conclusions drawn from the sample to be and how certain you want those conclusions to be. In making a precise statement, you could say, for example, that “from the 2010 Caucasus Barometer, our best estimate of the proportion of Tbilisi residents who have travelled to another country is 18.5%, and we are 95% sure that the true value is between 15.5% and 21.5%.” Technically speaking, 95% is our confidence level and our margin of error is 3%. Therefore, we are 95% sure that the true value lies within the range of our best estimate plus or minus 3%. To increase your level of confidence or reduce the margin of error, you would need a larger sample size — and more money to pay for the extra interviews.
Here is one more thing worth knowing about sampling. Imagine a country of 5 million people, and a village of 500 inhabitants (both with the same amount of variability). Let’s say you require a sample of 200 from the country to reach a 95% level of confidence and a 3% margin of error. How many inhabitants of the village should be sampled to reach that same level of confidence and the same margin of error? Take a guess.
Done? The number is surprisingly high: we still need to sample one hundred and forty three inhabitants from the village. So while the country is 10,000 times the size of the village, it only requires an extra 57 people in the sample to achieve the same margin of error at the same level of confidence. In other words, one entirely counter-intuitive aspect about sampling is that small populations may still require a large proportion to be sampled to get representative findings.
In summary, while population size is one of the four factors that influence the necessary sample size for any survey (and even more factors have to be considered for complex surveys like the CB), its influence is relatively negligible.
Do you have further questions? Write a comment and let us know.
