What are Survey Quotas?
A survey quota is the number of observations needed to meet a specified requirement, such as the number of men and the number of women to complete a survey.
Most customer feedback surveys employ quotas, sometimes referred to as strata or stratification. This is to ensure that the data is representative on key variables. While there is no strong theory regarding which variables to use for quotas, the norm is to use age, gender, and geography. Variants of quotas include: interlocking, non-interlocking quotas, soft, hard, and strata.
Interlocking survey quotas
The table below shows the quotas used in a commercial study of retail energy choice. Each quota involves multiple variables: household structure and age. Where a quota is defined by more than one variable, it is referred to as being interlocking. These variables are relevant to the energy market, as size of household relates strongly to appliance usage and water-heating.
Non-interlocking survey quotas
An alternative approach is to have separate quotas by each variable, non-interlocking quotas, also known as overlapping quotas. This is done in situations where there is insufficient knowledge to determine the required number of interviews for the interlocking quotas, or where there is a desire to save money (with interlocking quotas, it can be time-consuming to find people for all the quota groups).
Although popular, non-interlocking quotas are usually a bad idea. Most researchers conduct their surveys using online panels (databases of people who participate in market research studies in return for payment or entry into prize draws). These panels are generally massively skewed – they tend to over-represent white-collar workers, the young and the inert. Quotas are used to correct for these skews. However, the use of non-interlocking quotas actually creates further problems with the data (e.g., with quotas for males, females, old, and young, you may end up with only young females and old males in the sample).
Hard versus soft quotas
Sometimes researchers refer to quotas as being either “hard” or “soft”. Unfortunately, there is no standardized meaning for these terms. “Hard” can refer to a quota that needs to be filled exactly (i.e., 200 and no more or less), or an absolute minimum (200 or more). “Soft” can mean either an absolute minimum that researchers expect to be exceeded or a quota for which near enough is good enough.
Difference between strata and quotas
In the formal language of statistics, strata refers only to quotas where membership of the quota groups can be determined prior to the commencement of data collection. For example, if you have a list of email addresses by age, gender, and state, you can send invitations to specific quota groups, whereas you cannot if you need first to collect data to ascertain a person’s age, gender or state.
About Tim Bock
Tim Bock is the founder of Displayr. Tim is a data scientist, who has consulted, published academic papers, and won awards, for problems/techniques as diverse as neural networks, mixture models, data fusion, market segmentation, IPO pricing, small sample research, and data visualization. He has conducted data science projects for numerous companies, including Pfizer, Coca Cola, ACNielsen, KFC, Weight Watchers, Unilever, and Nestle. He is also the founder of Q www.qresearchsoftware.com, a data science product designed for survey research, which is used by all the world’s seven largest market research consultancies. He studied econometrics, maths, and marketing, and has a University Medal and PhD from the University of New South Wales (Australia’s leading research university), where he was an adjunct member of staff for 15 years.