Background
Quotas are normally set by counts (shown under the Quota column below). For example, suppose you are seeking 1,000 total responses among a population ages 21 to 50, and you want a balance of the younger and older halves of this range. You could set a quota maximum at 500 for ages 21 to 35, and 500 for ages 36 to 50. This would work fine, allowing up to 500 in each category, and begin terminating people once those limits are achieved.
However, this could theoretically allow the first 500 respondents to be all in the older group with 0 in the younger quota group. Having this kind of imbalance might be undesirable, even if only temporary. For example, if a story in the news or on social media related to the survey topic came out, you could wind up with that news only affecting the later arrivals. If those later respondents were all in the younger group, there would be no way to see if the news affected each group differently. Likely the survey results would be skewed. In addition, if one had to shut down the survey early for some reason, one would be left with an imbalance that would be difficult to correct or handle with weighting. For this reason, IntelliSurvey also allows setting quotas by percentage and variance instead of setting an absolute number as the target.
Quota targets set by counts, a.k.a. setting "caps", and quota targets set by percentages are mutually exclusive and not used together.
Quotas set using target percentages should always be set within the Quota Tile directly. The Upload feature in the Actions menu is currently designed for setting quotas by counts only.
Costs to consider with this approach
It is important to realize that setting quotas by percentage will tend to throw out more respondents than quotas set by counts. Continuing with the above example, suppose that at a certain point the survey has 200 people in the older group and 100 in the younger group.
To define a quota by percentage and variance, select the desired quota tile in the Quota Manager applet, click the Gear > Use target percentages, then enter the desired percents under the Target % column. Once the desired percents are entered, the Target Variance will automatically populate with 5%. This is the system default, but can be edited as needed.
In this example, users could instead set target percentages of 50% each for the younger and older groups, with a 5% allowed variance. This will help ensure that the respondents will never get too far out of balance.
Target % and Target variance should be used together. If a Target % is entered, but no Target Variance, the system will default to using a 5% variance. If a Target Variance is entered but no Target %, the system will assume and behave as if there is no quota in place, and the variance will be ineffective.
If the Compact View is too small, you can use the scroll bar at the bottom of the tile, or you can switch to Full View.
Returning to the original example with a quota set by counts, let's assume we now have 100 respondents allocated to the 2135 age group, and 200 allocated to the 3650 age group. Quota caps for each group are set to 500.
Quota Group  Total  Quota 

2135  100.00  500 
3650  200.00  500 
Total N  300.00 
Now, also suppose that a new respondent comes through in the older group. In the regular case of using counts, with quotas at 500 for each group, the respondent would be allowed to continue through the survey.
Quota Group  Total  Quota 

2135  100.00  500 
3650  201.00  500 
Total N  301.00 
However, if quotas are set by percentage with 50% each and 5% variance, we would terminate the respondent. This demonstrates that with quotas by percentage, some respondents will be removed before we reach our total target numbers.
Quota Group  Total  Target  Variance 

2135  100.00  50%  5% 
3650  200.00  50%  5% 
Total N  300.00 
This is a desirable result if the goal is to keep the sample in balance, but it could be a disadvantage if the goal is to fill all quota groups as quickly as possible  or if we need to successfully capture as many respondents as possible in a hardtoreach demographic.
In the U.S., representative samples generally use ±3% variance because panels tend to be larger. Countries outside of the U.S. may use ±3%  ±5% though. Check with your client, internal Project Consultant (PC), and panel provider for what is reasonable and feasible.
How the system determines effective quotas
When quotas are set numerically (by counts), the system does a simple calculation for each respondent – is the total number of expected Completes greater than or equal to the quota for the respondent's category for that quota? If so, remove that respondent and count them as Overquota; otherwise, let them continue.
When quotas are set by percentage and variance, the system does the same sort of calculation, but it must first convert the percentage and variance into a numerical quota. It does so at that moment, for each respondent that comes through. So in a way, quotas by percentage operate dynamically, as if the numerical quota is constantly changing, based on the total number of responses at that time.
How does the system then convert the percentage and variance into an absolute quota number? The basic calculation is simple:
 The total allowed in a quota group is normally: (Percentage + Variance) x Current N.
By "Current N", we mean: Total Completes + expected In Progress completes (using predictive closure) for regular quotas. For quotabase or click balance quotas (CBQs), the Current N is the number of records that match thequotabase
orcbq base
filter.
However, there are two exceptions, designed for managing the beginning and end of fielding:

For current N less than 300, the calculation is: (Percentage x Current N) + (Variance x 300).
We use 300 as a default basis for variance in order to allow more variance at the beginning of a survey. Note: If total cap is set and less than 300, we use the total cap as the default instead.  If a total cap is set, the quota is also limited by the Percentage x Total Cap.
Returning again to our younger/older quota groups example, suppose that the survey is just starting, so we have no Completes and no In Progress responses. That means that for each group, the effective quota is:
Quota = (Percentage x Current N) + (Variance x 300)
(50% x 0) + (5% x 300) = 15.
Therefore, respondents in either category will be allowed through.
Later, suppose we have 225 in the younger group, and 275 in the older group. This means N=500 (ignoring In Progress completes for instructional purposes). Let us further suppose that a new respondent in the older group comes through. The question then becomes, would they be allowed in?
Quota Group  Total  Target  Variance 

2135  225.00  50%  5% 
3650  275.00  50%  5% 
Total N  500.00 
At this point, the effective quota for the older group is:
Quota = (Percentage + Variance) x Current N
= (50% + 5%) x 500 = 275.
In this case, we have reached our quota, and the new, older group respondent will be marked as Overquota.
Now suppose we get one more younger person to complete.
Quota Group  Total  Target  Variance 

2135  226.00  50%  5% 
3650  275.00  50%  5% 
Total N  501.00 
In that case, if another respondent in the older group were to enter the survey, the quota for the older group would be:
Quota = (Percentage + Variance) x Current N
= (50% + 5%) x 501 = 275.55.
Since 275 < 275.55, we would let the next person in the older group through.
As a final example, suppose we are near the end of fielding, and we have 480 younger respondents and 500 in the older group. Now a new older respondent arrives. Will they be allowed to continue the survey?
Quota Group  Total  Target  Variance 

2135  480.00  50%  5% 
3650  500.00  50%  5% 
Total N  980.00 
Quota = (50% + 5%) x 980 = 539
In this case, they will be termed because we set a quota limit to 500 and 500 < 539.
Comments
0 comments
Please sign in to leave a comment.