Custom CheatSweep rules

Custom CheatSweep rules permit Survey Programmers (SPs) to add new rules to CheatSweep, or disable existing rules, for a given survey. 

Users should note the following when adding custom CheatSweep rules:

• Values of '0' and '1' are not to be used in custom rules by design. This is so that a single rule cannot be used to flag a respondent as definitely "good" or "bad." These integers are out of range and will not take any effect.
• Using values of 0.001 or 0.999 would be effective in most cases, but a rule does not ignore other existing rules or other CheatSweep mitigating factors.
• The rule will override the CSP score if the defined condition is met, and it can also be decreased or increased in small amounts by 'cs_good' or 'cs_bad' mitigating factors, or even “nullified” by another rule if not carefully written.

When creating custom rules, be conscious and think about all rules in effect in the survey source. For example, if one rule is defined as 0.001 and another as 0.999, you could wind up at 0.5 if both are true. In this case, the two rules likely would result in a score that no longer serves its intended purpose (“nullified”).  Another example is that, if you have one rule as 0.9999, but then the respondent is flagged with a few “good” factors, they could end up with a csp score of 0.975. In this example, users should be cautious of where they establish their CheatSweep threshold in order to effectively sweep the desired “bad” respondents. 

Syntax

Define new rule

The addrule tag expects a "hash" of data, i.e., a collection of certain named values. Heredocs are used here for multi-line support.

To include or modify a rule, add the following lines. The keys (shown to the left of each => arrow) are 'id', 'test', 'bad', and 'good' and should remain unchanged. Change the "values" (on the right side) of each key as appropriate.

enable cheatsweep 
add rule: <<END 
    id => Rule ID here, 
    test => Rule to test returning value, 
    bad => 'Description of failed rule', 
    good => 'Description of passed rule' 
END

The 'id' and 'test' keys are required, and at least one of either 'good' or 'bad'. When adding a rule, the 'test' should always evaluate to a number greater than 0 and less than 1.

Disable a rule

Disable a rule by referencing the rule ID in the disable rules tag.

enable cheatsweep 
disable rules: Rule ID here

If you have multiple rules to disable, provide them as a comma separated list, like disable rules: 101, 102. You must know the rule IDs in order to use this feature. 

Current rules

Examples

State does not match State derived from ZIP code

enable cheatsweep
add rule: <<END
    id => 1001,
    test => sub { ($_->{QSTATE_FROM_ZIP} != $_->{QSTATE}) ? 0.95 : undef },
    bad => 'zip code does not match state'     
END

Always assign a near-0 CSP score to Panel 100

enable cheatsweep
add rule: <<END
    id => 1001,
    test => sub { $_->{QPANEL}==100 ? 0.0001 : undef },
    good => 'panel 100 never includes cheaters'
END

Skip good mitigating factors

The following example will always assign a 99.99 csp score to respondents who answer a "HoneyPot trap" by skipping all 'good' mitigating factors.

enable cheatsweep
add rule: <<END
	id => 1001,
	test => sub { (length($_->{QHP1}) > 0) ? 0.9999 : undef },
	bad => 'likely bot; answered hidden honeypot QHP1)',
	skip => [51,55,61,102,103,104,105],
END

Flag for removal if fail 3+ attention checks

enable cheatsweep
add rule: <<END
    id   => 1003,
    bad  => 'Three or more attention checks failed',
    test => sub { ( $_->{attention_fail_count} >= 3 ) ? 0.95 : undef },
END