Dummy data

A common practice among Survey Programmers (SPs) is to review survey data after 5%-10% of the total N of a study is collected. In this review, SPs seek to verify that the survey programming satisfies the researcher's requirements, and that each respondent is seeing the appropriate questions. Two important questions SPs seek to answer are:

• Are the base sizes for each question correct? (Did respondents see or skip questions as intended?)
• Is logic for background variables working properly?

Dummy Data is a tool that allows programmers to answer these questions without potentially sacrificing costly sample. This is because the records generated by Dummy Data are subject to the same rules as an actual survey respondent. Dummy Data automatically generates survey records, populating all applicable survey fields as an actual respondent might. These system-generated survey records can be used to generate reports to verify that logic, skip patterns, validations, etc. are programmed correctly.

Generating dummy data

To create dummy data, expand the Data button group in the Survey Navigation menu, and click Dummy Data.

Users can choose between two options – to Run Dummy Data in Pub Mode, using the latest published version of the survey, or to Run Dummy Data in Dev Mode, using the latest development version of the survey. If the survey has yet to be published, the Run Dummy Data in Pub Mode will be disabled. 

For additional information, read Dev and Pub mode.

Regardless of whether you choose to test in Dev or Pub mode, the Dummy Data testing window will open in a separate browser tab, and the window will display the questions/tables with their IDs. The additional controls and information will be presented across the top of the page.

This control pane is where you can define and manipulate the data. The available fields and controls are described below.

Using count

When Dummy Data is opened, if defaults to having "10" entered in the Count field, but testers can specify whatever number of records they need run. Simply press Go, and Dummy Data will begin to generate the test records. When the tool is running, you can observe the current record's progress through the survey. The current record's number and progress is reported in the Dummy Data window. When the tool has stopped running, it will report the number of records generated.

Using copy from ID

Dummy Data selects question responses at random. In circumstances where questions require a specific response to qualify for the survey or force a specific answer for logical validation, this can lead to a large number of Dummy Data responses being terminated or stuck on a survey page. This isn't always undesirable: after all, the function of running Dummy Data is to check that programming is accurate, and termination/validation logic is a part of programming.

However, in the case of particularly stringent screening sections, Dummy Data may have a very low qualifying rate. To circumvent this issue, use Copy from ID. This functionality will copy all the data from a specific survey respondent, allowing the tester to quickly pass through the screener or the questions with strict validation rules. 

First, create a test record that answers the questions as needed for your testing purposes. Then, simply enter that record's ID in the Copy from ID box. After entering the Count of Dummy Data records you wish to generate, press Go. 

Using custom data

Custom data allows the tester to create customized rules which tell Dummy Data to use specific answers/values for the given questions instead of choosing answers at random. The custom rules can be simple or complex. If the Avoid simple terms button is active, the Enter custom data button will display a green checkmark, indicating that logic has already been generated to help bypass the simple termination points. You can then manually add the more complex rules for termination points, quotas, or questions with validations as needed.

To use custom data, select the Enter Custom Data button on the Dummy Data interface.

This launches the Enter custom data modal where testers can specify which responses should or should not be allowed for individual questions. A list of examples is provided to help with syntax. If need be, the modal can be resized by clicking on an edge or corner of the pop-up. 

Custom data syntax

There are a few different ways to specify the values that Dummy Data should use. Different techniques apply to different field types. Some are specific to a given question type, while others overlap.

Syntax	Description	Single Select	Multi-Select	Numeric Open End	Free Text Open End
Q1=5	Select option with an ID of 5/fill response with number 5.
Q1__loop__2=5	Select option with an ID of 5/fill response with number 5 in the 2nd iteration of a loop.
Q1=gt(5)	Select an option with an ID greater than 5/fill response with a number greater than 5.
Q1=ge(5)	Select an option with an ID greater than or equal to 5/fill response with a number less than 5
Q1=lt(5)	Select an option with an ID less than 5/fill response with a number less than 5.
Q1=le(5)	Select an option with an ID less than or equal to 5/fill response with a number less than 5.
Q1=ne(5)	Select any option with an ID not equal to 5.
Q1=or(4,5)	Select either option 4 or 5/fill response with number 4 or 5
Q1=[1,2,3]	Select options 1, 2, and 3.
Q1=or([1,2,3],5)	Select options 1, 2, and 3, OR select only option 5.
Q1='Orangutan'	Fill response with text "Orangutan".
Q1=exclude(1,2)	Exclude options IDs 1 and 2.
Q1=must_have(1,4,5)	Select all options out of the IDs specified – 1, 4, and 5.
Q1=must_have_one_of(1,4,5)	Select a single option out of the IDs specified – 1, 4, and 5.
Q1=no_conflict(1,2)	Do not select options specified at the same time.
Q1=min_checked(3)	Select at least 3 options.
Q1=max_checked(7)	Select at most 7 options.
Tables
T1=n_rows_have(2,1)	Table 2 should have at least two rows with option 1 selected.
T1=n_rows_have(4,[1,2,3])	Table 1 should have at least four rows with options 1, 2, and 3 selected. Other options may also be additionally selected. Exclusive options are avoided.

Setting data

Once you have added all values you wish to set, click Set Data. A green checkmark will appear next to Enter custom data, indicating that custom rules have been entered for the Dummy Data testing session. All new Dummy Data records will contain the entered custom data after you click Go.

To clear custom data, re-click Enter custom data and select Clear Data.

Using Run up through question

The Run up through question feature of Dummy Data is designed to get through a complicated screener section, test surveys with branching paths, or help the client obtain sample data for a specific section of the survey in order to review the reporting output. If you are using this feature, odds are you are interested in speed and efficiency, and you will likely combine this with the Enter custom data feature mentioned above.

In older versions of the software, this was the only method to generate test records in an automated process. Now, you can also use the Clone this Record feature in Testmode. However, if you need multiple test records, fast, the Run up through question feature is the most efficient, generating an unlimited number of records with data populated up to the specified question. Testers can then use these records to manually test the remaining survey paths. 

To use Run up through question, click the box to reveal a menu of all survey fields. The search field allows for text matching; testers can either enter field IDs or scroll through the menu to make a selection. Then, Run up through question will populate data up through the last page specified in this field.

Note that if more than one question is listed in the same line for the question you have selected (for example, Q10, Q10_DTA and Q10A below), these must be selected together because they are displayed on the same page. In this case, all three of these questions could populate with data (depending on the programmed logic in place).

After selecting the question, and entering the number of records needed in Count, click Go. Dummy Data will then generate new records up through the specified field.

Show IDs created

After a Dummy Data session has begun, a button for Shows IDs created appears. This will show only the test records created in the current Dummy Data session. 

To use the Dummy Data records in further testing, go to the Survey Dashboard navigation menu. Click on the chevron icon next to Pub: Run | Test. A field for entering a unique user ID appears. Type in the ID generated from your Dummy Data session and continue testing where you left off.

For more information about formatting test link URLs with the various testing levels, see Running the Survey and Running Surveys in Testmode.