Sampling for hold-out control groups

To determine the effectiveness of your offer, you can create a hold-out control group for cells assigned that offer. Hold-out control groups are non-contact groups that would have received the offer, but you have chosen to deliberately withhold from the target cell compare the “lift” or difference in response from the target cell that received an offer.

Controls are applied at the cell level. When you assign offers to cells, either in a contact process in a flowchart or from a target cell spreadsheet, you can optionally specify one or more cells that will act as a control cell for each target cell.

For details about specifying a cell as a control for another cell or cells, see the IBM® UnicaCampaign User’s Guide.

Creating a hold-out control group requires you include to a Sample process in your flowchart.

This section describes two ways in which you can sample for hold-out control groups:

*
Sample at the audience ID level.
This approach is the best practice to sample for hold-out control groups and is consistent with how control cells are used in Campaign. In this case, control cells are at the audience ID level rather than at the offer-audience ID level. The following limitations apply:
*
Contacts members of the control cell do not receive any offers. A contact’s offers are suppressed for offers being processed in the flowchart. Across multiple flowcharts, the same contact will not always be chosen to be in the control cell; therefore, a particular contact may still receive offers from other flowcharts.
*
Sampling at the customer ID level does not guarantee equal representation across all offers. For example, by randomly choosing that 10% of all customer IDs be treated within a flowchart with multiple offers, it is possible that no one in the control cell is targeted to receive offer X. When analyzing offer X, in this case, the control cell is empty.
*
Sample at the offer level.
An alternative way to randomly sample for hold-out control groups is to do this on a per-offer basis, rather than on a cell basis. While more work, this option guarantees that a particular percentage of a specific offer is being held out for measurement purposes; at the cell level, an offer that is infrequently given out may not have any representation in the control cell. This approach, therefore, may be preferred by marketers when the distribution of their offers is skewed.

When randomly sampling at the cell level, you must sample from groups that are statistically similar. For example, if you initially segment into high-value, medium-value, and low-value segments, you must re-create those segments before sampling for control groups. You must compare high-value customers not receiving any offers against high-value customers receiving offers, medium-value customers to medium-value customers, and so on. You should not randomly sample across all customers, regardless of their value, and compare them against your initial segments.

Prior to sampling, then, you must re-create the appropriate segments. This is not necessary if you created different randomly selected cells from the same population, purely for testing purposes (for example, for assigning different offers).

Copyright IBM Corporation 2012. All Rights Reserved.