Sampling for hold-out control groups
To determine the effectiveness of your offer, you can create a hold-out control group for cells assigned that offer. Hold-out control groups are non-contact groups that would receive the offer, but you chose to deliberately withhold from the target cell. You can then compare the “lift” or difference in response from the target cell that received an offer.
Controls are applied at the cell level. You can assign offers to cells in a contact process in a flowchart or from a target cell spreadsheet. When you assign offers to cells, you can specify one or more cells that act as a control cell for each target cell.
For details about specifying a cell as a control for another cell or cells, see the IBM® Campaign User’s Guide.
Creating a hold-out control group requires you to include a Sample process in your flowchart.
This section describes two ways in which you can sample for hold-out control groups:
*
Sample at the audience ID level.
This approach is consistent with how control cells are used in Campaign. In this case, control cells are at the audience ID level rather than at the offer-audience ID level. The following limitations apply:
*
Members of the control cell do not receive any offers. The offers for a contact that is part of a control cell are suppressed for any offers being processed within the specific flowchart. Across multiple flowcharts, the same contact is not always chosen to be in the control cell. Therefore, a particular contact might still receive offers from other flowcharts.
*
This sampling method is not the same as a universal hold-out control group that is withheld from all company communications. If you use a universal hold-out control group, you must identify members of your surviving contacts in that group and remove them. For example, by matching against a universal control strategic segment and then excluding those audience IDs. For more information about control groups, see the IBM® Campaign User’s Guide.
*
Sampling at the customer ID level does not guarantee equal representation across all offers. For example, you choose that 10% of all customer IDs be treated within a flowchart with multiple offers. It is possible that no one in the control cell is targeted to receive offer X. When analyzing offer X, in this case, the control cell is empty.
*
Sample at the offer level.
Another way to randomly sample for hold-out control groups is to sample on a per-offer basis, rather than on a cell basis. Although sampling on a per-offer basis takes more work to configure, this option guarantees that a particular percentage of a specific offer is being held out for measurement purposes. At the cell level, an offer that is infrequently given out might not have any representation in the control cell. You might prefer this approach when the distribution of your offers is skewed.
When randomly sampling at the cell level, you must sample from groups that are statistically similar. For example, if you initially segment into high-value, medium-value, and low-value segments, you must re-create those segments before sampling for control groups. You must compare high-value customers not receiving any offers against high-value customers receiving offers, and medium-value customers to medium-value customers. Do not randomly sample across all customers, regardless of their value, and compare them against your initial segments.
Before sampling, then, you must re-create the appropriate segments. Re-creating segments is not necessary if you created different randomly selected cells from the same population, purely for testing purposes (for example, for assigning different offers).
To sample at the audience ID level
To sample at the offer level
Copyright IBM Corporation 2013. All Rights Reserved.