Understanding Date/Time A/B Testing
For date/time A/B testing, the process of determining the winner needs to be done manually, as there is no automatic option available. After analyzing the results, the winner must be declared, and the winning email sent accordingly.
However, if there is no intention to declare a winner and simply send emails at different times, utilizing a 100% sample size can achieve this. It ensures delivery at two different times without the need to set specific winner criteria.
This is Not recommended when we are doing for winner criteria.
Is my understanding correct?
Anyone, please help me to understand the following concept:
Marketo recommends users to not set the test sample size to 100% while performing the AB test as there's a high chance that your winning email will not go out to anyone once the test period is over. If you’re using a smart list, setting the sample size to 100% sends the email to everyone in the audience at that time. When the email program runs again at a later date (once the test period is completed), any new people who qualify for the smart list will also receive the email since they’re now included the the period is over. It is due to this uncertainty paired with not being able to send the tested and better-performing email out to the audience, Marketo (and many of its users) doesn't recommend setting the AB test sample size to 100%. You could also use the random sample choice step condition to send all versions of emails to equal portions of the audience instead of using the email program.
Thank you