Received 2 Emails in the A/B Test: Test and Winner | Community
Skip to main content
KCS_Integration
KCS_Integration
New Participant
December 13, 2017

Received 2 Emails in the A/B Test: Test and Winner

  • December 13, 2017
  • 4 replies
  • 2186 views

Issue Description
After running an A/B test, some Lead records appear to have received both Test and Winner.

 

Issue Resolution

In an A/B test, the sample size of the audience is sent the variants, and then for the winner: the entire audience minus the test audience is sent the Winning email.

 

So let's say that you received both Test and Winner. This means you have duplicate records in the database.

 

Marketo is designed to not send duplicate leads the same email at the same time.

 

So one of the duplicates is qualified for the Test, and then when the Winner is sent, it's the total audience minus the test audience, the Winner is sent to the duplicate record.

 

This above situation can apply to Lead Records as well, because of duplicates with the same email address, but also duplicate records that may have a FWD set up between them (record B emails FWD to Record A email address).

 

Because this is expected from the design decision, the resolution is to deduplicate your database.


Is this article helpful ?

YesNo


This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

4 replies

Roxann_McGlump1
Roxann_McGlump1
New Participant
January 28, 2019

Hi Santiago,

I don't see why this would contradict the other article. Both articles are describing the same basic issue from different angles. Could you clarify what conflict you see in them?

Santiago_Muniag
Santiago_Muniag
New Participant
January 24, 2019

Hello Roxann! If I'm not mistaken this seems to contradict the following: An A/B test sent the "winner" email to some leads even though the test sample size was 100%

We have duplicate email addresses by design in our database, so fully deduping the base is not a viable solution for us - trying to decide what's the next best thing!

Roxann_McGlump1
Roxann_McGlump1
New Participant
March 16, 2018

No, testing 100% of the audience should not cause duplicate leads to receive both copies of the email, as the A and B emails are not sent in separate runs. Only Test and Winner sends are separate.

A
Anonymous
March 14, 2018

If you decide that your audience isn't big enough and decide to A/B test the entire smart list, is it likely that duplicate records in the A/B test would get both test A and test B since it's not the same email at the same time?

Terms & ConditionsAccessibility statement

Loading footer...