Incorrect traffic split on ab test

Question

We ran an ab-type experiment with 50/50 manual allocation with both the control and variant versions modified.

The control version also injects custom js/css.

We have noticed an anomalous split percentage, given that the variant has approximately double the number of users of the control.

We can't understand what the problem could be.

Do you have any idea?

abhinavbalooni · Accepted Answer

@jack_hala In addition to what @alexbishop mentioned, did you change the split midway through the activity or make major edits post activating the activity? Sometimes that also plays a part in this anomaly.

https://helpx.adobe.com/ca/target/kb/traffic_allocation_in_activity_in_target.html

alexbishop · Answer

Is it only in this activity that you've seen an unequal traffic split? Have you got Target integrated to an analytics platform where you could see what the distribution looks like? I'm just thinking if there is some sort of error occurring for users on certain browsers/devices for the Control version, for example.

I'd be tempted to run an A/A test with no changes, just to rule out a more fundamental error with traffic allocation/implementation

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded