How does "Random Sample" work? | Community
Skip to main content
P
PhillipWi1
New Participant
May 28, 2015
Solved

How does "Random Sample" work?

  • May 28, 2015
  • 5 replies
  • 22957 views

I've seen "Random Sample" in Flow steps when you use "Add Choice". But...is this self contained random samples, or is it reducing the amount of leads flowing through as it goes?

As an example:

Choice 1: If Random Sample is 33%, send email A

Choice 2: If Random Sample is 33%, send email B

Default: Send Email C

Would this work? Does the second random sample start afresh from the full amount of leads, or is it smart enough to remove anyone who qualified for Choice 1?

My theory is that it will remove as it goes, which means you would need to adjust the percentages. So if you started with 100 people:

* 33 people qualify for Email A and get that send. 67 remain.

* 33% * 67 = approx 23 people then qualify for Email B and get that send.

* The remaining 100 - (23 + 33) = 44 people will get Email C.

So if you wanted to get three splits of 33% each, the second choice should say 50%.

This is basically just a more difficult way to do an A/B test through that functionality, so I wouldn't necessarily use it like that - but I'd love to know 100% how the functionality works so I can think about how to use it.

Thanks, Phil

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by

    Hi @Phillip Wild​!

    I struggled with this at first too. You have the right idea though.

    With the random sample flow step, you'll need to be sure that all choices equal 100. In your example above, Choice A would get 33%, Choice B would get 33% and Choice C would have to get 34%. Also, you will need to add the various sample choices to a static list instead. You are unable to directly send the email in the random sample flow step. Once you add the members of each sample to a list, you create the email send flow step and add the constraint "member of list".

    You may want to add a small wait step depending on the size of the send. I typically like to add a 2 minute wait step.

    Hope that helps! Let me know if you have questions.

    5 replies

    Marisa_Rybar
    Marisa_Rybar
    New Participant
    September 18, 2017

    Hey everyone,

    I have a couple questions on this. I'm going to use this "random sample" for a different use case so I will need to keep this outside of an email flow step.

    Essentially what I want to happen is that there are 7 states which are going to comprise a new shared sales territory that will have a completely different assignment rule than the rest of our territories. This is what needs to happen:

    • Sales Team 1 should receive 32% of incoming leads in this territory
    • Sales Team 2 should receive 26% of incoming leads in this territory
    • Sales Team 3 should receive 22% of incoming leads in this territory
    • Sales Team 4 should receive 21% of incoming leads in this territory

    The way I'm thinking this needs to be set up is that I have a smart campaign with a trigger of "person is created" and the states defined as filters, then the flow steps set up with the random sample as established above.

    Or does the random sample flow step require a static list? In which case I would imagine the way it would need to be set up would be that I'd have a smart campaign listening for when new people are created in this shared sales territory, with a flow step to add them to a static list. Then, I'd have a second smart campaign set up with a trigger "Is Added to List" and the flow steps set up with the random sample as established above.

    My other question though is this. Will this random sample work as a triggered campaign on a static list as it grows, or can it only work once on a static list through a batch campaign? (In other words, will the random sample flow step recognize the changes in list size and adjust the %'s accordingly?)

    @Josh Hill​?

    A
    Anonymous
    July 16, 2015

    The trouble I have found (and Marketo Support confirmed this), is that if you add in choices other than the random steps, it can throw the randomness off.

    For instance, this was the flow step that prevented the random steps from working properly.

    R
    rachazen
    New Participant
    March 10, 2016

    Greg, can you please elaborate on why this didn't work properly? When I look at your screen shot, this is what I assume would happen:

    1. All active clients will NOT have anything happen
    2. All BDR reps will NOT have anything happen
    3. 33% of the remaining leads will receive the BDR Rep value ending with UZIA0
    4. 33% of the remaining leads will receive the BDR Rep value ending with GXiIAM
    5. Anybody left over (34%) will receive the BDR Rep value ending with TIAQ

    Please correct me if I'm wrong or if it doesn't work that way. I would like to know what actually happens so that I can make sure my flow steps work properly.

    Thanks!

    A
    Anonymous
    March 11, 2016

    Random works best when there are no other choices to deter it. So by having choices above it (1 and 2) that are empty, throws off the randomness of the flowstep. This is what Marketo support told me.

    Justin_Norris1
    Justin_Norris1
    New Participant
    May 28, 2015

    @Phillip Wild​ My experience is the same as @Anna Bruning​ in that you can use random sample directly in the "Send Email" flow step. Christina Reynolds , where did you encounter that restriction?

    The random sample divisions are based on the original total as Christina Reynolds described. Each percentage is a percentage of the original 100% and the "else" option sends to the balance of the sample after the previous percentages are removed.

      A
      Anonymous
      May 29, 2015

      Thanks @Justin Norris​ & @Anna Bruning​ for pointing that out! We first started using "random sample" last year for A/B testing in default programs, and we weren't able to add it as a choice in the email flow step for a default program. I had never gone back to look at it again (stuck in an old habit I guess). I saw your comments this morning, so I tested it out and IT WORKS! Hooray! Thank you for the comments.

      @Phillip Wild​ - Skip the add to list flow step and send straight from the send email flow step!

      Send Email

      Choice 1: If Random Sample is 33, Email A

      Choice 2: If Random Sample is 33, Email B

      Choice 3: If Random Sample is 34, Email C

      P
      PhillipWi1Author
      New Participant
      May 29, 2015

      That makes sense! It seemed weird that the option was there but I couldn't use it.

      Thanks all.

      Phil

      Anna_Bruning1
      Anna_Bruning1
      New Participant
      May 28, 2015

      Christina Reynolds​, are you talking about email (mailbox) programs? I know you can't use random samples in the smart lists for those programs, but I have used random samples in flow steps of regular programs to send emails so I can confirm that it works.

        A
        Anonymous
        Accepted solution
        May 28, 2015

        Hi @Phillip Wild​!

        I struggled with this at first too. You have the right idea though.

        With the random sample flow step, you'll need to be sure that all choices equal 100. In your example above, Choice A would get 33%, Choice B would get 33% and Choice C would have to get 34%. Also, you will need to add the various sample choices to a static list instead. You are unable to directly send the email in the random sample flow step. Once you add the members of each sample to a list, you create the email send flow step and add the constraint "member of list".

        You may want to add a small wait step depending on the size of the send. I typically like to add a 2 minute wait step.

        Hope that helps! Let me know if you have questions.

          P
          PhillipWi1Author
          New Participant
          May 28, 2015

          Hi Christina

          Interesting. So despite the fact you can add “Random Sample” to a choice step in a “Send Email” step in a flow, you still can’t actually send an email in that fashion?

          I think I know what you mean - but just to clarify, you would need to do the following in a flow series:

          • Add to List

          -Choice 1 - If Random Sample is 33%, then add to Static List A

          -Choice 2 - If Random Sample is 33%, then add to Static List B

          -Choice 3 - If Random Sample is 34%, then add to Static List C

          Wait 2 mins

          Send Email

          -Choice 1 - If Member of List Static List A, then send Email A

          -Choice 2 - If Member of List Static List B, then send Email B

          -Choice 1 - If Member of List Static List C, then send Email C

          Is that right?

          Thanks, Phil

          A
          Anonymous
          May 28, 2015

          Exactly right! That will work.

          Side note: Under the "send email" flow step listed above the last choice should be choice 3 - probably just a typo.

          Terms & ConditionsAccessibility statement

          Loading footer...