How can I "Hold Out" 20% of Leads from receiving the emails within a program? | Community
Skip to main content
R
RonnBu
New Participant
August 16, 2022
Question

How can I "Hold Out" 20% of Leads from receiving the emails within a program?

  • August 16, 2022
  • 1 reply
  • 2264 views

I have an Engagement Program that needs to be cloned and updated for a new 1-year period. This process needs to be duplicated each business year as we provide health insurance and this Retention program is built to send reminder email communications beginning at 105 days prior to renewal date and every 15 days (105, 90, 75, 60, 45 & 30) until the member reaches their renewal date.

 

I would like to now "Hold Out" 20% of all members and only send 80% of our members through this Retention nurture.

 

Goal: Randomly exclude 20% of qualifying records from receiving emails within this specific Engagement Program.

 

Purpose: To measure the effectiveness of our emails by comparing the 80% of qualified records that received the nurture versus the 20% that did not.

 

Will what I have built below utilizing "Random Sample" work to achieve this and/or what is the best way I should solve for this? 

 

Thanks in advance!

 

Best,

Ronn

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

1 reply

Michael_Florin-2
Michael_Florin-2
New Participant
August 18, 2022

We did hold-out groups or no-nurture groups or any other testing that you want to conduct over a longer period of time using "Random Cohorts". We just called it that - some MATs had this already built in like Silverpop/IBM Campaign.

 

It was essentially a Marketo Custom field which was populated by Random Sample flow steps the moment a lead was created in Marketo. It held values between 0 and 9, so it looked something like this:

 

That field was also blocked from all updates.

 

Now that you have your database tagged with these values, it's easy to suppress "Random Cohort = 0 OR 1" from whatever you're doing and you have an unchangeable flag on your hold-out group of 20% and you can monitor this cohort long term.

 

    R
    RonnBuAuthor
    New Participant
    August 18, 2022

    That's really smart! This is a very useful method to implement and utilize. That's bigger picture. I'll need to think about creating a program to batch and cohort tag the existing 20 million records. I like it and will definitely look to implementing in the near future.

     

    As for my initial question on this program specific random sample - will my logic work perfectly fine as I have it in the flow?

     

    Thank you so much!

     

    Michael_Florin-2
    Michael_Florin-2
    New Participant
    August 18, 2022

    20 million records - I feel you. 🙂 I backfilled this value for 45 million, IIRC.

     

    I'm not sure where I read that, but I tried to not exceed 2 million records per batch. And I went through the whole database alphabetically in chunks of "Email Address starts with A"... and so on. Marketo processes 1-2 million records with one Change Data Value in an hour or two, and I tried to use off times and not run too many batches at the same time to not clog the arteries of batch campaign processing.

     

    In a couple of days you'll be done with it.

    Terms & ConditionsAccessibility statement

    Loading footer...