Batch campaign run to enrich & update 200K+ records | Community
Skip to main content
Nina_Valtcheva5
Nina_Valtcheva5
Level 4
Mar 16, 2016 4:05 PM UTC
Question

Batch campaign run to enrich & update 200K+ records

We have a new field (Leadspace Architect score) that I need to enrich all marketable leads in our database with which is over 200K. I'm concerned about breaking the SFDC sync or causing issues if I set up a campaign to enrich all leads at once. So instead, I'm thinking of updating 20K leads in one campaign run, so then I need a way to ensure the first campaign run has finished before the next run starts to update the next 20K. What is the best way to set this up?

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

    6 replies

    Ed_Dunn
    Ed_Dunn
    Level 2
    Oct 25, 2019 4:56 PM UTC

    is there a way for Marketo support to change MKTO to SFDC sync to use 100 per API call rather than 200 records per API call? This would get around the Apex CPU time limit exceeded and System.LimitException: errors

    Translate
    Grégoire_Miche2
    Grégoire_Miche2
    Level 10
    Mar 16, 2016 10:11 PM UTC

    Hi Nina,

    There is no specific reason why you should not launch the field update all at once. Anyhow, Marketo will not perform that update in 5 minutes, not even in 10 not 15. So their will be many Marketo-SFDC syncs during the update, in parallel. Furthermore, Marketo-SFDC sync does not sync thousands of leads at once, it does it with smaller batches.

    Of course, pushing all this data to salesforce WILL take time, but if there are no other large transaction at the same time, Marketo will handle it. The main drawback is that, if you have other leads that need to be pushed to salesforce at the same time, they will pile up in the backlog and will have to wait. This might be a problem is this other lead needs to be pushed to SFDC rapidly.

    Bottom line is: you would rather do it over a week end to make it smoother. You can also do it on a first batch of 20k and measure the time it takes. Then decide if you can process the rest in 1 batch or divide it in smaller batches.

    -Greg

    Translate
    Robb_Barrett
    Robb_Barrett
    Level 10
    Mar 16, 2016 8:41 PM UTC

    I believe SFDC only allows 100 updates in a single job. We see this happen a lot when we have a program that updates large batches of leads. Here's a link to the error you'll receive: System.LimitException: Too many SOQL queries: 101

    https://help.salesforce.com/apex/HTViewSolution?id=000181404&language=en_US​

    You might want to look at doing the update in SFDC instead of Marketo.

    Translate
    Robb Barrett
    Grégoire_Miche2
    Grégoire_Miche2
    Level 10
    Mar 16, 2016 10:05 PM UTC

    Hi Robb,

    Sorry, but the limit you are referring to has nothing to do with the volume of leads to be updated. It's about the number of distinct queries fired in a single transaction. You are getting this error because there are too many triggers that are fired as a consequence of an update. It can also be due to the fact that some of your triggers are not correctly developed to accommodate multiple lead updates in a single transactions.

    More doc about the Governor limits in SFDC here: Salesforce Developers

    -Greg

    Translate
      Robb_Barrett
      Robb_Barrett
      Level 10
      Mar 16, 2016 10:11 PM UTC

      Hmmm.  We had a campaign that wanted to update a field on a couple thousand leads and we hit that error.

      Translate
      Robb Barrett
      Terms & ConditionsAccessibility statement

      Loading footer...