Use of Bulk API Export vs REST API | Community
Skip to main content
Jess_Ramos
Jess_Ramos
New Participant
February 11, 2021
Question

Use of Bulk API Export vs REST API

  • February 11, 2021
  • 1 reply
  • 4616 views

Hello! 


We are currently using the BULK API export to fetch update lead data for our Tableau dashboard.  We have ~600K lead records and 300 fields. 

 

We are using the python library 'marketorestpython' to make calls to the marketo API.  The python script fetches all the leads starting from June 2017 up until now and gets all the fields that are present in Marketo.

 

The script runs on weekends and we get the 500 mb limit exceeded error when the script has fetched all the leads ranging from July 2017 to about August 2020. We want to capture any changes and deleted records in this refresh. Not all fields or leads will have updates though. 

 

Marketo support has directed us to use the REST API instead of the BULK API for this process. Our developer is concerned that doing so will be too slow and create delays in refreshing our lead data in Tableau.  Does anyone have an opinion or similar experience that could help guide us in our best path forward here?  

 

Thank you!!!

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

1 reply

SanfordWhiteman
SanfordWhiteman
New Participant
February 12, 2021

If this is a one-time backfill — not representative of your ongoing weekly volume — why not continue to use Bulk Extract, but do it in parts?

 

You can also purchase more daily BE transfer volume, even on a temporary basis. That'll be more efficient than rewriting to use Bulk Export.

Jess_Ramos
Jess_RamosAuthor
New Participant
February 12, 2021

Thanks for the reply Sanford.  It's not a one time back-fill. It's an ongoing weekly refresh we do of our lead data table for our Tableau Dashboard. We are consistently hitting the bulk API limit and Marketo recommends we use REST API for these ongoing updates -- we are told by our developers it might be a very long update process. I'm not sure if that is a valid concern and not getting a clear answer from support. 

SanfordWhiteman
SanfordWhiteman
New Participant
February 12, 2021
Why do you have to go back to 2017 every week? That doesn't make sense.
Terms & ConditionsAccessibility statement

Loading footer...