A

Anonymous

Solved

Is there anything I can do to speed the getMultipleLeads API call?

Forum|Forum|13 years ago
January 17, 2013
16 replies
2534 views

I'd like to stay current on lead changes, so I've set up an app to call getMultipleLeads on an hourly basis, with a last modified threshold of an hour. Or half an hour. Or even ten minutes - it still usually takes over an hour to get any results. If I pull as little as the last three minutes' updates, this may complete in just five minutes. Obviously at that rate I won't be able to keep current.

We do have a fairly high volume of touches, as our data is synched with Salesforce on a near-real time basis. Still, a report in the web GUI for all leads updated in the last hour returned "about 377" records (though it took several minutes to return the first page of results). Surely 377 records shouldn't take minutes to process!

What can I do to keep current on lead changes? Is there a way to speed getMultipleLeads, or is there a better approach I should be using?

To be clear, the span of time I'm measuring is literally the length of time for the getMultipleLeads method to return. I'm using very verbose logging to pin this down.

Marketo

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by

You can't do UI Smart Lists, but you can use recurring batch campaigns to populate the static lists.

Are you pulling all the activities for each lead? That is probably the core area where we can speed this up. There is a getLeadChanges API call which gives you new activities. Many folks use that for capturing what is needed. Even better, filering out some activity types will help. This is probably the most important thing to look at.

You can have multiple connections to the Marketo API at once, although keep it in the lower numbers (less than 5).

-Erik

A

Anonymous

We're able to keep up with our volume in near-real time by running these queries twice an hour:

Get new leads (with a time horizon of 1.05 hours), max 10K records. This uses the getMultipleLeads() API method, using a LastUpdateAtSelector.
Get recent activities (with a time horizon of 1.05 hours), max 10K records. This uses the getLeaChanges() method and a limited set of activity types.
Get activities by lead, max 500 records. This uses getLeadActivity(), and checks specific lead IDs on a rotating basis.

A

Anonymous

D'oh. I meant Marketo.

A

Anonymous

I'm not familiar with Eloqua. Are you sure you're in the right forum?

A

Anonymous

Hi Jon, were you able to make any headway into speeding up your use of the Eloqua API? What did you end up doing? We're also trying to monitor all updated data fields and are also trying to find ways to cope with the slow API performance. Thanks in advance for any wisdom you can impart.

A

Anonymous

I did look at LeadSelector, however it's an abstract class, not instantiable. However, the LastUpdateAtSelector is usable, and it exposes latestUpdatedAt and oldestUpdatedAt fields. Very promising, I will test this!

A

Anonymous

You have to put it in the leadSelector object:

<xs:extension base="tns:LeadSelector">

<xs:sequence>
<xs:element name="latestUpdatedAt" type="xs:dateTime" minOccurs="0" maxOccurs="1" nillable="true"/>
<xs:element name="oldestUpdatedAt" type="xs:dateTime" minOccurs="1" maxOccurs="1" nillable="false"/>
</xs:sequence>

</xs:extension>
	
	We support the non-LeadSelector option so the old API calls still work against the new endpoint.

A

Anonymous

That would be groovtacular, but the WSDL marked "2_0" does not appear to offer such a filter. Here's the XML for that method's parameters struct:

<xs:complexType name="ParamsGetMultipleLeads">

<xs:sequence>

<xs:element name="leadSelector" type="tns:LeadSelector" minOccurs="0" maxOccurs="1" nillable="false"/>

<xs:element name="lastUpdatedAt" type="xs:dateTime" minOccurs="0" maxOccurs="1" nillable="false"/>

<xs:element name="streamPosition" type="xs:string" minOccurs="0" maxOccurs="1" nillable="true"/>

<xs:element name="batchSize" type="xs:int" minOccurs="0" maxOccurs="1" nillable="true"/>

<xs:element name="includeAttributes" type="tns:ArrayOfString" minOccurs="0" maxOccurs="1" nillable="false"/>

</xs:sequence>

</xs:complexType>

Being able to filter both start and end dates, and by date created or by date modified, would be very useful.

A

Anonymous

Well, can't you specify both start and end dates for getMultipleLeads? We added that to the 2.0 version of the WSDL. The various activity calls don't have that option yet, but you can at least try getting old leads that way.

A

Anonymous

The getLeadChanges method has been working well for me in keeping current, thank you.

However, I'd also like to backfill old data, which entails calling getMultipleLeads with an early lastUpdatedAt param. Unfortunately, this method takes forever to return the first rows for a large query, so there's no opportunity to break up the query into separate threads before it times out.

Have you successfully used the multi-threaded approach you mentioned? If I could specify both start and end dates, it wouldn't be a problem, I'd pick up data one week at a time (or whatever date range was manageable).

A

Anonymous

An auto-updated static list sounds like a good option, though if I can get adequate performance using other calls or multiple threads that would be even better. I'd prefer not to have to choose a population of leads and assume the rest are not interesting.

Ah, so getLeadChanges actually gets recent activities across all leads? That sounds like exactly what I need! Probably paired with separate calls to get new leads. I've set up some draft code to test this.

Yes, I'm filtering activity types. I'll work with the marketing group to see if we can pare down what they want to monitor.

I'd not noticed that API limit, but I think you're right, getLeadChanges will address that nicely.

Show more replies

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded