API, events delay

Hi @MonocleTech!

When you say some events do not get returned immediately, how many do get returned? For this endpoint, there is a maximum limit of 200 events per page. To get more than 200, you would have to paginate through the API response, gathering 200 events at a time using cursor-based pagination. 

Also, do you have a date-time filter in place? This would inherently affect the number of events that are passed in the response. Overall, it will help myself and other Community members understand the issue better if you would be willing to explain a little more about which events are being excluded from the response.

Best,

Brian

Hi @Brian Turcotte 
I’m using the python package to call the api like so:
 

kapi= KlaviyoAPI(api_key=store_key, max_delay=60, max_retries=10) 
 for....
          if next_link:
                return kapi.Events.get_events(
                    page_cursor=next_link,
                    filter=klaviyo_filter
                )
          ...

Where klaviyo_filter is for example:

'less-or-equal(timestamp,1685980368.936), greater-than(timestamp,1685980068.936),equals(metric_id,"HVD8JD")'

I do this for all the metrics in the store, I noticed the pagination and am addressing it in the rest of my script.

In this example I ran the same code 4 and 5 minutes after the end timestamp,
and I received different results, where the later run is bigger and this is the difference:

total num of events in this call: 564, new compared to running 1 minute earlier:
diff ('SrTRSL', 2) Closed Form
diff ('WAdXL2', 3) Received Email
diff ('Yscz2Q', 2) Viewed Form

also for a different store:
total num of events in this call: 12221, new compared to running 1 minute earlier:
diff ('HVD8JD', 1) Screen View
diff ('HVZkCC', 6) Received Email
diff ('QGJcz3', 1) Session End

As my use case requires events in a timely manner, and ideally all events,
I am waiting at least 30 minutes after the ending timestamp before running it,
I wanted to understand more about the situation to know better if my 30min decision is optimal.
Thanks

Hi @MonocleTech,

Would you be willing to try adding a sorting method to your API call, that lists the events from most recent to least recent? Then, you can try the same call shortly after, and if the top event is different, that will confirm that some events are delayed.

We would expect events to be syncing in near-real time, but there may be cases where events are delayed several minutes. The 30 minute period would likely be sufficient, but this also depends on what type of metric you are syncing. Native Klaviyo events are synced almost immediately, and would only be delayed several minutes in rare cases. However, if these other metrics you named like “Closed Form” and “Viewed Form” are custom, they may not adhere to the same syncing protocols, which would explain the delay.

I hope this helps to get you started, and thanks for using the Community!

- Brian

Yes the new events seem to arrive at the top, being most recent, or equally most recent to others

@MonocleTech

May I ask which events are at the top? Are they the “Closed Form” and “Viewed Form” events, and others that are custom/not native to Klaviyo?

That would confirm what I suspected earlier, that these custom events are synced/sent to Klaviyo differently than the native events, causing the delay that cannot be remedied from within Klaviyo. Therefore, to adjust for the delay, you’d either have to adjust the custom events from your own backend, or experiment with the time delay until you’ve established that sync time. 

Best,

Brian

Yes these 2 are among the most often delayed events along with ‘Received Email’, across stores.
There are some other delayed events that are rare events so it’s hard to immediately tell how often they are delayed .
I don’t personally have knowledge about what is custom/not native.