Hi community,
Background:
I’m reaching the rate limit on profiles and events due to high activity. Currently, I use the synchronous single request, but the activity is higher than the allowed rate limit (forcing us to move to bulk api). I must point out that on initial sync, I’m already using bulk, but the ongoing activity hit the rate limit, so the solution is to stop sending single requests and use bulk requests (enqueue the requests and send them together in the bulk request).
Question:
Now, how can we keep the order of the requests? We don’t want the newest request to be processed before an older one, otherwise, the final data on the profile will not be accurate (it will have old data).
Furthermore, I see in the Bulk Import Profiles API regarding “Ordering and duplicates” that Processing order is not guaranteed.
In the profiles update use case, it’s simpler because we can avoid sending duplicates (send only the oldest). But in the events, we need to send all (even duplications, if, for example, 2 purchases were made).
I don’t see in the documentation of the Bulk events any info regarding the asynchronous impact on the order. I do see that they offer a time filed for historical events.
So the question is, what exactly the time field in historical events works? If I send 3 events on the same profile (that include also profile properties, and add a time, does the profile will end with the property values of the event with the oldest time? Or it will have the values of the last processed event? (Or it knows to process them order by time filed?).
Second question, is the time field available also for the Bulk profiles? (seems that the answer is no based on the body params in the api doc.
To conclude, it will help a lot if you could help to clarify how the time field works in the bulk events request, and in general, if someone has an insight into how to overcome the ordering issue in events, in a way that the profile will end with the most up to date data I will very appreciate it. Thanks!