Pulling Data for analytics via API or 3rd party + Modelling data and enhancing segments

Hi, we had a similar issue with needing to migrate metrics / profiles between accounts. We ended up developing our own extraction tool extracting JSON data to ETL in to another data store and Klaviyo account. Unfortunately, for us the generic tools resulted in too much loss of data fidelity. Klaviyo’s flexibility worked against us. 

Happy to share more information if this is what you are after.

Derek Giles - TranslatorHQ  

Hi Derek,

Thank you for sharing! I would love to hear more.

Which tools did you try out before building your own?

What data storage do you use now and what do you use for anlytics?

Best,

Mathilde

Hey - @Mathilde  - we done the same as @TranslatorHQ . we tried Fivetran and Rivery but they were either too expensive or not stable. We ended up building our own solution for syncing klaviyo events\metrics with an ETL process.
there are some edge cases needs to be taken care of, most of them related to rate limiting and large events in very small timeframe (i.e million email events at midnights).
we use AWS redshift Kinesis, though not 100% happy with redshift …

would be great if klaviyo can add more webhooks for all types of events..

Hi @unique-username-007

Thanks for sharing. What are the reasons redshift is not optimal for you?

Yes, more webhooks would be great! Even though there are some more capabilites in the CDP it’s still way too limited to be worth the price point, sadly.

i think it varies by the use case, but there’s almost no support to deduplication of data, and also there’s weekly maintenance windows that can create 30 minutes of downtime which is not great for streaming solutions\production use cases

@unique-username-007 I see, data cleaning is definitely something important for my choice, so thats good to know!

what worked well for us is actually to do start with some 3rd party vendor like https://rivery.io/ so we can get some data and play with it quickly in relatively high amounts of data. it allowed us to be smarter when we actually build our own solution…

anyway, good luck! 

@unique-username-007 Thank you, yes that makes sense!