​@teej Can you explain more what you mean by that? I’m not sure what you’re referring to.

hi Kaila, thx for reaching out. it was kind of a technical DB question about AI data specifically. i was talking about the Custom Data Store data model including a vector index of some kind, which i assume your team built in house, so you can store AI data basically and so semantic and sentiment analysis that’s more nuanced than what regular analytics, or ML, can do. i’m guessing your team built a vector index into the Custom Data Store using open source tool like FAISS (Meta), or another one. Is that an easy answer to get or do you have an Engineer I can ask? thx

Perhaps a question for the Klaviyo Developer group :)

The Data Store integrates with Vector Stores as either an extension (e.g., pgvector for PostgreSQL) or through partnerships with standalone vector databases, depending on the implementation.

Thanks for responding George. So Klaviyo needs customers to provide vector store? I’m learning this but still a bit confusing. My understanding is custom data store is a unstructured doc store/data lake essentially provided by Klaviyo for capturing customer eCommerce data, so an in house DB from which Klaviyo can run analytics insights, is that right? i’m curious why they don’t have their own way to do similarity search natively, partnership with Weavaite or Pinecone type full VDB vendor. Oracle built Vector data model into their 23ai DB so it can be built but it’s a lot of work, i’d imagine Klaviyo has a partner, using tools like FAISS or something, Any thoughts?

Databases handle vector stores using specialized structures and indexing techniques designed for high-dimensional data, commonly used in applications like similarity search, embeddings, and AI-driven retrieval.

Here's how it generally works:

1. Storage: Vectors (e.g., from text, images, or audio embeddings) are stored in a database table or collection. Each row typically includes a unique ID and a vector field (often as arrays or binary blobs).

2. Indexing: Traditional indexes like B-trees don’t work well with high-dimensional vectors, so databases use specialized vector indexes such as:

   • FAISS (Facebook AI Similarity Search)

   • HNSW (Hierarchical Navigable Small World graphs)

   • Annoy (Approximate Nearest Neighbors)

   • IVF, PQ (used for compressed or partitioned search)

3. Search: Instead of exact match, vector search looks for nearest neighbors based on metrics like cosine similarity, Euclidean distance, or dot product.

4. Vector Databases: Some databases are built specifically for vectors (e.g., Pinecone, Weaviate, Milvus, Qdrant), while others (like PostgreSQL with the pgvector extension) add vector support.

5. Performance: Vector databases prioritize speed and scalability in similarity search, often supporting approximate search for faster results at scale.

Would you like to see an example using a specific database like PostgreSQL or Pinecone?

Maria, appreciate the insights, very through. Do you have a sense for what Vector store Klaviyo, the company it’s self, uses? I am not referring to customers using Klaviyo data in this case, it’s the Data Store itself. I’d imagine that Klaviyo does it’s own internal analysis using a VDB of some kind. Any thoughts on what they are likely using, is is FAISS, pgvecgor or SCann etc?