Postgres or a Vector Database? A Framework for Choosing

Every RAG project we start now includes the same conversation early on: does this need a dedicated vector database, or does pgvector inside the Postgres instance we are already running handle it? The honest answer is that most teams reach for a dedicated vector database before they need one, and a smaller number wait too long to switch off pgvector once they actually do.

Neither choice is free. Here is the framework we actually use.

Start with pgvector unless you already know why not

If you are already running Postgres for your application data, pgvector removes an entire piece of infrastructure from the system: no separate service to deploy, monitor, back up, and keep in sync with your source of truth. For corpora up to a few million vectors with moderate query volume, pgvector with an HNSW index performs well enough that the "dedicated database is faster" argument does not hold up in practice for most workloads we see.

The bigger win is consistency. When your embeddings live in the same database as the records they describe, you get transactional guarantees for free: update a document and its embedding in the same transaction, filter by metadata using the same SQL you already know, and never worry about a sync job silently falling behind.

Where pgvector actually falls short

The limits show up predictably. Query latency degrades as the corpus grows past the tens of millions of vectors, especially under high concurrent write load, because you are sharing resources with your transactional workload. Filtered search across high-cardinality metadata gets slower than purpose-built vector databases that index metadata and vectors together. And if you need approximate nearest neighbor search tuned aggressively for sub-10ms latency at massive scale, a dedicated system built for exactly that will out-perform a general-purpose database doing double duty.

We moved a client off pgvector onto a dedicated vector database exactly once in the last year, when their corpus crossed roughly 40 million embeddings and query latency under load had climbed past what their product experience could tolerate. Everywhere else, pgvector has stayed in place, including on the RAG work we describe in RAG Is Not a Feature. It Is a System. — the retrieval quality problems we ran into there were about evaluation and reranking, not the vector store itself.

The questions that actually decide it

• Is your corpus size in the low millions or below? Default to pgvector.
• Do you need complex filtered search across many metadata fields at low latency? Lean toward a dedicated store.
• Is operational simplicity worth more to your team than shaving milliseconds off retrieval? Default to pgvector.
• Have you actually measured retrieval latency in production, or are you optimizing for a scale you do not have yet?

That last question matters more than the other three combined. We have seen more engineering time wasted migrating to a dedicated vector database ahead of an actual scaling need than we have seen production incidents caused by staying on pgvector too long. Build the evaluation set first, as we argued in the RAG piece. Let the actual retrieval quality numbers, not the fear of hitting a wall you have not hit yet, decide when it is time to add a new piece of infrastructure.