So you wanna build a local RAG?

Here's the thing: deploying a local retrieval-augmented generation (RAG) setup in just 8 minutes sounds appealing, but the trade-offs can be significant. Skald's reliance on Postgres with pgvector for vector storage is a bold choice, especially when you consider the competition. Compared to dedicated vector stores like Pinecone or Weaviate, Postgres might limit your scalability as your document count grows. It's great for hundreds of thousands of documents, but what happens when you hit that ceiling? You might find yourself migrating later — and that’s a headache no one wants at 2 AM.

The flexibility to configure your own LLM and reranker is a double-edged sword. On one hand, it gives you control over your stack. On the other, it adds complexity that could overwhelm teams used to simply calling APIs. If you're not ready to manage your own models, you may end up with a setup that’s suboptimal or even underperforming compared to managed alternatives.

To be clear: while Skald offers the promise of privacy and self-hosting, it’s critical to scrutinize the benchmarks when they’re released. The current claims of performance do not provide enough context on how they compare to established players in the space. Without rigorous benchmarking, it's hard to gauge whether Skald can truly stand toe-to-toe with tools like Faiss or Langchain.

In essence, if you're a team that values privacy and is comfortable managing a more complex stack, Skald is worth considering for evaluation. However, if your priority is immediate performance and operational simplicity, you might want to look elsewhere — at least until Skald demonstrates solid benchmarking results and a clear competitive advantage over the established alternatives. Don’t rush into adopting something that may not yet be ready for production workloads without adequate verification of its capabilities.