RAG Pipeline Cost Breakdown: Where the Money Actually Goes

Short summary

The author systematically priced every stage of a RAG pipeline and found that embeddings — widely believed to be expensive — cost only ~$0.18 per 1,000 pages. The real cost drivers are the vector database infrastructure (running 24/7 at scale) and LLM inference during retrieval. Document deduplication and incremental re-processing of only changed chunks can dramatically reduce ingestion costs, since ~80% of knowledge bases stay unchanged between updates.

• •Embeddings are cheap (~$0.18 per 1,000 pages) — the real cost is vector DB infrastructure and LLM inference
• •Document deduplication and incremental chunking avoid re-processing unchanged content (~80% of docs)
• •Semantic caches were over-avoided due to the false belief that embedding costs are prohibitive

Generated with AI, which can make mistakes.