Building a RAG Chatbot with FastAPI and ChromaDB (that runs locally, no API key)

Short summary

An open-source RAG chatbot starter using FastAPI, ChromaDB, and SentenceTransformers that runs fully locally with Ollama—no API key required. The article covers the RAG pipeline (indexing and querying), key design decisions like chunk size and re-ranking, and common pitfalls such as scanned PDFs needing OCR. The full repo is MIT-licensed and deployable via Docker Compose.

• •Open-source RAG starter with FastAPI, ChromaDB, and Ollama runs locally with no API key
• •Covers chunk sizing, OCR fallbacks for scanned PDFs, and cross-encoder re-ranking for better retrieval
• •MIT-licensed repo deployable via single Docker Compose command with multilingual support

Generated with AI, which can make mistakes.