What this prompt does
This prompt drives an AI assistant to produce a complete, opinionated RAG architecture — not a generic overview, but a system spec tailored to your actual use case, corpus size, and accuracy target. The ten-section structure mirrors what you would actually spec in a design doc: from raw document parsing through embedding model selection, vector store configuration, retrieval strategy, prompt design, and finally a working code skeleton. Nothing is skipped because every section feeds the next.
The reason it works is the combination of specificity and tradeoffs. By requiring justification for embedding model choice (cost vs quality), chunk strategy (semantic vs fixed vs recursive), and retrieval method (hybrid + MMR), the template forces the model to reason rather than list. The [accuracy_target] and [optimization_priority] variables are the levers — setting accuracy to 95%+ with latency priority produces a different architecture than 80% with cost priority, as it should.
The citation format requirement in section 6 and source attribution in section 7 are load-bearing — they prevent the response from quietly dropping provenance, which is the single biggest production failure mode in RAG systems.
When to use it
- You are building an internal knowledge base assistant over engineering runbooks, Confluence pages, or Notion docs and need to decide between Pinecone, pgvector, and Qdrant before writing a line of code.
- You are designing a legal or compliance Q&A system where hallucinated citations are a liability, and you need faithfulness guardrails baked into the prompt layer.
- You are evaluating chunking strategies for a corpus of PDF technical manuals where tables and code blocks break naive fixed-size splitting.
- You are scoping a customer support bot and need a realistic cost estimate before pitching it internally.
- You are migrating an existing keyword-search system to hybrid search and need the re-ranking step specified correctly.
Example output
Chunking Strategy (for 50k engineering docs, mixed PDF + Markdown):
- Method: Recursive character splitting with semantic boundary detection
- Chunk size: 512 tokens, overlap: 64 tokens
- Rationale: Fixed-size at 256 tokens produced measurably lower recall on
multi-paragraph answers in testing; semantic splitting added ~340ms/doc
ingestion latency with only marginal recall gain — not worth it at this
corpus size.
Embedding Model: text-embedding-3-small (OpenAI)
- Cost: ~$0.02 per 1M tokens vs $0.13 for text-embedding-3-large
- Quality delta on retrieval benchmarks: small vs large gap is real but
typically under 5% on domain-specific corpora — start small and measure
before paying 6x more.
- Recommendation: upgrade only if faithfulness drops below 0.85 in evaluation
Vector Store: pgvector on existing Postgres
- Index: HNSW (m=16, ef_construction=64)
- Metadata filters: doc_type, department, last_updated
- Rationale: eliminates a managed vector DB dependency at < 5M vectors
Pro tips
- Set
[accuracy_target]as a number, not a word. "High accuracy" generates vague architecture. "Faithfulness > 0.90 on RAGAS" forces the model to include an evaluation pipeline that actually measures it. - Be explicit about
[doc_formats]. If your corpus has PDFs with scanned pages, name that — the ingestion section will include OCR routing, which is a real pipeline branch many architects skip until it breaks in production. - Use
[optimization_priority]as a constraint, not a wish. "Minimize per-query cost under 50ms p95 latency" produces a concrete trade-off analysis. "Fast and cheap" produces nothing useful. - Pair with a separate evaluation prompt. The section 8 metrics spec is a starting point — run it again with a dedicated prompt asking for a RAGAS test harness in your language to get runnable evaluation code.
- Re-run with a different
[vector_db]after the first output. Getting the architecture for both pgvector and Qdrant takes two minutes and often reveals the managed DB is not justified at your corpus size — a decision worth making explicitly.
