What this prompt does
This prompt asks the AI to build a semantic search engine over [content_type] content of about [corpus_size] documents. It defines the full pipeline: chunking and metadata extraction for [doc_format], embedding generation with [embedding_model] in batches, a vector index in [vector_store] with the right dimension and distance metric, and hybrid search that fuses vector similarity with BM25 keyword scoring. A re-ranking pass using [reranker] refines results, and query understanding handles expansion, typos, and intent.
The structure works because plain keyword search and plain vector search each miss cases the other catches, and combining them with a re-ranker is what beats either alone. By naming [embedding_model], [vector_store], and [reranker], you get a concrete, swappable stack instead of hand-waving. Faceted filtering on [filter_fields], passage highlighting, zero-result analytics, and evaluation against [eval_metric] mean you can prove the search is actually better, implemented in [language]. The chunking strategy for [doc_format] also matters more than it looks: chunk too coarsely and embeddings blur multiple topics together, too finely and you lose the surrounding context that makes a passage meaningful, so the prompt ties chunking to the document format and extracts metadata alongside each chunk.
When to use it
- You are adding search over docs, a knowledge base, or product data and keyword search keeps missing intent.
- You want hybrid retrieval plus re-ranking rather than betting everything on embeddings.
- You need faceted filtering on fields like
[filter_fields]alongside relevance ranking. - You want to measure search quality with
[eval_metric], not just ship and hope. - You care about zero-result queries and click-through analytics to guide tuning.
- You need an API with autocomplete, did-you-mean, and pagination ready for a frontend.
Example output
Expect an indexing pipeline (chunking, embedding, upsert into [vector_store]), a search API in [language] exposing hybrid search with re-ranking, faceted filters over [filter_fields], highlighting, and pagination. You also get analytics hooks for search terms and zero-result queries, plus an evaluation harness scoring [eval_metric] like NDCG@10 and MRR.
Pro tips
- Match
[embedding_model]dimensions to your[vector_store]index config; a mismatch is the most common setup error and fails silently or loudly. - Tune the chunking strategy to
[doc_format]— Markdown headings make natural boundaries, while flat HTML may need a token window. - Treat
[reranker]as the biggest quality lever; if results look close-but-wrong, the re-rank step is usually where to push first. - Build a small labeled query set early so
[eval_metric]numbers mean something; without ground truth, NDCG is just a number. - Watch zero-result queries from day one — they reveal vocabulary gaps and missing synonyms faster than any benchmark.
- Balance hybrid weighting deliberately; over-weighting BM25 reverts you toward keyword search, while over-weighting vectors loses exact-match precision.
