Skip to content

docs(cookbook): add zero-cloud local hybrid RAG recipe with SQLite FTS5 and phi-4-mini #571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Çağrı Giray Keşan (Cagrik34) wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(cookbook): add zero-cloud local hybrid RAG recipe with SQLite FTS5 and phi-4-mini #571

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ Follow these steps to get started using these resource :

- RAG with Azure AI Search
- [How to use Phi-4-mini and Phi-4-multimodal(RAG) with Azure AI Search](https://github.com/microsoft/PhiCookBook/blob/main/code/06.E2E/E2E_Phi-4-RAG-Azure-AI-Search.ipynb)
- [Zero-Cloud Local Hybrid RAG with SQLite FTS5 and phi-4-mini](https://github.com/microsoft/PhiCookBook/blob/main/code/06.E2E/E2E_Phi-4-mini_Local_Hybrid_RAG_SQLite_FTS5.ipynb)

- Phi application development samples
- Text & Chat Applications
Expand Down
237 changes: 237 additions & 0 deletions code/06.E2E/E2E_Phi-4-mini_Local_Hybrid_RAG_SQLite_FTS5.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 🚀 Hybrid RAG with Microsoft phi-4-mini & SQLite FTS5 (Zero-Cloud SLM)\n",
"\n",
"> **Author:** Çağrı Giray Keşan ([@Cagrik34](https://github.com/Cagrik34)) \n",
"> **Focus:** Small Language Models (SLMs), SQLite FTS5 BM25, Dense Embeddings, Reciprocal Rank Fusion (RRF)\n",
"\n",
"---\n",
"\n",
"## 📌 1. Motivation: The Keyword Recall Dilemma in Local SLMs\n",
"Standard RAG architectures relying purely on dense vector embeddings often fail to retrieve exact numerical tokens (e.g., `2,340,000 TL`, contract codes, account numbers). \n",
"Conversely, sparse lexical search (BM25) misses semantic synonyms and paraphrased questions.\n",
"\n",
"This cookbook demonstrates how to implement a **high-speed, in-memory Hybrid Retrieval engine** combining:\n",
"1. **Dense Vectors** (Cosine Similarity)\n",
"2. **Sparse Lexical Search** (SQLite FTS5 BM25)\n",
"3. **Reciprocal Rank Fusion (RRF, $k=60$)**\n",
"4. **Grounded Citation Generation (`[1]`, `[2]`)** with Microsoft `phi-4-mini`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import sqlite3\n",
"import numpy as np\n",
"from typing import List, Tuple, Dict, Any\n",
"\n",
"RRF_K = 60\n",
"TOP_K = 2\n",
"print(\"✅ Core dependencies loaded successfully.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🏗️ 2. Dual SQLite Schema (Dense Vectors + Virtual FTS5 BM25 Table)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class LocalHybridRAGStore:\n",
" def __init__(self, db_path: str = \":memory:\"):\n",
" self.conn = sqlite3.connect(db_path)\n",
" self._init_schema()\n",
"\n",
" def _init_schema(self) -> None:\n",
" with self.conn:\n",
" self.conn.execute(\"\"\"\n",
" CREATE TABLE IF NOT EXISTS document_chunks (\n",
" id INTEGER PRIMARY KEY AUTOINCREMENT,\n",
" source_file TEXT NOT NULL,\n",
" chunk_index INTEGER NOT NULL,\n",
" content TEXT NOT NULL,\n",
" embedding BLOB NOT NULL\n",
" )\n",
" \"\"\")\n",
" self.conn.execute(\"\"\"\n",
" CREATE VIRTUAL TABLE IF NOT EXISTS document_chunks_fts USING fts5(\n",
" content,\n",
" source_file UNINDEXED,\n",
" chunk_index UNINDEXED,\n",
" tokenize='unicode61'\n",
" )\n",
" \"\"\")\n",
"\n",
" def insert_chunk(self, source_file: str, chunk_index: int, content: str, embedding: List[float]) -> None:\n",
" vec = np.array(embedding, dtype=np.float32)\n",
" norm = np.linalg.norm(vec)\n",
" if norm > 0:\n",
" vec = vec / norm\n",
"\n",
" with self.conn:\n",
" self.conn.execute(\n",
" \"INSERT INTO document_chunks (source_file, chunk_index, content, embedding) VALUES (?, ?, ?, ?)\",\n",
" (source_file, chunk_index, content, vec.tobytes())\n",
" )\n",
" self.conn.execute(\n",
" \"INSERT INTO document_chunks_fts (content, source_file, chunk_index) VALUES (?, ?, ?)\",\n",
" (content, source_file, str(chunk_index))\n",
" )\n",
"\n",
" def search_dense(self, query_embedding: List[float], top_k: int = 5) -> List[Tuple[int, str, str, float]]:\n",
" q_vec = np.array(query_embedding, dtype=np.float32)\n",
" q_norm = np.linalg.norm(q_vec)\n",
" if q_norm > 0:\n",
" q_vec = q_vec / q_norm\n",
"\n",
" cursor = self.conn.execute(\"SELECT id, source_file, content, embedding FROM document_chunks\")\n",
" results = []\n",
" for doc_id, src, content, blob in cursor.fetchall():\n",
" doc_vec = np.frombuffer(blob, dtype=np.float32)\n",
" similarity = float(np.dot(q_vec, doc_vec))\n",
" results.append((doc_id, src, content, similarity))\n",
" results.sort(key=lambda x: x[3], reverse=True)\n",
" return results[:top_k]\n",
"\n",
" def search_sparse_bm25(self, query_text: str, top_k: int = 5) -> List[Tuple[int, str, str, float]]:\n",
" clean_tokens = [t for t in query_text.replace(\"'\", \"\").replace('\"', '').split() if len(t) > 1]\n",
" if not clean_tokens:\n",
" return []\n",
" fts_query = \" OR \".join(f'\"{t}\"' for t in clean_tokens)\n",
" cursor = self.conn.execute(\n",
" \"SELECT rowid, source_file, content, rank FROM document_chunks_fts WHERE document_chunks_fts MATCH ? ORDER BY rank LIMIT ?\",\n",
" (fts_query, top_k)\n",
" )\n",
" results = []\n",
" for doc_id, src, content, bm25_rank in cursor.fetchall():\n",
" bm25_score = 1.0 / (1.0 + abs(float(bm25_rank)))\n",
" results.append((doc_id, src, content, bm25_score))\n",
" return results\n",
"\n",
" def hybrid_search(self, query_text: str, query_embedding: List[float], top_k: int = TOP_K) -> List[Dict[str, Any]]:\n",
" dense_hits = self.search_dense(query_embedding, top_k=10)\n",
" sparse_hits = self.search_sparse_bm25(query_text, top_k=10)\n",
" fused_scores = {}\n",
" chunk_map = {}\n",
"\n",
" for rank, (doc_id, src, content, sim) in enumerate(dense_hits, start=1):\n",
" key = f\"{src}::{content[:50]}\"\n",
" chunk_map[key] = (src, content, \"vector\")\n",
" fused_scores[key] = fused_scores.get(key, 0.0) + (1.0 / (RRF_K + rank))\n",
"\n",
" for rank, (doc_id, src, content, bm25) in enumerate(sparse_hits, start=1):\n",
" key = f\"{src}::{content[:50]}\"\n",
" if key not in chunk_map:\n",
" chunk_map[key] = (src, content, \"bm25\")\n",
" else:\n",
" chunk_map[key] = (src, content, \"hybrid\")\n",
" fused_scores[key] = fused_scores.get(key, 0.0) + (1.0 / (RRF_K + rank))\n",
"\n",
" sorted_keys = sorted(fused_scores.keys(), key=lambda k: fused_scores[k], reverse=True)[:top_k]\n",
" output = []\n",
" for citation_idx, key in enumerate(sorted_keys, start=1):\n",
" src, content, match_type = chunk_map[key]\n",
" output.append({\n",
" \"citation_index\": citation_idx,\n",
" \"source_file\": src,\n",
" \"content\": content,\n",
" \"rrf_score\": fused_scores[key],\n",
" \"match_type\": match_type\n",
" })\n",
" return output\n",
"\n",
"print(\"✅ LocalHybridRAGStore class compiled successfully.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📊 3. Sample Ingestion & Execution Benchmark"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"store = LocalHybridRAGStore()\n",
"\n",
"sample_docs = [\n",
" (\"q3_financial_report.pdf\", 0, \"CodePulse engineering project total Q3 budget was allocated at 2,340,000 TL with 15 active developers.\", [0.8, 0.1, 0.2] + [0.0] * 1021),\n",
" (\"architecture_specs.md\", 0, \"Zenith AI leverages Microsoft phi-4-mini (3.8B parameters) for local zero-cloud inference.\", [0.2, 0.9, 0.1] + [0.0] * 1021),\n",
" (\"hr_policy_2026.docx\", 0, \"Remote work expense allowance is capped at 15,000 TL per employee quarterly.\", [0.1, 0.1, 0.8] + [0.0] * 1021)\n",
"]\n",
"\n",
"for src, idx, content, emb in sample_docs:\n",
" store.insert_chunk(src, idx, content, emb)\n",
"\n",
"query = \"What is the total allocated budget for the CodePulse project?\"\n",
"query_vec = [0.75, 0.15, 0.25] + [0.0] * 1021\n",
"\n",
"results = store.hybrid_search(query, query_vec, top_k=2)\n",
"for res in results:\n",
" print(f\"[{res['citation_index']}] {res['source_file']} ({res['match_type'].upper()}) -> Score: {res['rrf_score']:.4f}\")\n",
" print(f\" Content: {res['content']}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📝 4. Grounded Prompt Formulation for Microsoft phi-4-mini"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def construct_grounded_prompt(query: str, retrieved_chunks: List[Dict[str, Any]]) -> str:\n",
" context_blocks = []\n",
" for chunk in retrieved_chunks:\n",
" context_blocks.append(f\"[{chunk['citation_index']}] (Source: {chunk['source_file']})\\n{chunk['content']}\")\n",
" context_str = \"\\n\\n\".join(context_blocks)\n",
"\n",
" return f\"\"\"You are Zenith AI, an enterprise-grade local assistant.\n",
"Answer the user query strictly based on the provided context below.\n",
"Every factual claim must cite its source index like [1] or [2].\n",
"If the context does not contain the answer, respond: 'This information is not present in the indexed documents.'\n",
"\n",
"--- CONTEXT ---\n",
"{context_str}\n",
"--- END CONTEXT ---\n",
"\n",
"User Query: {query}\n",
"Answer:\"\"\"\n",
"\n",
"prompt = construct_grounded_prompt(query, results)\n",
"print(prompt)"
]
}
],
"metadata": {
"language_info": {
"name": "python",
"version": "3.11.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading