DIRIS — Data Injection and Retrieval System

A knowledge-graph-based document intelligence platform. Unlike plain RAG (vector similarity only), DIRIS builds an evolving knowledge graph from your documents and answers questions with hybrid retrieval (vector + keyword + graph), producing grounded, cited, multilingual, conversational answers that say "not in the documents" instead of hallucinating.

 Upload ─▶ Process ─▶ Chunk ─▶ Embed (ChromaDB) ─▶ Extract entities/relations (LLM)
   │                    │                                    │
   ▼                    ▼                                    ▼
 MySQL            Vectors (Chroma)                    Knowledge graph (Neo4j)
 (metadata,          │                                    │
  chunks,            └──────────┬───────────┬─────────────┘
  entities,                     ▼           ▼
  conversations)         Hybrid Retrieval (RRF)  ─▶  Grounded QA  ─▶  Cited answer

Architecture

Layer	Tech	Role
API	FastAPI	Auth (JWT), documents, search, entities, graph, /ask, conversations
Relational	MySQL	users, documents, chunks, entities/relationships, conversations (source of truth)
Vectors	ChromaDB	dense chunk embeddings (default local ONNX MiniLM)
Graph	Neo4j	knowledge graph projected from MySQL, traversal / multi-hop
LLM	Groq / Gemini / Anthropic	extraction + QA, provider-agnostic with automatic fallback

Each store sits behind an interface (VectorStoreBase, GraphStore, BaseLLM) so it can be swapped. See docs/data-model.md for what lives where.

Prerequisites

• Python 3.13
• Docker (for MySQL + Neo4j) — docker compose up -d
• At least one LLM API key (Groq / Gemini free tiers work; Anthropic optional)

Setup

python -m venv .venv && source .venv/Scripts/activate   # Git Bash on Windows
pip install -r requirements-dev.txt
cp .env.example .env        # add GROQ_API_KEY and/or GEMINI_API_KEY (or ANTHROPIC_API_KEY)
docker compose up -d        # MySQL (host :3307) + Neo4j (:7474 / :7687)
python -m alembic upgrade head
uvicorn diris.api.main:app --reload

Open http://127.0.0.1:8000/docs (Swagger), /graph/view (graph explorer), or http://localhost:7474 (Neo4j Browser, neo4j / dirispassword).

Try it (Swagger)

Register → Authorize → POST /documents (upload) → poll GET /documents/{id} until done → POST /ask {"question": "..."} → inspect citations, GET /entities, GET /graph.

LLM providers

Provider is chosen from whichever key is set (order: groq → gemini → anthropic), with the others as automatic runtime fallbacks. Configure in .env: DIRIS_LLM_PROVIDER, DIRIS_GROQ_MODEL, DIRIS_GEMINI_MODEL. Groq's free tier is fast but rate-limits (~8k tokens/min) → large ingests fall through to Gemini.

Tests & evaluation

pytest                          # full suite (DB tests skip if MySQL/Neo4j are down)
python -m ruff check .          # lint
python -m eval.retrieval        # retrieval recall@k / MRR on the labeled set
DIRIS_RUN_LIVE_LLM=1 pytest tests/test_extraction_live.py   # live LLM (uses quota)
python -m eval.hallucination    # live refusal-rate on out-of-scope questions

CI (.github/workflows/ci.yml) runs lint + migrations + the full suite against MySQL and Neo4j service containers on every push.

Feature map (PRD)

Requirement	Where
FR-1 users/auth	diris/api/routers/auth.py, diris/security.py
FR-2 upload & library	diris/api/routers/documents.py
FR-3 processing	diris/services/processing.py, diris/ingestion/
FR-4/5 entity/relationship extraction	diris/extraction/, diris/services/{extraction,resolution}.py
FR-6/9 knowledge graph	diris/graph/, diris/services/graph_projection.py
FR-7 hybrid retrieval	diris/services/retrieval.py (RRF)
FR-8/12 grounded QA + citations	diris/services/qa.py
FR-10 multilingual	diris/services/language.py
FR-11 conversation memory	diris/services/conversation.py
FR-13/15 graph viz & export	diris/api/routers/graph.py
FR-14 incremental updates	diris/services/processing.py (content-hash diff)

Known limitations (honest)

• Entity resolution uses exact/alias + embedding similarity with a high threshold (prefers under-merging); not perfect coreference.
• Cross-lingual retrieval pivots via English translation (good for English corpora); true any-to-any needs multilingual embeddings.
• Extraction is 1 LLM call/chunk — a whole book is slow and rate-limited on free tiers.
• No page-number citations yet (section headings only).
• Graph/keyword search have small eventual-consistency / cap caveats (documented in code).

The incremental build history is in ROADMAP.md; conventions in CONTRIBUTING.md. A legacy pure-Python MVP (python -m diris) remains as a reference implementation.