RAG Assistant

A Retrieval-Augmented Generation chat app that answers questions strictly from a document you provide, powered by Claude.

Overview

RAG Assistant is a Streamlit chat interface that demonstrates a complete Retrieval-Augmented Generation pipeline end to end. Upload a text/markdown document (or use the bundled sample), and the app chunks it, embeds the chunks with a local sentence-transformer model, indexes them in a ChromaDB vector store, and retrieves the most relevant passages for every question. Those passages are injected into Claude's system prompt so answers stay grounded in the source document instead of the model's general knowledge.

The bundled sample document, terms_of_service.txt, is a mock customer-support FAQ for a fictional bank ("WiBank"), used to showcase the assistant answering document-grounded support questions.

Features

• Document-grounded answers — Claude is instructed to answer only from retrieved excerpts, and to say so clearly when the document doesn't cover a question.
• Adaptive chunking — automatically switches between section-aware chunking (for documents with Category: headers) and fixed-size sliding-window chunking with overlap for unstructured text.
• Configurable retrieval — adjust how many chunks are retrieved per query directly from the sidebar.
• Model & sampling controls — pick from multiple Claude models, set max output tokens, and choose between temperature or top-p sampling.
• Streaming responses — Claude's output streams token-by-token into the chat.
• Debug mode — inspect the exact chunks retrieved for each question before/after generation.
• File upload — swap in your own .txt or .md document at runtime; the vector store rebuilds automatically.
• Conversation tools — clear the chat or save the full transcript (with retrieved chunks) to a JSON file.

Tech Stack

Layer	Technology
LLM	Anthropic Claude (Opus, Sonnet, and Haiku model options) via the anthropic Python SDK
Embeddings	sentence-transformers — all-MiniLM-L6-v2
Vector store	ChromaDB (in-memory EphemeralClient)
UI	Streamlit
Config	python-dotenv

Architecture

Document (.txt/.md)
      │
      ▼
   Chunking          section-based (Category: headers) or fixed-size w/ overlap
      │
      ▼
   Embedding         sentence-transformers (all-MiniLM-L6-v2)
      │
      ▼
   Vector Store      ChromaDB collection ("rag-documents")
      │
      ▼
   Retrieval         top-k similarity search against the user's question
      │
      ▼
   Generation        retrieved chunks injected into Claude's system prompt,
                      response streamed back to the chat UI

1. Ingest — a document is loaded from disk or uploaded through the sidebar.
2. Chunk — process_document() picks a strategy: chunk_by_sections() splits on Category: headers for structured docs, chunk_by_size() uses a 500-character sliding window with 50-character overlap otherwise.
3. Embed & Store — each chunk is embedded with all-MiniLM-L6-v2 and added to a ChromaDB collection.
4. Retrieve — on each question, ChromaDB returns the top-k most similar chunks (k is adjustable in the sidebar).
5. Generate — the retrieved excerpts are inserted into Claude's system prompt with an instruction to answer only from that context, and the response is streamed back into the chat.

Installation

git clone https://github.com/advayiscoding/RAG_application.git
cd RAG_application
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirement.txt

Create a .env file in the project root with your Anthropic API key:

ANTHROPIC_API_KEY=sk-ant-...

Usage

Run the app:

streamlit run app.py

Then, in the browser UI:

1. Use the bundled terms_of_service.txt sample, or upload your own .txt/.md file from the sidebar.
2. Pick a Claude model, adjust max tokens, retrieval depth (chunks per query), and sampling strategy (temperature or top-p).
3. Ask questions in the chat box — answers are generated only from the retrieved document excerpts.
4. Enable Debug mode to inspect which chunks were retrieved for each answer.
5. Use Save in the sidebar to export the conversation (including retrieved chunks) as JSON.

Skills Demonstrated

• RAG pipeline design — end-to-end retrieval-augmented generation flow from raw document to grounded LLM response.
• Vector search — embedding generation and similarity-based retrieval with ChromaDB.
• Prompt engineering — system prompts that constrain the LLM to grounded, context-only answers with explicit fallback behavior.
• Adaptive text chunking — content-aware strategy selection (structured vs. unstructured documents) with overlap handling.
• LLM integration — streaming completions via the Anthropic SDK with configurable sampling parameters.
• Interactive application development — stateful, multi-turn chat UI built with Streamlit.