Local RAG PDF Assistant

A local Retrieval-Augmented Generation (RAG) system for PDF documents using LangChain, ChromaDB, and OpenAI-compatible APIs with LM Studio.

Features

• PDF Processing: Extracts text from PDFs and filters out pages with less than 20 characters
• Document Chunking: Splits documents into manageable chunks for better retrieval
• Vector Storage: Uses ChromaDB for efficient similarity search
• Local LLM: Integrates with LM Studio for local language model inference
• Web Interface: Streamlit-based UI for easy interaction
• Batch Processing: Support for processing multiple PDFs or entire directories

Setup

1. Activate your virtual environment:

   python -m venv .openai_rag_venv

   source .openai_rag_venv/bin/activate

2. Install Dependencies:

   pip install -r requirements.txt

3. Configure LM Studio:

   • Start LM Studio and load your microsoft/phi-4-mini-reasoning model
   • Ensure it's running on http://localhost:1234
   • Update .env file if using different settings

4. Run the Application:

   streamlit run streamlit_app.py

Usage

Via Streamlit UI

1. Open the web interface
2. Test system connectivity
3. Upload PDF files or specify a directory
4. Ask questions about your documents

Via Python API

from rag_system import RAGSystem

# Initialize the system
rag = RAGSystem()

# Add a PDF
rag.add_pdf("path/to/document.pdf")

# Query the system
result = rag.query("What is the main topic of the document?")
print(result["answer"])

Configuration

Edit .env file to customize:

• OPENAI_API_BASE: LM Studio endpoint (default: http://localhost:1234/v1)
• OPENAI_API_KEY: API key (default: lm-studio)
• MODEL_NAME: Model name (default: microsoft/phi-4-mini-reasoning)

Architecture

• PDFProcessor: Handles PDF text extraction and chunking
• VectorStore: Manages ChromaDB operations and embeddings
• LLMClient: Interfaces with LM Studio for text generation
• RAGSystem: Orchestrates the complete RAG pipeline
• Streamlit App: Provides web-based user interface

Requirements

• Python 3.8+
• LM Studio running locally
• PDF files for processing