CreditMaxx — Prompt Cost Analyzer

Tools for measuring how prompt engineering affects Claude API token usage and cost — a CLI and a companion NiceGUI web dashboard, both built directly on the Anthropic Python SDK.

Overview

CreditMaxx answers a practical question for anyone writing prompts against the Claude API: does this prompt actually cost less, and why? Rather than assuming shorter prompts are always cheaper, it sends real requests, reads back actual input/output token counts from the API, and explains the result — including the common case where a longer, more detailed prompt produces a shorter response and ends up cheaper overall, since output tokens are priced several times higher than input tokens on every current Claude model.

The project has two parts that share the same core idea:

• credit.py — a single-file CLI demo (single prompt, two-prompt comparison, or multi-prompt document comparison modes).
• webapp/ — a fuller NiceGUI dashboard version of the same idea, with a live session dashboard, a prompt library, and an AI-assisted prompt optimizer.

Features

• Single Prompt mode — send one prompt to Claude and see the response alongside input/output/total tokens, estimated cost, and response time.
• Prompt Comparison — run two prompts meant to accomplish the same task and get a side-by-side token/cost/time table plus an efficiency verdict.
• Document Comparison — run two or more prompts against the same reference document (a fictional hospital discharge summary, auto-generated on first run) to see how prompt design affects cost when a large document is part of every request.
• Prompt Engineering Lab (webapp only):
  • Prompt Library — built-in poor-vs-optimized prompt pairs across six categories (Healthcare, Document Summarization, Data Extraction, Customer Support, Executive Reports, Email Generation), each with an explanation of what was wrong and why the rewrite helps.
  • Optimize Prompt — send your own prompt to Claude and get back a rewritten version plus a structured checklist of applied improvements, using the API's structured-output (JSON schema) support.
  • Run Comparison — runs the original and optimized prompt against the same document and reports real token/cost/time deltas.
• Efficiency analysis engine (webapp/services/efficiency_service.py) — decides which prompt actually won on cost (not just token count) and generates a plain-language explanation, correctly handling the case where input tokens grew but output tokens (and therefore cost) fell.
• Live session dashboard — running totals, averages, and charts (token usage, cost, cumulative cost) across every request made in the session.
• Configurable pricing and model — per-model pricing (input/output $ per million tokens) is editable from the Settings tab and persisted to settings.json; changes apply immediately to every cost calculation.
• CSV export and formatted currency/token/percentage display via webapp/utils/formatting.py.

Tech Stack

• Python 3.10+
• Anthropic Python SDK — anthropic.Anthropic (CLI, sync) and anthropic.AsyncAnthropic (webapp, async), including structured outputs via output_config/JSON schema
• NiceGUI — the web dashboard's UI framework (built on FastAPI + Vue/Quasar)
• python-dotenv — loads ANTHROPIC_API_KEY from a .env file
• pytest — test suite for the webapp (webapp/pytest.ini, webapp/conftest.py, webapp/test_repro_bug.py)
• pyright — static type checking (webapp/pyrightconfig.json)

Installation

git clone https://github.com/advayiscoding/creditmaxx.git
cd creditmaxx

Set your Anthropic API key, either by creating a .env file (in creditmaxx/ or creditmaxx/webapp/) containing:

ANTHROPIC_API_KEY=your-api-key-here

or by exporting it in your shell:

export ANTHROPIC_API_KEY="your-api-key-here"

CLI

pip install anthropic python-dotenv
python3 credit.py

Web dashboard

cd webapp
pip install -r requirements.txt
python3 app.py

Then open http://localhost:8080.

Usage

CLI — run python3 credit.py and choose a mode from the menu:

Choose a mode:
  1) Single prompt
  2) Prompt comparison mode
  3) Document comparison mode
  4) Quit

Document Comparison mode automatically writes sample_patient_record.txt — a fictional, clearly-labeled hospital discharge summary — next to the script on first run, so no manual test-data setup is needed.

Web dashboard — run python3 app.py inside webapp/ and use the tabs: Dashboard, Single Prompt, Prompt Comparison, Document Comparison, Prompt Engineering Lab, and Settings.

Architecture

The webapp is layered so all Anthropic API calls live in one place:

webapp/
  app.py                     Entry point — page layout, tab navigation only
  config.py                  Constants + persisted AppSettings (model, pricing, theme)
  services/                  Business logic — no UI code
    anthropic_service.py     The only module that calls the anthropic SDK
    prompt_templates.py      System prompt + JSON schema for prompt optimization
    prompt_library.py        Static poor-vs-optimized prompt examples
    efficiency_service.py    Pure comparison logic — which prompt actually wins on cost, and why
    document_service.py      Sample document generation + prompt/document combining
    history_service.py       In-memory session history for the Dashboard
  ui/                        UI code only — no direct Anthropic API calls
    theme.py, components.py, dashboard_tab.py, single_prompt_tab.py,
    comparison_tab.py, document_comparison_tab.py, prompt_lab_tab.py,
    settings_tab.py
  utils/
    formatting.py            Currency/token/percentage formatting, CSV export

Every UI tab calls only the functions exported from services/anthropic_service.py (send_prompt, optimize_prompt, calculate_cost), so the backend can be swapped or mocked without touching UI code. credit.py is a standalone, dependency-light CLI that implements the same core ideas independently of the webapp.

Skills Demonstrated

• Practical integration with the Anthropic API, including sync and async clients, structured outputs (JSON schema), and typed exception handling (AuthenticationError, RateLimitError, APIStatusError, APIConnectionError)
• Layered application architecture separating API access (services/), UI (ui/), and configuration (config.py) so the LLM backend is swappable without touching presentation code
• Building an interactive multi-tab dashboard with NiceGUI, including live charts, persisted settings, and refreshable components
• Designing a cost/efficiency model that reflects real API pricing mechanics (asymmetric input/output token pricing) rather than a naive token-count comparison, and generating a correct natural-language explanation for each outcome
• Test coverage and static typing setup (pytest, pyright) alongside application code
• Careful handling of demo data — the sample clinical document is explicitly and repeatedly labeled fictional, for demonstration purposes only