A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.
python machine-learning reinforcement-learning embeddings faiss rag llm semantic-cache llm-inference cache-eviction smart-evictions gptcache smartevict
-
Updated
Jul 28, 2026 - Python