kto

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).

Auditable human-feedback annotation, review, provenance, and frozen training-data export.

This project is built using the latest tech stack, including Compose for the UI, Ktor for backend communication, Flow for asynchronous operations, Koin for dependency injection, and MVVM for a clean architecture. Unit testing ensures reliability and code quality. The result is a modern and responsive Android app with a well-organized structure.

Advanced LLM fine-tuning techniques: SFT (LoRA, QLoRA, DoRA, P-/Prefix-Tuning), GRPO, DPO, ORPO, KTO & PPO; composable correctness/format rewards + LLM-as-a-Judge evals (DeepEval, Evidently AI) across math, multi-hop, medical & general QA on Llama 3, Mistral, Phi-4, Gemma & Qwen3. Built on TRL, PEFT & Unsloth.

Headless LLM fine-tuning in 3 lines — smart defaults, VRAM-aware batch sizing, multi-run SLAO, GGUF export for Ollama.

Open, MCP-native training-data foundry: turn permitted documents into traceable, quality-gated SFT and preference datasets.

Quick and fast rapid development with ktor

Preference optimization framework for text classification (DPO/ORPO/KTO), with SFT, encoder, and XGBoost baselines plus unified run pipeline and reproducible outputs.

Systematic comparison of DPO, IPO, KTO, and ORPO alignment methods

🚀 Optimize preferences effectively with ORPO, a framework for monolithic preference optimization without a reference model.

Structured game-agent distillation: distilling frontier-LLM gameplay reasoning into a small open student model (Qwen) through a typed MCP tool API, in a persistent 2D MMORPG. SFT + KTO data collection across 4 LLM harnesses, measured against a quest benchmark. (Joint work with Niral Patel.)

RLHF experiments with Unsloth: DPO, ORPO, SimPO, KTO. Training scripts, notebooks, and quick evaluation utilities.

Fine-tune Large Language Models using LoRA, QLoRA, and Unsloth on Windows with this verified, all-in-one setup guide.