AgentLight

Built with Llama

A coding agent built by fine-tuning a vanilla open-weight Llama into a reasoning + tool-using model — on a hobbyist Kaggle GPU budget.

Unlike a from-scratch project, AgentLight starts from a strong pretrained base (Llama 3.2 3B Instruct) and teaches it to reason and act through a three-stage pipeline. The showcase task is verifiable by construction: given a programming problem, the model reasons step by step, writes Python, runs it against unit tests, and revises until the tests pass.

It's a standalone sibling to the earlier GPTlight project (which trained a 97M model from scratch). AgentLight reuses GPTlight's hard-won Kaggle lessons (see docs/PLAN.md) but shares no code — it's self-contained.

Why a small model can still be a real showcase: the base 3B model already far outperforms a from-scratch model on every benchmark. The contribution here is the pipeline — reasoning-SFT → repair-SFT → general-SFT → GRPO — and the fact that its reward signal is objective: unit tests either pass or they do not. It is a focused code agent, not a general far-reaching assistant; that honesty is the point.

Status: training complete through general-SFT; GRPO not finished, and no evaluation published yet. The before/after pass@1 comparison this pipeline is built to produce has not been run to completion, so no result is claimed here. src/eval_code.py is the harness for it, and MODEL_CARD.md has the table waiting to be filled. Note that extract_code in agent/executor.py currently takes the last Python block in a reply, which mis-scores answers that end with a usage example — fix that before trusting any numbers it produces.

Pipeline

1. Reasoning-SFT — teach the <think>…</think> format on the verified code subset of R1-distilled OpenThoughts metadata, with overlong rows removed before training (open-thoughts/OpenThoughts-114k, Apache-2.0).
2. General-SFT replay — a smaller/lower-LR assistant replay phase (HuggingFaceTB/smoltalk, Apache-2.0), assistant-only loss.
3. GRPO — reinforcement learning with a verifiable reward: generate code, show one public MBPP interface example, and reward only against the remaining held-out tests. This is the RLVR recipe that made reasoning models strong at code.

Then an inference-time ReAct loop (agent/react_agent.py) turns generation into agency: execute → observe failure → revise.

Repository layout

config/config.py        Single source of truth (model, data, hyperparameters)
data/prepare_data.py    License-clean dataset loading & formatting
src/train.py            Pipeline: reasoning-SFT → general-SFT → GRPO (resumable)
src/eval_code.py        HumanEval pass@1 — the before/after showcase metric
agent/executor.py       Sandboxed code execution (GRPO reward + agent tool)
agent/react_agent.py    ReAct execute/retry coding agent
chat/local_chat.py      Local CLI chat against a trained adapter
kaggle/run.py           Kaggle entrypoint (install + sync repo + train)
kaggle/kernel-metadata.json
docs/PLAN.md            The 16h Kaggle plan, phase budget, and GPTlight lessons
THIRD_PARTY_NOTICES.md  Model + dataset licenses and compliance obligations
NOTICE                  Required attribution notices (Llama, MBPP)
MODEL_CARD.md           Ready-to-publish card for the trained adapter
licenses/               Verbatim Llama 3.2 license and acceptable use policy

Running it

Train on Kaggle (push from the repo root with the Kaggle CLI):

kaggle kernels push -p kaggle

Set the accelerator to GPU T4 x2. The run resumes automatically across sessions — see docs/PLAN.md and checkpoints/README.md. Run the two-GPU smoke procedure in HANDOFF.md before committing the full quota.

Evaluate (the RESULTS story):

python src/eval_code.py --adapter checkpoints/grpo --n 60
python src/eval_code.py --adapter checkpoints/grpo --n 60 --agentic
python src/eval_code.py --adapter checkpoints/grpo --n 164 --tts --k 4

Chat / solve locally:

pip install -r requirements.txt
python chat/local_chat.py --adapter checkpoints/grpo

Licence and attribution

Built with Llama. The code in this repository is MIT-licensed (LICENSE). Everything the project builds on keeps its own licence, and two of those impose obligations that continue to apply to anything trained here:

• Llama 3.2 Community License — the base model is Meta's Llama 3.2 3B Instruct, so any model trained by this pipeline is a derivative of it. A distributed model must be named beginning with "Llama" (the name reserved here is Llama-AgentLight), must display "Built with Llama", and must ship with a copy of the licence: licenses/.
• MBPP (CC BY 4.0) — supplies the GRPO reward signal, so trained weights are a derivative of it and must carry its attribution.

Meta's base weights and the raw datasets are never redistributed here; they are downloaded from their official sources at runtime.

Required attribution notices: NOTICE. Full analysis of every third-party component, the obligations it imposes, and how they are met: THIRD_PARTY_NOTICES.md. If you publish an adapter, use MODEL_CARD.md — it already carries the required notices.

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Safety note

The GRPO reward and the ReAct agent both execute model-generated Python (agent/executor.py). Run training, evaluation, and the agent in a disposable environment. Never execute model-written code unreviewed on a machine whose contents you care about.