lora-continual

A compact research project for continual learning with sequential LoRA adapters.

Study of catastrophic forgetting and interference in LoRA adapters trained sequentially on different domains.

Features

• Sequential training across multiple domains
• Interference metrics between LoRA adapters
• Catastrophic forgetting curves
• Metric and training-result visualization
• Small base model with a series of LoRA adapters
• Cheap experiments designed to run on CPU

Stack

• PyTorch
• HuggingFace Transformers
• PEFT
• Matplotlib

Quick start

pip install -e .
python run_experiment.py
python visualize.py

Project structure

.
├── src/
│   └── lcont/
│       ├── __init__.py
│       └── experiments/
└── tests/
    ├── __init__.py
    └── test_smoke.py

License

MIT