━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Experimental open-source desktop lab for local LLM pipeline exploration.
LLM Forge Studio is a Windows-first desktop application built with .NET 8, Avalonia, and a Python backend.
It provides a guided interface for exploring the main stages of a local LLM workflow:
Dataset gathering -> Dataset preparation -> Tokenization -> Model setup -> Training execution -> Evaluation artifacts -> Experimental export / handoff
The goal is not to claim that this project can already produce high-quality models out of the box.
The goal is to make the local LLM training process more visible, guided, inspectable, and easier to experiment with.
Current Version:
v1.0.7
Status:Beta / Experimental
Primary target: Windows 10 / Windows 11
Linux/macOS: possible in manual/experimental mode
LLM Forge Studio is currently focused on workflow execution, artifact generation, observability, and experimentation.
At this stage, the project does not guarantee that the models trained through the full pipeline will produce useful, correct, or high-quality outputs.
What has been mainly validated so far is the ability of the application to execute parts of the workflow, generate files, save logs, produce checkpoints/reports, and expose the process through a guided UI.
What is not yet fully validated is final model quality.
This means:
- a pipeline may run without crashing and still produce a poor model
- a generated output may exist but still be low quality or incorrect
- training parameters, tokenizer choices, dataset size, decoding settings, and hardware limits can heavily affect results
- several advanced workflows still need broader testing on different machines and datasets
This project should currently be seen as an experimental workflow studio, not as a proven model-quality solution.
This is the most important limitation of the current release.
The application can help you run and inspect a local LLM training workflow, but it does not yet prove that the final trained model is good.
The maintainer has not yet reached consistently useful generations from the full pipeline, even on simple cases.
Possible reasons include:
- small or weak datasets
- insufficient training time
- incorrect or non-optimal hyperparameters
- tokenizer/model mismatch
- decoding settings that need better defaults
- hardware constraints
- bugs or missing pieces in the pipeline
- architectural assumptions that need review from people with deeper ML experience
This is one of the main reasons the project is being shared openly.
Feedback from people with stronger ML experience, different hardware, larger datasets, or better training setups is extremely welcome.
LLM Forge Studio is an experimental desktop environment for:
- learning how local LLM pipelines are structured
- preparing and inspecting datasets
- training tokenizers
- configuring small local model experiments
- launching backend training processes
- collecting logs, manifests, checkpoints, reports, and artifacts
- testing generation outputs
- exploring evaluation and release-gate style reports
- experimenting with future fine-tuning and export workflows
It is meant for creators, developers, technical learners, and AI-assisted builders who want a more visual and guided way to explore local LLM workflows.
It is not intended to replace mature ML frameworks or expert workflows.
LLM Forge Studio is not currently:
- a production-grade LLM training framework
- a guaranteed way to create useful LLMs
- a replacement for Hugging Face tooling, Axolotl, llama.cpp, Ollama, or custom Python workflows
- a validated recipe for high-quality model training
- a fully tested multi-GPU or cluster training solution
- a finished fine-tuning platform
- a benchmarked ML research project
If you need a battle-tested production training stack, this is not that yet.
If you want to inspect, break, test, improve, and learn from an open experimental desktop pipeline, that is the current purpose of this project.
The current validation has been done mainly on small/local scenarios compatible with the maintainer's available hardware and time.
The following areas have received practical validation at limited scale:
- dataset import and preparation flows
- local project/session handling
- tokenizer workflow execution
- training process orchestration at small scale
- generation flow execution
- logs, manifests, status files, and artifact creation
- selected evaluation/reporting paths
- UI workflow gating and long-operation visibility
- save/load project behavior
This does not mean these areas are fully proven across all datasets, model sizes, hardware setups, or long-running workloads.
It means they have been exercised in local validation scenarios and are ready for broader testing.
The following areas should be treated as experimental and may break, behave incorrectly, or require substantial improvement:
- final model quality
- large dataset behavior
- long training runs
- fine-tuning workflows
- GGUF conversion and Ollama handoff/finalization
- multi-GPU behavior
- multi-machine / cluster workflows
- large-scale memory pressure behavior
- optimal training defaults
- optimal decoding defaults
- stronger automated test coverage
- reproducibility across different machines
- end-to-end validation with meaningful model outputs
Please report issues with as much context as possible:
OS
GPU / VRAM
RAM
Python version
dataset type / size
workflow used
training settings
logs
artifacts
reproduction steps
expected behavior
actual behavior
Local LLM training is hard to approach because the workflow crosses many areas at once:
- dataset preparation
- tokenization
- model configuration
- training parameters
- hardware limits
- checkpoints
- evaluation
- quantization / export
- generation settings
Most of these pieces are usually handled through scripts, terminals, config files, and scattered artifacts.
LLM Forge Studio tries to make that process more approachable by putting the workflow into a guided desktop interface and keeping the generated artifacts visible.
The project is also an experiment in AI-assisted software building: using AI development agents to build a real, inspectable, open-source application step by step.
This project was built through an AI-assisted development workflow.
The codebase was generated and iterated step by step with AI development tools, while the maintainer directed the product design, architecture goals, feature decisions, testing passes, debugging iterations, release process, and documentation.
This is disclosed intentionally.
The project should be judged as an open experimental software project: by what it actually runs, what it produces, what is documented, what is validated, and what can be improved.
Contributions, criticism, bug reports, and technical review are welcome.
Previous version demo ( At begin when it was only a tokenizer with training, working fine ): https://youtu.be/s9LBW09kp_8
The current intended workflow is:
1. Gather or import a dataset
2. Clean / prepare / validate the dataset
3. Train or configure a tokenizer
4. Configure model and training settings
5. Start backend training
6. Inspect logs, checkpoints, manifests, and reports
7. Test generation
8. Review evaluation / quality signals
9. Experiment with export or handoff paths
Some stages are more mature than others.
The workflow is intentionally visible so users can inspect where things fail, where assumptions are wrong, and where better defaults are needed.
- file/folder dataset import
- basic cleaning and deduplication paths
- source staging concepts
- license/compliance acknowledgement paths for selected flows
- dataset merge and provenance concepts
- dataset analytics / readiness reporting foundations
- tokenizer workflow execution
- Byte-level BPE / Unigram / WordPiece oriented options
- tokenizer state persistence
- routing based on tokenizer metadata
- safer handling for larger inputs where possible
- guided training configuration
- backend training process launch
- optimizer / scheduler / checkpoint oriented settings
- mixed precision / curriculum / advanced settings foundations
- progress and runtime state visibility
- logs, manifests, checkpoints, and reports
- generation from produced artifacts/checkpoints
- decoding settings exposure
- output inspection
- experimental quality feedback loop
- evaluation summary artifacts
- scorecard-style reporting
- regression/history concepts
- release-gate style artifact checks
These areas exist in the project but are not yet fully validated:
- fine-tuning workspace
- SFT / DPO / RLHF foundations
- GGUF conversion paths
- Ollama handoff bundle generation
- deterministic export/finalization reports
Treat these as experimental until broader validation is completed.
Ollama/GGUF support should currently be considered an experimental handoff/export area.
When enabled, the app is intended to prepare an export folder such as:
<run-directory>/exports/ollama
The target bundle may include files such as:
model.gguf
Modelfile
status / notes / validation files
The app does not write directly into Ollama internal storage paths such as blobs or manifests.
At this stage, Ollama/GGUF workflows still need stronger end-to-end validation.
LLM training is hardware-sensitive.
Small experiments may run on consumer hardware, but meaningful training quickly becomes expensive in time, memory, and GPU resources.
Hardware requirements vary based on the result quality and model scale you want to achieve. Use the table below as a practical reference.
CPU-only execution is supported for basic workflows, but it is significantly slower than GPU execution and not comparable for training speed.
Approximate rule-of-thumb:
| Model Scale | Params | Inference VRAM Approx | Training VRAM Approx | Practical Notes |
|---|---|---|---|---|
| Tiny | 10M | < 1 GB |
2-4 GB |
Educational/testing scale |
| Small | 100M | 1-2 GB |
8-16 GB |
Useful for experiments |
| Compact | 500M | 2-4 GB |
16-40 GB |
Needs careful tuning |
| Base | 1B | 4-8 GB |
32-80 GB |
Often hard on single GPU |
| Medium | 7B | 14-20 GB |
120-300+ GB |
Usually multi-GPU territory |
| Large | 13B | 26-34 GB |
250-600+ GB |
Data-center territory |
| XL | 70B | 140-180 GB |
1.4-3.5+ TB |
Cluster-scale infrastructure |
Actual memory usage depends on sequence length, batch size, precision, optimizer states, gradient checkpointing, framework overhead, and implementation details.
The project includes links to permissive-license dataset discovery pages for convenience.
Always verify dataset licenses yourself before use.
Useful starting points:
- Hugging Face MIT datasets: https://huggingface.co/datasets?license=license%3Amit
- Hugging Face Apache-2.0 datasets: https://huggingface.co/datasets?license=license%3Aapache-2.0
- Hugging Face CC0 datasets: https://huggingface.co/datasets?license=license%3Acc0-1.0
- GitHub MIT dataset search: https://github.com/search?q=dataset+license%3Amit&type=repositories
- GitHub Apache-2.0 dataset search: https://github.com/search?q=dataset+license%3Aapache-2.0&type=repositories
- Zenodo: https://zenodo.org/search?page=1&size=20&type=dataset
- OpenML: https://www.openml.org/search?type=data
- UCI Machine Learning Repository: https://archive.ics.uci.edu/
- Data.gov: https://catalog.data.gov/dataset
Project validation is ongoing.
Useful documents:
The validation checklist should be read as a living document, not as a claim that every feature is complete or fully proven.
src/LLMForgeStudio.App Avalonia UI + C# core services
backends/python Python training/generation backend
tests/LLMForgeStudio.App.Tests Core tests
samples/validation Sample datasets and validation material
docs/images UI images and branding
dotnet restore
dotnet run --project src/LLMForgeStudio.AppThe prebuilt Windows executable may trigger a Windows SmartScreen or Microsoft Defender warning.
This is expected for new unsigned desktop applications, especially open-source tools distributed outside the Microsoft Store.
It does not automatically mean the application is malicious, but users should only run software they trust.
At the moment, the Windows build is not code-signed yet, so it may not have Microsoft SmartScreen reputation.
For transparency:
- the project is open source
- the source code is available in this repository
- release binaries should be treated as unsigned builds
- users can inspect the code before running it
- users can build the app locally instead of using a prebuilt executable
To run from source:
dotnet restore
dotnet run --project src/LLMForgeStudio.AppTo publish a local Windows executable:
dotnet publish src/LLMForgeStudio.App/LLMForgeStudio.App.csproj -c Release -r win-x64 --self-contained true /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=trueIn the future, code signing may be added to reduce SmartScreen warnings for prebuilt Windows releases.
The software includes automated setup paths, but manual setup is also possible.
Backend dependency setup now includes supply-chain protection controls:
- centralized policy file:
security/dependency-policy.json - trusted index allowlist enforcement for pip installs
- block on non-trusted package indexes
- requirements policy checks (
--index-url,--extra-index-url,--trusted-hostblocked) - optional block for direct URL / editable dependencies
- vulnerability gate with
pip-audit --strict - policy execution mode:
enforcementMode = strict | warn(default:strict)
Recommended for releases: keep enforcementMode set to strict.
cd backends/python
python -m venv .venv
# Windows
.venv\Scripts\activate
# Linux/macOS
source .venv/bin/activate
pip install -r requirements.txtIf you install manually, use the same trusted sources defined in security/dependency-policy.json to stay aligned with in-app protections.
Then in the app:
Training -> set Python interpreter path
Training -> set Run directory
Training -> start backend training
Example Windows interpreter path:
backends\python\.venv\Scripts\python.exe
A scenario-pack gate can be run over completed runs:
python3 backends/python/e2e_release_gate.py --scenario-pack samples/validation/e2e_scenarios/quick_suite.json --output-dir runs/default/e2e_gate_suiteGenerated artifacts may include:
e2e_suite_summary.json
e2e_suite_summary.md
per-scenario reports
This is intended for artifact validation and workflow checks, not as proof of final model quality.
You can build a local Windows executable with:
dotnet publish src/LLMForgeStudio.App/LLMForgeStudio.App.csproj -c Release -r win-x64 --self-contained true /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=trueOutput:
src/LLMForgeStudio.App/bin/Release/net8.0/win-x64/publish/LLMForgeStudio.App.exe
Prebuilt release binaries are currently unsigned, so Windows SmartScreen may show a warning. See the SmartScreen / Defender section above.
- Fine-tuning / export and large-scale training flows remain experimental and require broader real-world validation.
The most useful feedback right now is not generic praise.
Useful feedback includes:
- workflows that crash
- workflows that complete but produce bad artifacts
- confusing UI states
- incorrect training assumptions
- poor tokenizer defaults
- weak decoding defaults
- bad or misleading evaluation signals
- hardware-specific failures
- large dataset failures
- long-run instability
- fine-tuning/export failures
- documentation gaps
- suggestions from people with real ML training experience
If you test the project, please include logs, artifacts, dataset shape, hardware details, and reproduction steps when possible.
## Environment
OS:
CPU:
GPU:
VRAM:
RAM:
Python:
.NET:
## Workflow
Dataset type / size:
Tokenizer settings:
Model settings:
Training settings:
Generation settings:
## What happened
## What you expected
## Logs / artifacts
## Reproduction steps
MIT License.
