flores-200

Here are 5 public repositories matching this topic...

Rizzwanth / NLLB-Translation-Evaluation

A machine translation evaluation pipeline prioritizing Meta's NLLB-200 model and the FLORES-200 dataset, featuring Kaggle-optimized data loading and UMAP embedding visualization.

python nlp deep-learning machine-translation transformers pytorch artificial-intelligence text-embeddings generative-ai flores-200 nllb-200 sacrebleu

Updated May 9, 2026
Jupyter Notebook

Kenny0bi / owoori

Star

What 204 languages pay for the same sentences: the tokenizer tax, measured across 12 tokenizers on FLORES-200 parallel text

multilingual nlp data-visualization yoruba tokenization fairness flores-200

Updated Sep 2, 2026
Python

Furious-Meteors / quorum

Star

Multilingual LLM-as-a-judge framework with anchored pairwise comparison, CI-driven sampling, and human-calibration support.

python modal gemini llms openai-gpt4 flores-200 llm-as-a-judge claude-sonnet-4-6

Updated Jul 2, 2026
Python

aungthuhein2005 / myanmar-tokenizer-bench

Star

How many tokens do AI models spend on Burmese? Benchmarks GPT, Claude, Gemini, Llama, Qwen, DeepSeek & more on the same FLORES-200 sentences. Burmese costs up to 11.7× more tokens than English, and more than Thai, Hindi, Vietnamese and Chinese with every tokenizer tested.

benchmark myanmar tokenization burmese southeast-asia llm flores-200

Updated Sep 28, 2026
Python

Bekafi01 / english-amharic-transformer-nmt

Star

From-scratch Transformer NMT for English ↔ Amharic (PyTorch). Data pipeline → BPE → training → beam search → FLORES-200 eval → live demo.

Updated Oct 6, 2026
Python

Add this topic to your repo

To associate your repository with the flores-200 topic, visit your repo's landing page and select "manage topics."

Learn more