TR AI Evals
Benchmarks for evaluating AI models on Turkish-language tasks.
Our mission is to contribute to the Turkish AI ecosystem. We give developers, researchers and organizations working in Turkish reliable evidence of how AI models perform on Turkish-language tasks. The benchmarks, code and results are public, so anyone can use them, repeat them and build on them.
Most AI benchmarks are in English, and English results say little about how a model handles Turkish.
This project evaluates models on Turkish-language tasks, the way someone choosing a model for Turkish-language work needs: reproducibly, and under the same conditions for every model.
Reproducible: every run records the exact dataset revision, prompts, settings and model weights. Published versions are never edited; changes become new versions.
Same conditions for every model: all models in a benchmark get the same prompts and settings, with no tuning for any one model.
Benchmarks
Turkish University Entrance Exam
Academic knowledge and reasoning, using 4,858 multiple-choice questions from ÖSS, YGS and LYS exams held in 2006–2017.
Read the report
Text from the project’s README on GitHub (opens in a new tab).