TeenyTinyLlama: small open-source language models trained in Portuguese 🦙

Large language models (LLMs) have significantly advanced the field of natural language processing, but this progress still isn’t evenly distributed across languages. In this project, we document the development of a pair of open-source language models, adapted for use in low-resource settings, trained in the Portuguese language. This is the TeenyTinyLlama pair: two compact models for text generation in Brazilian Portuguese. We released them under a permissive license (Apache 2.0) on GitHub and Hugging Face for community use and further development.

The study below was developed as part of the RAIES initiative (Network for Safe and Trustworthy Artificial Intelligence), a project supported by FAPERGS (Foundation for Research Support of the State of Rio Grande do Sul), Brazil.

Access it here.