‹ back to 2026-07-16

Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

arXiv cs.CL LLMs & Foundation Models Hailay Kidu Teklehaymanot, Debela Desalegn Yadeta, Wolfgang Nejdl 2026-07-16

TL;DR - VEXMLM is a vocabulary-extended XLM-R variant that adds 30,000 Ge'ez-script subwords to fix poor tokenization on Amharic and Tigrinya, substantially boosting QA, sentiment, and NER performance and transferring gains to 17 related African languages.

Method: Trains a language-specific SentencePiece tokenizer on curated Amharic/Tigrinya corpora, extends XLM-R's vocabulary by 30K Ge'ez subwords, and initializes new embeddings by averaging constituent-subword embeddings from the original tokenizer.
Two-stage training: Continued masked language modeling over the extended vocabulary, then supervised fine-tuning on QA, NER, and sentiment analysis (SA).
Results: Amharic/Tigrinya QA reaches 87.0 EM / 90.0 F1 (vs. 66.0/78.0 for XLM-R, 74.0/78.0 for Glot500); SA hits 80.0% accuracy; NER OOV-token entity accuracy rises from 81.4% to 94.3% (avg over 11 languages).
Transfer: Gains from augmenting two high-resource languages carry over to 17 typologically related, unaugmented African languages (19 total), evaluated on both tokenization metrics and downstream tasks.

view merged work →