🛰️ Daily AI Frontier
‹ back to 2026-07-16

Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

Research LLMs & Foundation Models

Merged summary

TL;DR - VEXMLM is a vocabulary-extended XLM-R variant that adds 30,000 Ge'ez-script subwords to fix poor tokenization on Amharic and Tigrinya, substantially boosting QA, sentiment, and NER performance and transferring gains to 17 related African languages.

  • Method: Trains a language-specific SentencePiece tokenizer on curated Amharic/Tigrinya corpora, extends XLM-R's vocabulary by 30K Ge'ez subwords, and initializes new embeddings by averaging constituent-subword embeddings from the original tokenizer.
  • Two-stage training: Continued masked language modeling over the extended vocabulary, then supervised fine-tuning on QA, NER, and sentiment analysis (SA).
  • Results: Amharic/Tigrinya QA reaches 87.0 EM / 90.0 F1 (vs. 66.0/78.0 for XLM-R, 74.0/78.0 for Glot500); SA hits 80.0% accuracy; NER OOV-token entity accuracy rises from 81.4% to 94.3% (avg over 11 languages).
  • Transfer: Gains from augmenting two high-resource languages carry over to 17 typologically related, unaugmented African languages (19 total), evaluated on both tokenization metrics and downstream tasks.

Sources (1)

Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

arXiv cs.CL Hailay Kidu Teklehaymanot, Debela Desalegn Yadeta, Wolfgang Nejdl 2026-07-16 arXiv:2607.15209

TL;DR - VEXMLM is a vocabulary-extended XLM-R variant that adds 30,000 Ge'ez-script subwords to fix poor tokenization on Amharic and Tigrinya, substantially boosting QA, sentiment, and NER performance and transferring gains to 17 related African languages.

  • Method: Trains a language-specific SentencePiece tokenizer on curated Amharic/Tigrinya corpora, extends XLM-R's vocabulary by 30K Ge'ez subwords, and initializes new embeddings by averaging constituent-subword embeddings from the original tokenizer.
  • Two-stage training: Continued masked language modeling over the extended vocabulary, then supervised fine-tuning on QA, NER, and sentiment analysis (SA).
  • Results: Amharic/Tigrinya QA reaches 87.0 EM / 90.0 F1 (vs. 66.0/78.0 for XLM-R, 74.0/78.0 for Glot500); SA hits 80.0% accuracy; NER OOV-token entity accuracy rises from 81.4% to 94.3% (avg over 11 languages).
  • Transfer: Gains from augmenting two high-resource languages carry over to 17 typologically related, unaugmented African languages (19 total), evaluated on both tokenization metrics and downstream tasks.
item →