The Illusion of Cross-Lingual Safety in Low-Resource Languages

Imagine you're developing a language model that needs to be safe and effective across multiple languages, especially in regions where resources are limited. Currently, most safety measures are designed with English in mind, which can lead to serious issues when these models are applied to languages like Twi, Hausa, Amharic, and Swahili. The problem is that while these models may understand the language, they often fail to apply the same safety protocols, leading to harmful outputs. This is what's called cross-lingual safety transfer, and it turns out that it doesn't work as well as we hoped; harmful prompts in these languages retain less than 10% of the safety signals that are effective in English. This gap highlights a significant vulnerability in how we assume safety mechanisms will generalize across languages. To address this, the authors introduce a new dataset called LoDNA, which pairs literal translations with culturally relevant prompts, and they propose a latent geometric framework to analyze how well these models refuse harmful prompts. Their findings indicate that while the models can recognize the concepts, they struggle to route them through the safety mechanisms effectively. This research underscores the need for more nuanced safety alignment strategies that consider the unique challenges of low-resource languages, moving beyond the assumption that what works in English will automatically work elsewhere.

The exploration of cross-lingual safety transfer in low-resource languages is a significant extension of existing safety alignment research.

The findings are based on a new dataset and a robust evaluation framework, though the sample size may limit generalizability.