Manar Alharbi, Asma Alsakkaf, Reem Bazarah and AI Astrolabe team (work done as interns at AI Astrolabe) Introduction In our previous research , we introduced ASAS (أساس) - the first human-rated Arabic safety benchmark for evaluating LLMs in Modern Standard Arabic (MSA). The results revealed significant safety gaps across frontier models, with the best-performing model achieving only 68% safe responses. Building on that work, we now present a critical follow-up study: evaluati

Arabic is spoken by hundreds of millions, yet most models fall short; not just in fluency, but in safety, cultural grounding, and contextual understanding.