Blinded panels of clinicians and hospital executives compared LLM-X against a leading global LLM across real Emergency Department cases and executive briefings. They preferred LLM-X on every scored dimension. This is the full story of how it was validated, and why it matters for every hospital in the region.
Healthcare is where language stops being a feature
In most industries, an AI model that gets the language slightly wrong produces an awkward sentence. In a hospital, it produces a clinical note a physician does not trust, a discharge instruction a patient misreads, or a safety risk that goes unflagged. Arabic-speaking clinicians across the GCC have spent years working with AI that thinks in English and translates on the way out. The output reads like a translation, and clinicians treat it like one.
That is the problem HIKMA AI and Arabic.AI set out to close, and it is why our newly announced strategic partnership was not built on a demo or a benchmark screenshot. It was built on one of the most rigorous Arabic clinical AI validations conducted in the region to date: blinded panels, printed paper packets, specialist-authored gold standards, and real Emergency Department cases.
Two layers, one system
The partnership unites two complementary layers of the healthcare AI value chain.
HIKMA AI provides the execution intelligence layer. Its platform operates above the systems hospitals already run, including EMR, laboratory, radiology, pharmacy and finance, and orchestrates clinical and operational workflows across them, with ICD-10-AM coding, AR-DRG grouping and NPHIES integration engineered specifically for Saudi and GCC health systems.
Arabic.AI provides the intelligence layer through LLM-X, our flagship Arabic-first large language model, ranked first on the Stanford HELM Arabic Enterprise benchmark. LLM-X was built on 18+ years of Arabic language expertise, and it understands the language, the clinical register and the cultural context of the clinicians and administrators it serves.
HIKMA AI turns hospital data into coordinated action. LLM-X makes sure every word of it is native, precise and trusted. Neither layer is an adaptation of imported technology. Both were built in the region, for the region.
Phase one: the pilot that set the bar
The first validation phase ran in July 2026 as a structured clinical pilot: eight real-world Emergency Department scenarios of escalating complexity, from routine presentations to high-acuity cases. An independent senior clinician evaluated LLM-X’s Arabic clinical output against specialist-authored reference notes across six clinical dimensions.
The result was 98% clinician satisfaction, with LLM-X rated superior to a leading global LLM on all seven comparison dimensions. Just as important, LLM-X independently flagged three medication-safety risks, including an allergy cross-reactivity, a dosing error and a QT-prolongation risk, none of which the global model caught without explicit prompting.
Here is how the two models compared, dimension by dimension:
| Evaluation dimension | LLM-X (Arabic.AI) | Leading global LLM |
|---|---|---|
| Arabic clinical register | Native. Reads as if written by an Arabic-speaking clinician | Translated-English structure, distrusted by clinical staff |
| Saudi MOH terminology | Aligned with HIKMA AI’s 729-term verified clinical library | Inconsistent hybrid terms, non-standard in KSA |
| Medication safety | Independently flagged allergy cross-reactivity, a dose error and a QT-prolongation risk | Missed all three without explicit prompting |
| Differential prioritization | Clinically correct ranking in 8 of 8 cases | Missed stump appendicitis as the primary differential |
| Protocol alignment | Average 8.5 of 9 specialist actions matched; 10 of 10 on acute MI | Average 6 of 9; missed resuscitation-readiness items |
Phase two: blinded, at scale, on paper
A pilot with a single evaluator is a signal. The second phase was designed to remove every excuse: a blinded, larger-scale evaluation built to withstand clinical scrutiny.
Thirty Emergency Department clinical cases were scored by an independent panel of six clinicians: three ER physicians and three senior ICU and ER nurses. The panel worked from printed paper packets and was blind to which output came from which model. Every case was judged against a gold-standard reference prepared separately by a senior emergency-medicine physician.
The outcome: LLM-X-generated clinical assessments were rated trustworthy at the bedside in 89 of 90 physician-panel-scored instances, and the panel preferred LLM-X over the leading global LLM on every scored clinical dimension, including Arabic clinical language quality, terminology alignment with regional standards, and overall clinical accuracy.
Trustworthy at the bedside is the phrase that matters. It is not a score about fluency. It is clinicians saying they would act on what they read.
Beyond the bedside: the executive test
Hospitals do not run on clinical notes alone. They run on decisions made every morning by CEOs, COOs, CFOs and clinical leadership, and those decisions are only as good as the operational picture in front of them.
So a parallel evaluation tested something no Arabic model had been formally scored on before: daily executive briefings of hospital state, generated in Arabic from structured operational data. Fifteen hospital-state briefings were scored by an external five-member panel of hospital executives, under the same blinded, printed-paper format, against a specialist gold-standard reference, producing 75 scored instances in total.
All 75 instances rated the LLM-X brief as ready to send to executives without a single edit. And once again, LLM-X was preferred over the same leading global LLM on every scored dimension: factual fidelity to the underlying operational data, cultural grounding of the recommended actions, and register naturalness in executive Arabic.
From validation to production in 24 hours
Rigor usually comes at the cost of speed. Not here. LLM-X was live inside HIKMA AI’s Arabic-mode clinical routing layer within 24 hours of API delivery, and it now powers Arabic clinical intelligence across the HIKMA AI platform. The same model the panels validated is the model running in production.
What sovereign AI means when it reaches a hospital
“AI for the Arabic-speaking world must be built with Arabic at its core, and nowhere is that more consequential than healthcare, where a clinician’s trust in what they read can change a patient’s outcome. This is what sovereign AI actually means in practice: intelligence built in the region, in the language of the people it serves, validated by the clinicians who use it. Our work with HIKMA AI proves that Arabic-first AI is not an adaptation of someone else’s technology. It is the standard,” said Nour Al Hassan, Founder and CEO of Tarjama and Arabic.AI.
“HIKMA AI was built on a conviction: healthcare is transformed not by technology, but by how it empowers caregivers to do their clinical, operational, and administrative work safely, in their own language, at the moment it matters. Arabic.AI gives us the finest Arabic intelligence in the world; HIKMA turns it into action a hospital can run on. Together, we are setting the standard for how an entire region’s health systems will work,” said Charbel Rizkallah, Founder and CEO of HIKMA AI. “For the first time, our clinicians read AI output in Arabic and trusted it. That is the difference between a tool clinicians adopt and one they abandon.”
The partnership directly supports the Kingdom’s data-residency and AI-sovereignty direction: Arabic-native intelligence paired with regional workflow execution, rather than imported AI adapted after the fact. As governments across the GCC formalize requirements for where health data lives and what intelligence is allowed to touch it, this architecture is not a preference. It is the direction of travel.
What comes next
Following the joint announcement, HIKMA AI and Arabic.AI will publish a co-branded clinical case study documenting the full validation methodology and results, and will advance a strategic partnership framework for co-marketing to health systems across the GCC.
For health systems, ministries and hospital groups across the region, the message is simple: Arabic-native, clinician-validated AI is no longer a roadmap item. It is running in production today.
To explore what LLM-X and the Arabic.AI platform can do for your organization, visit arabic.ai or contact our team. Read the official announcement HIKMA AI and Arabic.AI Announce Strategic Partnership to Deliver the First Arabic-Native, Clinician-Validated Healthcare AI for Arab Hospitals.
More on HIKMA AI: https://hikma-ai.com · LinkedIn
