Arabic-First AI Assistants with RAG: Retrieval, Morphology, Evaluation and GovernanceArabic-First AI Assistants with RAG: Retrieval, Morphology, Evaluation and Governance

Arabic-speaking organizations are moving quickly from experimenting with chat interfaces to a harder question: can an assistant answer accurately from our own policies, contracts, manuals and knowledge bases, in Arabic, with sources, and without revealing what it should not? Retrieval-augmented generation (RAG) is the pattern most teams reach for. When these assistants underperform in Arabic, the language model is often not the weakest link. The retrieval pipeline around it was simply designed with English assumptions.

What RAG does, and where Arabic changes the equation

A RAG system retrieves the passages most relevant to a question from your own content, passes them to a language model as context, and asks it to answer from that context with citations. Done well, it keeps answers current without retraining, makes them traceable, and lets you enforce who can see what.

Arabic adds specific challenges at every stage:

• Rich morphology: conjunctions, prepositions, the definite article and pronouns attach directly to words, so a single concept appears in many surface forms, such as عقد، العقد، بالعقد، وعقودهم.
• Orthographic variation: hamza forms on alef, taa marbuta written as haa, alef maqsura written as yaa, optional diacritics and decorative tatweel.
• Mixed scripts: product names, codes and technical terms in Latin script inside Arabic sentences, and occasionally Arabizi in informal queries.
• A register gap: users ask in Saudi dialect, while policies are written in formal Modern Standard Arabic.

Ingestion: get the text right before anything else

Many Arabic PDFs extract poorly: characters in reversed order, presentation-form glyphs instead of standard letters, or tables flattened into noise. Scanned documents need OCR that genuinely supports Arabic. Before tuning any model, sample the extracted text from every source type and read it. If a person cannot read it, retrieval will not find it.

• Preserve structure: headings, article and clause numbers, tables and lists define natural chunk boundaries.
• Capture metadata: owning department, effective date, version, classification level and language.
• Keep the original text for display and citations, alongside a normalized version for indexing.

Normalization and tokenization

Apply one normalization function consistently at indexing time and at query time: strip tatweel and diacritics, unify alef variants, and make explicit decisions about taa marbuta, alef maqsura, and Western versus Arabic-Indic digits. Inconsistency between the two sides is a common and silent cause of missed results.

For lexical search, use an Arabic-aware analyzer with light stemming that removes common prefixes and suffixes. Aggressive root-based stemming can merge unrelated words that happen to share a root, which hurts precision. Also measure how your chosen language model tokenizes Arabic: many tokenizers split Arabic text into more tokens than equivalent English, which affects context budgets, latency and cost. Test this on your own corpus rather than relying on assumptions.

Retrieval: hybrid by default

• Combine dense embeddings (chosen after testing on Arabic text, not just on multilingual leaderboards) with lexical BM25 search for exact terms such as policy numbers, product codes and names.
• Add a re-ranking step with a model that handles Arabic well; it is often among the most effective improvements available.
• Chunk by meaning: one article, clause or procedure step per chunk, with modest overlap and the parent heading attached.
• Rewrite queries: expand acronyms, map dialect phrasing to formal terms, and handle Arabic questions over English documents when the corpus is mixed.

Evaluation: build an Arabic test set before launch

Create a golden set of real questions, written by domain experts in both formal Arabic and the dialect your users actually type, each linked to the passage that should answer it. Then measure the two halves of the system separately:

• Retrieval quality: does the correct passage appear in the top results (recall at k)?
• Faithfulness: is every statement in the answer supported by the retrieved context?
• Citation accuracy: do the cited sources actually contain what the answer attributes to them?
• Abstention: when the answer is not in the documents, does the assistant say so instead of improvising?
• Language quality: is the Arabic clear, appropriately formal or conversational, and free of awkward translated phrasing?

Automated scoring with a language model acting as judge speeds this up, but calibrate it regularly against human review of a sample, and confirm that its judgments agree with those of your experts.

Data governance is architecture, not a policy document

• Permission-aware retrieval: enforce document access rights at query time, filtering before retrieval rather than hoping the model withholds information afterwards.
• Classification-based routing: decide which data may be sent to an external model API and which must stay on infrastructure you control, including in-Kingdom hosting where required.
• Personal data: assess your obligations under Saudi Arabia’s Personal Data Protection Law (PDPL) with your legal advisors, and minimize or redact personal data in indexes, prompts and logs.
• Prompt injection: treat retrieved documents as untrusted input; instructions embedded in a document must never override system rules.
• Auditability: log questions, retrieved sources and answers with defined retention, so behavior can be reviewed and explained.

An assistant that cannot show where its answer came from is not ready for the enterprise.

A pragmatic rollout

1. Start with one well-bounded domain, such as HR policies or an internal IT knowledge base, with a clear content owner.
2. Build the golden set and baseline measurements before the pilot opens.
3. Release to a small internal group with a simple feedback control that captures why an answer was wrong.
4. Fix the weakest stage first (frequently extraction or retrieval), then expand to the next domain.
5. Make content owners part of the system: an assistant is only as current as the documents behind it.

Built this way, an Arabic AI assistant becomes a dependable interface to institutional knowledge rather than a novelty, and the same pipeline extends naturally to customer service, internal search and decision support.