Muhammad Zubair: Engineering the First Khattak Pashto LLM & Pashto AI
Muhammad Zubair
2026-08-20 • 12 min read
Introduction: Advancing Pashto AI
I am Muhammad Zubair, a Software Engineer specializing in AI Infrastructure and Cloud Platforms. While Large Language Models (LLMs) have mastered major world languages, Pashto AI remains a frontier with significant challenges. Most existing models fail to recognize regional dialects, which is why I engineered Khatta-ka AI—the first-ever Pashto LLM specifically fine-tuned for the Khattak (Khatak) dialect spoken in Karak, Kohat, and Nowshera.
Standard Pashto AI models often fail to understand regional dialects. Khatta-ka-LLM was developed to bridge this gap. It understands the unique vocabulary, spelling, and grammar of the Khattak Pashto dialect (e.g., using "شوشته" and "ایس نه کاون").
Built on top of junaid008/qehwa-pashto-llm (Qwen2.5-7B), Khatta-ka AI has been fine-tuned using LoRA via Unsloth to adapt to the Khattak dialect. It understands regional Khattak vocabulary, grammar shifts (such as using "یں" instead of "یم" for the first-person copula, and "کاوں" instead of "کوم"), and phonetic variations.
The Official Khattak Dialect Grammar & Rulebook
To build a truly effective Pashto LLM, I digitized the unique grammatical rules of the Khattak dialect. Here is the complete summary of the rules this AI has been trained on:
1. Pronouns & Possession
- My / Mine: Uses مو والا (Mo wala) instead of standard Zama.
- To me: Uses موته (Mota) instead of standard Mala.
- To you: Uses توته (Tota) instead of standard Tata.
- From me: Uses رونه (Rona) instead of standard Rana.
- We: Uses موخ (Moxh) instead of standard Mung.
2. The "Noon Ghunna" (ں) Rule (Crucial!)
- Verbs (First Person): When saying "I do something", the verb ends in a nasal Noon Ghunna (ں) instead of the standard "M" (م).
- I do: کاوں (Kaon)
- I wash: وینزاں (Wenzan)
- I write: لېکاں (Lekan)
- I will come: روساں (Rosan)
- The "I am" Copula: Standard Pashto uses یم (Yam). Khattak uses یں (Yay + Noon Ghunna). Example: Za khwash یں (I am happy).
- Nouns: Nouns do not use the Noon Ghunna. They keep the hard "N" (ن). Example: Asmon (اسمون), Baron (بارون), Pashtun (پښتون).
3. Tenses & Verbs
- Future Tense: Uses بو (Bo) instead of standard به (Ba). Example: Za bo sabo char kaon (I will do work tomorrow).
- "To Say/Tell": Uses یای (Yai) for "he says" and ویاوں (Wayaon) for "I will tell".
- "To Read": Uses یای (Yai) instead of standard Lwali. Example: Agha kitob yai (He is reading a book).
4. Phonetics & Vowel Shifts
- The "O" Shift: Many standard "A" sounds shift to a deep "O" sound.
- Asmaan ➡️ Asmon (اسمون)
- Baraan ➡️ Baron (بارون)
- Bazaar ➡️ Bazor (بازور)
- Maakham ➡️ Mashom (ماښوم)
- Yaada ➡️ Yoda (یوده)
- Masculine Singular Nouns: Words that end in "i" in standard Pashto often end in "a" in Khattak.
- Gadi (Car) ➡️ Gada (ګاډه)
- Kanri (Stone) ➡️ Konrha (کاڼه)
- Halak/Kakai (Boy) ➡️ Orhka (اوړکه)
5. Unique Khattak Vocabulary
- Family: Grandmother = انیو (Anyo)
- People: Boy = اوړکه (Orhka), Girl = اوړکې (Orhkay)
- Animals: Snake = منګور (Mangor)
- Nature: Day = ورېز (Wrez)
- Actions: Stand up = چګ شه (Chag sha), Flying = نغېږي (Neghezxhi), Crying = ژوړې (Zorhay)
- Adjectives: Big = استر (Astr), Bad/Dirty = کسو (Kso), Tired (Singular) = ستړه (Starha)
- Items: Tea leaves = پتې (Pathay)
Technical Architecture & Implementation
As a Software Engineer, I utilized the Qwen2 7B architecture and the Unsloth framework to ensure the model is both accurate and computationally efficient.
Installation & Setup
pip install transformers accelerate torch peft
# For faster inference (Recommended)
pip install unsloth
Usage Method: Unsloth (2x Faster) codePython
from unsloth import FastLanguageModel
import torch
model_name = "Muhammad-Zubair796/Khatta-ka"
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = model_name,
max_seq_length = 2048,
dtype = None,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.
### Instruction:
{}
### Response:
"""
inputs = tokenizer(
[alpaca_prompt.format("Where did the boy go?", "")],
return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]
print(response.split("### Response:\n")[-1].strip())
Training Details This model was trained using Unsloth for optimized, memory-efficient fine-tuning by Muhammad Zubair.
| Parameter | Value |
| --------------- | -------------------------- |
| Base model | junaid008/qehwa-pashto-llm |
| Epochs | 4 |
| Sequence length | 2,048 tokens |
| LoRA Rank (r) | 16 |
| Learning rate | 2e-4 |
| Optimizer | adamw_8bit |