Isharapal Real-Time AI Recognition Of Pakistani Sign Language

Pakistan Sign Language (PSL) is a primary communication medium for the deaf and hard-of-hearing community in Pakistan, but limited public awareness creates substantial communication barriers. This work presents a real-time isolated PSL recognition system for 40 Urdu alphabet signs using temporal sequence modeling. The system extracts hand landmarks with MediaPipe and lightweight per-frame visual features, aggregates them into fixed-length temporal sequences, and performs temporal modeling with a compact Temporal Convolutional Network. A browser-based frontend captures webcam video; a Python Flask backend running PyTorch performs real-time inference using a sliding-window approach with temporal smoothing and automatic buffer reset on hand disappearance to stabilize predictions. The model was trained on a custom dataset of video sequences and images; class imbalance was mitigated via weighted loss and data augmentation (temporal shifts, landmark perturbation). Experimental evaluation under controlled conditions demonstrates high recognition accuracy on the test set while maintaining real-time performance. Results indicate the feasibility of browser-deployed, TCN-based PSL recognition, with future work targeting word- and sentence-level recognition, signer independence, and bidirectional communication support.