Featured case study
Offline-first trilingual assistive communication for Sri Lanka, with on-device supportive facial expression recognition. Built as a final-year engineering system, not a PDF dump.
Thesis
Smart AAC System with Facial Expression Recognition for Autism
Stack
Flutter, TensorFlow Lite, offline-first
Languages
Sinhala, Tamil, English
Problem through impact, as documented in the final report.
Problem
In Sri Lanka, many children with limited speech need affordable AAC that works in Sinhala, Tamil, and English, offline, on ordinary Android phones. Imported English-first tools are often too expensive or a poor cultural fit.
Research
Final-year Software Engineering project (CS6P05ES) at ESOFT Metro Campus. Literature review, caregiver and therapist conversations, and clinical interest from Pragathi Centre / National Hospital Galle shaped the requirements.
Architecture
Offline-first Flutter client with no backend in the submitted build. Local vocabulary, on-device TensorFlow Lite models, and no camera-frame upload. Earlier cloud dashboard ideas were deferred to keep the deployable core honest.
Development
Core AAC flows (registration, home, categories, favourites, settings) plus a supervised camera-expression screen. Companion simple_aac_app path for caregiver-customisable cards. EfficientNetB0 primary TFLite model with MobileNetV2 fallback.
Testing
TFLite compatibility checks, release APK verification, manual real-device walkthroughs, Google Play internal testing, and public-dataset model evaluation (52.9% test accuracy; weighted F1 0.54). Anonymous early tester questionnaire n=11. No children recruited for clinical evaluation.
Impact
A working foundation for free or low-cost supportive AAC and observation in Sri Lanka, with a formal letter of support for a future pilot after ethical clearance. Not a diagnostic device and not clinically approved.
Children with limited speech need a way to express everyday needs. In Sri Lanka that means Sinhala and Tamil as first-class languages, English where useful, offline use on ordinary phones, and a price families can actually afford. The thesis positions this as a free or low-cost supportive tool, not a subscription product aimed at Western English-only markets.
The technical contribution is integration: trilingual AAC plus cautious on-device facial expression recognition for observation support. It is not a novel emotion algorithm, and it is not a diagnostic system.
Trust boundaries
Supportive collaboration with Pragathi Centre / National Hospital Galle. Not clinical approval, not medical certification, and not a deployed medical device.
Formal letter confirming collaboration on the Pragathi AAC App and interest in a future pilot after ethical clearance. Correspondence frames the app as supportive observation and learning, including for medical students.
Early discussions and openness during field exposure. Informal feedback from speech-language therapists and parents on the AAC interface.
Clinical interest, design input, and a documented pathway toward a supervised pilot once ethics and health-sector approvals are in place.
Not Ministry of Health approval. Not a completed clinical pilot. Not diagnosis of autism or emotion. Not production healthcare deployment. No child facial-expression dataset was collected for this thesis.
Understanding the context in which AAC support may be used.
The project was informed by exploratory field exposure at Karapitiya Teaching Hospital, providing contextual insight into environments in which AAC support may be used.
This visit was exploratory field exposure rather than a clinical trial or formal clinical validation study.
Field Exposure
AAC demonstration during field exposure.
Field exposure video
Privacy-redacted field-session recording. Faces are obscured; audio is omitted.
A demonstration of the developed application.
On-device FER through TensorFlow Lite. EfficientNetB0 is primary; MobileNetV2 is the fallback if load or tensor validation fails. Gates include face detection, confidence checks, top-1/top-2 margin checks, and explicit uncertain states.
Reported public-dataset test accuracy 52.9% (weighted F1 0.54). Useful only as a cautious supportive cue under caregiver supervision. Confusion matrices and TFLite verification evidence are in the thesis gallery below.
Figures, screenshots, clinical photos, and testing evidence extracted from the thesis PDF. Nothing here was invented for the website.
Source documents that exist in the project archive.
Trilingual Offline Smart AAC with On-Device Facial Expression Recognition for Autism in Sri Lanka. Venue: EICON 2026, ESOFT International Conference. Paper ID: FPC21. Status: Full paper submitted; major revision completed; camera-ready manuscript prepared (July 2026). Submission is not the same as acceptance or published proceedings unless later confirmed.
