Senara
Senara is an assistive mobile application designed to bridge the communication gap for individuals with sensory disabilities. It utilizes the device camera to scan and translate SIBI (Sistem Isyarat Bahasa Indonesia) sign language gestures in real time using integrated machine learning models operating directly on-device.
Project Overview
Senara serves as a real-time translator for the Indonesian Sign Language System (SIBI), designed to empower deaf and hard-of-hearing communities in their daily communication.
The Problem
Deaf and mute individuals using SIBI sign language face communication barriers when interacting with the general public who do not understand sign language.
The Solution
A mobile application that captures real-time video of sign language gestures through the device camera, processes video frames locally using a quantized TensorFlow Lite model, and converts them to readable text and spoken audio.
System Architecture
Built on cross-platform Flutter (Dart). The machine learning pipeline utilizes TensorFlow Lite for on-device gesture recognition, with Google Firebase for data management.
Key Technical Challenges
Achieving high-accuracy gesture detection with minimal inference latency on mobile devices without causing thermal throttling.
Lessons Learned
Optimizing the neural network architecture by quantizing the TFLite models reduced latency while maintaining high recognition accuracy.