Stack
React Native (Expo), Python, PyTorch, FastAPI, YOLOv11, EasyOCR, OpenCV
Background
Clarity started as a way to help visually impaired users read text around them. The main thing I realized while building it was that reading something close up, like a label or small print, is very different from trying to read a sign across the street. Regular OCR works pretty well for nearby text, but distant text can become so small in the image that it struggles to detect it at all.
Overview
Clarity is a React Native mobile app with two modes depending on how far away the text is. Near mode is for things like labels and small print. It reads the detected text aloud, and you can tap any individual line to hear just that line.
Far mode is for things like signs in the distance. Since EasyOCR struggled to find very small text directly, I fine-tuned a YOLOv11 model on around 1,200 images from the ICDAR scene-text dataset to locate the text first. Those regions are then merged where needed, cropped out of the image, and passed to EasyOCR to actually read them.
The results are sorted back into reading order and sent to the phone, which reads them aloud.