ASL recognition app

A web app that recognizes American Sign Language in a live video feed and shows the matching letter or word on screen. Built for the Congressional App Challenge, District NJ-01.

Nov 2024
  • YOLOv8
  • Roboflow
  • Computer vision
  • Web app

From sign images to live recognitionModel pipeline

  1. Data
    Sign imagescollected and prepared
  2. Label
    Annotationbounding boxes in Roboflow
  3. Train
    YOLOv8custom object detection model
  4. Deploy
    Web applive video feed
  5. Output
    Letter or wordshown on screen

feedback loopTesting with members of the Deaf community fed back into the dataset and the model.

What I did

I owned the model end to end, from raw images to a working app.

  • Prepared and annotated the full image dataset in Roboflow.
  • Trained a custom YOLOv8 object detection model on it.
  • Deployed the model in a web app that detects signs in live video.
  • Tested with Deaf community members and refined the model to meet real accessibility needs.