VisionVoice

Inspiration

We are both visually impaired, and come from immigrant families with histories of bad eyesight. Fortunately, we have the privilege of owning glasses and contact lenses. However, many people in the world do no have access to these items, such as our parents before they moved to the US, forcing them to stay visually impaired for life. We built our Chrome Extension with these people in mind. Our singular goal is to enhance and simplify the browsing experience of those who are visually impaired.

What it does

Our tool mainly accomplishes two tasks. When your cursor hovers over any text, our extension will read the text out loud, and also magnify it by utilizing a popup. When you move off of the text, the audio will stop and the popup will disappear. This is useful for a multitude of reasons. Magnification of the text will make it easier to read for those visually impaired, and audio will make it easier for them to follow along with the text.

How we built it

The first thing we had to do was establish the structure of our chrome extension. This includes creating the manifest.json file to declare the extension, and creating basic popup abilities for our extension. We then moved on to tackle the hardest part of creating this extension, audio text to speech abilities. To do this, we utilized Chrome's TTS API. As TTS is a background activity, while text detection is a content activity, this process was greatly complicated. We had to send messages in between our contentScript.js file and our background.js file. The background file would take in the text form the contentScript.js file, and then say the audio. After finishing the TTS, we moved on to do our magnification. The magnification ended up being very hard as well, due to the fact that mouse event sensing would not work. However, in the end, we found a solution and made it work.

Challenges we ran into

We ran into a multitude of challenges throughout this entire process. At the start, we struggled with console logging, which essentially ensures that our extension is actually activated and working. We struggled for an hour or so before we finally found a solution and were able to move on to the actual coding. Another issue we ran into was coding the TTS. We had to find a way to detect when the mouse was hovering over text and when it wasn't. To do this, we created listeners, for when the mouse was over the text and when it left. This enabled us to successfully implement TTS. When implementing magnification, we ran into the issue of mouse event sensing. However, we were able to resolve the issue and successfully implement the magnification.

Accomplishments that we're proud of

Coming into this hackathon, both of us had minimal web development experience. However, overnight we were able to greatly improve our app building and web development skills.

What we learned

Both of us learned a tremendous amount of web development skills. We enhanced our HTML and CSS skills, while refining our Javascript skills. We also learned an entirely new concept in Json.

What's next for VisionVoice

We plan to publish this Chrome Extension on the Chrome Web Store, and also hope to work on an app for Apple and Android Phones that has the same features as VisionVoice.

Built With

• chromeapis
• chromeextensions
• chrometts
• css
• github
• html
• javascript
• json
• vscode