gLitter

Inspiration

Following the theme track of social good one of the biggest issues that affects not just us but the environment as a whole is litter. Our project aims to give more of an incentive for people looking to try to keep their community and local environment a little cleaner. By adding game mechanics such as levels and leaderboards for cleaning up litter we hope to achieve that goal.

What it does

What gLitter aims to do is to make users more aware of the environment by giving collected litter a value and a means to compete against other people. Giving a value to litter will make people more aware of any trash they could pass by without even noticing. With a way to score the collected litter, we hope this would drive users to collect as much as they can to get a higher score leading to a cleaner environment as a side effect. But there is no point in gaining a higher score without having a way to show you have the highest score. With the implementation of a leaderboard, it adds a layer of competitiveness within the app hoping to increase users' drive to collect more litter. Also with every photo that is taken a fact about recycling and or the environment is shown.

How we built it

gLitter is built with React on the web host with Google Cloud. gLitter includes a map for users to search and mark where areas of litter and trash are using Google Maps API. The browser uses tracking data given by the user to place markers on the map. To submit and gain points for collecting trash, the user takes a picture in the app and a machine vision model scans the image for various types of trash to award points. By using Roboflow and a dataset of 1500 annotated pictures of litter was used to train different available models on Roboflow.

Challenges we ran into

The first main challenge was getting location data for users to add markers to the map. We first thought we would have to use geotags when a user took an image on the app but since this is a web app the browser can simply request location data from the user.

The Second main challenge was adding the trained vision model into the app to be used to scan and detect trash from the uploaded photos. Roboflow offers a way to add the model within the web but with how the app is hosted on Google Cloud some issues couldn't be solved. The javascript add-on that Roboflow provides has limited functionality for the problem we had run into so had to use Openai GPT-4 API with some prompt engineering as a backup.

Accomplishments that we're proud of

One accomplishment we are proud of is the potential of our app to make a change for the environment even in the slightest. While using multiple technologies to implement even a basic draft of our idea.

What we learned

From making gLitter we have learned how to make a React web app that uses location data, takes a photo with direct upload in the browser, how to make data sets to train an AI, uses Google Maps API, Google Cloud for hosting, and Google database for storing users.

Riley: I had never used Google Cloud to host a React app before, nor had I ever used BigQuery to manage databases. After this hackathon, I can confidently say that I have learned a LOT about building apps within Google Cloud, as well as expanding my database knowledge with BigQuery.

Aryan: Before this, I had never used TypeScript or React on a real project before. It was kind of tough to catch up, but I learned a lot in the process about web development.

Ryland: I have never made a dataset for making or even training an AI before. I scoured the Roboflow documentation for making and hosting a machine vision model for the web. I have also come across ways to even train models in Jupyter and Google Colab notebooks. I have also found a model called Detectron2 by Meta that includes various ways to train your own open-source vision models. After this hackathon, I will definitely be exploring making my own models for projects in the future.

Averi: Before this Hackathon, I did not even have a GitHub account but quickly learned all that it offers. From being able to make stylized ReadMes to quickly sync all code and art assets into the project. Also learned how to make pixel art in Aseprite despite never have done so before.

What's next for gLitter

Future updates for gLitter included weekly challenges, a friends list, an in-game shop such as decorations for user profiles, and local events. The biggest update would be moving from a web app to mobile apps on both Android and Apple devices.