MetaNight shade

example of how night shade effects image generation

Second video explaining our Nightshade script

Inspiration

The primary inspiration behind this was Nightshade. Since it is able to edit images to prevent Deep Fakes. A Deep Fake is where they use photos of you to create new photos of you to pretend to be you. link ![]https://imgur.com/a/f0a1Rkf)

What it does

Our project goes to a public Instagram account scrapes the photos off the account. Runs through the Nightshade GUI. Uploads the results to a MongoDB server. Then lets you view the results on a web or mobile app.

How we built it

We didn't create the Nightshade GUI, but we had to use it. The GUI was interacted with the python pyautogui library feeding it file data. Later the Instagram web scraper was built using the library. Then after the scraping was done it was uploaded to our MongoDB server using pymongo library. With displaying data using the dart framework Flutter. Everything else was connected with Flask to function as the backend server to request to the Nightshade GUI pyautogui script to start it up and also to start scraping an Instagram account.

Challenges we ran into

Some of the challenge face include storing images on MongoDB. We did not know how to do that. We found out that you could use python pickling on the images. We could not just do command line to use Nightshade, so we had to use the GUI and we had to detect when it was finished with a batch of images. MongoDB Atlas was not working on flutter. It seems like there is not Flutter support for that yet. So we used Flask as a middle man between MongoDB server and Flutter app. This script only works on public Instagram profiles. MongoDB doesn't like you using two computers. Additionally it would have been nice if the nightshade model had just been open source or had some kind of api that could be interacted with rather than the hacky way we automated it. With a lot more time and knowledge it would have been nice to just create our own version of the nightshade model but time absolutely did not allow that.

Accomplishments that we're proud of

Probably the most impressive thing was that we were able to run a batch of on a Instagram account and successfully display the images on the Flutter app. It isn't always a guarantee to that you can link a back-end, database, and a front-end in just a weekend. It was also the first time using computer vision libraries. It wasn't at all that bad and I can think of a couple projects I want to do personally now.

What we learned

We learned a lot about Flask and sending information over. Learned a bit more about Flutter. Learned a lot about scripting and file finding using pyautogui and the os libraries. Got a bit of web scraping experience as well. It was our first time using some pre-built models. Creating and training models from scratch is something we've done before with tensorflow but it was nice using open-cv to use prebuilt models that can easily be deployed. However the model that was used wasn't as good as I would have liked it to be, but hey it's fast.

What's next for MetaNight shade

Up next would ideally be to implement a solution for private accounts and get a public app out. The current database is not optimized for keeping photos secured. Granted these photos we store on the database are all Nightshade edited photos which shouldn't be usable for Deep Fakes. We are aware of how to use our program to pull data for private instagram accounts but would need to implement many security measures as it involves passing passwords. Additionally it would be nice to make the nightshade server multithreaded. A handler function was designed with this in mind but ultimately not implemented in the end because of how intensive just a single application is. To process 1 image with lowest poisoning strength and lowest render quality takes about 3 minutes minimum.