OneSkin

• 

  AI Derm chatbox - displaying sample questions that the bot will answer

• 

  Disclaimer

• 

  Results of AI scan -- this is an example of what shows up when it is a benign lesion

• 

  Response to one question - these are all best practice guidelines used by Dermatologists

• 

  Instructions to take accurate photo pt 2

• 

  Instructions to take accurate photo

• 

  Intro Screen

• 

  Review photo of skin lesion

Inspiration

With a background in healthcare, I have always sought out to build something that uses technology to improve the healthcare system. Five years ago, my brother was diagnosed with Hodgkin's Lymphoma. Luckily, we had a healthcare professional in our family who was able to order the testing necessary to diagnose his condition early, but this access to early detection is not available to a vast majority of Americans. This project stemmed from the idea of providing that same level of access my brother was able to receive to others.

What it does

OneSkin is a mobile app that allows users to take a picture of a skin lesion on any part of their body and get an AI-based analysis from a proprietary model. The results display to users whether the lesion is benign or malignant, as well as providing next steps for monitoring and following up with a provider.

How we built it

We built the mobile application in a cross-platform framework called Flutter. We have an API built with express and an image categorization model built on Vertex AI, all hosted in a Google Cloud project.

Challenges we ran into

Initially, we created a categorization model using TensorFlow, so we could train locally. We ran into issues when training the model with a relatively large dataset, so we had to compromise at first in the size of the sample. We also encountered some challenges when trying to host it and serve it as an API on the cloud. We then quickly turned to Vertex AI and leveraged its ability to host a model in just a few clicks, as well as train a model with a larger dataset in a fraction of the time.

Accomplishments that we're proud of

We were set at the beginning that we wanted to create a model instead of using existing AI technologies like chatGPT. So that we could provide value not only through our app, but break the barriers of our use case and integrate into other platforms, augmenting our mission to facilitate early and accurate detection. We are also proud of being able to stick to a delivery schedule, and stick to the strategy to create a first iteration of a proof of concept, and then execute incremental enhancements that would lead to a finished product on schedule.

What we learned

We learned a lot about training a model and the vast capabilities of AI/ML, especially in the healthcare field. Given the importance of the real world application of this product, we performed a lot of research - referencing medical databases, consulting Dermatologists, Primary Care providers, etc. Through this project, we gained a deep understanding of the healthcare system and how we can continue leveraging technology to improve it.

What's next for OneSkin

Our main goal is to search for resources in order to continue building upon this initial concept and train the model with more data. In the future, we plan to build features to store and track skin lesions so users are able to pinpoint when lesions grew or exhibited other concerning characteristics. Another thing we would love to implement is the ability to take a larger image (for example, take an image of the full back) and identify/ map out concerning skin lesions from that one image. For individuals with many moles, this could be extremely beneficial to identify areas of concern without scanning every individual skin lesion or mole.

Built With

• apis
• ci/cd
• express.js
• figma
• flutter
• github
• google-cloud
• materialdesign
• node.js
• vertexai