Cadu | Devpost

Cadu's increases patient safety by making medical information more accessible
An admin interface to monitor the iterative AI generation

Background

In the United States today, the healthcare system faces a significant challenge with over 7 million misdiagnoses annually, leading to approximately 2.6 million cases of harmful mistreatments. There’s an overreliance on an understaffed industry, leading to lasting damages for patients who are unfortunately harmed. In marginalized communities, these problems are even more pronounced.

To combat these misdiagnoses, we imagined what healthcare would be like if every single patient had a medical research team – high quality, source-verified research that assists both the doctor and the patient in making informed healthcare decisions.

User Personas

We developed user personas to better understand our target audiences and guide our thought processes

That's why we’re introducing Cadu — a revolutionary approach to patient diagnostic safety.

How Cadu Works

Cadu surpasses contemporary LLMs by employing a multi-agent model. This model, akin to human cognitive processes, breaks down complex tasks like providing a second medical opinion into smaller, manageable subtasks, enhancing effectiveness.

Multi-Agent Model

Additionally, Cadu incorporates Retrieval-Augmented Generation, allowing it to access and utilize health datasets from authoritative sources like the NIH and CDC. This feature ensures that Cadu's outputs are firmly rooted in up-to-date and accurate medical information.

Our vision with Cadu is to pioneer in the field of Machine Learning architecture through Iterative Generation, where AI can autonomously evaluate and refine its outputs. By combining these innovative elements, Cadu stands as a next-generation tool in the realm of medical research and diagnosis.

Cadu, step-by-step

Cadu seamlessly integrates into medical workflows with transcription powered by Whisper, as well as manual data entry.
The multi-agent workflow splits this task into subtasks to assign to research agents
Searches across multiple authoritative sources like government medical paper databases to find possible diagnoses
The quality assurance agents verify this information across the other sources and ensure it was accurately described
The software compiles a report in a matter of minutes citing where it finds information

Design Docs

Lo-Fi

Our initial lo-fi designs focusing on three stages: Input, Research, and Output

Hi-Fi

Hi-Fi Designs The final high fidelity designs we used on our web app

Tech Stack

Cadu’s frontend was built with a wide array of frontend technologies. Our stack included React, Next.js, Typescript, Tailwind CSS, Shadcn, AWS, and HTML/CSS. This combination supercharged our ability to move quickly and develop UI effectively. One of our frontend developers was a first-time hacker, so having an accessible tech stack also enabled us to work well together.

Tech Stack

In addition, we leveraged Transformers.js and Whisper Web to bring powerful, client-sided Speech-to-Text transcription into our project. With Whisper’s transcription, we’re able to provide a more seamless integration for medical professionals, as well as a manual text entry for patient symptoms.

Our backend was built with Javascript, Python, Node.js, Gemini, and OpenAI. We built a hierarchical multi-agent system on top of Gemini and OpenAI, enhanced by retrieval augmented generation. We also learned how to work with AWS and eventually deployed our application to AWS Lightsail.

Cadu Architecture

Challenges we ran into

The most complex component was our real time display of our multi-agent model. It was challenging to develop the structure and correct ordering for rendering each agent’s responses in real-time to the users. But after lots of hard work, using WebSockets, we were able to stream the interactions between agents and show their iterative communications. Although it proved challenging, we felt that the effort was necessary as it best demonstrated the capabilities of a multi-agent model and kept users engaged in the autonomous process.

What we learned

We delved deep into the fascinating realm of artificial intelligence and machine learning. We focused on mastering Retrieval-Augmented Generation (RAG), which combines the power of information retrieval with advanced language generation techniques. This approach enhances the ability of models to generate more informed and contextually rich responses. Additionally, we explored the intricate world of embeddings, understanding how these mathematical representations effectively capture the nuances of language and semantic relationships in a multi-dimensional space.

Our exploration of Large Language Models (LLMs) provided us with a comprehensive understanding of how these powerful tools process and generate human-like text, pushing the boundaries of natural language understanding and generation. Furthermore, we developed a multi-agent system, a complex and dynamic structure where multiple agents interact, learn, and make decisions, demonstrating the practical applications of AI in simulating real-world scenarios.

What's next for Cadu

We’re really excited about Cadu’s potential to assist in democratizing and increasing access to medical information for both patients and doctors. Still, we know there’s a variety of additional use cases for a powerful multi-agent model; moving forward, we’d love to explore how hospitals could integrate and utilize Cadu. Beyond improvements to the model, we want to explore how we can best integrate Cadu into existing workflows, as well as making it more accessible to more people.