Contrail

a graph generated from a snippet about counting vowels in a sentence.

Video here: https://www.loom.com/share/b02e719ed147494bade8f08e13d57b11?sid=2fb3abc2-a905-4d8c-a077-70a03cc96da8

Inspiration

Our inspiration comes from debugging complex code, where logical flow is incredibly important. We found that there is a gap in visualizing code logically, and wanted to use the power of Gemini to translate from well documented code to Mermaid, a popular flowchart design language.

Often, in languages like C or C++, there is not a stack trace when a program exits with a segmentation fault, making it even more important to have a thorough grasp on the logical intricacies of a piece of software. As logic and codebases grow, it becomes increasingly difficult to keep this all mentally, or in the format of comments strewn throughout. That's where Contrail comes in!

Contrail isn't just for debugging, though. We wanted to build an AI coding assistant that breaks away from the usual copy-and-paste we are all used to by now. Instead, Contrail allows you to input multimodal problems into Gemini and generate a flowchart that way, too. This way, AI-generated code becomes explainable and modular. In this mode, double clicking the flowchart will paste the associated generated code into the editor, seamlessly integrating with existing code.

What it does

Contrail is a VSCode extension that makes logic visualization easy. All it takes is selecting a code snippet, opening Contrail, and a detailed flowchart appears side-by-side with your code editor. You can then double-click any box in the flowchart to highlight the corresponding code, to pinpoint any logical errors.

A second feature in contrail leverages Gemini's multimodal capabilities to solve problems on its own, generate code, and generate a flowchart from that. Double clicking on this flowchart's boxes pastes the relevant AI-generated code into the editor.

How we built it

Contrail is very lightweight, and is built completely in TypeScript and CSS for the VSCode extension frontend and in python with poetry and fastAPI for the backend that interfaces with the Gemini API. We had to learn about the Mermaid language rules, but this is entirely wrapped in Gemini's functionality.

Challenges we ran into

This was our first time making a VSCode extension, so setting up the infrastructure for that to wrap our functionality was a big challenge. This project also involved many hours of prompt engineering Gemini for our desired output formats, to enforce Mermaid language rules, and to reformat into API-friendly payloads. We did not have much prompt engineering experience coming in either, so that was a big feat to overcome.

Accomplishments that we're proud of

We are very proud that we overcame what seemed to be a task suited only for a fine-tuned LLM through prompt engineering, and that we could realize our visions for VSCode user-facing interactions.

What we learned

We learned that as these models continue to accept larger and larger context windows, and as they draw from multimodal data sources, finetuning tasks and prompt engineering tasks may converge in the future. We grew a lot together as a team that had never worked together before, and each carved out our own niche in this project throughout the night and morning, and we are very grateful for the opportunity!

What's next for Contrail

Contrail will be launching on ProductHunt in the near future, and we are working to get minor features more ready for the end user. We are excited to use it in our own development workflows, and we hope you are too.