Jeopard.ai

Inspiration

Playing games is a great way to learn. Not only does it provide a fun way to study, but can also force you to think about content in a new way that will help you recall what you need to know later. We're big fans of the popular game show "Jeopardy", but the work involved in manually creating games to study often takes too much time that could be used to study instead.

What it does

We leveraged Google's newly released Gemini 1.5 Pro API to generate Jeopardy game-boards from a diverse set of image and text inputs, taking full advantage of the model's 1M token context window and support for many modalities. Users can upload both PDFs and images, and jeopard.ai will generate a brand new jeopardy-style game consisting of 6 categories with 5 questions each. Users can play solo, or add up to 3 other players and provide the questions to various answers worth $200-$1000. Upon responding with the question, we further leverage Gemini to judge the correctness of the player's response in a natural way that won't punish small typos or different answering styles (e.g. "Who is Jefferson?" when the full question is "Who is Thomas Jefferson?").

How we built it

Gemini 1.5 does not natively support PDF file uploads. However, since students likely will scan or export their notes as PDFs, we felt it important to support this file type. We achieved this by stripping any recognizable text from the PDF, as well as rendering each page of the PDF to a PNG image to also be included in the context window. As such, we are able to generate rich games from typed notes, handwritten notes, and even diagrams/drawings in rich text articles.

Our application features a lightweight Flask API to provide game generation and answer checking. We use the newly released Material Joy UI to provide a sleek and modern-looking game interface.

Challenges we ran into

We found that Gemini sometimes struggled with providing game output in the exact format we'd expect. Sometimes, the JSON output would be slightly malformed (missing a quote, a random character somewhere). We were able to find an open-source project for json repairing, and integrated this into our project to have more robust output. Designing prompts was also a challenge, since we needed a specific amount of questions for the game and were limited by API rates to only 1 query per game in order to have faster response times.

Accomplishments that we're proud of

We’re proud of making this product. Even when we were dev-testing this, we realized how much of an impact this could be for studying and understanding exams. In fact, some of the questions that Gemini reproduced when looking at my LLM lecture notes, matched exactly to some questions from the practice midterm. It’s always important to be considering the use-case for AI, and we think this app could be an invaluable tool for education.

What we learned

This was the third hackathon (for each of us) that we competed in and I think we learned how to more quickly get things done. Last MHacks for example, we ended up working through the night in order to get the integration done, but this time we were able to spend our extra time cleaning up the UI. Also, I felt we were much more familiar with the frameworks, and manipulating them to really make the project look how we wanted it to. Like always though, we learned that with more time we could always improve the project.

What's next for Jeopard.ai

The next stage for Jeopard.ai is to store previously generated gameboards in a database so that users can also take advantage of what other people have created. We could incorporate the same sort of guidelines, ranking systems, and reinforcement learning system that quizlet does. Another feature we want to implement is a “contesting” feature so that the user could argue with the chatbot if they believe they did in fact get the answer correct. Finally, we designed this game with the idea of playing it locally, but a natural extension would be to allow for multiplayer multi-system support with “buzzing in” to “question the answers!”

https://www.loom.com/share/674c77651bb7484491a8f0af3428f6bb?sid=4ea5114f-4487-481a-b112-a397534b9259