Gemini Bridge

Inspiration

The education system lacks personalization, featuring large lectures that don't cater to most students' pace, and lacks resources for individualized guidance to address knowledge gaps. If a student desires personalized instruction, they must pay exorbitant amounts of money for private tutoring. For students where accessibility is an issue, asynchronous online formats are helpful, but lack the interactiveness that drives learning. Furthermore, students with ADHD can benefit greatly from having an evaluation system that checks their grasp of key learning objectives and tailors their experience to their level of understanding. Gemini Bridge revolutionizes this space as a highly accessible, personalized, and interactive product to ensure authentic learning.

What it does

Gemini Bridge revolutionizes learning with its tailored approach, offering concise educational segments, interactive quizzes, and engaging chats. Through AI-generated questions aligned with learning objectives, students immerse themselves in the material and reinforce their comprehension. What sets Gemini Bridge apart is its personalized feedback system, pinpointing knowledge gaps and seamlessly directing learners to relevant sections within the video content. Powered by Gemini AI, the platform constructs a knowledge graph derived from instructors' materials, effectively mapping core concepts and detecting gaps in a students knowledge of pre-requisites. This intelligent tool not only shapes subsequent content but also tailors questions to address individual learning needs, ensuring a customized learning journey for every student.

How we built it

We built a web application with a JavaScript-based frontend that uses React and MaterialUI to render an elegant, minimalist learning page alongside a Flask backend that interacts with Gemini AI and communicates with the frontend with REST APIs

Challenges we ran into

Uploading video frames was difficult and slow until we understood how to send multiple frames at once after uploading them. Prompting the model to answer questions on topic without getting distracted was difficult, and we spent a lot of time exploring how to prompt Gemini to stay focused on the video and learning objectives.

Accomplishments that we're proud of

The best achievement is integration with video content for a seamless learning experience that can customize learning for a vast array of educational content and student backgrounds. We designed robust prompts and system information that allows the model to remain in the context of our learning application and target the proper questions and feedback. Furthermore, we achieved streaming in the chat output that achieved a natural, fluid interaction element.

What we learned

We learned that Gemini can handle video frames well for guiding content, and with that it can understand knowledge gaps. We learned about React, MaterialUI, Flask for Python-based Rest APIs, Organizing API-FrontEnd architecture, and Gemini API calls, and how to leverage AI for both coding support and ambiguous use cases. Additionally, we learned how to design and implement full stack applications effectively.

What's next for Gemini Bridge

We aim to add an instructor view for advanced learning analytics, so that they can understand where knowledge gaps are common among students, supported by a breakdown of questions and topics that a student struggled with. Furthermore, expansion to additional education platforms to streamline the process of checking comprehension and learning objectives can be an invaluable tool to both educational institutions and private users. We also seek to have better integration with specific timestamps so that, based on identified gaps in knowledge, the application can point students to specific areas of the video to review before moving on.