PlantPartner

Plant Partner exploring the outside

Inspiration

We took inspiration from existing plant robots but emphasized putting them into the hands of the average consumer. We wanted to create a plant assistant to navigate a garden or greenhouse rather than being rooted in place. PlantPartner can help people grow their food daily without the headache of constant monitoring while keeping it at a reasonable price.

We also believe that everyone deserves to have access to food. Home gardening increased significantly in 2022, notably in urban and suburban areas. Growing food helps families, contributes to local food systems, reduces food miles, and promotes green spaces in urban environments.

What it does

Growing your food is a daunting task, typically involving intense research and constant attention. PlantPartner automates this process by routinely checking up on plants for various issues, including under and over-watering, temperature, AI plant health analysis, and pests.

How we built it

We built PlantPartner by first limiting our scope to assessing plants and providing actionable information rather than interacting with the plants and trying to do everything. We decided on a must-have functionality that would enable this and started designing the software to recognize issues with plants alongside the physical design of the robot. We ensured that individual aspects of the hardware, such as motors, the frame, sensors, power, and drivers, worked individually. We also ensured we could accurately assess plants using GPT-4 and the soil sensor. Then, after testing individual components, we combined all of these elements to create the working robot.

Challenges we ran into

Our robot has a lot of moving parts (figuratively and literally). Getting each idea to work independently is quite the challenge, though integrating all components into one cohesive system was one of the biggest battles. The use of outlines and rigorous planning beforehand helped tremendously, although integration always brings its own unexpected problems.

Along with integration came time management. Understanding when to cut your losses and pivot a design idea was difficult to experience, as needed aspects might not be working by the deadline, and working on that version of the implementation would harm the team. We had to be flexible, and the team showcased that greatly.

Accomplishments that we're proud of

The PlantPartner system's core is recognizing plant issues based on images. The pipeline for approaching a plant, collecting images and moisture data, and then using OpenAI’s API to upload and assess the image was a challenge to get right, and we are proud of making it work. One thing that we’re incredibly proud of is how we used prompt engineering to make sure that we were getting useful, accurate, and actionable information from the model. We are also proud of making the robot successfully drive, insert, and remove the soil probe and utilize all its sensors.

What we learned

Working on PlantPartner helped inform us of the current existing solutions and shortcomings facing the world today when it comes to food. We also learned how to use LLMs to solve a specific problem, organize code for a complex collaborative project, make hardware and software work together, and wire various sensors and motor drivers.

What's next for PlantPartner

PlantPartner still has some ways to go in order to become an even more reliable machine. The next goal is for PlantPartner to move autonomously. This would involve using a 360 camera attached to the robot so that it can spot the location of plants through a live feed. The python library OpenCV is a useful application that could be trained by using the camera input to locate the plants and move towards it. For now, PlantPartner only detects if there are bugs on the plants or not. The next step would be to fine tune ChatGPT4 to identify various invasive bugs and how to treat each type of species. Additionally, the detection of plant species could be more accurate, and so can the identification of wilting.

The aim is to use PlantPartner through all weather events, so weatherproofing the robot so that it doesn’t deteriorate through rainfall or extreme heat/sun would be ideal. These next steps should be enough to have PlantPartner be useful to gardneres, and encourage more people to take care of plants, grow their own crops, and overall enjoy the nature that Earth has to provide.