Squash It

Inspiration

The inspiration of the project comes from the struggle of keeping track of casual and professional squash games, The aim of this project was to fix that by recording scores, ensuring players keep to the rules and also announcing handovers, the score and the game status

What it does

The project keeps track of a game of Squash by regulating and announcing the state of the game / if someone won, and handover times all whilst using computer vision to keep track of the score

How we built it

We built it using a range of techniques from Computer vision, image stream synchronisation, AWS and more

It starts off with multiple image streams frame by frame being merged into one big frame with the two previous frames that is then sent to AWS S3 Bucket via an AWS SQS service from a Raspberry Pi.

In the cloud, we then use computer vision and the latest deep learning models specifically focusing on the differences between frames to track the ball and bounces on the walls

The retrieved result from AWS is then retrieved and used to determine the direction, velocity, position and bounce of the squash ball. This is used for checking whether the ball is in bounds and whether it hit the wall or not and also for determining which player hit it.

We then track the score and game progress using golang and then the processed information is delivered to the players via speakers using a state-of-the-art AI voice

Challenges we ran into

Due to the complex nature of the infrastructure and project, we faced many problems ranging from sending data to the Raspberry Pi but more specifically the Bounce detection with computer vision had to most issues with shaky video and needed multiple iterations to get right

Accomplishments that we're proud of

We are proud of the extremely complex infrastructure and techniques we used to create the project and the overall result of a real-time functional squash tracker.

What we learned

We learned to make sure we get the required video ASAP, The difficulties of detecting a moving ball and when it bounces and how much work new computer vision ideas take to work

What's next for Squash It

Next up would be using more cameras to more precisely capture the ball and movements within the game. Potentially even extending to making new minigames by tracking the ball and hitting certain points on the wall using a projector.

Built With

• amazon-web-services
• deep-learning
• opencv
• python
• raspberry-pi