Cal Poly Pomona Hand Sign Detection

Hi everyone! We are BANSHEE UAV, a research project team at California State Polytechnic University, Pomona (Cal Poly Pomona) College of Engineering. Once again, we are teaming up with ROBOTIS this summer to utilize their Dynamixel technology. We are very grateful to have ROBOTIS’s support and hope the community can give us feedback on the achievements we will make this summer.

I am Winson Zhu and this is my 2nd year with BANSHEE UAV. I am the project lead for the team this summer and will keep the community updated with our work biweekly. Since everyone is just starting with their tasks, this week’s updates will be short.

6/9/24 Update
We attended the first ROBOTIS meeting and discussed the 3 sub-teams we have created and their tasks for the week.

Electrical Team:

  • Research what microcontrollers to use to control the arm and camera with pros and cons for each method.
  • Research what could power the whole system.

CAD Team:

  • Research about 3D model robotic hands and compare them (pros/cons).

Code Team:

  • Research on how we would be able to create a hand detection software that does not use YOLO (You only look once).

We also went over the overview of the internship and the goals we have in mind for it.

Overview: Use a robotic arm(with hand) with an OpenCV camera to detect a hand sign (high-five, fist bump, handshake)

Goals:

  • Code the camera to detect a hand sign (high-five, fist bump, or a number sign) and robotic hand/arm to have motor control​
  • Create a 3D model of robotic hand and implement with motors​
  • Integrate all systems and have robotic hand mimic hand sign and perform gestures​
2 Likes

Hello! My name is Jamison and as a member of the internship, I’ll be dishing out the updates from here on out:

6/23/24 Update

We had our 2nd meeting with ROBOTIS to provide an update on progress made during the week as well as discussing possible solutions to problems that have been encountered:

Electrical Team:

  • Created a broad schematic how the whole project will be powered using pin layouts of each component of the system

  • Connect the hand motors to the pi and work with the CAD team to incorporate those motors to the hand

CAD Team:

  • Worked on 3D printing the fingers with the gears and incorporate it with the motors to start testing out the design

  • Worked with EE team on configuring the motors together

Code Team:

  • Worked on hand detection

  • Started to work with the motors in the arms

  • Begun livestreaming camera to the website

1 Like

Hello! Hope everyone had a great 4th of July Weekend! Here’s our latest update for Week 5:

7/07/24 Update

Electrical Team:

  • Designed a PCB to make the connections between the Arduino and motors more organized and streamlined

  • Wired the hand motors to test out 3D printed finger prototype

CAD Team:

  • Printed the updated pieces for the 3 linkage system fingers

  • Started to work on the thumb linkage system that would fit into the palm

  • Designed the camera mount

Code Team:

  • Continued testing for action/motion detection

  • Wrote code for hand signs such as “Yes”, “Hello”, “Thank You” etc.

Hello everyone! We are down to the wire and approaching the final weeks of the project. All the hard work is coming to fruition with the results of this week. Here are the updates:

7/21/24 Update

CAD Team:

Assemble the hand with the palm and fingers

CAD Hand 2

Below is the demonstration/test of the finger’s range of motion:

CAD Finger Movement

The CAD team is working to fine tune some parts such as the gears as the model requires a better print in order for the gears to work smoothly

Code Team:

Completed coding the following ASL signs:

Yes
YES ASL Sign Video GIF

Hello
HELLO ASL Sign Video GIF

In tandem with the arm, the team has also made progress with detecting distinct gestures:

Movement Gesture Detection:
Movement Gesture Detection GIF

Gestures via Depth Sensing Camera:
Gestures_DepthSensing GIF

All that is left is fine tuning the code and integrating the hand, arm and camera together to fully realize the project. See you soon!