Tasks from First meeting
· Team assignment
· Set up GITHUB
· Extend code for more faces
· Calculate the face distance from the camera
So let’s jump straight in, from our first meeting the teams were assigned, Deepak Zhuo and I are team vision. The resulting tasks above are the key tasks to complete from our first meeting. As a task outline of what is needed from the vision team, a vision system sheet was created to Lampbotics sheet on google sheets. This is intended for tracking bugs, adding ideas and identifying the key variables used in Python. The sheet is broken into subtasks with the configured settings required for each known variable.
Need access, Padraig is owner. (Blogger has access i.e Wit blogger account)
Github location: https://github.com/Terry200/Lampbotics
The original code from proof of concept blog, was altered to allow more than one face to be detected. This will send the information from Python to the mbed via serial.
What to send and how?
The variables; x and y are simple to determine due to Haar cascading, so formatting them was the decider. I formatted the x & y from a pixel distance to a degree integer, this was completed by creating vectors from the centre point of the detected face to the centre of the screen. This is divided by a ratio relating the known width of the screen to the range of a servo 175o. This is stored as a coordinate pair.
xygo = ((cam_c-center_point)/(vid_width/CONVERT),(cam_c-center_point)/(vid_height/CONVERT) )
The resulting coordinates is the difference needed to position the centre point of the screen to the centre of the face. The ability to detect multiple faces was completed. A need to ID each was important, a flag and number is generated based on each detection of the Harr classifier.
The distance of each face was needed. To do this OpenCV documentation recommends calibrating the camera, this requires a lengthy process to complete. The calculation is not simply the right-angle triangle, as the lamps height is changing which is not recorded. Members of the vision team had a discussion to overcome this. Utilising a previous lab in Physics on lenses and focal length shown in the image below. I developed the calculation that could create a rough estimate of the distance an object was away from the camera; shown in the table below. To convert the math to code a fixed point was used as reference at 80cm, measuring the size of the head in pixels and cm.
The calculation is out by about 5 -> 10 cm with a range of 350 cm. Action complete, we have something to work with.
Now we have 5 parameters to pass to Mbed. To keep the length of the buffer the same each time it was padded to ensure the maximin bit length by default. The buffer has a structure of:
Buffer length received on Mbed is 18 due to ‘quotes around string’ & EOF char.
· The F is a flag that defines the detection
o i.e this will change to O for object,
o 00 identifies the face detected
· x is a delimiter
o 000 is a degree position needed for x
· Y is a delimiter
o 000 is a degree position needed for y
· d is a delimiter
o 000 is a cm distance an object is from camera
Areas for consideration for the project:
- Outline system status over nest sprint.
- . Yalefaces is a data base the detects emotions this could be used to send info to the LCD face.
a. Known data base needed.
b. Can extend to ID people (Known people)
- . Detect an object to track
- . Count objects (create colour shapes and make Vision count)
a. Shapes different colour
b. Send result and Mbed can output on speaker
- . Develop 1 -> 3 into a game
a. Find Jason
i. Walk into shot with 2 people, cover face and move
ii. Mbed points lamp in on new position.
iii. Mbed do some movement
iv. Speaker found you
b. Counting game
c. Track the ball
i. freeze lamp
ii. pass ball between two people
iii. after said passes lamp points to person with ball.
Reflections on sprint 2.
Sprint 2 was team assignment and the beginning of the project. Padraig took key notes and shared on the drive so we can add collective ideas to the project. The team assignment went well, although my final year project is heavy in code along with DSP and microcontroller. I wanted to do a bit more hardware than software. I felt pushed into the vision team during the meeting but I’m happy to be doing it. Over the next few meetings, I felt like I contributed well to the discussion. While discussing some plans, I felt frustrated. I noticed that while I was frustrated my tone of voice may come across as aggressive. This has made me more aware of my tone of voice when I am feeling frustrated. Jason gave some feedback from meetings, from this he advised to slow down as the basics are needed before more thoughts for the project. I think adding a vision sheet will help design the code for OpenCV and develop ideas.
The tasks reflection. Getting the few extra variables was straight forward. The only one that seemed to be troublesome was the distance to the object. I attempted it twice only to fail. I thought this would be a big issue since reading online recommends a second camera for this. Then on the usual walk home debrief Zhuo and I discussed how to solve this. I got home and pulled out old Lab reports, and discovered calculations on focal lengths and distance approximations. Pulling out certain bits of data and converting it to code and Bingo! It worked.
Sprint 1: Link