Author: Philip O Connor
Date: 21/04/2016Personal Reflection:
The goal of this of this project was to advance lamp-bot design which has been built in a previous year. The key object of this project was implementing the lamp-bot design using a raspberry pi to control the different functions which have been incorporated in this year’s lamp-bot design. My role in this project mainly involved working of the vision system which is used to detect and track a person’s face and this produces three variables X, Y and Z which can then be used to control the servos located on the lamp-bot. Also as this was a team based project another one of my roles in this project involved working as a team to accomplish a common goal which was the building of the second version of the lamp-bot design. Additionally each of the different functions which will be implemented in this design will uses an RTOS application which is essential used to switch between tasks so that the different tasks appear to be operating in parallel.
The key objects at the beginning of this sprint were to finish incorporating the threading design created by Kamil for both the face detection and the face recognition systems. When this threading design was be implemented in the last sprint an initial issue which arose in the face recognition section of the design was a path issues. This was due to an incorrect assignment of the path to the face cascade which is used with the face recognition section of the design to check if a detected face is stored in the face recognition data base. This issue was solved by reconfiguring the path location of the Face cascade in the program to the path where the main threading program is called. Also as this was the final sprint many of the different applications which have been design where intergraded together which involved communication to the various outputs tags (Face detected, X,Y and Z face positions and the subject number) which are generated in the Face detection/ recognition thread design.
The basic idea behind an RTOS application is to allow sharing of resources in a controlled and predictable manner. This can result in a design which appears to be running all processes/task at the same time but in fact each task is execute individually were the scheduling function (multitasking) of an RTOS application is used to switch between the different tasks/processes.
The RTOS application which was implemented in the lampbotics design used a threading system to switch between different tasks where each function/process is given a thread which can then be scheduled for execution in the main processing program. Shown below is how different threaded programs are introduced into the main program for scheduling so that they maybe execution and the example below only executes two threaded designs which are the Face detection/ recognition thread design and the servo thread design which converts the X, Y and Z positions of a detected face into a 0 to 1 range value which can then be used by servo team to control the movements of the lamp-bot. Also if a face has been detected a True state is sent using a tag to the sound thread which outputs a sound to indicate that a face has been either lost or has been detected.
Additionally in each of these designs is a need for communication the X,Y and Z positions generated in the Face detection/ recognition thread design which need to be outputted to the servo thread design and this is accomplished though the use to tags. These are specified within each of the threaded design in the def init(self): function and the tags which are used by the Face detection/ recognition thread design are:
The facePos tag outputs the X, Y and Z position values obtained from the face detection cascades into the object variables x,y and z which will be used by both Joe’s servo calculation thread and Luke’s colour detection. The faceDet outputs a True (face detected) or False (face not detected) into the object variable Face and the resulting output from this tag is used with the sound tag which outputs if a face has been detected using the speakers. The last tag faceRec is used to output a number which relates to a person which is stored in the Face recognition data base and this value is stored in the object variable Subject.
Face Detection Testing Results:
Shown below are the results from the execution of the face detection section of the design which shows the resulting outputs for one cycle of both the Face detection section of the design and the servo calculation thread design.
Face Recognition Testing Results:
Also it should be noted that the recognition part of the Face detection/ recognition thread design also uses the face cascade to load an image to determine if that face is recognized but will not be used to control the movements of the servos. This due to the high level of processing needed to execute this process which creates a long lag time for each cycle of this process. However the recognition part of this design could be used to pass the subject number to the sound thread which would speak that person’s name and if a face has been detected and this section of the design could be easily implemented but due to time restrictions this section of the design was not implemented.
The resulting output from the recognition part of the design once a face has been detected is shown below which produces three outputs which are a classification to indicate if the image being processed has been identified correctly. Next the faceRec tag is used to output the subject number associated with that recognized person. Then that person’s name is printed out on both the command prompt and on the processed image. Lastly using the image capture data base these images are processed to match the processed imaged with an image stored in the image data base library. This produces a value which is associated with the level of confidence that the matched image is correct (confidence). So then this means that a low value of confidence will mean that there is a high probability that the processed image matches the image which is taken for the image data base.
The resulting faceRec tag outputs which are produced from this process could then be used with the sound thread which would output the name of the recognized person on the speakers. The code which is used to implement this version of the combined Face detection/ recognition and the servo threaded design can be found here:
Also the following YouTube link demonstrates the operation of this design and can be view using the following link:
As this was the final sprint many of the different design which had been created for this design of the lamp-bot were combined which involved communicating with the different groups so that the correct tags could be sent to different tasks which involved varying levels of teams work. The tags which needed to be sent from the Face detection/ recognition thread design were the face position values and if a face had been detected. The face position tags were communicated to Joe’s servo calculation thread which then outputted his tags to Ian’s Pi communication thread so that they could be used by the servo threads. Also the face detected tag was also communicated to David’s sound thread so that the speakers could output a sound. Also as Kamil was assign the task of implement the RTOS thread design he was a great help with pushing this design to its final design. It was also necessary to keep people update on the progress of my task so that they that they would know when to expect the final design.
Initially design which was implemented used the face detection section of the design to pass tags to Joe’s servo calculation thread which is then used to control the servos. Also if a face has been detected David’s sound thread would be used to output a detection sound. The final design of the Face detection/ recognition thread design fully integrated Kamil’s threading design which is used to control the different sections of the lampbotics project using a RTOS thread design. And although all of the different sections of the project could not be fully implemented into one design the final design demonstrate the key objects of the project which where face track to control movements, lighting effects, sound effects and Pi communications. Also due to the high processing required by the Face recognition section of the design this process would not be used to control the movements of the servos.
I would also like to thanks all the members of the project team which were involved in this project and I found that though the participation of this group project provided to be an excellent learning experience in relation to improving my communications skills, team work skills and also allow me to obtain a better understanding of an RTOS application.
Blog entrie list:
Team ESD (Face Detection/Recognition)
Philip O Connor entry 7,