Team: ESD (Face detection/recognition)
Authors: Robert Burke and Philip O connor
For this sprint the face detection module of the lampbotics project will be improved and face recognition will also be incorporated into the design. The face detection section of the design will be improved primarily through the implementation of using two face detection cascades to demonstrate the improved face detection. There are many different types of detection cascades which can be implement and the key face detection cascades which were examined are as follows:
- The first type of face detection cascades which will be examined are the Lbp cascades which are faster than the haar cascades but are less accurate (approximately around 20% less accurate). Additionally these cascades use integers values to calculate the positioning of the detected face. This makes these types of cascades more ideal for embedded systems due to the fact that the required processing will be deceased when compared to the processing required by haar cascades which uses floats to determine the positioning of the detected face.
- The second type face detection cascades which will be examined are the haar cascades which are more accurate than the Lbp cascades but require more processing power which can become an issue when implementing these cascades on the raspberry pi.
This sprint will also incorporate face recognition into the design which simply uses a range of trained images which are stored in a database. This database can then be used to process detected faces to determine if that person recognized. A good face recognition design will require fast processing and a high level of accuracy to determine if a face is recognized. Face recognition can be implemented using a number of different approaches which are:
- Holistic Matching Methods uses the entire face as an input to determine if that face is recognized and is the most commonly used form of face recognition.
- Feature-based (structural) Methods uses features of a person's face (mouth, eyes and nose) as an input to determine if that face is recognized. This approach improves the accuracy of the face recognition module but increases the amount of processing.
- Hybrid Methods using a combination of Feature-based and Holistic Matching methods as an input to determine if that face is recognized. This allows this type of face recognition to incorporate the advantages of the Feature-based and Holistic Matching methods which means that a high level of accuracy can be achieved while using a lower level of processing when compared with the required process which is used to implement the Feature-based method.
- Philip will focus on implementing an improved version of the face detection and will also collaborated with some sections of the face recognition design.
- Robert will focus on implementing the face recognition design and will also collaborated with some sections of the face detection design.
Initial design results:
Analysis of proposed improvements to the face detection section of the design:
The initial design results which have been obtained using two face detection cascades to track a person's face is shown below and to demonstrate the effects of using two face detection cascades the below images also demonstrates the implementation of a single face cascade:
From the above figure the results demonstrate the improvements of the face detection module when two cascades are used. The primary improvements have have been obtained from the implementation of the two cascades is the detection of the face when the person's face is tilted up or down. Also face detection using the Lbp cascades was also examined and the testing scenario which was performed to examine the effects of using the Lbp cascades was implemented by quickly reposition a face to a to a different section of camera frame. This testing scenario was also implement using the haar cascades to highlight the increased processing which is obtained from using the Lbp cascades. In conclusion the the haar cascades demonstrate the best detection of a person's face with a higher processing requirement while the Lbp cascades demonstrated a small reduction in the processing requirements but comes at the cost of decreasing face detection.
Analysis of face recognition section of the design:
OpenCV provides a FaceRecognizer class for face recognition. They currently provide functions to allow for the use of EigenFaces, FisherFaces and Local Binary Patterns Histograms face recognition algorithms. The FaceRecognizer is then trained for a given database of faces from which it builds a model for what each subject looks like using one of the three algorithms. A image of a face can then be passed to the FaceRecognizer function, where it will again analyse the face using the selected algorithm and compare the model of the face to the faces in the database it was trained for. The function then returns a prediction as the label of the face most like it in the database as well as a confidence.
Due to the limited time available to implement such a program, and the wealth of information and tutorials available for similar applications on the Raspbian OS, the decision was made to examine some tutorial and example code available online. The first example that was examined was the Magic Treasure Box project. The project uses facial recognition to open a box latch using a servo only for particular users. The code was reduced to just the face recognition functionality and run on the RPI.
This allowed for an insight into what will be needed to implement such a system for the lampbotics application. The Magic Treasure Box project uses OpenCv and Python on the RPI, where the AT&T Laboratories Database of faces is used along with the Eigenfaces algorithm.
The AT&T Laboratories Database of faces consists of 10 images of 40 distinct subjects with varying lighting conditions and facial expressions. This database is then used as a negative database of faces which are people who cannot open the box in this particular application. A separate positive database is then created of the users who can open the box.
Figure 1: Example images from the AT&T Laboratories Database of faces
Both databases are first analysed where the face is first extracted from the image using face detection and label also extracted. The FaceRecognizer function is then trained using this data. The function is then ready for an input. In this application a user's face is detected using the RPI camera, and an image of the face passed to the FaceRecognizer function and the prediction and confidence returned.
Figure 2 below shows the outputs to the command window for the various scripts run to implement face recognition. 12 images of myself where saved to the positives folder. Both databases are then analysed, extracting faces and labels and both the positive and negative databases saving this data as training.xml, then used to train the FaceRecognizer function The program then waits for an input image which it performs the same analysis of, before returning a positive detection and the confidence
Figure 2: Output to command window
OpenCV + Python + Raspberry Pi + Visión Artificial
Face recognition (C++)
Face recognition (python) (Magic Treasure Box)
[Tutorial] Simple Face Recognition (OpenCV, Python)
Principles and Methods for Face Recognition white paper