Sunday, 12 February 2017

Proof of Concept Servo response, Python & OpenCV Terry


Develop a facial detection program, that will enable a pan and tilt response from servos controlled by an Mbed device.

  • Python OpenCV working program
    • Face Detection 
  • Define centre
  • Serial communication
  • Servo 180 degree
  • Servo response from OpenCV
  • Read readme from Source code


A quick reference to the information gathered. These sites below were used to develop code produced. Originally I began with a head droid example online [1] this looked as a great point to start, alas the development of Python has removed C++ code in OpenCV library to numpy arrays. Trying to convert  Ian Ozsvald code was a step i started with as there was no point in reinventing the wheel. However during development due the changes in OpenCV, I decided to scrap this conversion and start from basics and understand this better.  The main setup tool is taken from openCV  [2].      

The software versions to complete Face detection software:
  • Python 2.7.3 64 bit [3] (DL Anaconda for preset IDE & # of packages)
    • Add Path to root
  • OpenCV 2.4.9 [4]
    • DL .zip 
    • From extracted folder find the version needed. (Me 64bit)
    • copy file cv2.pyd into python site packages
    • Eg of my directory
      • (C:\Opencv2_4_9\opencv\build\python\2.7\x64)
  • pip (This is a way for easy installation of libraries. Pip included in Anaconda)
    • numpy, Pyserial
  • Code supplied Below
    • "haarcascade_frontalface_default.xml" in file location

OpenCV uses haarcascade_frontalface_default.xml for face detection. This was developed by Intel, later released as open source. Yale have developed facial expressions database [5] this will be of use in the future.

Explanation of code:


The code is basic setup, outlined in guide [2]. The video image 640*480  I found to be a good scale of the video frame. Scaling this down more increases the smoothness of the tracking, but difficult to determine the distance from the centre. Variable DEBUG = true will print particular values for Debug reasons.

When a face is detected a rectangle enclosing the face is scaled to a pixel point Dxy. The vector from origin to Dxy is used, the difference determines how far from the centre the face is located. The pixels range of the image is converted into a angler value (Based on 175^o range of Servo). This is passed via serial to the Mbed with a tag pos, used to flag the first face (Program is blocked from detection of 2 faces at the moment). More data can be added to the data packet, following the structure of the string.


Code is a basic read serial as buffer, this takes the serial string, splitting it into tokens (Const_Char), Variable x & y cast this Char to integer value. plots it on the LCD screen. This is the difference to the centre. Default position of servos 80^o as range 0->175^o. The vaule x+80 is passed into servopan to respond to the difference in the x axis, likewise for servotilt.

Issues noted:

  1. Movement odd oscillation, (NEED Delays)
  2. Smother control from Mbed side (Change to float  1 Degree = 4 pixels)
  3. Serial method (TeraTerm cannot access the serial com LCD used for DEBUG)


  1.  This program only deals with one face, easily add more.
  2.  Detect Face expressions (Yalefaces)
  3.  Detect objects
  4.  Detect distance
    1. This could be used as a motion for centre arm (Distance & Angry face from Y.faces move back)
  5.  OpenCV has the potential to train (Some real thought needed)
    1. This could be used to detect certain people for interaction

 Video for example


My initial thoughts to this was not to reinvent the wheel, I decided to look online for examples of pan and tilt, reading older blogs was a great starting point, the integration between python and the processor was an exciting task. There have been some learning curves, Real Time OS is expressing the difficulty of cross platform integration. These are subtle and include the ever evolving of open source software. 

The main point I have decided to express was when I began reading blogs and looking at examples. Code was available, however head droid [1] needs conversion from the old python cv library (No legacy left) for C++, where cv2 is based numpy array.

I took the route of building from scratch having a notion of what to do, opencv's documentation and notpad++ was all i needed. I'm a final year student that is capable of this task with the skills I have developed.




No comments:

Post a Comment