This blog post is aimed in demonstrating a simple hello world lampbot based on mbed RTOS and a computer vision system as part of an individual class assignment. The blog post gives simple overview of the design, code explanation (both for vision system and motor control) and the final operational Lampbot.
The Lampbot project was designed based on simple movement of two servo motors (pan and tilt) driven by a vision system input. At first, I kick started in choosing a vision system that will work for me. After much thought between roborealm and openCV, I choose openCV with python because of all the code experience I could get and also the flexibility it offered. Simultaneously, worked on the servo motor control using the mbed application board and programming it based on RTOS concepts.
Face and Eye Recognition using OpenCV with Python:
As a rookie in python programming, I had to use all the open source material on openCV and YouTube videos to write my code. The below image is simple demonstration of my final working code.
Breaking the code to give an overview of how the code works. For object detection in python we use something called Haar Cascades classifiers and with these classifiers one could detect face, eyes, smile, etc. and these files are available as an open source for people to use. In my program I have used face and eye cascades and it was defined as below:
Next, a typical loop is created to detect faces and eyes inside the faces and then finding there sizes, location of faces, drawing a rectangle and the same is done for eyes too. All these few lines code was enough to make it work.
But at the later part, it was found that the frame size was too big and it caused to slow down the processing of the images and hence I had to scale it down to give a smooth movement. I also had to send some data to the system and hence had to find a way to do that. With some assistance from my classmates was able to find the centre of the image and send that over a serial bus to the mbed setup. The below code addresses was finally fused with the original to complete the vision system.
Now, that I had the vision system working, I had to interface this with the servo motors controlled by mbed application board. The output is given as x and y coordinates measured from the centre of the rectangle or the face detected. This data is transmitted to the mbed as serial input to drive the motors and move accordingly. The x and y data will be used to pan and tilt the servos respectively.
Servo Motor Control using RTOS:
Even though there was an easier way to program the servos, we were asked to use RTOS as part of module requirement. To give a quick introduction on RTOS, the common services used are threads, MUTEX, semaphore, signals and Memory pools and queues. In my design, I have only used two threads to control servos (pan and tilt).
In the above code, each thread is an object from class thread. The function motor_thread and motor1_thread I what is run by the operating system and one is used to for pan and the other for the tilt. The wait is used to give a delay before it performs the next and all this run in a loop to have a continuous process. The process is not complete as I had challenges in setting up the serial input to drive the motors.
A real-time operating system (RTOS) is an operating system intended to serve real-time applications to process data in real time without any delays. They work on two main aspects i.e. event driven or time shared. Event driven systems switch between tasks based on their priorities while time sharing systems switch the task based on clock interrupts.
A thread is a class which can used to define, create and control a thread function in the system. A thread as different states like running, ready, waiting or Inactive. It also works on priority, there is always a main() thread which is given the highest priority.
A Mutex is used to synchronize the execution of threads, for example to protect the access to a shared resource. Stdio_mutex is an object from the class mutex and this enables the use of mutex in our programs. Mutex provide protections against race conditions, but at the same time incorrect use of mutexes causes problems such as deadlock, starvation and priority inversion.
A Semaphore manages thread access to a pool of shared resources of a certain type. It is used for signalling from an interrupt to a task.
Queues and Memory Pools:
A Queue allows you to queue pointers to data from producer threads to consumer threads and Memory Pool class is used to define and manage fixed-size memory pools. It helps us specify amount of memory we need and so on. Queues help us in managing these memory pools.
Who Am I???
They call me D-Pack (but actually Deepak), but I am cool with it. I am a final year BEng (H) in Electronic Engineering student. I love technology and all that it can offer and that’s motivated to study electronics. I love volunteering for good causes, makes me feel good. I love to travel, did a lot of travel in south India, Hong Kong, London, Philippines, Dubai, Abu Dhabi, etc. and that’s what took me to study in Ireland and have enjoyed it so far.
My motto for life is “Be good for something, so you can contribute”.