Face and Smile Detection using OpenCV C++

Finding and tracking a face is always an excitement to work with, Opencv library comes with list of feature classifiers which help us to find an object in the frame. Before moving further let understand what is frame and how this classifier works on the video stream.

Video stream is a series of images changing one after another. If the series of images have progressive change then that makes of video. Our human eyes can process up to 12 images per second individually, anything higher the frame rate will be perceived as motion. In opencv we apply the classifier on every frame of the video to find and track the objects.

What kind of classifiers are available


  • Face classifier: haarcascade_frontalface_alt.xml
  • Eye classifier: haarcascade_eye.xml
  • Smile classifier: haarcascade_smile.xml
  • Cat face Classifier: haarcascade_frontalcatface.xml

These classifiers are based on the machine learning concepts where they used lot of positive and negative images to train the classifier and extract the feature from the trained model. Those extracted features were mentioned in those classifiers xml’s. These features are just like kernel where it will be mapped over the frame to decide the success/failure case.

Since these features are large in number applying every feature is tedious. So to avoid that they come up with the boosting cascade method. In boosted cascade they group the features based on the effectiveness (strong/weak) to determine the object. If the frame is failed in first cascade (strong) then stop going further and if success then go further with next cascade. This avoids huge number of calculation.

The following is the sequence of steps to be followed to get the job done.

  • Capture the frame.
  • Convert the frame into grayscale.
    • cv::cvtColor( frame,frameGray,cv::COLOR_BGR2GRAY );
  • Equalize the frame using histogram.
    • cv::equalizeHist(frameGray, frameGray);
  • Apply classifier to get the list of detected objects.
    • faceCascade.detectMultiScale(frameGray, faces, 1.1, 2, 0|cv::CASCADE_SCALE_IMAGE, cv::Size(30,30)); This function always gives back std::vector<cv::Rect>  of objects.  These objects have x, y, height and weight as its member. Using that we can draw around the objects.
  • Draw and show the image
    • cv::Point center(faces[i].x + smile[j].x + smile[j].width*0.5, faces[i].y + smile[j].y + smile[j].height*0.5)
    • cv::ellipse(frame,center,cv::Size(smile[j].width*0.5, smile[j].height*0.5), 0, 0, 360, cv::Scalar(0,255,255), 4, 8, 0);
    • cv::imshow(windowName, frame);
#include "opencv2/opencv.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>

std::string faceCascadeName = "haarcascade_frontalface_alt.xml";
std::string smileCascadeName = "haarcascade_smile.xml";

cv::CascadeClassifier faceCascade;
cv::CascadeClassifier smileCascade;

std::string windowName = "Face Detection";
void detectAndDisplay(cv::Mat frame);

int main(int argc, char **argv){
    cv::VideoCapture vid(0);
    if( !faceCascade.load(faceCascadeName) ) {
        std::cout<<"face cascade error loading"<<std::endl;
        return 1;
    }
    if( !smileCascade.load(smileCascadeName) ) {
        std::cout<<"eyes cascascade error loading"<<std::endl;
        return 1;
    }
    if(!vid.isOpened()){
        std::cout<<"Camera is not opened"<<std::endl;
    }
    while(1) {
        cv::Mat frame;
        vid >> frame;
        if (frame.empty()) {
            break;
        }
        detectAndDisplay(frame); 
        int c = cv::waitKey(10);
        if(char(c) == 'b') {
            break;
        }
    }
    return 0;
}


void detectAndDisplay(cv::Mat frame) {

    std::vector<cv::Rect> faces;
    cv::Mat frameGray;

    // convert into grayscale
    cv::cvtColor( frame,frameGray,cv::COLOR_BGR2GRAY );
    // equalize
    cv::equalizeHist(frameGray, frameGray);

    faceCascade.detectMultiScale(frameGray, faces, 1.1, 2, 0|cv::CASCADE_SCALE_IMAGE, cv::Size(30,30));

    for(size_t i =0; i< faces.size(); i++ ) {
        cv::Point center(faces[i].x + faces[i].width*0.5, faces[i].y + faces[i].height*0.5);
        cv::ellipse(frame, center, cv::Size(faces[i].width*0.5, faces[i].height*0.5), 0,0, 360, cv::Scalar(255,0,255), 4,0,0);

        cv::Mat faceROI = frameGray( faces[i] );
        std::vector<cv::Rect> smile;

        //detect smile
        smileCascade.detectMultiScale(faceROI, smile, 1.1, 2, 0|cv::CASCADE_SCALE_IMAGE, cv::Size(60,60));

        for(size_t j =0 ; j< smile.size(); j++) {
            cv::Point center(faces[i].x + smile[j].x + smile[j].width*0.5, faces[i].y + smile[j].y + smile[j].height*0.5);
            cv::ellipse(frame,center,cv::Size(smile[j].width*0.5, smile[j].height*0.5), 0, 0, 360, cv::Scalar(0,255,255), 4, 8, 0);
        }
    }
   cv::imshow(windowName, frame);
}

Hope this post helped you. let discuss if you have any queries.

Leave a Reply

Your email address will not be published. Required fields are marked *