## The challenge
I'm trying to replicate the experience from this campaign, but in realtime:
- http://www.gettyendlesspossibilities.com/
## What I have now
- I'm able to detect faces and (somewhat naively) mark the regions I want: right and left eyes, cheeks, forehead, mouth, nose and so on.
- I run the same algorithm against a database of face images (5k images from Getty Images) and store image slices for each region.
- I'm just converting to gray and storing some metadata for each face slice. I suspect I could get better matches applying some kind of filter.
- I made a web app using Django Rest Framework that captures a snapshot using the web cam, locate your face and try to match each slice against slices of the same part of the face in the database.
## The current problem
The matching is not very good. Sometimes, the best match for a face slice is a false positive from the face recognition algorithm (I got a low percentage of false positives and I'm curating the database by hand). I've tried the following algorithms in order to find the best match for each face feature from `skimage.measure`:
- compare_ssim
- compare_psnr
- compare_mse
- compare_nrmse
Best results are from Structural Similarity, but still far from ideal. Bellow are an example of the app returning a false positive as the best match for my mouth:
 
As you may have guessed I don't have a clue about what I'm doing - unable to figure out why it is not picking from hundred of mouths in the database.
↧