How to Train the AI Face Recognition Engine
Zoom allows you to train the face recognition engine to recognize one or more faces with a custom label that represents a person’s name. Once you finish training, you can then use Zoom’s AI Hub to recognize all the faces in images that match the trained faces. After that, you can search by person’s name across millions of images and find matching faces in these images.
In this article, we cover how to train our face recognition engine. For each person that you wish to recognize, you will need to create 5-10 cropped images for each age group for a person.
The process
Before you begin
You should have access to a library of images that represent a person. You should be able to use the Zoom photo cropping tool or have access to a photo editing software such as Adobe Photoshop.
Prerequisites
- You must have access to the Zoom desktop apps
- You must be a superadmin who can execute AI face training jobs using the Zoom Asset Browser app on the desktop.
- You must be prepared to iterate over the training process a few times with a set of test images till you are happy with the recognition.
- Follow the guidelines below for creating the training set.
Benefits
The Zoom face Training process allows you to:
- Create a grouping of cropped faces to be used as a training set.
- Once you add sufficient cropped images for a person, you can let the AI recognize the person from any number of images you have in the Zoom repository.
Procedure
Step 1: Create a folder structure for training set
We recommend arranging the cropped images in age specific folder structure in Zoom. Create a project vault or a subfolder under an existing Zoom Project vault such as “AI Training” that will hold the cropped images you will use for training. This will be called the ‘Training set’.
Step 2: Name your crop angle
Under the age based sub folder, add your cropped images (PNG or JPEG) with meaningful names or use a Zoom custom metadata field to track the shot angle. For example:
Step 3: Label the faces
For each of the cropped images per the guidelines below, after ingesting them into the Zoom repository, please attach a label using a metadata field: AI CONTROL::Face Label
Step 4: Submit the Training Set
Once you have completed creating the training set, submit it to the Zoom AI hub using the Asset Browser › Create Job › AI Face Training.
Step 5: Configure the AI Face recognition
Your media manager needs to setup the Zoom AI Hub to pick up your unrecognized images and run AI Face Recognition using the training set you created.
Guidelines for Training set
As mentioned above, the training set contains the cropped images of faces for one or more persons that you will use to train the AI Face Recognition engine to recognize faces and tag them with your custom labels.
General face cropping guidelines
- Prefer cropped images from professional photo shoots if available.
- Minimum cropped face dimension must be 50×50 pixels. Higher would be better.
- Use a face that occupies a large proportion of the crop. Training set images where the face occupies a larger proportion of the image lead to greater accuracy during the recognition phase.
- Cropped face should include a full head, hair, ear, neck and some part of the shoulders.
- Avoid faces with hats, glasses, headbands, and masks.
- Avoid obscure or concealed faces.
- Make sure at least 70% percent of the face is visible in the cropped image.
- Avoid faces with partial shadows, prefer faces with flat lighting.
- A cropped image must contain only a single person’s face.
- To cover various life stages from childhood to adulthood, include crops from these age brackets (years): 0-1, 1-2, 3-5, 5-10, thereafter every decade such as 11-20, 21-30 and so on.
- Include between 5 to 10 cropped images per age group
Face angle guidelines
You will need at-least 5 crops with varying face angles: yaw, pitch & straight-on. Please ensure you include at least 1 cropped image from the below list of 5 face angles:
Face Straight on Crop
- face crop straight on
Face Pitch Crop
- face tilted down with a pitch of 30 degrees or less
- face tilted up with a pitch of 45 degrees or less
Face Yaw Crop
- face turned left with a yaw of 45 degrees or less.
- face turned right with a yaw of 45 degrees or less
After adding at-least one set of images with above angles, try to add a few images of the person in an age group with a beard or braces if available.
Examples of Bad Crops
Examples of Good Crops
Training set gotchas
- Detected face name cannot be renamed. Make sure the metadata field has the correct person’s name before you start the training. Otherwise the images will need to be submitted again for training.
- Removing a face from a training set after it has been used for face recognition, will not remove the metadata for earlier recognised images.
- To clear out AI metadata for recognized faces, just re-run the AI Face recognition job on the selected assets or folders. This will delete previous face recognition metadata and insert new metadata based on the recognition job
- Keep in mind the cost of AI is proportional to both the number of images as well as how many iterations of AI Face recognition your perform
Next Steps
Setup your Project & Media Store Folders
Before you start editing projects using the Zoom Desktop app or Plugins, you need to setup your Zoom workspace folders. This article will walk you through the setup along with common tips & tricks.
- Last modified on