SI470: ML and DS CDR Jatho Policy Resources Datasets

Facial Recognition

In this lab, you will build a facial recognition system. Like other large-scale neural net projects, the creation and training of an embedding system for faces that would work impressively on real images is probably beyond the scope of our available time and expense. However, we can certainly use some pre-trained networks, and build a facial recognition system from them.

A facial recognition system normally consists of two networks, one which detects faces, and the other which embeds faces. We’ll be using the open-source project facenet to do this.

Here’s your notebook.

Installing a pre-trained network

Pytorch doesn’t have a pre-trained network for facial recognition, so we’ll be using a github repo with pre-trained weights.

Getting MIDN Portraits For now I will provide a small dataset for you to work with, prelimenary_data. Full MIDN official picture repository is here.

Opening an image

If the image is available locally, this can look like this:

from PIL import Image
img = Image.open('path/to/image.jpg')
img.load()

Displaying an image

If it’s a PIL Image or numpy array, you can do this with

import matplotlib.pyplot as plt
plt.imshow(img)

If it’s a torch tensor, it keeps RGB in a different order, so you need to do this:

import matplotlib.pyplot as plt
plt.imshow(img.permute(1,2,0))

Cropping an image using a face detector

Assume img is a PIL Image.

from facenet_pytorch import MTCNN
mtcnn = MTCNN(keep_all=True)
#You can take out the keep_all parameter if you only want one face from each image
mtcnn.eval()
cropped_img = mtcnn(img)

Building an embedding of a cropped face

Assume cropped_img is a tightly-cropped picture of a face.

from facenet_pytorch import InceptionResnetV1
resnet = InceptionResnetV1(pretrained='vggface2')
resnet.eval()
embedding = resnet(img_cropped)

Everyone taking SI470

There is a portion of the lab which requires you to collect the images of all members of SI470. We might expand this to instead be your CS cohort.

What you should know after doing this lab (aside from the coding part)