nxp/facenet512-imx

FaceNet512

Introduction

FaceNet is a neural network for face re-identification based on the Inception architecture. It regresses a face feature vector (embedding) that can be matched against a database of known face vectors. In this version the embedding has dimension 512. The weights come from the DeepFace implementation.

Note: This is not a face detection model. Faces must be detected and cropped before being passed to FaceNet.

Key Features

• Compact 512-dimensional face embedding suitable for verification and identification tasks.
• Fully quantized int8 model with uint8 input and float32 output.
• Compiled for i.MX 93 Ethos-U65 NPU and converted for i.MX 95 / i.MX 952 Neutron NPU.

Model Description

Modifications

The original Keras model from the DeepFace library is converted directly to TensorFlow Lite with full int8 quantization using 100 calibration images. The input type is uint8 (values 0-255) and the output type is float32.

Model Information

Information	Value
Input shape	RGB face crop (1, 160, 160, 3), uint8
Output shape	Face embedding vector (1, 512), float32
FLOPS	2.84 GOPS
Number of parameters	23,497,424
Source framework	TensorFlow/Keras (DeepFace 0.0.75)
Target platform	MPUs (i.MX 8M Plus, i.MX 93, i.MX 95, i.MX 952)

Tested Configurations

The quantized int8 model has been tested on i.MX 8M, i.MX 93, i.MX 95 and i.MX 952 using benchmark-model (see i.MX Machine Learning User Guide).

Training and Evaluation

The original model was trained and evaluated on the LFW dataset, achieving 99.65% accuracy according to the DeepFace source.

We re-evaluated the model on LFW deep-funneled images from the Kaggle LFW dataset using the pairsDevTest split (1000 pairs, cosine distance threshold 0.6):

Model	Accuracy
FaceNet512 Keras float32	97.5%
FaceNet512 TensorFlow Lite int8	97.5%

The evaluation script is evaluate.py.

Conversion/Quantization

The original Keras model is converted directly to TensorFlow Lite with full int8 post-training quantization. 100 random images are used as calibration data. The conversion is performed using TensorFlow 2.x with:

• inference_input_type = tf.uint8
• inference_output_type = tf.float32
• OpsSet.TFLITE_BUILTINS_INT8

The i.MX 93 model is compiled from the int8 TFLite model using Vela tag lf-6.18.20_2.0.0 with --accelerator-config ethos-u65-512 --optimise Performance.

The i.MX 95 and i.MX 952 models are converted using eIQ Neutron SDK version 3.1.3.

Download and Run

To regenerate the TFLite model fully quantized in int8 (uint8 input, float32 output), run:

bash recipe.sh

An example showing how to compute the cosine similarity between two face images is in example.py:

python example.py -m original_model/facenet512_uint8_float32.tflite -i face.jpg -i2 face2.jpg

To evaluate the model on the LFW pairsDevTest split, download the required data and run:

# Download LFW deep-funneled images
wget https://vis-www.cs.umass.edu/lfw/lfw-deepfunneled.tgz
tar xzf lfw-deepfunneled.tgz
# Download pairs file
wget https://vis-www.cs.umass.edu/lfw/pairsDevTest.txt
python evaluate.py -m original_model/facenet512_uint8_float32.tflite

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/serengil/deepface/

Original article: Schroff, Florian, Dmitry Kalenichenko, and James Philbin. "FaceNet: A Unified Embedding for Face Recognition and Clustering." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Schroff_FaceNet_A_Unified_2015_CVPR_paper.html