MARK 5018 Update – Face Recognition, Face Landmark Detection

In our Christmas wishes blog post last month we hinted at new M.A.R.K update, bringing a long-awaited feature. And here it comes!

Face recognition is a very important (and a very controversial) field of Computer Vision, that gained a lot of traction in recent years, thanks to advancements in Deep Learning. Now it is possible to recognize a person’s face among millions of others, just from a digital picture.

M.A.R.K had image recognition and object detection functions from the first version. We then added local training, custom user models as new features through OTA updates. But face recognition function was missing. The reason for that is face recognition is actually very complicated and involves using not one and not even two, but three model pipeline.

• The first model detects faces
• The second model detects face landmarks, such as eyes, nose and mouth corners. Detected faces then aligned to a standardized positions using these landmarks.

• Finally, the third model extracts so-called feature vectors from images of the faces. Feature vector is a string of numbers, that contains encoded information about face structure. The face recognition models are trained in that way, so feature vectors for similar faces are “close” to each other and for different faces are “far” from each other. We use Euclidean distance for distance calculation between different vectors.

• The last step is to compare feature vector from a face that is currently detected in camera image to vectors in the database – and if distance is lower then preset threshold (called similarity in user interface of Codecraft), then we conclude it is the same face!

The beauty of M.A.R.K is in simplicity though. First and foremost M.A.R.K is a robot for STEM education, so it is important that teachers and parents can demonstrate Machine Learning and Computer Vision concepts without going too much into complicated code. So, while you can use Face recognition, Face Landmark detection and Neural Network Face Detection functions in 5018 update with Micropython API, you can also use them from a beginner-friendly graphical programming environment, Codecraft:

Visit TinkerGen’s online course platform Make2Learn to see an example lesson, demonstrating how to use these new blocks in Codecraft!

Here is full CHANGELOG for 5018 version:

20210107: cyberEye-20210107-2000.3000.4000.5018.kfpkg

1. add a visible notification “sound … is playing” so user would now that M.A.R.K is not frozen
2. rectangle detection bug fix
3. improving memory consumption and garbage collector logic
4. added face recognition blocks and model
5. added face keypoint detection blocks and model
6. added NN face detection blocks and model
7. added language switch and QR code for error messages
8. updated MaixPy codebase to 88bfd01, made relevant changes to _boot.py, ESP32 ADC/Network, board_info
9. ESP32 firmware updated to 1.4.1
10. moved Lego detection model to SPIFFS

Note: In the 88bfd01 commit of MaixPy MicroPython firmware, the method of reading and writing files to the internal file system (SPIFFS module) was changed. This can cause intermittent delays when uploading code from Codecraft to MARK and when using register face block. When uploading the code, please do not unplug the data cable or restart MARK! We will investigate this issue and correct it in the next update.

You can see the source code both for M.A.R.K Micropython firmware(a fork of MaixPy Micropython port) and M.A.R.K Micropython filesystem in TinkergGen’s Github. And if you are already purchased M.A.R.K, feel free to share the feedback and your own projects in our Facebook group!

About Author