Eye lash fitting app with face recognition and WooCommerce integration

The task was to build a virtual fitting room for eyelashes that could scale to a large number of clients. This had to be a responsive web app, integrated into a WooCommerce store as a plug-in. The main challenges were scaling analysis, photo manipulation (Exif data, rotation, scaling, plotting) face recognition, eye detection, lash positioning and scaling, drag-n-drop implementation.

smartmockups_k7cizkzgApp on a tablet, integrated into a WooCommerce store

smartmockups_k7citz5nPhoto taken from an iPhone with slight head rotation

The knowhow of the solution was to develop the entire application in javascript (including thel face recognition logic). With all the computations being handled on the client’s side the app could easily scale with no additional resources – a huge cost saver.

We decided to use Tensorflow.js to detect facial features and a set of specifically trained models that showed the best results.

A lot of work had to be done to scale and rotate the photos appropriately, as well as apply eye lash transformations and a a drag-n-drop functionality.

Custom carousel was added to pull products from WooCommerce using the API.

In order to showcase the project to the client a full fledged WooCommerce staging environment had to setup and configured inside a docker container.

smartmockups_k7cifdcwApp working on mobile with a touch interface
smartmockups_k7cjig9k

Fullscreen desktop view