Knowledge Base

Webcam-Based Eye Tracking Technology

By Tess den Uyl and Amogh Gudi, Research Consultants, Vicarious Perception Technologies BV

Next to facial expressions analysis, eye tracking analysis can also be performed with a regular webcam. Our eyes are our windows to the world. Our gaze reveals what we are looking at and this often indicates what we find important. Eye gaze can be stimulus driven and reveal what implicitly draws attention, but is also driven by explicit motivation.

Therefore, eye tracking is used in a wide domain of fields, such as psychology, marketing, and usability research. Usually, eye tracking technology consists of an infrared camera, attached to the computer (with different price ranges and accuracy, depending on the requirements). However, using a simple RGB camera enables eye tracking to be done from every computer or laptop with a camera, thus increasing flexibility greatly.

This white paper discusses the methodology and accuracy behind the technology. It also gives a validation example for a specific application and indicates how to start using it in your own project.

How It Works

The EyeReader algorithm estimates the gaze direction and relates that to an image on the screen. The first step is face analysis to locate key-points in the face (such as eyes, nose, etc.) and estimate the head pose.

The eye image and head pose information are then used by a pre-trained neural network (a convolutional neural network or CNN) to produce a 3D gaze vector (a line segment in three directions, x, y, z) representing the direction of the person's eye gaze. The neural network is trained with many labelled datasets of screen locations and video recordings to learn the relation between the image of the eyes and the gaze vector.

To go from a 3D model of the gaze to a location on a screen one needs to know exactly where the screen is. With the help of a calibration task in which the participant follows dots on a screen, the 3D vector can be mathematically decoded to 2D gaze x, y points on the screen.

Accuracy of EyeReader

EyeReader has been tested on a validation dataset which includes data from 17 participants (12 males/5 females, 9 aged 18-29/8 aged 30-59) using EyeReader and a Tobii Nano eye tracker under natural lighting conditions. Participants performed a calibration task and a few psychological tasks. They were situated at a distance of approximately 75 cm in front of a 1920x1080 23-inch screen and also performed a calibration task on a 15-inch laptop (without eye tracker).

The results on the 32-dot calibration task (different dots used for calibration and testing) on a desktop computer show that the system predicts gaze points on screen with an average deviation of 2.4 +/-0.6 cm (or an average of 113.6 pixels, and an average of 5.2% deviation of the screen).

The results also demonstrate that a large amount of sections of the screen can be discerned with high accuracy. The system can estimate a gaze location in 1/16th of the screen with 96% accuracy, albeit not in the range of hardware eye tracking systems (which achieve over 99% accuracy for 1/16th of the screen). If one wants to study a difference in fixation between two points on the screen, it is ideal if they are in different 1/16th sections of the screen.

The results are comparable with competitor systems on the market, although it should be noted that direct comparison is not possible due to different screen sizes (making noting errors in pixels problematic) and different testing settings.

Laptop versus Desktop

The system is more accurate on the laptop (Mean Absolute Error: 2.3 +/-0.5 cm) in comparison with the desktop (MAE: 2.5 +/-0.7 cm). This is likely due to the smaller screen and the fixed ideal alignment of the in-built webcam of the laptop.

The results are similar for brown eyes (MAE: 2.5 cm) in comparison to blue eyes (MAE: 2.4 cm), and for those not wearing glasses (MAE: 2.4 cm) in comparison to those wearing glasses (MAE: 2.7 cm). However, this is based on only 3 people who are wearing glasses (with thin lenses); when there is reflection of the surrounding light this will likely cause issues.

How EyeReader Can Be Used

An important but simple finding within eye tracking research is that people look longer at what they find interesting. Participants were asked to perform two simple choice tasks, where they viewed a few images of items (e.g. cookies, chocolate, etc.) and were allowed to choose one afterwards.

A comparison of the EyeReader gaze estimations with the eye tracker estimations shows a high correlation between the X and Y coordinate estimation (X: Mean r = 0.87, Y: Mean r = 0.77).

Gaze estimations can be classified into being a saccade (fast movements with less visual information processing) or fixation (slower eye focus relevant for information processing) based on the speed of the gaze movements. For each category that was presented, the total duration of fixation time was calculated.

Validation of This Application

Both systems resulted in the same interpretation of the results: 79% of participants looked longest at the category that they preferred (choice as a reward for participation), with EyeReader giving an average of 9.3 seconds on the chosen category and 2.0 seconds on not chosen categories (versus 11.1 seconds and 2.6 seconds for the eye tracker).

The slightly lower estimates in EyeReader are likely due to a slightly higher percentage of failed eye localizations. In addition, the total fixation duration for each category indicated strong correlation between the two systems (Mean r = 0.91).

How to Implement EyeReader

The results show that eye tracking via the webcam can be a useful and easily applicable research tool. Under good conditions, varying individual and screen characteristics do not cause large differences in performance. Generally speaking, there is often a trade-off between ecological validity and measurement precision.

If high precision is required, a lab setting is more appropriate where participants can also put their head in a headrest and have long calibration sessions. EyeReader is very suitable for researching realistic behaviors where you expect clear discernible effects.

EyeReader can be used in a flexible and mobile lab environment or within the FaceReader Online platform for online testing. If you are already a FaceReader Online user, make sure you are using FaceReader 9. When you open a project, click on the tab 'details' and make sure version 9 is selected as analysis engine for the participants' recordings. You can then turn gaze tracking on in your experiment design.

---

Download the original PDF: White paper - FaceReader - EyeReader (PDF)