Facial Action Coding System (FACS)

Facial expressions represent the most effective way to convey information on one's emotional state. At the same time, these expressions are very difficult to measure objectively. Ekman and Friesen [1] have developed the Facial Action Coding System (FACS) for exactly this purpose. The FACS is based on Action Units (AUs), roughly to be interpreted as the smallest visible units of muscular activity in the face, which produce changes in the facial appearance.

Extensive research shows that certain combinations of Action Units are linked to the six universal or basic facial patterns of the emotions anger, disgust, fear, sadness, surprise, and happiness [1]. Interpretation of the reported activated units, however, is not covered in the FACS, but in separate systems, such as EMFACS (Emotions FACS) [2] or FACSAID (Facial Action Coding System Affect Interpretation Dictionary) [3]. These are methods for objectively scoring and interpreting emotional or affective expressions.

Facial Action Coding System (FACS)

By concentrating completely on muscular activity and relinquishing interpretation, the FACS produces a purely descriptive account of facial expressions. Therefore, the FACS is one of the most widely used and comprehensive coding systems for facial expression analysis.

Roughly, there are three ways to score facial action units:

1. Manually
2. With use of facial expression recognition software
3. With use of coding software for observational data

Objective Results with FACS

The FACS manual describes the criteria for observing and coding each Action Unit. It also describes how AUs appear in combinations. You can code for example, inner brow raise (AU1), nose wrinkle (AU 9), and upper lip raise (AU 10). There are 28 main Action Units in the face.

Using the FACS, facial configurations are decoded using specific, quantitative, and sharable rules. In scoring, it will be necessary to apply slow motion and frame-by-frame viewing to identify the AUs that occur, always alternating with real time viewing. As such, manually FACS coding is very time intensive.

Who Uses FACS Coding?

Coding facial actions is used in many different settings, for example in science, teaching, and by animators as well. It enables a greater awareness to subtle facial behaviors, which is very useful for psychologists, therapists, interviewers, and when working in communications. FACS is unobtrusive. It can be applied without the participant's awareness that their faces are being analyzed.

FACS Coding

The Facial Action Coding System is based on two main pillars: emotional facial expressions are universal, and facial expressions arise spontaneously from unconscious processes.

Distinguish Genuine Smiles from Fake Smiles

Of all human expressions, a smile is the most universal. FACS can be used to distinguish two types of smiles: a genuine, or 'felt' smile, and fake ones, also known as the "say cheese" smiles. The real ones are also called Duchenne smiles, named for Guillaume Duchenne, who first characterized the facial muscles needed to make the expression happen [4]. According to their research, felt smiles activate muscles in the eyes and false smiles do not. In addition, false smiles have been shown to be more unilateral, meaning the mouth curls more to one side.

Automated Analysis of Facial Expressions

Recent advances in computer vision have allowed for reliable automated facial action coding. To gain accurate automated facial action coding, FaceReader is a software tool that can help you out. FaceReader can classify twenty Action Units.

For more information about the validation of the Action Unit Module of FaceReader, you can request the white paper Validation Action Unit Module at your convenience.

Action Units

When an action is active, its intensity is displayed in five categories A (Trace), B (Slight), C (Pronounced), D (Severe), E (Max). Output will be presented on this scale with different colors and can be exported for further analysis in detailed log as numerical values.

Action Unit classification can add valuable information to the facial expressions classified by FaceReader. The emotional state Confusion is, for example, correlated with the Action Units 4 (Brow lowerer) and 7 (Eyelid tightener) [5]. Most Action Units are unilateral, that is, they can be visible in the left and/or right part of the face. FaceReader analyzes left and right Action Units separately. This feature distinguishes the intensity of the active muscles at the left and the right side of the face.

Using Action Units to Measure Arousal

Arousal is important in regulating consciousness, attention, alertness, and information processing. It is crucial for motivating certain behaviors. For example, it accomplishes whether we intend to fight or flight when we are in a dangerous situation [6].

Within FaceReader, arousal is based on the activation of 20 Action Units (AUs) of the Facial Action Coding System (FACS) and indicates whether the test participant is active or not active [7].

Create Your Own Custom Expressions

Custom Expressions are facial expressions or mental states that you can define yourself by combining the facial expressions and Action Units that FaceReader can recognize. The Custom Expression function can be used for a wide range of applications, for example:

• Analyze the positive expression 'Awe' using a combination of Action Unit 1, 5, 25 and 26
• Research related to pain, based on Action Unit 4, 6, 7, 9, 10, 25, 26, 27, 43
• Measure affective states: Interest, boredom, and confusion are provided by default

FACS Coding for Infants

Baby FACS, developed by H. Oster [8] is a modification for infants and young children of the adult FACS. It is ideally suited for research on changes and continuities in facial expressions of emotion, infants' responses to taste, odor, and other sensory stimuli, cognitive information processing, and expressive behavior occurring in naturalistic and experimental situations, and during parent-child interactions.

Infants cannot provide verbal feedback, making facial expressions particularly insightful. For example, the taste of something sweet will result in a facial relaxation, indicating that the infant experienced the sweet taste as pleasant. In contrast, a non-sweet taste will involve certain facial muscle actions indicating an unpleasant experience. Baby FaceReader can automatically measure facial expressions in infants ranging in age from six to 24 months old.

Manage the FACS with The Observer XT

With The Observer XT it is also possible to manage the whole FACS or parts of it, allowing comprehensive coding when needed. Video images can be displayed frame-by-frame in full resolution to enable detailed logging. Coding can be refined as many times as you like, without losing previously coded measurements. You can also select time segments of the video recording for coding, for example select time segments of 5 seconds after a certain stimulus has been applied.

Study Facial Expressions in Combination with Other Data

After coding your observations and importing external data, such as physiological data or eye tracking data, you can start to analyze your results. Specify the parts which are relevant for analysis, for example, determine if the subject raises the upper lip more frequently in conversation with a same-sex person than with an opposite-sex person. Peak events of specific action unit activation can be put against one another to assess facial response to a certain stimulus.

The Observer XT supports the entire workflow of your project, from setting up a coding scheme to building an elaborate filter to analyzing data. As a result, you will be able to make all the necessary preparations to collect rich and meaningful data.

When using FaceReader to automatically classify facial expressions, you can import the FaceReader data into The Observer XT and calculate statistics like the mean and maximum classification value for each of the emotions, making advanced analysis possible.

Examples Using FACS

• Forestell and Mennella (2012) used the FACS to objectively quantify infant's facial expressions. As a result, they gained insight in the relation between infant behavior and the assessment made by his or her mother concerning the infant's temperament.
• Researchers involved in the FIRB and NARSAD projects from the Vita-Salute San Raffaele University and the Sigmund Freud University in Milan, Italy, conducted a study to characterize a set of emotionally charged stimuli in terms of the induced facial activation. In the project, FaceReader along with a FACS coder were used to investigate the differences between the two.
• Jan Zumhasch studied at the Technical University Dresden in Germany and finished with a master's degree in communication science. He validated version 7.1 of FaceReader, including the Action Unit Module. To specify this investigation part of his research, he used selected study materials, including photos and videos of the "Amsterdam Dynamic Facial Expression Set" (ADFES) and material taken from the Facial Action Coding System (FACS), the holy grail for learning about the face.

Check Out What Facial Action Units Look Like

Of 44 FACS Action Units that Ekman and Friesen defined, 30 AUs are anatomically related to the contractions of specific facial muscles: 12 are for upper face, and 18 for lower face. AUs occur either singly or in combination.

The 20 AUs FaceReader can detect are visualized on the Noldus website. Some images have been zoomed in on the area of interest to explicitly show what muscle movement corresponds to the specific Facial Action Unit.

References

1. Ekman, P.; Friesen, W. (1978). Facial action coding system: A technique for the measurement of facial movement. Palo Alto, Ca.: Consulting Psychologists Press.
2. Friesen, W.; Ekman, P. (1983). EMFACS-7: Emotional Facial Action Coding System. Unpublished manual, University of California, California.
3. Ekman, P.; Hager, J. (2002). Facial Action Coding System Affect Interpretation Database (FACSAID). Retrieved from http://face-and-emotion.com/dataface/facsaid/description.jsp, September 2006.
4. www.psychologytoday.com/us/blog/thriving101/201001/what-science-has-say-about-genuine-vs-fake-smiles
5. J. F. Grafsgaard, J. B. Wiggins, K. E. Boyer, E. N. Wiebe and J. C. Lester (2013). Automatically recognizing facial expression: Predicting engagement and frustration. In Proceedings of the 6th International Conference on Educational Data Mining (pp. 43-50).
6. https://en.wikipedia.org/wiki/Arousal
7. P. W. Ekman, V. Friesen and J. C. Hager (2002). FACS manual. A Human Face.
8. Oster, H. (2016). Baby FACS: Facial Action Coding System for infants and young children. Unpublished monograph and coding manual. New York University.

---

Download the original PDF: FaceReader and the Facial Action Coding System (PDF)