Benefits of implicit and explicit measurements of engagement

In this blog post, Tess den Uyl, PhD from VicarVision shows how to measure engagement in different ways.

Everyone wants to create engaging content and learn if people are truly engaged. But how do we accurately measure engagement? Nicholas Keatch’s research project, part of his academic focus on social psychology, aimed to explore the intricate nature of engagement.

In his study, he examined how people interact with movie trailers—emotionally, cognitively, and behaviorally. Importantly, this study included both self-reported and implicit measures.

What is engagement?

Engagement refers to an active involvement and emotional connection with a product, brand, or experience. It goes beyond attention—it's about how someone is interacting with a brand.

Engagement plays a vital role in how people interact with different types of content, whether it's advertisements, movie trailers, or educational videos. By studying engagement, marketers and UX researchers have an indication of how much someone is interested in their content. In fact, for most video marketers, engagement is the most important metric.

Different types of engagement

In this study, Keatch examined how engaged people were with different movie trailers. To study engagement, he used a model based on Ben-Eliyahu’s (2018) framework and previous work by Noldus intern Dayenne Sarkol. This model divides engagement into three key dimensions:

• Cognitive engagement: how well viewers concentrate on the content and maintain their attention
• Affective engagement: emotional responses to the content, such as interest, positive attitude, involvement, and motivation
• Behavioral engagement: actions taken as a result of engagement, like recommending, sharing content, or interacting with the brand

In Sarkol’s work on engagement, she found that a real-time expression of engagement was too complex to capture all three aspects at once. Therefore, the current project focused on measuring the separate components of engagement.

Measuring engagement

Keatch recruited 75 participants for this study, through platform Prolific and personal invitations. Participants watched three movie trailers from different genres: My Spy: The Eternal City (action-comedy), Fly Me to the Moon (romantic-drama), and Paddington in Peru (family-comedy).

Each trailer was divided into two parts. Participants had the option to either continue watching or switch to another trailer after the first part. Their decisions provided an implicit measure of behavioral engagement.

Additionally, we used FaceReader Online to measure implicit cognitive and affective engagement. We did so by detecting facial expressions like happiness, sadness, and attention. We also measured explicit cognitive, affective, and behavioral engagement through self-reports.

By using both implicit and explicit measures, we aimed to understand engagement more completely.

Engagement when staying vs. switching

As expected, self-reported engagement was higher for participants who chose to continue watching rather than switch trailers. Decisions to switch were roughly around 50%.

Specifically, behavioral and affective engagement were significantly higher in those who continued watching the trailers. However, the cognitive component (e.g., concentration) showed only marginal significance.

Interestingly, this pattern was not consistent across all trailers.

• Participants who chose to keep watching the Paddington trailer scored higher on all engagement components.
• Those who continued watching Fly Me to the Moon only showed higher affective engagement.
• For My Spy, there was no difference in explicit engagement between the group that stayed and the group that switched.

Implicit engagement: facial expressions reveal different dynamics

Facial expression analysis provides an implicit way to measure reactions that are automatic and beyond conscious control.

Using FaceReader Online, we analyzed happiness, which is indicated by smiling and laughter. Both of these behaviors are important to measure affective engagement. In addition, we used FaceReader Online to measure cognitive engagement, by estimating the visible attention participants gave to the screen.

The implicit responses showed interesting differences between trailers.

• In the Paddington trailer, people who decided to continue watching had, on average, 15% higher attention scores. This indicates increased cognitive engagement.
• For Fly Me to the Moon, affective engagement was evident. People who kept watching showed a mean happiness level that was three times higher than those who switched.
• For My Spy, there was no difference in mean facial expressions between those who stayed and those who switched. However, an interesting emotional dynamic emerged—participants found a particular joke funny, but it did not persuade them to continue watching.

One advantage of facial expression analysis is its ability to capture immediate, real-time responses. That's how we learned that across all trailers, larger peaks in happiness were observed for those who decided to continue watching.

You may also like to read about how perception of brand authenticity affects brand performance or how to increase attention in commercials for Gen-Z.

The value of combining implicit and explicit measures

Engagement is a complex, multidimensional construct. The varied responses to each trailer highlight the difficulty of measuring engagement and the importance of understanding the context.

Using both implicit and explicit responses allows for a more accurate and nuanced understanding of engagement. This dual approach makes it possible to predict viewer behavior more effectively. As a result, professionals in fields like entertainment, advertising, and education can improve their content creation strategies.

By employing tools like FaceReader Online, content creators can test the effectiveness of videos with implicit and explicit measures to measure engagement comprehensively.