Knowledge Base

Analyze Voice Characteristics

Aim

To switch on voice analysis.

Prerequisite

You have the Voice Analysis Module.

Procedure

1. Choose File > Settings > Analysis options.
2. Under Optional Classifications, select Voice analysis.

Note

The option to analyze the voice must be switched on, which is by default if you have the Voice Analysis Module. However, when you upgrade a basic FaceReader license with the Voice Analysis Module, you must switch the option on manually.

Visualize Voice Characteristics

Aim

To display the voice characteristics during or after the analysis.

Prerequisites

• Your FaceReader license includes the Voice Analysis Module.
• Voice analysis is switched on in the Settings. See Analyze voice characteristics
• You are running an analysis, or viewing the analysis afterwards.

Voice Expression Intensity

The Voice Expression Intensity chart displays the intensity of the four basic emotions that can be recognized in the voice (Neutral, Happy, Sad and Angry). You will see the bars change over time, reflecting the changes in expressions in the voice.

To view the Voice Expression Intensity Chart

Click the Select window button in one of the analysis windows and select Voice Analysis and then Voice Expression Intensity Chart.

Voice Expression Line Chart

The Voice Expression Line Chart shows the detected emotions over time. There are gaps in the line chart if the VAD does not detect voice. This detection method cannot be adjusted. If there is noise in the audio signal, the Voice Expression Line Chart may show continuous lines even though the participant did not speak all the time.

To view the Voice Expression Line Chart

Click the Select window button in one of the analysis windows and select Voice Analysis and then Voice Expression Line Chart.

There are gaps in the line chart at the time points when the volume was below the threshold.

Voice Valence And Arousal Line Chart

The Voice Valence indicates whether the emotional state of the subject is positive or negative. 'Happy' is the only positive emotion, while 'Sad' and 'Angry' are considered to be negative emotions. Valence is calculated by subtracting the intensity of the strongest negative emotion from the intensity of 'Happy'.

Arousal is the weighted average of Loudness and Speech rate. See How is Loudness calculated? and How is Speech rate calculated?

To view the Voice Valence and Arousal Line Chart

Click the Select window button in one of the analysis windows and select Voice Analysis and then Voice Analysis and Arousal Line Chart.

There are gaps in the line chart at the time points when the volume was below the threshold.

Voice View

The Voice View shows a two-second audio waveform. The color indicates the detected emotions. If a volume threshold is crossed, the two meters on either side of the waveform show the volume-independent Loudness and the Speech Rate.

To view the Voice View

Click the Select window button in one of the analysis windows and select Voice Analysis and then Voice View.

How Is Loudness Calculated?

Loudness is a custom volume-independent loudness measure. Variations in microphone sensitivity and speaker positioning can significantly affect the absolute volume of a recording. To address this, the loudness measure is computed using a continuously updated, normalized audio signal. For each frame in the video, the system analyzes the past second of audio. Within this one-second window, the audio is normalized so that its maximum absolute amplitude is 1.0. This ensures that the measurement reflects the speaker's vocal dynamics, i.e. how energetically a person is speaking within their own dynamic range, rather than the recording conditions. After normalization, the RMS (root mean square) value is calculated to capture the average energy of the speech in that time frame. Because the signal is normalized before computing the RMS, the resulting loudness value is always scaled between 0 and 1, where 0 corresponds to silence and 1 represents a sustained signal at maximum amplitude.

How Is Speech Rate Calculated?

The speech rate measure estimates how quickly a person is speaking. The algorithm detects peaks in the audio signal, typically corresponding to syllables, by identifying bursts of vocal energy. These peaks are counted over the same one-second window used for loudness analysis. The count is then normalized to a value between 0 and 1, using an upper bound suitable for capturing various emotional states [1,2]. A higher value indicates faster speech, while a lower value reflects slower or more deliberate speaking.

1. Arnfield, S., Roach, P., Setter, J., Greasley, P., Horton, D. (1995) Emotional stress and speech tempo variation. Proc. ESCA/NATO Workshop on Speech under Stress, 13–15.
2. Braun, Angelika & Oba, Reiko. (2007). Speaking Tempo in Emotional Speech - a Cross-Cultural Study Using Dubbed Speech.

Source: FaceReader 10 Reference Manual (Help), Noldus Information Technology