Emotion Recognition: Ethical Or Invasive?

The global market for emotion recognition technologies (ERT) is rapidly expanding, and there is increasing concern about their ethics. Ethical considerations include the risk of biased and unfair outcomes, the sensitivity of emotion data, and the risk of harm in consequential settings, including employment, education, healthcare, and policing.

Privacy and personal control

The use of emotion recognition technology (ERT) raises several ethical concerns regarding privacy and personal control. ERT relies on the collection and analysis of personal data, including facial images, physiological signals, and contextual information, which raises questions about privacy and data protection. There are concerns about who has access to this information and how it is used. To address these concerns, companies using ERT should implement strict privacy policies, secure data storage methods, and obtain informed consent from individuals.

The use of ERT in public spaces or through personal devices without individuals' knowledge or consent is a significant ethical issue. It raises questions about the right to privacy and the potential for surveillance and intrusion into personal emotions and behaviours. The accuracy and potential biases of ERT are also questionable, as emotional expression can be highly subjective and culturally varied, leading to potential misinterpretations and negative consequences, especially in security and law enforcement contexts.

ERT has the potential to be used for

Group privacy and soft biometrics

Soft biometrics are physical or behavioral features that can be described by humans, such as height, weight, hair color, and ethnicity. They are not unique to an individual but can be aggregated to provide discriminative biometric signatures. Soft biometrics can be typically described using human-understandable labels and measurements, allowing for retrieval and recognition based on verbal descriptions. They can be obtained at a distance without subject cooperation and from low-quality video footage, making them ideal for use in surveillance applications.

Soft biometrics are often used to improve the performance of traditional biometric systems and allow identification based on human descriptions. They can be extracted from very low-quality data such as those generated by surveillance cameras. They also require limited cooperation from the subject and can be non-intrusively obtained, making them ideal in surveillance applications. One of the main advantages of soft biometrics is their relationship with conventional human descriptions; humans naturally use soft biometrics to identify and describe each other.

Soft biometrics are usually extracted and utilized in surveillance applications, where the evidence attributes exhibit the adverse influence of low-resolution issues due to the long distance. They are also used in healthcare, human resources, and education. However, there are ethical concerns regarding the use of soft biometrics. These concerns include the risk of biased and unfair outcomes due to the faulty bases and problematic premises of soft biometrics, the sensitivity of emotion data used, and the risk of harm that arises from the technologies in consequential settings, including employment, education, healthcare, and policing.

Variability of expression and mental representation

The variability of human expression and mental representation of emotions is a key consideration when discussing the ethics of emotion recognition technology. This variability is influenced by a range of factors, including cultural norms, context, and individual differences. This section will explore the implications of this variability for the ethical deployment of emotion recognition technologies.

Cultural Norms and Context

Cultural norms and context play a significant role in shaping how emotions are expressed and perceived. Cultural values, for example, provide rules or guidelines for the expression of emotions, with "display rules" dictating how emotions should be expressed in different social situations. These norms vary across cultures, and the situational context in which an emotional expression occurs also influences how emotions are perceived. As a result, inferring emotions based solely on facial expressions or other limited sets of cues can be problematic and unreliable.

Individual Differences

In addition to cultural and contextual factors, individual differences also contribute to the variability of emotion expression and perception. For instance, the ability to recognize emotions can vary among individuals, with some people being more adept at interpreting emotional cues than others. Additionally, certain mental health conditions, such as autism spectrum disorders, can affect an individual's ability to express and perceive emotions, further contributing to the variability of emotion expression and perception.

Implications for Emotion Recognition Technology

The variability of emotion expression and perception has significant implications for the ethical deployment of emotion recognition technologies. Here are some key considerations:

• Risk of Biased and Unfair Outcomes: Emotion recognition technologies that rely on a limited set of cues, such as facial expressions, may produce biased and unfair outcomes due to the variability in how emotions are expressed and perceived. This can lead to inaccurate assessments of an individual's emotional state, which may have negative consequences in sensitive settings such as employment, education, healthcare, and policing.
• Sensitivity of Emotion Data: Emotion data, including facial expressions and other biometric signals, is considered sensitive information. The collection and use of such data raise privacy concerns, especially when it is linked to personal information or used without an individual's consent.
• Risk of Harm in Consequential Settings: The use of emotion recognition technologies in consequential settings, such as employment, education, healthcare, and policing, carries a risk of harm. In these settings, inaccurate assessments of an individual's emotional state can have significant negative consequences, such as discrimination, denial of services, or adverse legal outcomes.
• Need for Ethical Design and Implementation: The variability of emotion expression and perception underscores the need for ethical design and implementation of emotion recognition technologies. This includes ensuring a defined scope for the use of these technologies, ethical decision-making, fairness, non-discrimination, and privacy protections.
• Addressing Variability: To address the variability of emotion expression and perception, emotion recognition technologies should incorporate a broader range of contextual and cultural factors. This may include considering the influence of cultural norms, situational context, and individual differences, such as mental health conditions, when interpreting emotional cues.
• Informed Consent and Transparency: Due to the sensitivity of emotion data and the potential for harm, it is essential to obtain informed consent from individuals before collecting and using their data. Transparency about the use of emotion recognition technologies is also crucial to ensuring individuals' privacy and autonomy.
• Evaluation and Validation: The performance of emotion recognition technologies should be rigorously evaluated and validated across diverse populations and contexts to ensure accuracy and fairness. This includes assessing the technologies' ability to handle variability in emotion expression and perception and mitigating potential biases and harms.

Norms of emotion expression

Additionally, gender differences exist in the perception of the injunctive norms of emotion expression. Females tend to rate the expression of positive emotions as more appropriate than males on Facebook, Twitter, and Instagram. However, there are no significant differences between males and females in the expression of negative emotions, except for expressing worry on Facebook.

Norms of attitudes

The use of emotion recognition technologies (ERT) has raised ethical concerns, especially as their commercial development for widespread use continues. Norms of attitudes towards ERT can be summarised as follows:

• Risk of Biased and Unfair Outcomes: ERT is based on the faulty bases and problematic premises that emotions can be universally recognised from facial expressions. This has been debunked by research, which shows that emotions are better understood as flexible patterns shaped by cultural and social factors. Therefore, ERT can lead to biased and unfair outcomes.
• Sensitivity of Emotion Data: ERT relies on sensitive emotion data, which is often linked to mental data. This data is considered highly sensitive, and there are concerns about its misuse.
• Risk of Harm: ERT can cause harm in consequential settings, including employment, education, healthcare, and policing. This is due to the risk of biased and unfair outcomes and the sensitivity of emotion data.

Attitudes Towards Ethical Use

ERT raises significant ethical issues, and attitudes towards its use should consider the following:

• Ethical Design and Implementation: ERT should have a defined scope, and its use should involve ethical decision-making, fairness and non-discrimination, and privacy.
• Addressing Bias: ERT training data should be diverse and representative to avoid biased outcomes that disproportionately affect marginalised groups.
• Privacy: ERT should not be used to invade privacy or infringe on individual rights and freedoms.
• Regulation: There is a need for regulation to ensure ethical use, especially in sensitive contexts such as law enforcement and surveillance.
• Human Dignity: ERT should not objectify individuals or treat them as a means to an end. It should respect human dignity and autonomy, and not restrict fundamental rights and freedoms.
• Common Good: ERT should benefit society as a whole, not just powerful corporations or governments. It should be used to improve the well-being of all people and promote human-centred design.
• Transparency: There should be transparency about how ERT is used, who has access to data, and how it is stored securely.
• Consent: Individuals should provide informed consent for the collection and use of their data.

Frequently asked questions