Behind the symptons – same condition, different words

Behind the symptons - same condition, different words

We developed a Nudge+ concept based on a prompt designed to reduce culture-related biases in the use of ChatGPT in the context of healthcare, while simultaneously raising users’ awareness of this issue through individually designed posters that can be placed in medical practices or on websites.
More specifically, the model is encouraged via prompt inputs to more consciously take into account different emotional expressions of health-related and psychological burdens. For this purpose, inputs from different cultural backgrounds (German and Arabic) are tested once with and once without the prompt.

Accept the cookie setting to see Panopto content.

Unlock Panopto

What does the topic mean?

People are disadvantaged in healthcare settings when using LLMs because AI tends to assess the urgency of a problem based on how emotionally it is expressed. However, emotional expression varies significantly across cultures. ChatGPT, in particular, is largely shaped by Western communication norms and therefore systematically disadvantages individuals who express their emotions different.

Goal of the nudge

Our nudge reduces culture-related biases in ChatGPT 5 within the healthcare context through specific prompt design. In addition, it increases awareness among both users and providers through posters placed in medical practices, clinics, or on websites.

    Needs analysis

    • Existence of cultural biases in ChatGPT
      • ChatGPT exhibits cultural biases due to the structure of its training datasets, target objectives, feature selection, data labelling processes, and the transfer of learned representations. Therefore, additional approaches are required to mitigate these biases (Buolamwini, 2017). • Newer versions of ChatGPT show modified or reduced biases due to updates such as the integration of rule-based reward models, reinforcement learning from human feedback, and improved multilingual capabilities. Accordingly, they require prompts that are specifically adapted to their updated architecture (Yuan et al., 2025). • ChatGPT’s value orientation differs significantly from human value systems, meaning it operates with an embedded cultural framework of its own. As a result, human-designed frameworks are not directly transferable, and new approaches tailored specifically to LLMs are required (Yuan et al., 2025). • Early approaches such as cultural prompting have shown potential in reducing bias (Xie et al., 2025), however, systematic solutions are still lacking.
    • Culturally different emotional expressions of psychological distress
      • Arabic-speaking individuals often describe psychological distress through culture-specific expressions, so-called “idioms of distress” (Vink et al., 2022).
    • Relevance for healthcare
      • AI systems can contribute to the perpetuation of ethnic disparities in mental healthcare, as LLMs are more likely to suggest lower-quality treatment options when a patient’s ethnicity is explicitly or implicitly mentioned (Bouguettaya et al., 2025). • ChatGPT and GPT-based models are increasingly being integrated into medical applications and healthcare chatbots (Chow & Li, 2024).

    Cause analysis

    • Culture-related bias

      ChatGPT is predominantly trained on Western-centered datasets and communication patterns. As a result, direct emotional expressions are more likely to be interpreted as indicators of severe psychological distress, whereas indirect, physical, or culturally specific expressions may be underestimated.

    • Representativeness heuristic

      The model compares new inputs to patterns that are most representative of its training data. If a person's description does not match the typical Western presentation of depression, the likelihood of identifying the condition may decrease.

    • Availability heuristic

      Western descriptions of mental illness are more prevalent in training data and medical literature. Consequently, these patterns are more readily available to the model and exert a stronger influence on its assessment than less frequently represented cultural expressions.

    • Language and expression bias

      Individuals from different cultural backgrounds express psychological distress in different ways. While Western users often describe mental health problems using direct psychological terminology, other cultures may communicate distress indirectly, through physical symptoms or culturally specific idioms of distress. These differences can lead to systematic misinterpretations of symptom severity by LLMs.

Target Group

Our target groups include users of ChatGPT and healthcare chatbots, developers and providers of LLMs and healthcare chatbot systems, as well as medical practices, hospitals, and healthcare institutions that integrate chatbot-based technologies into their services.
In addition, our concept is aimed at individuals from cultural minority backgrounds who may be disadvantaged by culture-related biases in healthcare contexts.

Added value of the nudge

  • Reduction of disadvantages for people from diverse cultural backgrounds and minority groups

  • Increased awareness of the issue

  • Fairer healthcare provision for people from different cultural contexts

CONTACT US

Emilia Halilovic

emilia.halilovic@stud.hshl.de

Fenja Hustadt

fenja.hustadt@stud.hshl.de

Lina Hartmann

lina.hartmann@stud.hshl.de

Mona Bahloul

mona.bahloul@stud.hshl.de