FairCredit

Rethinking AI decisions

The nudge targets credit officers, analysts and risk managers in banks. When an AI-based credit scoring system calculates a person’s score to a significant degree based on residential data (e.g. postal code), an automated warning appears next to the score.
This prompts the user to review the individual income profile manually before a decision is made. If the employee nonetheless decides to follow the AI’s recommendation, they must provide a written justification. Targeted reflection questions are designed to encourage more conscious engagement with the algorithmic output and to make potential biases visible.

Accept the cookie setting to see Panopto content.

Unlock Panopto

What does the topic mean?

Representation bias in AI-assisted credit scoring refers to the phenomenon whereby AI systems absorb and perpetuate historical social inequalities embedded in their training data. Modern scoring models often rely on financial data that reflects systematic disadvantages, such as those based on ethnicity or socioeconomic status.
A particularly problematic example is geoscoring: creditworthiness is assessed based on residential data such as postal code or regional averages. This causes applicants to receive worse credit terms, higher interest rates or complete exclusion from lending, regardless of their actual ability to repay.
Additionally, automation bias leads credit employees to accept algorithmic decisions uncritically, perceiving them as more objective than human judgement.

Goal of the nudge

The goal of the nudge is to actively combat representation bias in AI-assisted credit lending. By making the influence of residential data on the score visible through a warning, the nudge prevents historically entrenched inequalities from being reproduced unreflectively through algorithmic decisions.
Automation bias, the uncritical adoption of AI recommendations, is used as a lever: by deliberately interrupting this mechanism, more conscious and individually reviewed credit decisions are encouraged.

    Needs analysis

    • Discrimination through geoscoring
      Individuals are systematically rated worse based on their postal code or residential area, regardless of actual creditworthiness.
    • Financial exclusion
      Affected applicants face higher interest rates, worse credit terms or outright rejection.
    • Lack of transparency
      Borrowers are not informed that their place of residence was a decisive factor in their rejection. .

    Cause analysis

    • Representativeness heuristic (Kahneman & Tversky)

      AI systems judge by statistical
      patterns from historical data – individuals from lower-income areas are categorised
      as "high-risk" based on stereotypes rather than individual assessment.

    • Automation bias

      Credit staff perceive algorithmic scores as more neutral and
      precise than their own judgement, and adopt discriminatory decisions without
      scrutiny.

    • Statistical discrimination

      Group characteristics (place of residence) are used indirectly as a risk proxy, even though individual financial factors should be the basis for decisions.

Target Group

The direct target group consists of credit officers, analysts and risk managers in banks and financial institutions who evaluate AI-generated scoring results and make final lending decisions. The indirect target group are credit applicants, particularly those from neighbourhoods systematically disadvantaged by geoscoring.

Added value of the nudge

  • Fairer lending decisions

    The nudge interrupts automation bias and encourages
    individual review, reducing the unjustified rejection of marginalised applicants.

  • Reduced legal and reputational risk

    Mandatory written justifications lower the risk
    of discriminatory decisions and associated litigation or reputational damage.

  • Improved profitability

    Creditworthy customers who were previously rejected due to
    geoscoring can be identified and gained as profitable borrowers.

CONTACT US

Anna Bredemeier

anna.bredemeier@stud.hshl.de

Emily Schütz

emily.schuetz@stud.hshl.de

Louisa Siebrasse

louisa.siebrasse@stud.hshl.de