Explainability

Planning Building Deploying Monitoring
The technical logic of algorithms is complex, which make recommendations unclear. People involved in designing and deploying algorithmic systems have a responsibility to explain high-stakes decisions that affect individuals’ well-being.

Have you considered...?

  • Surveying individuals on whether they comprehend and trust the recommendations made by your model
  • Factoring in the stakes of decisions (e.g., recommending a movie vs. approving a home loan)
  • Choosing models that are easier to interpret (e.g., logistic regression, random forest)
  • Modeling counterfactual scenarios that would enable individuals to understand what would need to change to receive a desired result

Case study

The Fair Credit Reporting Act (FCRA) gives U.S. consumers the right to know and correct information about themselves. These rights may be violated if a complex model assigns credit scores but cannot justify them or provide information on how to improve them.

Have you engaged with...?

  • Affected communities
  • Legal counsel
  • UX researchers

Resources

What is missing?

Your suggestions

AI Blindspot Cards

PURPOSE

AI systems should make the world a better place. Defining a shared goal guides decisions across the lifecycle of an algorithmic decision-making system, promoting trust amongst individuals and the public.

REPRESENTATIVE DATA

For an algorithm to be effective, its training data must be representative of the communities that it may impact. The way that you collect and organize data will benefit certain groups while excluding or harming others.

ABUSABILITY

The designers of an AI system need to anticipate vulnerabilities and dual-use scenarios by modeling how bad actors might hijack and weaponize the system for malicious activity.

PRIVACY

AI systems often gather personal information that can invade our privacy. Systems storing confidential data can also be vulnerable to cyberattacks that result in devastating data breaches to access personal information.

DISCRIMINATION BY PROXY

An algorithm can have an adverse effect on vulnerable populations even without explicitly including protected characteristics. This often occurs when a model includes features that are correlated with these characteristics.

EXPLAINABILITY

The technical logic of algorithms is complex, which make recommendations unclear. People involved in designing and deploying algorithmic systems have a responsibility to explain high-stakes decisions that affect individuals' well-being.

OPTIMIZATION CRITERIA

There are trade-offs and potential externalities when determining an AI system's metrics for success. It is important to balance performance metrics against the risk of negatively impacting vulnerable populations.

GENERALIZATION ERROR

Between building and deploying an AI system, conditions in the world may change or not reflect the context in which the system was designed, such that training data are no longer representative.

RIGHT TO CONTEST

Like any human process, AI systems carry biases that make them subjective and imperfect. The right to contest an algorithmic decision can surface inaccuracies and grant agency to people affected.

OVERSIGHT

Ethical principles, standards, and policies are futile unless monitored and enforced. A diverse oversight body vested with formal authority can help to establish and maintain transparency, accountability, and sanctions.

CONSULTATION

The first, last, and every step in-between should include public participation. AI practitioners must enable meaningful input, explanations, and disclosures to ensure that AI systems promote human flourishing and mitigate harms.

BLANK TEMPLATE

Create a new card.

ABOUT

The AI Blindspot cards were developed by Ania Calderon, Dan Taber, Hong Qu, and Jeff Wen during the Berkman Klein Center Assembly program.

Learn more about the team.

COPYRIGHT

This work is licensed under a Creative Commons Attribution 4.0 International License.