AAP 2026 CONFERENCE

Recent debates on the ethical use of AI in medicine have gradually shifted from asking whether explainability as an epistemic property matters morally in terms of adopting a medical AI that cannot be fully understood by humans to determining how much explainability is required across different clinical contexts. This shift recognises that explainability is a matter of degree, and that the ethical adoption of medical technologies does not always require a full understanding of their underlying mechanisms. Following this view, some suggest that the level of explainability required should be determined by how a medical AI system would affect a person’s life — the greater the irreversibility, invasiveness, or risk of an intervention, the higher the demand for explainability.

This paper challenges that position. Using recent research on sepsis scoring systems and their use in the clinical context as a case study, I argue that the level of explainability required for adopting a diagnostic tool do not track these clinical factors. Instead, the degree of explainability ethically required should depend on the epistemic objectives the tool is designed to fulfil. In some contexts, a high level of explainability may be essential even when clinical risks are low.