Explainable artificial agents: : Considerations on trust, understanding, and the attribution of mental states

Abstract

The use of artificial agents (i.e., artificial intelligence and physical robots) is increasing in a wide range of application contexts, many of which already concern the daily lives of non-expert users. For artificial agents to be socially accepted, it is fundamental that users place calibrated trust (i.e., not too much, not too little) in them. In turn, this depends on several factors, which include artificial agents performance, accuracy and, importantly, understandability.This dissertation addresses a set of challenges that need to be overcome to successfully make artificial agents explainable and understandable by non-expert users. To this end, as explanations represent a fundamental form of social communication which has been thoroughly studied by social sciences, the first challenge tackled by this dissertation is of multidisciplinary nature. Here, we aim to integrate findings from social sciences into the design of explainable artificial agents. In particular, drawing from Karl Weick’s ‘sensemaking theory’, this dissertation proposes a model for explanatory interactions with artificial agents.Furthermore, this dissertation identifies factors that influence trust development over time. Additionally, the beginning of an interaction and the occurrence of unexpected events are found to be the situations that most likely require artificial agents to provide explanations for. Accordingly, this dissertation reports the results of an experimental study on which these theoretical considerations are tested by means of a mixed methodology investigation. Our main findings concern explanations’ positive role as a trust restoration strategy, as well as the influence of ‘institutional’ cues and individuals’ personality traits (e.g., propensity to take risks) in determining trust development.Finally, this dissertation discusses how explanations typically refer to either intentional (i.e., intentions, reasons etc.) or unintentional (i.e., accidental, natural etc.) factors. This is of particular relevance for artificial agents, as they do not possess the genuine mental states required by biological intentionality and yet people easily attribute such qualities to them. This dissertation states that the attribution of intentionality to artificial agents is ethical, as long as their artificial nature is manifest. However, at the same time, artificial agents should support users, by means of explanations, with adopting the most adequate interpretative framework for each situation.