Normative Narrator: Guiding and Explaining Reinforcement Learning Agents

Abstract

A normative supervisor is an external module that uses a formal reasoning engine to impose normative constraints on re inforcement learning agents, by either dynamic action masking or feeding the agent additional punishments when violations of norms occur (or both). In this paper, we use a normative supervisor implemented with a solver for deontic answer set programming (ASP) — deolingo — as a basis for the construction of a normative narrator, which uses deolingo’s ability to interface with the explainable solver xclingo to construct a module capable of both regulating behaviour through action masking and additional punishments, and providing contrastive explanations of why a given action was allowed while others were not, relative to the normative system being enforced. The explanations are modelled after the two tiers – internal and external – of explanation. We demonstrate this approach’s ability to provide understandable and descriptive explanations in a scenario where a taxi driver agent must act in accordance to a normative system governing its normal duties and provisions that must be made in case of an emergency.