Guardrails for Human-AI Ecologies: A Design Theory for Managing Norm-Based Coordination

Coordination in human-AI ecologies is made possible by social norms for interaction. Managing human-AI ecologies, therefore, requires decisions about norm specification, while allowing for the emergence of norms in ecologies in such a way that enables coordination in unspecified, unstructured situations. Yet it is also important that emergent norms are not harmful and that they follow guardrails set by managers under consideration of values. We integrate predictive processing theory and social norm theory to explain how existing norms are enacted and reinforced through agents' predictive models but also how new norms emerge as agents update their predictive models in response to prediction errors in uncertain situations. Rooted in this perspective, we propose a set of design principles for managers to encode norms to evolve in human-AI ecologies, monitor outcomes, and intervene when necessary.