New framework measures how much control pilots should give to AI copilots

<p>In industry and academia, the autonomy level of computerized systems is today being characterized by a singular dimension ranging from fully human labor (zero automation) to full automation with no human involvement. However, with more capable artificial intelligence supporting the human, it is increasingly relevant to understand what kind of cognitive control work that is automated. A system that can perform advanced cognitive tasks (e.g., adaptation to new control situations) is very different from one that can only perform basic tasks (e.g., lane keeping). These questions become relevant when designing or evaluating human-AI systems, namely how the cognitive work shall be distributed among the human and the AI agent, and, crucially, what kind of cognitive work they will perform together. Therefore, in this article, the aim is to support the inclusion of the cognitive dimension in systems assessment. As an analysis tool, we propose to include the Levels of Autonomy in Cognitive Control (LACC) jointly with the established and widely used Levels of Automation (LOA). We describe the approach and its bidimensional LACC-LOA matrix assessing six digital assistant concepts in the aviation domain with different levels of cognitive power. In our study, we considered one key scenario from each use case, through an online workshop format. In sum, the LACC-LOA assessment method granted considerable granularity to our understanding not simply of "who/what is in charge?," but "in charge of what?" Most of the use cases examined straddled not one but several LOAs, and the LACC added a useful dimension showing where the core "cognitive work" resided in these LOAs for human and AI. More generally, this extended mapping has implications for determining how the introduction of AI-based systems could affect human agency in the system. [GRAPHICS]</p>