Interaction Substitution for Mobile Games Accessibility and Exergaming

Speaker: 
Dragan Ahmetovic, University of Milan, Italy

Abstract:
Mobile device interaction is based on touchscreen usage, which poses accessibility barriers to people with upper extremity motor impairments. Existing mobile accessibility features, provided as software services, substitute the default touchscreen interaction paradigm with simpler alternatives. This is commonly achieved by replacing direct touchscreen access to interface elements with their sequential traversal, until a target element is reached. However, this sequential access is slower, and therefore not suited for apps with time-constrained interaction, such as games. In many cases alternative interaction paradigms cannot be used at all because common third-party software development kits do not advertise their interface elements to the mobile operating system.
To address these issues, we investigate a novel interaction substitution approach based on the direct mapping of alternative triggering actions to previously annotated touchscreen interaction events. The proposed approach can be used as an accessibility layer to make existing and popular games accessible without directly intervening on them. Within this framework, we explore the design space of the alternative triggering actions, proposing four input modalities that can be personalized to specific user needs. Through a series of studies with representative users, we assess the accuracy, time overhead, workload and appreciation of the proposed alternative inputs, compared to non-mediated touchscreen access by people without disabilities. Finally, we explore the applicability of the approach to telerehabilitation by transforming existing mobile games into full-fledged exergames.

Biography:
I am an associate professor at the Department of Computer Science of the University of Milan. My research focuses on assistive technologies designed to augment the cognitive, sensing and functional capabilities of users and overcome barriers that limit their access to information and physical world. I investigate embodied sensing, knowledge retrieval, intelligent user interfaces and human computer interaction, mediated by mobile and pervasive devices.
I conducted my doctoral studies at EveryWare Lab of the University of Milan and I received my Ph.D. in March 2015. Afterwards, I have worked as a postdoctoral fellow and project scientist at the Cognitive Assistance Lab of Carnegie Mellon University, Pittsburgh, USA and as a postdoctoral researcher at the “S. Polin” Laboratory for Research and Experimentation with Novel Assistive Technologies for STEM at the University of Turin, Italy.

Host: Prof. Monica Landoni