Detecting gaze direction using robot-mounted and mobile-device cameras

Two common channels through which humans communicate are speech and gaze. Eye gaze is an important mode of communication: it allows people tobetter understand each others’ intentions, desires, interests, and so on. The goal of this research is to develop a framework for gaze triggered events that can be executed on a robot and mobile devices and allows to perform experiments. We experimentally evaluate the framework and techniques for extracting gaze direction based on a robot-mounted camera or a mobile-device camera that are implemented in the framework. We investigate the impact of light on the accuracy of gaze estimation, and also how the overall accuracy depends on user eye and head movements. Our research shows that light intensity is important, and the placement of a light source is crucial. All the robot-mounted gaze detection modules we tested were found to be similar with regard to their accuracy. The framework we developed was tested in a human-robot interaction experiment involving a job-interview scenario. The flexible structure of this scenario allowed us to test different components of the framework in varied real-world scenarios, which was very useful for progressing towards our long-term research goal of designing intuitive gaze-based interfaces for human robot communication.