Interactive robot operation using multi-modal man-machine communication

Small- and medium sized enterprises (SME) in Western Europe are currently in a challenging situation. On one hand, they are small, flexible and have low overhead costs compared to bigger enterprises. On the other hand, they are often dependent on manual work force, and thus sensible for the increased wage- and social costs. To improve competitiveness and EHS, SMEs have to invest into industrial robot systems. However, the industrial robots have to be easy to use and reconfigure, with the capability to include human interaction during run time to adapt to unforeseen changes in the environment. Currently, the lack of intuitive and efficient communication between the robot system and the user, is one of the main limitations to use industrial robots in SMEs. With a more modern technology, the user could easier learn, operate, reprogram and monitor the robot, and thus, utilize it even in small series production. In this PhD, the focus is to extend the channels for man-machine communication by utilizing multiple human senses in parallel, i.e., vision, audio, haptic etc. By utilizing this multimodal man-machine communication (4MC), the intuitiveness of the man-machine communication can be enhanced to a new level of robot operation by significantly increasing the effectiveness and transfer rate of information between the human and the robot system. The PhD will focus on basic research on 4MC; develop new, 4MC based man-machine communication and demonstrate it by selected industrial related robot applications. This project, Interactive robot operation using multi-modal man-machine communication, is a collaboration between the University of West England (Bristol, UK) in their R&D organization, Bristol Robotics Laboratory and the high tech, research based, industrial robot system integrator, PPM AS in Trondheim, Norway, which is the project leader.

Outcomes and impacts: - Virtual fixtures based control for a monitoring robot to inspect industrial processes in difficult scenarios. The framework allows autonomous motion and safe navigation in tele-operation mode, in contrast with other approaches that focus on autonomous behaviour only. - Study on the effects of the virtual fixtures control on performance and navigation for a static inspection task with untrained participants. - Investigation on the positive impact of multimodal feedback modalities for the control of the monitoring robot. The selected feedback (haptic and visual) has shown positive effects on the navigation performance while not affecting the cognitive load for the operators.

Small- and medium sized enterprises (SME) are currently in a challenging situation. On one hand, they are small, flexible and have low overhead costs. On the other hand, they are dependent on manual work force which makes their competitiveness dependent on the wage level. Due to this, the SMEs have a high pressure to invest in industrial robot systems to reduce the wage. To lower the threshold for SMEs to invest in robot systems, the robots have to be easy to use and be capable to utilize human knowledge during run time. However, this puts high requirements on efficient and intuitive communication between the operator and the industrial robot. The current robot technology, offers a good solution for large volume production. However, small series productions, requires frequently reconfiguration, and requires a more efficient and intuitive interface for man-machine communication. This PhD will develop new technology for man-machine communication which is utilizing multiple human senses in parallel, i.e., vision, audio, haptic etc. By this, the intuitiveness and the efficiency of man-machine communication is intended to be brought to a new level. The Industrial PhD project, will focus on basic research on multi-modal man-machine communication and demonstrate it on selected industrial related robot applications. The focus will be industrial applications where the bottleneck is the man-machine communication, such as man-in-the-control loop, robot systems where the human operator is interacting in real time with a running robot system. This project, Interactive robot operation using multi-modal man-machine communication, is a collaboration between the University of West England and the high tech, industrial robot system integrator, PPM AS in Trondheim. The project is intended to run during September 2016 - August 2019.