Virtual reality environment for motion optimisation using haptic devices. Application for Assembly/Disassembly operation simulation by hand gestures recognition

Title: Virtual reality environment for motion optimisation using haptic devices. Application for Assembly/Disassembly operation simulation by hand gestures recognition

Supervisor(s): Peter MITROUCHEV (MCF-HDR, SIREP G-SCOP, IMEP-2)

Financing - Context - Partnerships

Financing : Scholarship application MESR  (Ministry of Higher Education and Research) for 3 years.

Context: The proposed topic fits into one of the main research themes of SIREP team of G-SCOP Laboratory, namely optimization, modeling and simulation of disassembly operations. It is in the continuation of some works already done in the SIREP team, related to: the identification of contacts and disassembly trajectory generation; optimization in the generation of disassembly sequences and their integration into a Virtual Reality environment.

Partnership: The proposed research subject falls under a commune set of research themes, within the Work-package WP9, Task 9.1. Interaction and Manipulation within Virtual interactive Scenes of the European Infrastructure VISIONAIR (http://www.infra-visionair.eu/) in cooperation with ISTIC, Université de Rennes 1, l'EPI VR4i -  l'IRISA (http://www.irisa.fr/), and the PERSYVAL Lab, Research actions, AAR Authoring Augmented Reality, WP2, Real-time capture and simulation of the real world. Representation and editing of virtual prototypes. Natural interaction with the augmented world, (http://www.persyval-lab.org/index.html) in cooperation  with GIPSA-Lab. Moreover, it fits into a common theme of research within the framework of a collaboration, begun two years ago, with Shanghai Key Laboratery of Mechanical Automation and Robotics at the University of Shanghai, China.

Description of the subject:   

Context: The existing platforms of assembly/disassembly (A/D) simulation by hand gestures recognition are often badly integrated in the Products Development Process (PDP). Some approaches for modelling of disassembly were proposed but they do not allow validating them because they do not take account of the effective geometry of the parts constituting the mechanical assembly. They do not take account of the physiological state of the operator for varied conditions of request (postures, efforts...) either. In the last decade, virtual reality (VR) technology has evolved to a new level of sophistication. Now it combines several human - computer interfaces (HCI) to provide various sensations such as: visual, auditory, haptic, which enables users to become immersed in a computer generated platform. Muscle-computer interaction (muCI), for instance, represents a useful tool to address hand gestures recognition and forearm muscle actions, particularly for characterizing the involved fingers and the exhaustion level during grasping. Adding such a technique in HCI to improve platform of immersion (VR) represents a scientific challenge for research today.

Scientific challenges: Current simulation platforms do not offer the necessary information and versatility required for a complete A/D process simulation, including human/operator data management based on physiologic data (ElectroMyoGraphy-EMG signal). In order to improve the quality of a real-time A/D simulation environment, the present research subject is focused on two elements: a better haptic devices integration and mobility module evaluation including physiologic data about the musculo-skeletal state. The later should be able to generate the mobility of a component from an assembly with respect to its surroundings and the muscular capabilities of the operator. As a result, more transparent access behaviour of the components in the simulation environment will be obtained. An immersive platform that integrates one or more haptic devices, using the model data, cognitive constraints, efforts in the muscles and their optimization, mobility information and the state of the musculo-skeletal system, should offer a more realistic simulation process through kinematic guided movements and human muscular control assessment. Nowadays, VR environments have significantly evolved towards the A/D simulation, highlighting new requirements for the preparation stages and their integration. Many of these platforms use haptic feedback and are facing difficulties while simulating the insertion/extraction operations realised by a human.

Objectives:

In this context, the main objective of this research is to improve the A/D process simulation through better haptic devices integration including physiologic data. To this end, a series of tests with a 6 degrees of freedom (DOF) haptic device and EMG data are necessary. The aim is: i). to provide a robust acquisition technology associated to an appropriated EMG signal processing, based on the use of EMG network sensors (localisation, tolerance in wrong positioning, optimal number of electrodes) in order to improve the time delay of detection for the separation and the classification of different hand gestures. ii). to propose a mobility module able to model contact relations between elementary components of a product and to determine the relative mobilities of the assembly components. A model including the physiologic state of the operator, including the quantification of the muscular fatigue for example, for planning and simulation of A/D operations will be proposed as well. It will be validated via its integration in a constrained virtual environment allowing the simulation of A/D operations within the framework of the existing data-processing environment, as its integration in the PDP. The main issues addressed in this proposal are: i). to perform A/D simulation tests with a 6DOF haptic device; ii). to introduce muCI as supplementary data in A/D simulation; iii). to evaluate the possibility of implementing a mobility module in a real-time A/D simulation environment; iv). to interact naturally with the augmented world via human motion modelling and simulation.

Contact(s)

Peter MITROUCHEV, HDR

Laboratoire G-SCOP, Sciences pour la Conception, l'Optimisation et la Production de Grenoble,

UMR-5272 CNRS,

 46,  av. Félix Viallet,

38031,  GRENOBLE Cedex 1, FRANCE

téléphone         : + 33 (0)4 76 57 47 00

fax                   : + 33 (0)4 76 57 46 95

e-mail               : Peter.Mitrouchev@g-scop.inpg.fr