Virtual reality environment for movement optimization with haptic device. Application for Assembly/Disassemby operations simulation by hand gesture recognition

Title: Virtual reality environment for movement optimization with haptic device. Application for Assembly/Disassemby operations simulation by hand gesture recognition
Thesis Director : Peter MITROUCHEV (MCF-HDR, SIREP, G-SCOP)
PhD School : Ingénierie - Matériaux - Mécanique - Environnement - Energétique - Procédés - Production (IMEP-2)
Start date : October 2016
Proposed funding : Request MESR allowance (Ministry of Higher Education and Research) for 3 years.
Context: The proposed PhD subject fits into one of the main research themes of SIREP team of G-SCOP Laboratory, namely: optimization, modeling and simulation of disassembly operations. It is in the continuation of some works already done in the SIREP team, related to: the identification of contacts and disassembly trajectory generation; optimization in the generation of disassembly sequences and their integration into a Virtual Reality environment.

Partnership: The proposed research subject falls under a commune set of research themes, within the Research actions, AAR Authoring Augmented Reality, WP2, Real-time capture and simulation of the real world. Representation and editing of virtual prototypes. Natural interaction with the augmented world, of LABEX PERSYVAL Lab (http://www.persyval-lab.org/index.html) in cooperation with GIPSA-Lab and et INRIA-LIG. Moreover, it fits into a common theme of research within the framework of a collaboration that had begun six years ago with Shanghai Key Laboratory of Mechanical Automation and Robotics at the University of Shanghai, China.

Brief Description:

Context: The existing platforms of assembly/disassembly (A/D) simulation by hand gestures recognition are often badly integrated in the Products Development Process (PDP). Some approaches for modelling of disassembly were proposed but they do not allow validating them because they do not take account of the physiological state of the operator for varied conditions of request (postures, efforts, fatigue,…). In last decades, virtual reality (VR) technology has evolved to a new level of sophistication. Now, it combines several human – computer interfaces (HCI) to provide various sensations such as: visual, auditory, haptic, which enables users to become immersed in a computer generated platform. Muscle-computer interaction (muCI), for instance, represents a useful tool to address hand gestures recognition and forearm muscle actions, particularly for characterizing the involved fingers and the exhaustion level during grasping. Adding such a technique in HCI to improve platform of immersion (VR) represents a scientific challenge for research today.

Scientific challenges: Current simulation platforms do not offer the necessary information and versatility required for a complete A/D process simulation, including human/operator data management based on physiologic data (ElectroMyoGraphy-EMG signal). In order to improve the quality of a real-time A/D simulation environment, the present research subject is focusing on two elements: a better haptic devices integration and mobility module evaluation including physiologic data about the muscular-skeletal state. The latter should be able to generate the mobility of a component from an assembly with respect to its surroundings and the muscular capabilities of the operator. As a result, more transparent access behaviour of the components in the simulation environment will be obtained. An immersive platform that integrates one or more haptic devices, using the model data, cognitive constraints, efforts in the muscles and their optimization, mobility information and the state of the muscular-skeletal system, should offer a more realistic simulation process through kinematic guided movements and human muscular control assessment. Nowadays, VR environments have significantly evolved towards A/D simulation, highlighting new requirements for the preparation stages and their integration. Many of these platforms use haptic feedback and are facing difficulties while simulating the insertion/extraction operations realised by a human.

Objectives: In this context, the main objective of the proposed PhD subject is to improve the A/D process simulation through better haptic devices integration including physiologic data. To this end, a series of tests with a 6 degrees of freedom (DOF) haptic device and EMG data are necessary. The aim is: i). to provide a robust acquisition technology associated to an appropriated EMG signal processing, based on the use of EMG network sensors (localisation, tolerance in wrong positioning, optimal number of electrodes) in order to improve the time delay of detection for the separation and the classification of different hand gestures, ii). to propose a mobility module able to model contact relations between elementary components of a product and to determine the relative mobilities of the assembly components. A model including the physiologic state of the operator and quantifying the muscular fatigue for example, for planning and simulating of A/D operations will be proposed as well. It will be validated via its integration in a constrained virtual environment allowing the simulation of A/D operations within the framework of the existing data-processing environment, as its integration in the PDP. The main issues addressed in this thesis are: i). to perform A/D simulation tests with a 6DOF haptic device, ii). to introduce muCI as supplementary data in A/D simulation, iii). to evaluate the possibility of implementing a mobility module in a real-time A/D simulation environment, iv). to interact naturally with the augmented world via human motion modelling and simulation.


Contact(s) :

Peter MITROUCHEV, HDR
G-SCOP, Sciences for Design, Optimisation and Production Laboratory of Grenoble
UMR-5272 CNRS,
46, av. Félix Viallet,
38031, GRENOBLE Cedex 1, FRANCE
phone : + 33 (0)4 76 57 47 00
fax : + 33 (0)4 76 57 46 95
e-mail : Peter.Mitrouchev@g-scop.inpg.fr