Project - ErgoWin - EN | Sébastien PASCAULT

Development of an application in mixed and augmented reality. Work ergonomics - Analysis and recognition of movements

Sébastien PASCAULT

Engineer: Computer scientist - Automatician - Computer graphics designer

Specialities: Human Machine Interfaces - Mixed Reality - Serious Gaming - Unity 3D

Mobile: 06 31 10 01 56
E-mail: sebastien.pascault@outlook.com
Team: https://teams.live.com
Skype: live:sebastien.pascault

About me:

Passionate about digital technology and simulated environments, I'm constantly looking for innovations and challenges. My passion for IT and computer graphics has motivated me throughout my studies and still motivates me today to continue my career in software development.

Resume:

I - Context:

Why ergonomics?

Musculoskeletal disorders are pathologies affecting muscles, tendons and nerves, accounting for 87% of occupational illnesses.

The ErgoWIN solution :

With the aim of preventing and avoiding Musculoskeletal Disorders within companies, ABMI is developing an ergonomic offer dealing with the biomechanical aspect of these disorders.

What does this offer consist of?

ErgoWIN is a gesture and posture diagnostic tool that uses a Kinect camera and associated analysis software to perform ergonomic ratings on a workstation.

The tool can therefore be used to carry out ergonomic diagnostics:
- Ergonomic analysis of workstations
- Ergonomic score
- Critical posture
- Extremely stressed limb

What are the benefits of augmented reality?

First and foremost, augmented reality enables digital simulations of virtual environments to be carried out in real time. It is then possible to superimpose virtual objects on reality in order to carry out various ergonomic analyses in the absence of equipment or when it is necessary to intervene on a customer's premises.
It's also possible to get a quick overview of a workstation to assess its validity and compliance with regulations, thanks in particular to the expertise of an ergonomist who can work upstream using an augmented reality application.

II - Objective :

The aim of the application to be developed in augmented reality on HoloLens will be, first and foremost, to create a simplified virtual scene of a workstation. It should include virtual elements that can be manipulated by a user. Different control functions will need to be developed, depending on the nature of the object and the action to be performed. The scene should allow the user to follow an execution scenario. This scene should also offer two different workstations: a non-ergonomic workstation and an ergonomic workstation. These workstations will have to be identical to a real workstation that will be set up on site in Chalon-sur-Saône. The scene will then have to be superimposed on the real workstation. Finally, it will have to be as faithful as possible to the different positions to be adopted when handling the objects. The user will therefore have to make movements and gestures identical to those in the real world.
The next stage will be to carry out ergonomic analyses using the ErgoWIN software developed by the Lyon office and the Kinect camera at the Chalon-sur-Saône workstation. Based on the results and data received, comparative analyses will have to be carried out between the real scenes and the augmented reality scenes. The long-term aim is to understand and define the adjustments that need to be made to this type of augmented reality application to ensure that it is as close to reality as possible. The various technological obstacles, such as the capabilities of the Microsoft HoloLens glasses and the loss of signal from the Kinect camera, will need to be taken into account.
The final stage will involve improving the scene, and in particular optimising the various interfaces so that it is understandable and accessible to the average user.

III - Results :

1. Final rendering of the virtual scene projected in the HoloLens glasses :

2. Overview of the real workstation :

3. Combining the two technologies, Kinect and HoloLens, and ergonomic experiments:

4. Examples of data produced following ergonomic ratings carried out on the AR simulation :

Unity3D, CAO, Blender

I - CAD:

Les différents éléments de la scène ont été reproduits en CAO et ajoutés à la scène virtuelle

II - Controls and user interfaces :

1. User help interface:
A user help interface has been incorporated into the scene. It takes the form of animated gifs and instructions for gripping objects in the scene.

ergowin_05

2. Grabbing complex objects:
The basic grabbing functions of the HoloLens glasses have been modified and adapted so that it is now possible to grasp complex objects.

ergowin_04

3. Position silhouettes:
To help users position their hands in the right place in the scene, and in particular as far as possible in front of the goggle camera (to ensure optimum detection), hand silhouettes have been implemented on the workbench.

ergowin_10

4. Cursor management:
To help users position their hands in the right place in the scene, and in particular as far as possible in front of the goggle camera (to ensure optimum detection), hand silhouettes have been implemented on the workbench.

III - AR Framework and calibration :

1. Vuforia and Calibration spheres :

vuforia
Vuforia is a plugin compatible with Unity 3D that allows you to track the position of a specific image in a real environment. To retrieve the plane vector parallel to the wall and the ground and calculate the coordinates of the normal to the wall, 2 spheres need to be placed on the wall. Position indicators about 20 cm apart should be placed on the wall to show the user where each sphere should be positioned (approximately). The function that manages the calibration of the scene automatically corrects the level if the spheres are not at exactly the same height.