Project - Airbus CIAO - EN | Sébastien PASCAULT

The research project proposed here is the CIAO project, an R&T project developed in-house at Airbus Opération SAS. The aim is to set up a collaborative platform for autonomous movement using two laser scanners (radar sick) to locate and avoid positive or negative obstacles. The platform is equipped with a UR10 arm on which a pneumo-electric screwdriving head and a camera-based vision system are implemented, capable of detecting and repositioning the arm to ensure that the screw tightening operation is carried out correctly.

The aim of this project is to contribute to the automation of data acquisition and enrichment for simulation, and to simulate robot trajectories in 3D as a function of the operations to be carried out and the constraints linked to its environment. This project also aims to simulate in virtual reality the perception of sensors and the behaviour of controllers, so as to be able to define reactions as expected, according to different types of hazard. The various prediction methods (machine-learning and deep-learning) will enable these behaviours to be automated, particularly for unidentified behaviours.

Sébastien PASCAULT

Engineer: Computer scientist - Automatician - Computer graphics designer

Specialities: Human Machine Interfaces - Mixed Reality - Serious Gaming - Unity 3D

Mobile: 06 31 10 01 56
E-mail: sebastien.pascault@outlook.com
Team: https://teams.live.com
Skype: live:sebastien.pascault

About me:

Passionate about digital technology and simulated environments, I'm constantly looking for innovations and challenges. My passion for IT and computer graphics has motivated me throughout my studies and still motivates me today to continue my career in software development.

Resume:

I - INTRODUCTION:

The first robots that were useful to humans appeared with the emergence of assembly lines, where robots could perform repetitive, arduous and dangerous tasks for operators. In factories, these robots are usually positioned in dedicated areas where operators cannot go when they are in operation. This rule helps to ensure the safety of workers, but also makes the organisation of an assembly line rather inflexible. There are also tasks that require hybrid skills between those of a robot (repetitiveness, strength, precision) and those of an operator (intelligence, adaptability). The use of collaborative robots is therefore a solution that is increasingly being studied by manufacturers. These robots operate in a shared space with the operator, and can work on the same task, in collaboration, or on two separate but co-present tasks (Figure 1). They illustrate the factory of the future, which will be more efficient, more effective and more productive. This close cooperation raises issues relating to human-robot interaction and communication, as well as the risks to humans.

airbusciao_16 airbus_ciao17

Figure 1. Coprésence homme/robot et reproduction des tâches de serrage

II - PROJECT DESCRIPTION:

1. Contexte:

The CIAO (Collaborative Instrumentation for Assembly Optimization) project is part of a drive to optimise industrial processes. Its objective is to set up a collaborative robot on the A350 programme for a floor assembly station, so that it can carry out screw tightening operations on the entire central section.

airbusciao_00
Figure 2. Overview of the centre section of an AIRBUS A350 aircraft.

To achieve this, it is necessary to simulate, using real-time 3D simulation, various operations such as floor assembly, clamping, platform movement and robot arm trajectories, and therefore optimisation. All in a virtual environment where humans and robots coexist.
The aim of this project is to provide a response to the issues of quality, traceability of operations, time saving, ergonomics and MSD prevention, and to reduce the arduousness of the work.

2. Cobot Airbus:

The PMA01 platform is a holonomic collaborative robot, because it can adopt any position in the same plane space (x y theta). It is capable of both translational movement and rotation on itself. It is also equipped with an UR10 robotic arm (developed by Universal Robots), which is fitted with a pneumo-electric screwdriving head and a camera vision system capable of detecting and repositioning the arm to ensure that the screw tightening operation is carried out correctly.

airbus_ciao_cobot01

Figure 3. PMA01 platform designed by AIRBUS

3. Objectives:

The objectives of the CIAO project are as follows:

1. To contribute to the automation of data acquisition and enrichment for simulation.

2. To simulate robot trajectories in 3D as a function of the operations to be carried out and the constraints linked to its environment (Off-Line programming). These trajectories could be optimised using techniques such as "Path Traveling Salesman Problem - TSP" or "Monte Carlo Localization for Mobile Robots".

3. Virtual reality simulation of the robot's trajectories in order to adapt them locally, when it is necessary to take into account a nearby manual operation.

4. Virtual reality simulation of the perception of sensors and the behaviour of controllers, so as to be able to define reactions as expected, according to different types of hazard. The various prediction methods (machine-learning and deep-learning) would enable these behaviours to be automated, particularly for unidentified behaviours.

5. Generate the robot program resulting from the simulation, which will have to be interpreted by the middleware in order to transform the mission into a machine instruction (PIK to ROS).

6. Provide specifications for the acquisition, enrichment, simulation and execution functions.

III - PROTOTYPE:

1. Strategies and functionalities:

The strategy used to set up an initial prototype is as follows:

The application first searches for the barycentres of each floor and instantiates a target position at the position found. Then, depending on the size of the floor, it instantiates other targets at a distance corresponding to the maximum coverage width of the robot's end effector. The aim is to cover the entire panel.
When a target is selected (the one closest to the robot in this first version), the mobile platform moves until the centre of the work area is in line with the centre of one of the targets. When the work zone comes into contact with a screw hole, the program registers the accessibility of the hole and tells the robot that it can carry out its tightening operation on the detected hole. This operation is repeated for each of the holes (green dots fig.43 bis) located in the zone. This identifies all the holes covered by the robot in relation to its current position. It can then start its clamping sequence calculated according to the number of holes detected.

airbusciao_08

Figure 4. Overview of the different functions of the Unity prototype.

Once the clamping sequences for a given panel have been completed, the panel is validated and turns green. A colour system has been introduced to identify the current status of each floor and whether it is in process, validated or faulty.

Figure 5: Identification of the status of each floor using a colour code

2. Final interface for the Unity application:

Figure 6. All the prototype functions accessible from the interface (Monitoring).

CAO, Unity3D, C#, CATIA, Autodesk 3ds Max, Autodesk Maya

I - LINE PROCESS ANALYSIS

The first phase of the project was devoted to in-depth research into the various techniques and skills involved in the design and manufacture of an aircraft. This research was limited to the various activities centred around floor laying, in connection with the CIAO project. The information gathered involved regular visits to the assembly lines to gather as much information as possible from the operators about their knowledge of the assembly processes. All this data was then sorted and analysed to obtain the data required to start developing the simulation and set up the first experiments.

II - BASE DE DONNEE:

DeASI (Design Assistant for Systems Integration introduction) is a webservice, developed within Airbus, enabling users to retrieve from a database the various torque tolerances for tightening screws. These tolerances are defined as a function of the type of screw/nut combination and are also a function of the method used: screwing in by the head or screwing in by the nut. In the DeASI interface, these values are listed and displayed in various fields via a browser-accessible interface.

Figure 7. DeASI webservice operating principle

It was therefore necessary to develop midleware to retrieve this data from Unity. The final interface makes it possible to send a screw/nut combination to the server by indicating their reference in a field, then to check the data returned by the server in a list of results. Once validated, this data is then injected into the simulation to pre-configure the scene.

Figure 8. Overview of the middleware developed in Unity to recover DeASI data

airbusciao_01 airbusciao_02

Figure 9. Overview of references and 3D data of interest extracted from the AIRBUS database.

III - SETTING UP PHYSICS AND EVENTS:

On the platform, the first physical behaviours have been put in place, particularly on the wheels, to simulate the reaction of the platform in the event of a collision with an obstacle.

The FinalIK asset was used to animate the reverse kinematics of the U10 arm, enabling different rotation limits to be selected for the various joints.

IV - CONVERSION TOOLS

The following is a brief description of the methods used to convert CATIA digital mock-ups downloaded from the AIRBUS database in order to make them compatible and light enough for viable use in simulation.

1. Pixyz :

PiXYZ STUDIO is a unique 3D data preparation tool that provides the best-in-class tesselation tool (cutting the mesh into simplified polygons) for transforming CAD data from market-leading solutions (Catia, NX, SolidWorks, etc.) into optimised, lightweight meshes.
PiXYZ STUDIO's interactive graphical user interface provides access to a wide range of functions and algorithms. It also incorporates a Python API, enabling advanced users to create powerful scripts to automate complex data preparation tasks (import/export functions for batch processing not included).
PiXYZ is a solution implemented directly in Unity, making it an advantage over other conversion tools.
Important: please note that the PiXYZ tool does not produce a model in FBX format, so it is impossible to process the converted model in any other software.

airbusciao_13
Figure 10. Aperçu de l’asset PiXYZ dans Unity

2. CATIA + 3DS Max/Maya

Another approach to converting a CATIA CAD model is to combine the "Tesselated" option available when exporting directly from CATIA with the use of professional CAD software such as Autodesk 3DS Max or Maya. The model. CATProduct model is first converted to step or .cgr, then post-processed and simplified in 3DSMax or Maya, and finally saved in .fbx format for Unity (Figure 12).

Figure 11. Simplified CATIA model, extracted from the complete model of the A350 aircraft section

Figure 12. Overview of the model converted in Autodesk 3DSMAX

Figure 13. Overview of the complete model in Autodesk Maya, after data enhancement and conversion to Step from CATIA, then fbx from 3DS Max.

III -SEQUENCING AND CYCLES

For the sequencing of operations and the implementation of the various scripts, a solution was envisaged for programming with automatic-oriented logic. This solution is called "Behaviour Tree". It makes it possible to program with a logic similar to Grafcet. Rather than using a simple switch to move between the different states of a cycle or sequence, this library can be used to implement state selectors and conditioned sequencers, which eliminate the need for conventional IF conditions.