I - CONTEXT:

1. Virtual Reality disassembly concepts

The automated generation of CAD assembly semantic is a subject highly addressed in the last decades. Techniques used in the literature are different, especially according to the format of the analysed 3D objects, which are represented by their surfaces (B-Rep) or by their volumes (CSG) [4]. In general, three main categories of methods are distinguished, namely: geometry based methods, graph based methods and feature based 
methods [5]. the proposed approach will focus on this latter.

2. Feature based approaches

A review of the literature shows that, although it is possible to recognise certain manufacturing features simply by considering the local geometric properties of the components, the detection of more complex functional properties requires the geometric model to be considered from a wider angle, which also covers the interaction between the different components.

To generate assembly joint from 3D models, assembly features representation are also necessary. Assembly features describe the relation of components in an assembly  group. A feature is defined here as “a partial form or a product characteristic that is  considered as a unit and that has an engineering semantic meaning for assembly design,  process planning or manufacture "[6]. To extract or recognize semantic features in CAD models many approaches exist. Hasan and Wikander [7] differentiate between three approaches: internal approaches, external approaches and ontology-based approaches. In internal methods, the Application Programming Interface (API) of the CAD software is used in order to extract topological, geometrical and assembly information related to a part or an assembly. While in external methods, a CAD model file is exported in a neutral data format (e.g. STEP-AP 214). C.M. Costat [8] focuses on internal approach to generate assembly sequence planning. A. Neb [2] combined external approaches with internal approaches to get the features directly from the 3D assembly model. Other authors use ontology base approaches to exploit geometric and shape information but those methods will not be covered here.

Figure 6. Diagram of formfeature classes

3. Keypoints : main specification for physics engines

To provide disassembly process in a virtual world, physics engines are used. Those engines provide an approximate simulation of certain physical systems such as rigid body dynamics, soft body dynamics or fluid dynamics. There are many available physics engine on the market such as Bullet, PhysX, ODE or XDE. Knowing our study focuses on VR we decided to use both PhysX and XDE which are direct kinematics engine well suitable for it. XDE Physics on the first hand is an interactive physics engine featuring precise collision detection, multibody and beam dynamics, at rate compatible with haptic rendering [9]. PhysX on the other hand is a scalable multi-platform physics engine providing the same features and already integrated into some of the most popular game engines, including Unreal Engine and Unity3D. Both XDE and PhysX offer a simple programming interface where assembly features can be used and easily implemented. To compute dynamics, most of the previously named physics engine have the ability to rely on specific “point features” in the assembly model allowing them to reason on kinematic links rather than collisions. Those “point features” are referred to as “key points” [10]. A KeyPoint (KP) can be defined as a reference point in the space associated with a part of an assembly that characterize a kinematic link or a mate [Fig 2]. KPs are specified by recognition algorithms according to certain mathematical base or specific rules.

II - FEATURE RECOGNITION AND EXTRACTION:

1. Feature Extraction from CAD models

To solve data exchange between CAD design and other software two extraction framework were developed, following the approach of internal B-rep CAD analysis. A first framework has been developed to extract the features from the B-Rep topology. Open CASCADE technology (OCCT) was used to extract these features from a step file (AP 242). The resulting dataset describes the different B-Rep shells of the model and all the associated data namely: transformation matrix, nature of the surfaces (plane, cone, cylinder, b-spline, etc.) and basic geometry characteristics (normal, radius, axis, orientation, bounding-box, center of mass and inertia). In order to describe VR behaviour of a product and identify the symbolic relationships between parts a second framework was created to extract the different primitive mates from a SW CAD assembly (all assembled). Those mates were extracted trough a SW API. The resulting mates dataset combined with the previous B-Rep dataset was finally imported in Unity3D to be exploited with a physics engine.

III - SEMANTIC RULES ENGINE:

1. Definition:

Based on input from CAD dataset recognition stage above an assembly design Semantic Rules Engine (SRE) is proposed. This SRE describes a novel approach based on features recognition and detection rules to model assembly semantic meaning. The proposed SRE, is a decomposition into three semantic levels (or layers) inspired by Hasan work [13], namely: Geometric Level, Kinematic Level and Behaviour Level.

• On the Geometric Level regroups two type of features : Shell Feature and Form Features. Shell Feature are related to B-Rep entities extracted from the CAD modeller in order to detect specific surface shape for each assembly part. These shell features are then analysed according to simple heuristic rules based on the analysis of conventional naming or conventional geometric configuration (nearest neighbours, shell dimension or symmetry axis). This set of rules results in a form classification thus called Form Features (e.g., hole form, cylinder form, thread form, etc.).

• On the Kinematic Level, mating features are further analysed in order to determine joining features. Each imported mates is linked to two entities that corresponds to part in the assembly. The first entities represents the origin part where the mates is attached and the second entities represents another targeted part giving a first assembly
  knowledge for potential Joining Feature between them.

• Lastly, Behaviour Level regroups all the behaviour rules aimed to detect applicationspecific features whose semantics are carried by KeyPoints. Those KPs are classified into three different types. Handling KP firstly represents position on parts’ surface where object can be grabbed by user hands or tools. Interest KP then defines specific positions inside parts where a kinematic joint exists. Lastly, Transition KP describes specific coordinates in parts where mates state and related degrees of freedom (DOF) can change according the part position in assembly at a given time.

Figure 3. Multi-level decomposition of generated semantic data

The objectives of this SRE is to reproduce with the machine the recognition mechanism of a human engineer. Our approach differs from other methods in the scientific literature by its effectiveness to be applicable in real time. Its interest for the application in VR is that it makes possible to set up aids for the immersed user such as position ghosts, movement indicator or even part snapping close to their final position.

2. Decision tree :

Figure 3. Rules priority diagram

3. Detection of possible matches :

Once the assembly has been enriched by the imported constraints and the newly generated interfaces from the B-REP data, new rules are defined to exploit this information and identify possible kinematic pairs. These potential links are then visualised in the Unity prototype using a colour code. Red to identify potential contact and green to identify potential interfaces.

During this stage, performance tests were carried out by comparing the detection of interfaces using semantic rules and by comparing this method with a simple detection of distance between surfaces using the PhysX engine (Figure ). The execution of scripts based on the SRE took on average half the time of scripts based on the PhysX engine.

Hypothesis: 
Analysing possible matches in post-processing should be more efficient than doing it dynamically and, above all, less greedy in terms of resources in order to avoid possible slowdowns during immersion in VR.
 

IV - RULES FOR GENERATING KINEMATIC LINKS AND POINTS:

V - USE CASE:

To experiment the SRE, the use-case of a complex Honda engine CFR250X modeled in SW was chosen. The objective was to redo the complete disassembly process of a 3D model to compare our automatic approach with manual VR preparation methods. To identify the necessary semantic links inside the engine model, a unique disassembly sequence scenario has been defined following the Honda’s official manual [Fig 1].

Based on this scenario, a "control sample" was prepared in a "standard" manual way where each kinematic link and each KP have been configured manually through the physic engine interface in Unity3D. We then tried to achieve a similar preparation process using the automatically generated feature and KPs from our SRE.

To describe in a more details the behaviour of our SRE on this specific use case, we will focus on the screws and their disassembly sub-process. The SRE starts by analysing each B-Rep data previously extracted and loads the Shell Features. Then the potential class (belong percentage) of the current part is deducted according to its geometry (nature of shell, positions and bounding boxes) as well as its specific naming rules in this case the term “bolt” used multiple time for each screw of the model. While the object class is identified, the SRE uses the mates data to deduce the type of joint between the screw and the part in contact with this latter. Then the SRE generate the KP according to the nature of the joints, the position of the mates in the part and lastly the nature of the surrounding shells. For every screw three KP are generated. A first Transition KP is instantiated at the extremity of the thread, which will determine the screw DOF according to its position. Then an Interest KP is generated at the position of the coincident mates and on the concentric mate’s axis to describe the helical joint. Lastly, a Handling KP is generated to specify where the ratchet wrench tool should be position on the screw to be disassembled [Fig 2].

VI - RESULTS:

The process time to prepare a VR scene in a manual way was estimated at five days, approximately 40 hours of preparation time (human time). The results obtained using our SRE were in accordance with the control sample and the positions and natures of about 60% of the generated KPs appeared to be exact (71 well generated KPs on 117 for an assembly containing 45 parts : 81% Interest KPs, 75% Handling KPs and 33% Transition KPs). Using the SRE allowed us to prepare the scene in about 8 hours and reduced the engineering time by 80%.

III - REFERENCES:

1. Hasan, B., Wikander, J.: Features Extraction from CAD as a Basis for Assembly Process Planning, In: DoCEIS, pp 144-153, (2017)

2. Neb, A.: From 3D product data to hybrid assembly workplace generation using the AutomationML exchange file format, (2019). https://doi.org/10.1016/j.procir.2019.03.011.

3. Saniuk, S.: Personalization of Products in the Industry 4.0 Concept and Its Impact on Achieving a Higher Level of Sustainable Consumption, In: Energies, vol. 13(22), pp. 5895 (2020). https://doi.org/10.3390/en13225895.

4. Piegl, L., Tiller, W.: The NURBS Book, Springer Science & Business Media, (1997)

5. Tangelder, A.: Survey of content based 3D shape retrieval methods. Multimedia tools and applications, 39(3), pp. 441–471, (2008). http://doi.org/10.1007/s11042-007-0181-0.

6. Hasan, B.: Product Feature Modelling for Integrating Product Design and Assembly Process Planning, In: IJENS, vol.4 (10), (2017).

7. Hasan, B., Wikander, J.: A review on Utilizing Ontological Approaches in Integrating Assembly Design and Assembly Process Planning (APP), In: IJME vol.4 (11), (2017).

8. Costa, C.M.C.: Automatic Generation of Disassembly Sequences and Exploded Views from SolidWorks Symbolic Geometric Relationships, In: ICARSC(18th) (2018).

9. Merlhiot, X.: XDE Physics: https://www.researchgate.net/project/XDE-Physics-multibodydynamics- with-intermittent-contacts-and-flexible-bodies, last accessed 2021/07/24.

10. Spezialetti, R., Salti, S.: Performance Evaluation of 3D Descriptors Paired with Learned Keypoint Detectors. In: AI, vol.2, pp. 229–243, (2021). https://doi.org/ 10.3390/ai2020014.

11. Liu, Z., Tan, J.: Constrained behavior manipulation for interactive assembly in a virtual environment
In: Advanced Manufacturing Technology, vol.32, pp.797-810, (2007)

12. Bourdot, P.: VR–CAD integration: Multimodal immersive interaction and advanced haptic
paradigms for implicit edition of CAD models, (2008).