Table of Contents

  1. Contribution
  2. Architecture
  3. CAD Visualizer
  4. Visual Results
  5. Video
  6. Acknowledgement
  7. Citation

CAD-SIGNet: CAD Language Inference from Point Clouds using
Layer-wise Sketch Instance Guided Attention

Mohammad Sadil Khan1· Elona Dupont1 · Sk Aziz Ali1,2 · Kseniya Cherenkova1,3
Anis Kacem1 · Djamila Aouada1

1SnT, University of Luxembourg · 2German Center of Artificial Intelligence (DFKI AV Group) · 3Artec3D

CVPR 2024 (Highlight)

Paper
Results

Figure: Full design history recovery from an input point cloud (top-left) and CAD-SIGNet - user interaction (bottom-left and right)

Contribution

We propose CAD-SIGNet, an end-to-end trainable and auto-regressive architecture to recover the design history of a CAD model represented as a sequence of sketch-and-extrusion from an input point cloud. Our model learns visual-language representations by layer-wise cross-attention between point cloud and CAD language embedding. In particular, our main contributions are

  1. An end-to-end trainable auto-regressive network that infers CAD language given an input point cloud.
  2. Multi-modal transformer blocks with a mechanism of layer-wise cross-attention between point cloud and CAD language embedding.
  3. A Sketch instance Guided Attention (SGA) module which guides the layer-wise cross-attention mechanism to attend on relevant regions of the point cloud for predicting sketch parameters.

Architecture

Architecture

Figure: Method Overview. CAD-SIGNet (left) is composed of \(\mathbf B\) Multi-Modal Transformer blocks, each consisting of an \(\operatorname{LFA} \) module to extract point features, \(\mathbf F_{b}^v\), and an \(\operatorname{MSA} \) module for token features, \(\mathbf F_{b}^c\). An SGA module (top right) combines \(\mathbf F_{b}^v\) and \(\mathbf F_{b}^c\) for CAD visual-language learning. A sketch instance (bottom right), \(\mathbf I\), obtained from the predicted extrusion tokens is used to apply a mask, \(\mathbf M_{\text{sga}}\) during the cross-attention in SGA module (top-right) to predict sketch tokens.

CAD Visualizer

Currently in build!

Visual Results

We evaluated CAD-SIGNet on two reverse engineering scenarios -

  1. Design History Recovery
  2. Conditional Auto-Completion from User Input.

For scenario (1) DeepCAD is used as baseline. For Scenario (2), SkexGen and HNC have been used. Click on the button below for visual results.

Task Description: Given an input point cloud, the task is to infer the CAD design sequence.
Note: All the models are trained on DeepCAD dataset.
Design History for DeepCAD
Design History for CC3D
Design History for Fusion360
Task Description: This task is considered from a reverse engineering perspective and consists of recovering the ground-truth CAD construction history given a complete point cloud and a partial CAD sequence. All the models are trained on DeepCAD and tested on the same dataset.
Design History

Video

Coming Soon!

Acknowledgement

The present project is supported by the National Research Fund, Luxembourg under the BRIDGES2021/IS/16849599/FREE-3D, IF/17052459/CASCADES and Artec3D.

Citation

If you like our work, please cite.

@misc{khan2024cadsignet,
title={CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention},
author={Mohammad Sadil Khan and Elona Dupont and Sk Aziz Ali and Kseniya Cherenkova and Anis Kacem and Djamila Aouada},
year={2024},
eprint={2402.17678},
archivePrefix={arXiv},
primaryClass={cs.CV}
}