Edges are one of the most basic parametric primitives to describe structural information in 3D. In this paper, we study parametric 3D edge reconstruction from calibrated multi-view images. Previous methods usually reconstruct a 3D edge point set from multi-view 2D edge images, and then fit 3D edges to the point set. However, noise in the point set may cause gaps among fitted edges, and the recovered edges may not align with input multi-view images since the edge fitting depends only on the reconstructed 3D point set.

To mitigate these problems, we propose SketchSplat, a method to reconstruct accurate, complete, and compact 3D edges via differentiable multi-view sketch splatting. We represent 3D edges as sketches, which are parametric lines and curves defined by attributes including control points, scales, and opacity. During reconstruction, we iteratively sample Gaussian points from a set of sketches and rasterize the Gaussians onto 2D edge images. Then the gradient of the image loss can be back-propagated to optimize the sketch attributes. Our method bridges 2D edge images and 3D edges in a differentiable manner, which ensures that 3D edges align well with 2D images and leads to accurate and complete results. We also propose a series of adaptive topological operations to reduce redundant edges and apply them along with the sketch optimization, yielding a more compact reconstruction. Finally, we contribute an accurate 2D edge detector that improves the performance of both ours and existing methods.

Experiments show that our method achieves a new state-of-the-art accuracy, completeness, and compactness on ABC-NEF dataset. SketchSplat is the first method to achieve both accuracy (A) and completeness (C) below 7.0mm, while also exceeding 90% in recall (5mm), precision (5mm), and F-score (5mm).

Overview of SketchSplat pipeline: (a) We extract edge images using a novel approach. (b) We then obtain initial edge points using EdgeGS and (c) fit parametric sketches defined as lines and third-order Bezier curves. The core step of our approach is to optimize the sketches by (d) sampling and rasterizing them in a differentiable manner and back-propagating gradients of the image loss to (e) sketch parameters. In addition, during training we apply (f) a set of topological operations to improve compactness and connectivity of the reconstructed sketches. (g) After training, we take the optimized geometric parameters of sketches as the reconstructed 3D edges.

Visualization results on DTU. Our method achieves overall comparable visual quality with EdgeGS and shows smoother results. We also use the least number of edges.

Visualization results on Replica. Our method achieves overall comparable visual quality with EdgeGS and costs the least number of edges, which validates the compactness of our method.

Quantitative evaluation on ABC-NEF dataset. Our method achieves the state-of-the-art performance on most metrics. A: Accuracy (mm), C: Completeness (mm), Rx: Recall (threshold x mm), Px: Precision (mm), Fx: F-score (mm). SketchSplat is the first method to achieve both accuracy (A) and completeness (C) below 7.0mm, while also exceeding 90% in recall (5mm), precision (5mm), and F-score (5mm).

Ablation on 2D edge detector. We ablate the depth prediction methods (2DGS, DepthPro). 2DGS shows better performance since it produces multi-view consistent depth maps. The core insight of our new 2D edge detector (2DGS-SN) is that geometric cues (e.g., depth and normal) offer higher pixel-level accuracy and are thus more robust than pure learning-based edge detectors (e.g. DexiNed, PiDiNeT). Based on this observation, we demonstrate that other geometry estimation methods such as DepthPro can be alternative components of our 2DGS-SN. We replace 2DGS depth maps with the ones from DepthPro and test its performance. DepthPro is significantly faster than 2DGS, as it avoids per-scene optimization. However, its accuracy is lower due to unstable edge detection caused by multi-view inconsistent depth scales. In contrast, 2DGS is slower but provides depth maps with a multi-view consistent scale, therefore a single threshold is sufficient to detect consistent edges. Choosing 2DGS (higher accuracy) vs. DepthPro (faster) is therefore a trade-off between efficiency and quality.