FusionBridge: Enhancing Multi-View Multi-Modal Sensing and Perception for Edge Intelligence

Heterogeneous sensors (e.g., 2D cameras and LiDAR) provide a novel opportunity to leverage multiple modalities in collaborative artificial intelligence (AI)-based video analytics pipelines. Such applications use sensors that are frequently attached to resource-limited edge devices which can hinder the execution of multimodal and deep DNN models. While powerful edge devices can still benefit from multimodal fusion to enhance robustness, joint training of such models for generalizable applications is often infeasible due to the lack of large-scale multimodal datasets and the prohibitive cost involved in annotating those datasets. To address this, we introduce FusionBridge: a lightweight fusion framework that combines the capabilities of independently trained 2D (image-based) and 3D (LiDAR-based) perception models to improve object detection at the edge. FusionBridge extracts mid-level features from single modality 3D models and performs cross-modal fusion via a lightweight transformer-based adapter. This enables hints to be exchanged without requiring joint end-to-end training. By bridging modality-specific experts, our approach maintains modularity, supports model reuse, and allows scalable deployment across heterogeneous sensor configurations with zero calibration or sensor alignment effort. Evaluations on simulated and real world deployments demonstrate that FusionBridge achieves up to a 57% F1-score improvement over any single-modality baseline, while only incurring a 15% latency overhead and 0.4KB/frame transmission overhead compared to the baseline.