Robust & Explainable Seafloor AI

SeafloorAI, the first large AI-ready dataset for seafloor mapping using sonar imagery.

Abstract: Characterizing seafloor morphodynamics is essential for naval applications relying on extensive geoacoustic and environmental data over broad spatiotemporal scales. Understanding dynamics and uncertainty among numerous variables presents a significant out-of-distribution challenge, as AI/ML models often struggle with unseen distributions, leading to fragile predictions and unreliable explanations. To overcome these challenges, this project will develop new robust and explainable optimization methods using newly-curated seafloorAI datasets. The research outcomes—including an AI-ready multi-site seabed morphodynamic database, innovative trustworthy AI/ML optimization methods, and scalable implementations—will be linked to the Ocean Biogeographic Information System and Seabed 2030 to benefit a broad range of communities and stakeholders.

Publications:

• [ICML'25] Fengchun Qiao, Yanlin Chen and Xi Peng. Structure-informed Risk Minimization for Robust Ensemble Learning. In International Conference on Machine Learning, 2025. [PDF] [Code]
• [AAAI'25 Oral] Kien X. Nguyen, Tang Li, Xi Peng. Interpretable Failure Detection with Human-Level Concepts. In Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), 2025. [PDF] [Code]
• [NeurIPS'24] Kien X. Nguyen, Arthur Trembanis, Xi Peng. SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey. In Proceedings of Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2024. [PDF] [Dataset]
• [ICML'24] Fengchun Qiao and Xi Peng. Ensemble Pruning for Out-of-distribution Generalization. In International Conference on Machine Learning, 2024. [PDF] [Code]
• [ECCV'24 Strong Double Blind] Tang Li, Mengmeng Ma, Xi Peng. DEAL: Disentangle and Localize Concept-level Explanations for VLM. In European Conference on Computer Vision, 2024. [PDF] [Code]
• [CIKM'24] Kien X. Nguyen, Fengchun Qiao, Xi Peng. Adaptive Cascading Network for Continual Test-Time Adaptation. In Conference on Information and Knowledge Management, 2024. [PDF] 

Open-Sourced Data:

• SeafloorAI Dataset: It includes 696,000 sonar images, 827,000 annotated segmentation masks, 696,000 detailed language descriptions and approximately 7M question-answer pairs. We make this dataset publicly available:  [https://sites.google.com/udel.edu/seafloorai/homes]
• Prediction Rationale Dataset for ImageNet: We construct a new rationale dataset that covers all 1,000 categories in the ImageNet. For each category, we generate an ontology tree with a maximum height of two. Combining attributes and sub-attributes, this dataset contains over 4,000 unique rationales. [https://github.com/deep-real/DCP/tree/main/Rationale%20Dataset]

Open-Sourced Software:

• Ensemble Pruning for OoD generalization: A Toolkit for ensemble-based robust optimization against distribution shifts. Github Repo [https://github.com/deep-real/TEP]
• Ordinal Ranking of Concept Activation (ORCA): A lightweight, interpretable failure detection toolkit based on concept activation rankings. Github Repo [https://github.com/Nyquixt/ORCA]
• Distributionally Robust Explanations (DRE): A framework to enhance machine learning (ML) model robustness against out-of-distribution data. Source code and pretrained models at [https://github.com/deep-real/DRE]. 
• DisEntAngle and Localized (DEAL): An interpretation toolkit that disentangles and localizes fine-grained, concept-level explanations for visual models. Github Repo [https://github.com/deep-real/DEAL]