Unification of Closed-Open Industrial Detection Scenarios

Large-scale Visual-Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large-scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text-visual interaction for fine-grained understanding. To address these challenges, we introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M) containing over one million samples across 14 super-categories, 29 industrial scenes, and 351 defect subcategories. To our knowledge, MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection, providing valuable pre-training data for LVLMs in industrial scenarios. Furthermore, we propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert-assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy-based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text-visual interaction module that enhances cross-modal semantic alignment and understanding. Extensive experiments demonstrate that RTVPNet achieves state-of-the-art performance on MMIOC-1M, LVIS, and COCO benchmarks while maintaining computational efficiency. The dataset and code are available at https://github.com/hellozzk/MMIO.

Descrição

Zhang, Z., Zhang, J., Chen, Q., Li, G., Chen, D., Jing, S., Wang, H., Li, D., Liu, C., Bai, C., & Chen, S. (2026). Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks, Challenges and Baselines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(48), 9571-9588. https://doi.org/10.1109/TPAMI.2026.3680569

Palavras-chave

Industrial Open Detection Large Scale Industrial Benchmark Visual Language Model Software Computer Vision and Pattern Recognition Computational Theory and Mathematics Applied Mathematics Artificial Intelligence SDG 9 - Industry, Innovation, and Infrastructure

URI

http://hdl.handle.net/10362/204450

DOI

10.1109/TPAMI.2026.3680569

Coleções

NIMS: MagIC - Artigos em revista internacional com arbitragem científica (Peer-Review articles in international journals)

Métricas Alternativas

Ver registo completo