Towards Real-time Object Detection for Safety Analysis in an ML-Enabled System Simulation
RTOD for Safety Analysis in an ML-Enabled System Simulation
Keywords:
collaborative robot, object detection, simulation, machine learning, risk analysisAbstract
Machine learning (ML)-equipped critical systems such as collaborative artificial intelligence systems (CAISs), where humans and intelligent robots work together in a shared space are increasingly being studied and implemented in different domains. The complexities of these systems raise major concerns for safety risks because decisions for controlling the dynamics of the robot during the interaction with humans must be done quickly driving the detection of potential risks in form of collision between a robot and a human operator using information obtained from sensors such as camera or LIDAR. In this work, we explore and compare the performance of two You Only Look Once (YOLO) models - YOLOv3 and YOLOv8 - which rely on convolutional neural networks (CNNs) for real-time object detection in a case study collaborative robot system simulation example. The preliminary results show that both models achieve high accuracy (≥ 98%) and real-time performance albeit requiring a GPU to run at such speed as 40FPS. The results indicate the feasibility of real-time object detection in a CAIS simulation implemented with CoppeliaSim software.
References
L. E. Lwakatare, A. Raj, J. Bosch, H. H. Olsson, and I. Crnkovic, “A taxonomy of software engineering challenges for machine learning systems: An empirical investigation,” in Agile Processes in Software Engineering and Extreme Programming (P. Kruchten, S. Fraser, and F. Coallier, eds.), (Cham), pp. 227–243, Springer International Publishing, 2019.
Jellyfish, “AI Engineer vs. Software Engineer,” 2023.
M. Camilli, M. Felderer, A. Giusti, D. T. Matt, A. Perini, B. Russo, and A. Susi, “Risk-Driven Compliance Assurance for Collaborative AI Systems: A Vision Paper,” pp. 123–130, 2021.
J. G. Adigun, M. Camilli, M. Felderer, A. Giusti, D. T. Matt, A. Perini, B. Russo, and A. Susi, “Collaborative artificial intelligence needs stronger assurances driven by risks,” Computer, vol. 55, no. 3, pp. 52– 63, 2022.
Universal Robots, “New Technical Specification on Collaborative Robot Design,” 2018.
E. Commission, D.-G. for Research, and Innovation, Industry 5.0: human-centric, sustainable and resilient. Publications Office, 2021.
C. Birchler, S. Khatiri, B. Bosshard, A. Gambi, and S. Panichella, “Machine learning-based test selection for simulation-based testing of self-driving cars software,” Empirical Software Engineering, vol. 28, p. 71, Apr. 2023.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 7, pp. 138–148, 2015.
M. A. Santana, R. Calinescu, and C. Paterson, “Risk-aware realtime object detection,” in 2022 18th European Dependable Computing Conference (EDCC), pp. 105–108, 2022.
W. McCulloch and W. Pitts, “A logical calculus of ideas immanent in nervous activity,” Bulletin of Mathematical Biophysics, vol. 5, pp. 127– 147, 1943.
P. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University, Cambridge, MA, USA, 1974.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533– 536, 1986.
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541–551, 1989.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2012.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 211–252, 2015.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” European Conference on Computer Vision, vol. 9905, pp. 21–37, 2016.
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for´ dense object detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, 2018.
J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” 2018.
G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” Jan 2023.
Coppelia Robotics, “CoppeliaSim.”
R. Ierusalimschy, “Lua,” 1993.
Tzutalin, “LabelImg.” Git code, 2015.
Moses, “Imageai, an open source python library built to empower developers to build applications and systems with self-contained computer vision capabilities,” mar 2018–.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An Imperative Style, HighPerformance Deep Learning Library,” in Advances in Neural Information Processing Systems 32 (H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alche Buc, E. Fox, and R. Garnett, eds.), pp. 8024–8035, Curran´ Associates, Inc., 2019.
S. Bozinovski and A. Fulgosi, “The influence of pattern similarity and transfer learning upon the training of a base perceptron B2.,” Proceedings of Symposium Informatica, vol. 3-121-5, 1976.
F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A Comprehensive Survey on Transfer Learning,” 2020.
O. Ben-Kiki, C. Evans, and B. Ingerson, “YAML ain’t markup language (YAML) (tm) version 1.2,” 2009.
M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information Processing Management, vol. 45, no. 4, pp. 427–437, 2009.
R. Padilla, S. L. Netto, and E. A. B. da Silva, “A survey on performance metrics for object-detection algorithms,” in 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 237–242, 2020.
D. Shah, “Mean Average Precision (mAP) Explained: Everything You Need to Know,” 2022.
T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollar, “Microsoft coco:´ Common objects in context,” 2014.
K. Liu, H. Tang, S. He, Q. Yu, Y. Xiong, and N. Wang, “Performance validation of yolo variants for object detection,” in Proceedings of the 2021 International Conference on Bioinformatics and Intelligent Computing, BIC 2021, (New York, NY, USA), p. 239–243, Association for Computing Machinery, 2021.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Jubril Gbolahan Adigun, Patrick Aschenbrenner, Michael Felderer
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
License Terms:
Except where otherwise noted, content on this website is lincesed under a Creative Commons Attribution Non-Commercial License (CC BY NC)
Use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes, is permitted.
Copyright to any article published by WiPiEC retained by the author(s). Authors grant WiPiEC Journal a license to publish the article and identify itself as the original publisher. Authors also grant any third party the right to use the article freely as long as it is not used for commercial purposes and its original authors, citation details, and publisher are identified, in accordance with CC BY NC license. Fore more information on license terms, click here.