CLASSIFICATION OF 3D POINT CLOUD MODELS OF FISH BASED ON POINT TRANSFORMER APPROACH
-
摘要:
为实现对不同鱼类的精准分类, 研究共采集110尾真实鱼类的三维模型, 对获取的3D模型进行基于预处理、旋转增强和下采样等操作后, 获取了1650尾实验样本。然后基于Point Transformer网络和2个三维分类的对比网络进行数据集的分类训练和验证。结果表明, 利用本实验的目标方法Point Transformer获得了比2个对比网络更好的分类结果, 整体的分类准确率能够达到91.9%。同时对所使用的三维分类网络进行有效性评估, 3个模型对于5种真实鱼类模型的分类是有意义的, 其中Point Transformer的模型ROC曲线准确率最高, AUC面积最大, 对于三维鱼类数据集的分类最为有效。研究提供了一种可以实现对鱼类三维模型进行精准分类的方法, 为以后的智能化渔业资源监测提供一种新的技术手段。
-
关键词:
- 点云处理 /
- Point Transformer /
- 三维模型 /
- 鱼类分类
Abstract:Phenotypic data serve as the foundation for effective monitoring of fish species. Currently, fish classification heavily relies on expertise from relevant professionals, leading to issues such as low efficiency, high errors, potential damage to fish bodies, and susceptibility to subjective factors affecting data quality. In this study, we developed a simplified device for acquiring three-dimensional models of fish, pioneering the creation of a dataset comprising authentic three-dimensional fish models. By leveraging the Point Transformer algorithm, we can rapidly, efficiently, and accurately extract phenotypic features from three-dimensional fish bodies, enabling precise classification of different fish species. A total of 110 authentic fish three-dimensional models were collected in this research, resulting in 1650 experimental samples after preprocessing, rotation enhancement, and downsampling operations on the acquired 3D models. Subsequently, through classification training and validation using the Point Transformer network and two comparative networks for three-dimensional classification, the results indicate that the proposed Point Transformer method outperforms the two comparative networks, achieving an overall classification accuracy of 91.9%. Simultaneously, an effective evaluation of the utilized three-dimensional classification networks was conducted, demonstrating the meaningful classification of the three models for five authentic fish species models. The Point Transformer model exhibited the highest ROC curve accuracy and the largest AUC area, proving its effectiveness in classifying three-dimensional fish datasets. This study presents a method for accurately classifying three-dimensional fish models, offering a new technological approach for intelligent monitoring of fisheries resources in the future.
-
-
![]()
图 1 鱼类三维点云数据采集装置
左. 现实点云数据采集装置; 右. 点云数据采集装置CAD平面图
Figure 1. Fish 3D point cloud data acquisition device
Left. realistic point cloud data acquisition device; Right. CAD plan of point cloud data acquisition device
![]()
图 3 鱼类三维数据清洗示例
A图代表鱼类数据清洗之前; B图代表清洗后的模型
Figure 3. Example of fish 3D data cleaning
A represents the fish data before cleaning; B represents the model after cleaning
![]()
图 4 边缘收缩: 突出显示的边缘收缩为单个点。阴影三角形将在收缩中并被移除
Figure 4. Edge contraction: the highlighted edge is contracted into a single point. The shaded triangles are removed during the contraction
![]()
图 6 基于Point Transformer的三维鱼类分类模型
Figure 6. A 3D fish classification model based on Point Transformer
![]()
图 7 利用ROC曲线和AUC (曲线下面积)评估分类模型性能
Figure 7. Assessing classification model performance using ROC curves and AUC (area under the curve)
![]()
图 8 利用loss曲线可视化不同分类模型的训练过程
Figure 8. Visualisation of the training process of different classification models using loss curves
表 1 在鱼类数据集上的实验结果
Table 1 Experimental results on fish dataset (%)
算法
Algorithm输入
Import模型点数
Model points模型个数
Number of models总体分类准确率
OA (%)平均准确率
m-Acc (%)F1分数
F1 scoreMeshnet 面片 2048 550 81.4 73.1 82.7 Pointnet 点坐标 2048 550 75 80 76.4 Point transformer 点坐标 2048 550 91.9 91.7 91.9 
下载: 导出CSV
表 2 在五种鱼类三维模型上的实验结果
Table 2 Classification results on a three-dimensional model of five fish species (%)
算法
Algorithm草鱼
Ctenopharyngodon idella达氏鲌
Culter dabryi蒙古鲌
Culter mongolicus翘嘴鲌
Culter alburnus鲫
Carassius auratusMeshnet 95 74.4 85.4 60 85 Pointnet 71.1 89.8 27.8 88.9 87.9 Point transformer 97.6 88.8 92.9 85.7 93.3 over-accuracy 87.9 84.3 68.7 78.2 88.7
下载: 导出CSV
-
[1] Yang X, Zhang S, Liu J, et al. Deep learning for smart fish farming: applications, opportunities and challenges [J]. Reviews in Aquaculture, 2021, 13(1): 66-90. doi: 10.1111/raq.12464
[2] Longépé N, Hajduch G, Ardianto R, et al. Completing fishing monitoring with spaceborne Vessel Detection System (VDS) and Automatic Identification System (AIS) to assess illegal fishing in Indonesia [J]. Marine Pollution Bulletin, 2018(131): 33-39. doi: 10.1016/j.marpolbul.2017.10.016
[3] Chang A X, Funkhouser T, Guibas L, et al. Shapenet: An information-rich 3d model repository [J]. arXiv preprint arXiv, 2015(3012): 1512.
[4] Villon S, Mouillot D, Chaumont M, et al. A deep learning method for accurate and fast identification of coral reef fishes in underwater images [J]. Ecological Informatics, 2018(48): 238-244. doi: 10.1016/j.ecoinf.2018.09.007
[5] Hu J, Li D, Duan Q, et al. Fish species classification by color, texture and multi-class support vector machine using computer vision [J]. Computers and Electronics in Agriculture, 2012(88): 133-140. doi: 10.1016/j.compag.2012.07.008
[6] Banan A, Nasiri A, Taheri-Garavand A. Deep learning-based appearance features extraction for automated carp species identification [J]. Aquacultural Engineering, 2020(89): 102053. doi: 10.1016/j.aquaeng.2020.102053
[7] Mathur M, Vasudev D, Sahoo S, et al. Crosspooled FishNet: transfer learning based fish species classification model [J]. Multimedia Tools and Applications, 2020(79): 31625-31643. doi: 10.1007/s11042-020-09371-x
[8] Ishaq O, Sadanandan S K, Wählby C. Deep fish: deep learning–based classification of zebrafish deformation for high-throughput screening [J]. SLAS Discovery: Advancing Life Sciences R & D, 2017, 22(1): 102-107.
[9] Salman A, Jalal A, Shafait F, et al. Fish species classification in unconstrained underwater environments based on deep learning [J]. Limnology and Oceanography: Methods, 2016, 14(9): 570-585. doi: 10.1002/lom3.10113
[10] Xu W, Zhu Z, Ge F, et al. Analysis of behavior trajectory based on deep learning in ammonia environment for fish [J]. Sensors, 2020, 20(16): 4425. doi: 10.3390/s20164425
[11] Li X, Shang M, Hao J, et al. Accelerating Fish Detection and Recognition by Sharing CNNs with Objectness Learning [C]. OCEANS 2016-Shanghai. IEEE, 2016: 1-5.
[12] Shah S Z H, Rauf H T, IkramUllah M, et al. Fish-Pak: Fish species dataset from Pakistan for visual features based classification [J]. Data in Brief, 2019(27): 104565. doi: 10.1016/j.dib.2019.104565
[13] 段延娥, 李道亮, 李振波, 等. 基于计算机视觉的水产动物视觉特征测量研究综述 [J]. 农业工程学报, 2015, 31(15): 1-11.] doi: 10.11975/j.issn.1002-6819.2015.15.001 Duan Y E, Li D L, Li Z B, et al. Review on visual characteristic measurement research of aquatic animals based on computer vision [J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(15): 1-11. [ doi: 10.11975/j.issn.1002-6819.2015.15.001
[14] Chuang M C, Hwang J N, Williams K. A feature learning and object recognition framework for underwater fish images [J]. IEEE Transactions on Image Processing, 2016, 25(4): 1862-1872.
[15] Ali-Gombe A, Elyan E, Jayne C. Fish Classification in Context of Noisy Images [C]. Engineering Applications of Neural Networks: 18th International Conference, EANN 2017, Athens, Greece, August 25–27, 2017, Proceedings. Springer International Publishing, 2017: 216-226.
[16] Feng Y, Feng Y, You H, et al. Meshnet: Mesh Neural Network for 3d Shape Representation [C]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 8279-8286.
[17] Liang Z, Guo Y, Feng Y, et al. Stereo matching using multi-level cost volume and multi-scale feature constancy [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(1): 300-315.
[18] Su H, Maji S, Kalogerakis E, et al. Multi-View Convolutional neural Networks for 3d Shape Recognition [C]. Proceedings of the IEEE International Conference on Computer Vision, 2015: 945-953.
[19] Maturana D, Scherer S. Voxnet: A 3d Convolutional Neural Network for Real-time Object Recognition [C]. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015: 922-928.
[20] Guo Y, Wang H, Hu Q, et al. Deep learning for 3d point clouds: A survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(12): 4338-4364.
[21] Wu Z, Song S, Khosla A, et al. 3d Shapenets: A Deep Representation for Volumetric Shapes [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1912-1920.
[22] Charles R Q, Hao S, Mo K, et al. Pointnet: Deep Learning on Point Sets for 3d Classification and Segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 652-660.
[23] Uy M A, Pham Q H, Hua B S, et al. Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-world Data [C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 1588-1597.
[24] Wang J, Li X, Pan L, et al. Parametric 3D modeling of young women’s lower bodies based on shape classification [J]. International Journal of Industrial Ergonomics, 2021(84): 103142. doi: 10.1016/j.ergon.2021.103142
[25] Zhou S, Xiao S. 3D face recognition: a survey [J]. Human-centric Computing and Information Sciences, 2018, 8(1): 1-27. doi: 10.1186/s13673-017-0124-3
[26] Presti L L, La Cascia M. 3D skeleton-based human action classification: A survey [J]. Pattern Recognition, 2016(53): 130-147. doi: 10.1016/j.patcog.2015.11.019
[27] Zuffi S, Kanazawa A, Jacobs D W, et al. 3D Menagerie: Modeling the 3D Shape and Pose of Animals [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 6365-6373.
[28] Yang G, Vo M, Neverova N, et al. Banmo: Building Animatable 3d Neural Models from Many Casual Videos [C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 2863-2873.
[29] Rüegg N, Zuffi S, Schindler K, et al. Barc: Learning to Regress 3d Dog Shape from Images by Exploiting Breed Information [C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 3876-3884.
[30] Mathis M W, Mathis A. Deep learning tools for the measurement of animal behavior in neuroscience [J]. Current Opinion in Neurobiology, 2020(60): 1-11. doi: 10.1016/j.conb.2019.10.008
[31] Li C, Ghorbani N, Broomé S, et al. hSMAL: Detailed horse shape and pose reconstruction for motion pattern recognition [EB/OL]. arXiv preprint arXiv: 2106.10102, 2021.
[32] Zuffi S, Kanazawa A, Black M J. Lions and Tigers and Bears: Capturing Non-rigid, 3d, Articulated Shape from Images [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3955-3963.
[33] Labuguen R, Matsumoto J, Negrete S B, et al. MacaquePose: a novel “in the wild” macaque monkey pose dataset for markerless motion capture [J]. Frontiers in Behavioral Neuroscience, 2021(14): 581154. doi: 10.3389/fnbeh.2020.581154
[34] Garland M, Heckbert P S. Surface Simplification Using Quadric Error Metrics [C]. Proceedings of the 24th annual conference on Computer graphics and interactive techniques, 1997: 209-216.
[35] Engel N, Belagiannis V, Dietmayer K. Point transformer [J]. IEEE Access, 2021, 9: 134826-134840. doi: 10.1109/ACCESS.2021.3116304
[36] Qi C R, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space [J]. Advances in Neural Information Processing Systems, 2017, 30.
[37] 胡海彦, 狄瑜, 赵永锋, 等. 蠡湖 4 种鲌鱼形态特征的比较研究 [J]. 云南农业大学学报: 自然科学版, 2011, 26(4): 488-494.] Hu H, Di Y, Zhao Y, et al. Comparative study on the morphological characteristics of four species of culter and culterichthys in Lihu Lake [J]. Journal of Yunnan Agricultural University (Natural Science), 2011, 26(4): 488-494. [
[38] 王亚龙, 李昊成, 何勇凤, 等. 长湖 5 种鲌摄食器官形态学的比较 [J]. 淡水渔业, 2016, 46(6): 26-32.] doi: 10.3969/j.issn.1000-6907.2016.06.005 Wang Y L, Li H C, He Y F, et al. Morphological variations of feeding organs of five species of culter from Changhu Lake [J]. Freshwater Fisherie, 2016, 46(6): 26-32. [ doi: 10.3969/j.issn.1000-6907.2016.06.005
[39] Chao Z, Daming X, Kai L, et al. Evaluation of fish feeding intensity in aquaculture based on near-infrared machine vision [J]. Smart Agriculture, 2019, 1(1): 76.
[40] Fernandes A F A, Turra E M, de Alvarenga E R, et al. Deep Learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile tilapia [J]. Computers and Electronics in Agriculture, 2020(170): 105274. doi: 10.1016/j.compag.2020.105274
[41] Chen L, Yang X, Sun C, et al. Feed intake prediction model for group fish using the MEA-BP neural network in intensive aquaculture [J]. Information Processing in Agriculture, 2020, 7(2): 261-271. doi: 10.1016/j.inpa.2019.09.001
[42] Xu M, Wu Y. Fish diseases diagnosis based on rough set and neural network [J]. Computer Engineering and Design, 2009, 30(7): 1738-1741.
[43] Darapaneni N, Sreekanth S, Paduri A R, et al. AI Based Farm Fish Disease Detection System to Help Micro and Small Fish Farmers [C]. 2022 Interdisciplinary Research in Technology and Management (IRTM). IEEE, 2022: 1-5.
-
期刊类型引用(17)
1. 叶毅铭，何亮银，廖青青，林永翔，刘敏，陈虹坚，文宇鑫. 大黄鱼变形假单胞菌外膜蛋白OMPH和OMPW的原核表达及其多克隆抗体制备. 宁德师范学院学报(自然科学版). 2023(01): 57-64 . 
百度学术
2. 汪浩，汪玮，许文军，施慧，何杰，谢建军，王庚申. 大黄鱼源杀香鱼假单胞菌外膜蛋白OmpA的原核表达及免疫原性分析. 浙江海洋大学学报(自然科学版). 2019(01): 23-29 . 百度学术
3. 黄华，刘锡胤，张秀梅，王鹤，王晓飞. 鱼类鮰爱德华菌与鮰爱德华菌病研究进展. 渔业信息与战略. 2019(04): 272-278 . 百度学术
4. 段丽华，冯建军，郭松林，陈义航，阮彩章. 迟钝爱德华氏菌外膜蛋白基因工程表达产物的免疫原性. 华中农业大学学报. 2016(05): 97-104 . 百度学术
5. 王二龙，秦振阳，汪开毓，陈德芳，王均，贺扬. 鲁氏耶尔森氏菌外膜蛋白ompF基因的分子克隆、生物信息学与免疫原性分析. 南方水产科学. 2016(03): 24-34 . 百度学术
6. 黄艳青，刘港彪，王利. 鮰爱德华菌外膜蛋白OmpLC基因的生物信息学分析. 生物信息学. 2016(02): 61-70 . 百度学术
7. 王二龙，王兴丽，杨帆，秦振阳，汪开毓，陈德芳，耿毅. 鮰爱德华菌外膜蛋白ompN2基因的克隆表达、分子特性与免疫原性分析. 中国兽医学报. 2016(01): 56-65+69 . 百度学术
8. 陈亚楠，袁玲. 黄连根茎浸提物对隆线溞的急性毒性作用. 环境科学. 2015(10): 3892-3895 . 百度学术
9. 王玉，冯建军，郭松林，林鹏. 嗜水气单胞菌与迟顿爱德华氏菌二联外膜蛋白的表达及其初步免疫原性. 华中农业大学学报. 2015(01): 96-102 . 百度学术
10. 郭松林，冯建军，陆盼盼，曹伟棋，赵金平. 鳗鲡病原性嗜水气单胞菌与爱德华氏菌二联表达外膜蛋白的免疫原性. 中国水产科学. 2015(01): 113-120 . 百度学术
11. 郭松林，王玉，冯建军，林鹏. 鳗鲡病原菌二联外膜蛋白基因工程表达载体的构建. 集美大学学报(自然科学版). 2014(05): 330-338 . 百度学术
12. 胡林玲，郭松林，关瑞章，冯建军. 爱德华氏菌疫苗的研究进展. 安徽农业科学. 2013(25): 10353-10355 . 百度学术
13. 甘玲玲，王蔚芳，雷霁霖，刘新富，温海深. 鲆鲽类渔用疫苗研究现状及展望. 渔业科学进展. 2013(02): 125-131 . 百度学术
14. 郭松林，王玉，关瑞章，冯建军，林鹏，杨求华. 鳗鲡病原性气单胞菌外膜蛋白基因全长的克隆与抗原决定簇分析. 生物技术通报. 2013(07): 144-152 . 百度学术
15. 熊静，关瑞章，郭松林，黄文树，关淑芳. 不同方法提取鳗鲡病原菌DNA模板的差异分析. 集美大学学报(自然科学版). 2012(03): 161-166 . 百度学术
16. 张崇文，毛芝娟，于涟. 大黄鱼三种病原弧菌外膜蛋白交叉保护性抗原筛选. 生物工程学报. 2012(12): 1460-1472 . 百度学术
17. 沈文英，傅玲琳，李卫芬. 枯草芽孢杆菌表达的VP28对南美白对虾免疫力和抗病毒感染的影响. 水生生物学报. 2012(02): 375-378 . 本站查看
其他类型引用(18)
计量
- 文章访问数: 149
- HTML全文浏览量: 20
- PDF下载量: 33
- 被引次数: 35