A method for wildlife behavior video detection based on multi-scale neural network：a case study of wild Panthera tigris altaica

LIU Mengyu; DAI Wenrui; LIU Shucong; MA Guangkai; JIANG Guangshun

doi:10.16829/j.slxb.151082

2026 , Vol. 46 >Issue 4: 586 - 596

DOI: https://doi.org/10.16829/j.slxb.151082

METHOD AND TECHNOLOGY

A method for wildlife behavior video detection based on multi-scale neural network：a case study of wild Panthera tigris altaica

LIU Mengyu ,
DAI Wenrui ,
LIU Shucong ,
MA Guangkai ,
JIANG Guangshun

Expand

1 College of Computer and Control Engineering, Northeast Forestry University, Harbin 150040, China;
2 Feline Research Center of National Forestry and Grassland Administration, College of Wildlife and Protected Area, Northeast Forestry University, Harbin 150040, China

Received date: 2025-03-31

Revised date: 2025-05-08

Online published: 2025-05-08

Fold

Abstract

Behavioral detection of wild Panthera tigris altaica is a critical component for in-depth analysis and studying the expression of their wild behaviors and the variations in behavioral rhythms. It is essential for comprehensively understanding their living habits and ecological requirements. Currently, trap cameras have become a vital tool for capturing behavioral image data of wild P. t. altaica. However, influenced by the change of shooting distance, the sizes of individual Panthera tigris altaica vary in the video images. Moreover, the behavioral patterns of P. t. altaica exhibit significant dynamic differences in the temporal dimension, characterized by high variability in time series, which affects the detection accuracy of the behavioral patterns of P. t. altaica. Aiming at the image data of wild P. t. altaica captured by trap cameras, this study proposes a multi-scale neural network-based approach for wildlife behavioral video detection. This method uses the 3D MViTv2 network to model video frame sequences of P. t. altaica in the wild, extracting spatiotemporal multi-scale behavioral features. Through the weighted bi-directional feature pyramid network (BiFPN), we achieve adaptive fusion of multi-scale spatial contextual information, while enhancing cross-scale temporal feature modeling capabilities by integrating concepts from the temporal pyramid network (TPN), thereby realizing precise detection of P. t. altaica behaviors. Additionally, the Bytetrack multi-object tracking algorithm is implemented to track tiger trajectories and facilitate statistical analysis of behavioral durations. Our research establishes a behavioral dataset containing four categories (walking, standing, lying down, and jumping) derived from 200 field-collected infrared camera videos of wild P. t. altaica. Experimental results demonstrate that the proposed spatiotemporal multi-scale neural network achieves a mean average precision (mAP) of 93. 8% in behavioral pattern detection, indicating its superior performance in P. t. altaica behavioral pattern detection tasks and potential as a reliable technical framework for other wildlife behavior analysis studies.

Key words： Behavior pattern detection; Multiple object tracking; Deep learning; Trap camera; Multi-scale visual network

Cite this article

LIU Mengyu , DAI Wenrui , LIU Shucong , MA Guangkai , JIANG Guangshun . A method for wildlife behavior video detection based on multi-scale neural network：a case study of wild Panthera tigris altaica[J]. ACTA THERIOLOGICA SINICA, 2026 , 46(4) : 586 -596 . DOI: 10.16829/j.slxb.151082

References

Alibhai S K,Gu J Y,Jewell Z C,Morgan J,Liu D,Jiang G S, 2023. 'I know the tiger by his paw':a non-invasive footprint identification technique for monitoring individual Amur tigers(Panthera tigris altaica) in snow[J]. Ecological Informatics, 73:101947. DOI:10.1016/j. ecoinf. 2022. 101947.
Anderson D J,Perona P,2014. Toward a science of computational ethology[J]. Neuron, 84 (1):18-31. DOI:10.1016/j. neuron. 2014. 09. 005.
Bernardin K, Stiefelhagen R, 2008. Evaluating multiple object tracking performance:the clear mot metrics[J]. Journal on Image and Video Processing,1-10. DOI:10.1155/2008/246309.
Bottou L,2012. Stochastic gradient descent tricks[M]//Montavon G, Orr G B,Müller K R eds. Neural networks:tricks of the trade. Berlin, Heidelberg:Springer, 421-436. DOI:10.1007/ 978-3-642-35289-8_25.
Burton A C,Neilson E,Moreira D,Ladle A,Steenweg R,Fisher J T,Bayne E,Boutin S,2015. Wildlife camera trapping:a review and recommendations for linking surveys to ecological processes[J]. Journal of Applied Ecology, 52 (3):675-685.DOI:10.1111/1365-2664. 12432.
Cao J, Pang J, Weng X, Khirodkar R, Kitani K, 2023. Observation-centric sort:Rethinking sort for robust multi-object tracking[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada,9686-9696. DOI:10.1109/CVPR52729. 2023. 00934.
Chen Y H, Zhang Z, Cao Y, Wang L W, Lin S, Hu H, 2020. Reppoints v2:Verification meets regression for object detection[C]. Advances in Neural Information Processing Systems, 33:5621-5631. DOI:10.5555/3495724. 3496196.
Cheng M,Yuan H,Wang Q,Cai Z,Liu Y,Zhang Y,2022. Application of deep learning in sheep behaviors recognition and influence analysis of training data characteristics on the recognition effect[J]. Computers and Electronics in Agriculture, 198:107010. DOI:10.1016/j. compag. 2022. 107010.
Chollet F,2017. Xception:Deep learning with depthwise separable convolutions[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 1251-1258. DOI:10.1109/CVPR. 2017. 195.
Deng S,Tang G,Mei L,2022. Wild mammal behavior recognition based on gated transformer network[C]. Proceedings of the International Conference on Cyber-Physical Social Intelligence,Nanjing, China, 739-743. DOI:10.1109/ICCSI55536. 2022. 9970674.
Fan H,Xiong B,Mangalam K,Li Y,Yan Z,Malik J,Feichtenhofer C,2021. Multiscale vision transformers[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, 6824-6835. DOI:10.1109/ICCV48922. 2021. 00675.
Feichtenhofer C, 2020. X3d:Expanding architectures for efficient video recognition[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA,203-213. DOI:10.1109/CVPR42600. 2020. 00028.
Fuentes A,Yoon S,Park J,Park D S,2020. Deep learning-based hierarchical cattle behavior recognition with spatio-temporal information[J]. Computers and Electronics in Agriculture, 177:105627. DOI:10.1016/j. compag. 2020. 105627.
Girshick R,2015. Fast r-cnn[C]. Proceedings of the IEEE International Conference on Computer Vision, 1440-1448. DOI:10.1109/ICCV. 2015. 169.
Gu Z, Zhang H, He Z, Niu K, 2023. A two-stage recognition method based on deep learning for sheep behavior[J]. Computers and Electronics in Agriculture, 212:108143. DOI:10.1016/j. compag. 2023. 108143.
He K M, Gkioxari G, Dollár P, Girshick R, 2017. Mask r-cnn[C]. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2961-2969. DOI:10.1109/ ICCV. 2017. 322.
He K M,Zhang X Y,Ren S Q,Sun J,2016. Deep residual learning for image recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV,USA,770-778. DOI:10.1109/CVPR. 2016. 90.
He T, Zhang Z, Zhang H, Zhang Z, Xie J, Li M, 2019. Bag of tricks for image classification with convolutional neural networks[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA,558-567. DOI:10.1109/CVPR. 2019. 00065.
IUCN,2022. Panthera tigris[R]. The IUCN Red List of Threatened Species 2022:e. T15955A214862019. DOI:10.2305/IUCN. UK. 2022-1. RLTS. T15955A214862019. en.
Jiang Huilong, 2023. Spatio-temporal overlapping of niche among sika deer, roe deer, and wild boar in Muling Reserve, Heilongjiang Province[D]. Harbin:Northeast Forestry University.DOI:10.27009/d. cnki. gdblu. 2023. 000755. (in Chinese with English abstract)
Li K,Fan D,Wu H,Zhao A,2024. A new dataset for video-based cow behavior recognition[J]. Scientific Reports, 14 (1):18702. DOI:10.1038/s41598-024-65953-x.
Li Y,Wu C Y,Fan H,Mangalam K,Xiong B,Malik J,Feichtenhofer C, 2022. Mvitv2:Improved multiscale vision transformers for classification and detection[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,New Orleans,LA,USA,4804-4814. DOI:10.1109/CVPR52688. 2022. 00476.
Lin T Y,Dollár P,Girshick R,He K,Hariharan B,Belongie S, 2017. Feature pyramid networks for object detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2117-2125. DOI:10.1109/CVPR. 2017. 106.
Lin T Y,Maire M,Belongie S,Hays J,Perona P,Ramanan D, Dollár P,Zitnick C L,2014. Microsoft coco:common objects in context[C]. European Conference on Computer Vision,zurich, Switzerland, 13 (1), 740-755. DOI:10.1007/ 978-3-319-10602-1_48.
National forestry and grassland administration,ministry of agriculture and rural affairs,2021. List of National Key Protected Wild Animals[J]. Chinese Journal of Wildlife,42 (2):605-640. DOI:10.19711/j. cnki. issn2310-1490. 2021. 02. 045.
Qi J Z,Holyoak M,Ning Y,Jiang G S,2020. Ecological thresholds and large carnivores conservation:Implications for the Amur tiger and leopard in China[J]. Global Ecology and Conservation,21:e00837. DOI:10.1016/j. gecco. 2019. e00837.
Qi Miao,Xu Hui,Li Sen,Zhang Yu,Sun Hui,2023. An action recognition method based on two-stream network[J]. Journal of Jilin University Science Edition, 61 (2):347-352. DOI:10.13413/j. cnki. jdxblxb. 2022040. (in Chinese with English abstract)
Tanioka H, Miura T, Matsuura K, Karungaru S, 2023. Development of semi-automatic image annotation using object recognition[C]. Proceedings of the 14th IIAI International Congress on Advanced Applied Informatics. Osaka, Japan, 546-549.DOI:10.1109/IIAI-AAI59060. 2023. 00109.
Tan M, Pang R, Le Q V, 2020. Efficientdet:Scalable and efficient object detection[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 10781-10790. DOI:10.1109/CVPR42600. 2020. 01079.
Wang L,Xiong Y,Wang Z,Qiao Y,Lin D,Tang X,Gool L V, 2016. Temporal segment networks:Towards good practices for deep action recognition[C]. European Conference on Computer Vision, Amsterdam, Netherlands, 20-36. DOI:10.1007/ 978-3-319-46484-8_2.
Wojke N, Bewley A, Paulus D, 2017. Simple online and realtime tracking with a deep association metric[C]. Proceedings of the 2017 IEEE International Conference on Image Processing, Beijing,China,3645-3649. DOI:10.1109/ICIP. 2017. 8296962.
Yang C,Xu Y,Shi J,Dai B,Zhou B,2020. Temporal pyramid network for action recognition[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 591-600. DOI:10.1109/CVPR42600. 2020. 00067.
Zeng R,Huang W,Tan M,Rong Y,Zhao P,Huang J,Gan C, 2019. Graph convolutional networks for temporal action localization[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 7094-7103.DOI:10.1109/ICCV. 2019. 00719.
Zhang H,Wang Y,Dayoub F,Sunderhauf N,2021. Varifocalnet:An iou-aware dense object detector[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 8514-8523. DOI:10.1109/CVPR46437. 2021. 00841.
Zhang Y,Cai J,Xiao D,Li Z,Xiong B,2019. Real-time sow behavior detection based on deep learning[J]. Computers and Electronics in Agriculture, 1-10. DOI:10.1016/j. compag. 2019. 104884.
Zhang Y,Sun P,Jiang Y,Yu D,Weng F,Yuan Z,Luo P,Liu W,Wang X,2022a. Bytetrack:Multi-object tracking by associating every detection box[C]. European Conference on Computer Vision, Tel Aviv, Israel, 1-21. DOI:10.1007/ 978-3-031-20047-2_1.
Zhou D, Fang J, Song X, Guan C, Yin J, Dai Y, Yang R, 2019. Iou loss for 2d/3d object detection[C]. Proceedings of the 2019 International Conference on 3D Vision,Québec City,QC, Canada,85-94. DOI:10.1109/3DV. 2019. 00019.
Zhang Xueying,Zhang Haolin,Han Yingying,Weng Qiang,Yuan Zhengrong,Yao Yuan,2022. Research progress of the wildlife monitoring and identification based on deep learning[J]. Chinese Journal of Wildlife,43 (1):251-258. DOI:10.19711/j. cnki.issn2310-1490. 2022. 01. 032. (in Chinese with English abstract)
Zuerl M, Stoll P, Brehm, I, Raab R, Zanca D, Kabri S, Happold J,Nille H,Prechtel K,Wuensch S,Krause M,Seegerer S,Fersen L,Eskofier B,2022. Automated video-based analysis framework for behavior monitoring of individual animals in zoos using deep learning:a study on polar bears[J]. Animals, 12 (6):692. DOI:10.3390/ani12060692.
乔征磊,韦钦国,2015.东北虎描述性全行为谱的构建[J].黑龙江畜牧兽医,(9):207-209. DOI:10.13881/j. cnki. hljxmsy. 2015. 0717.
齐妙,徐慧,李森,张宇,孙慧. 2023.一种基于双流网络的行为识别方法[J].吉林大学学报 (理学版),61 (2):347-352.DOI:10.13413/j. cnki. jdxblxb. 2022040.
江辉龙,2023.黑龙江穆棱林区梅花鹿、狍和野猪生态位的时空重叠研究[D].哈尔滨:东北林业大学. DOI:10.27009/d. cnki.gdblu. 2023. 000755.
张雪莹,张浩林,韩莹莹,翁强,袁峥嵘,姚远,2022.基于深度学习的野生动物监测与识别研究进展[J].野生动物学报,43(1):251-258. DOI:10.19711/j. cnki. issn2310-1490. 2022. 01. 032.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References