A method for wildlife behavior video detection based on multi-scale neural network：a case study of wild Panthera tigris altaica

doi:10.16829/j.slxb.151082

Abstract

Abstract: Behavioral detection of wild Panthera tigris altaica is a critical component for in-depth analysis and studying the expression of their wild behaviors and the variations in behavioral rhythms. It is essential for comprehensively understanding their living habits and ecological requirements. Currently, trap cameras have become a vital tool for capturing behavioral image data of wild P. t. altaica. However, influenced by the change of shooting distance, the sizes of individual Panthera tigris altaica vary in the video images. Moreover, the behavioral patterns of P. t. altaica exhibit significant dynamic differences in the temporal dimension, characterized by high variability in time series, which affects the detection accuracy of the behavioral patterns of P. t. altaica. Aiming at the image data of wild P. t. altaica captured by trap cameras, this study proposes a multi-scale neural network-based approach for wildlife behavioral video detection. This method uses the 3D MViTv2 network to model video frame sequences of P. t. altaica in the wild, extracting spatiotemporal multi-scale behavioral features. Through the weighted bi-directional feature pyramid network (BiFPN), we achieve adaptive fusion of multi-scale spatial contextual information, while enhancing cross-scale temporal feature modeling capabilities by integrating concepts from the temporal pyramid network (TPN), thereby realizing precise detection of P. t. altaica behaviors. Additionally, the Bytetrack multi-object tracking algorithm is implemented to track tiger trajectories and facilitate statistical analysis of behavioral durations. Our research establishes a behavioral dataset containing four categories (walking, standing, lying down, and jumping) derived from 200 field-collected infrared camera videos of wild P. t. altaica. Experimental results demonstrate that the proposed spatiotemporal multi-scale neural network achieves a mean average precision (mAP) of 93. 8% in behavioral pattern detection, indicating its superior performance in P. t. altaica behavioral pattern detection tasks and potential as a reliable technical framework for other wildlife behavior analysis studies.

Key words: Behavior pattern detection, Multiple object tracking, Deep learning, Trap camera, Multi-scale visual network

摘要： 野生虎东北亚种（Panthera tigris altaica）的行为识别是深入剖析和研究其野性行为表达以及行为节律变化的关键环节，对于全面了解它们的生活习性和生态需求至关重要。目前，自动红外相机是捕捉野生虎东北亚种行为影像数据的重要工具。然而，受拍摄距离变化的影响，虎东北亚种个体在视频影像中的大小存在差异，且虎东北亚种行为模式在时间维度上呈现出显著的动态差异，具有时序多变性的特点，影响虎东北亚种行为模式识别精度。针对由自动红外相机拍摄的野生虎东北亚种影像数据，本研究提出一种基于多尺度神经网络的野生动物行为影像识别方法，该方法利用3D MViTv2网络对野外虎东北亚种的视频帧序列进行建模，提取行为的时空多尺度特征，通过加权双向特征金字塔网络（weighted bi-directional feature pyramid network，BiFPN）实现多尺度空间上下文信息的自适应融合，并借鉴时间金字塔网络（temporal pyramid network，TPN）的思想强化跨尺度时序特征建模能力，实现野生虎东北亚种行为的精准识别。此外，本文采用Bytetrack多目标跟踪算法对虎东北亚种的轨迹进行跟踪，实现虎东北亚种行为持续时间的统计分析。本研究基于自动红外相机野外拍摄的200段野外虎东北亚种视频影像，构建包含走、站立、卧息和跳4种行为的野生虎东北亚种行为数据集。结果显示，所提出的时空多尺度神经网络取得了93.8%的平均精度均值（mean average precision，mAP）的行为模式识别精度，表明了其在虎东北亚种行为模式识别任务中具有良好的性能，可以为其他野生动物行为识别研究提供可靠的技术支持。

关键词: 行为模式识别, 多目标跟踪, 深度学习, 红外相机, 多尺度视觉网络

CLC Number:

LIU Mengyu, DAI Wenrui, LIU Shucong, MA Guangkai, JIANG Guangshun. A method for wildlife behavior video detection based on multi-scale neural network：a case study of wild Panthera tigris altaica[J]. ACTA THERIOLOGICA SINICA, 2026, 46(4): 586-596.

刘梦雨, 戴文锐, 刘淑聪, 马光凯, 姜广顺. 一种基于多尺度神经网络的野生动物行为影像识别方法——以野生虎东北亚种为例[J]. 兽类学报, 2026, 46(4): 586-596.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.mammal.cn/EN/10.16829/j.slxb.151082

https://www.mammal.cn/EN/Y2026/V46/I4/586

References

Alibhai S K,Gu J Y,Jewell Z C,Morgan J,Liu D,Jiang G S, 2023. 'I know the tiger by his paw':a non-invasive footprint identification technique for monitoring individual Amur tigers(Panthera tigris altaica) in snow[J]. Ecological Informatics, 73:101947. DOI:10.1016/j. ecoinf. 2022. 101947.
Anderson D J,Perona P,2014. Toward a science of computational ethology[J]. Neuron, 84 (1):18-31. DOI:10.1016/j. neuron. 2014. 09. 005.
Bernardin K, Stiefelhagen R, 2008. Evaluating multiple object tracking performance:the clear mot metrics[J]. Journal on Image and Video Processing,1-10. DOI:10.1155/2008/246309.
Bottou L,2012. Stochastic gradient descent tricks[M]//Montavon G, Orr G B,Müller K R eds. Neural networks:tricks of the trade. Berlin, Heidelberg:Springer, 421-436. DOI:10.1007/ 978-3-642-35289-8_25.
Burton A C,Neilson E,Moreira D,Ladle A,Steenweg R,Fisher J T,Bayne E,Boutin S,2015. Wildlife camera trapping:a review and recommendations for linking surveys to ecological processes[J]. Journal of Applied Ecology, 52 (3):675-685.DOI:10.1111/1365-2664. 12432.
Cao J, Pang J, Weng X, Khirodkar R, Kitani K, 2023. Observation-centric sort:Rethinking sort for robust multi-object tracking[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada,9686-9696. DOI:10.1109/CVPR52729. 2023. 00934.
Chen Y H, Zhang Z, Cao Y, Wang L W, Lin S, Hu H, 2020. Reppoints v2:Verification meets regression for object detection[C]. Advances in Neural Information Processing Systems, 33:5621-5631. DOI:10.5555/3495724. 3496196.
Cheng M,Yuan H,Wang Q,Cai Z,Liu Y,Zhang Y,2022. Application of deep learning in sheep behaviors recognition and influence analysis of training data characteristics on the recognition effect[J]. Computers and Electronics in Agriculture, 198:107010. DOI:10.1016/j. compag. 2022. 107010.
Chollet F,2017. Xception:Deep learning with depthwise separable convolutions[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 1251-1258. DOI:10.1109/CVPR. 2017. 195.
Deng S,Tang G,Mei L,2022. Wild mammal behavior recognition based on gated transformer network[C]. Proceedings of the International Conference on Cyber-Physical Social Intelligence,Nanjing, China, 739-743. DOI:10.1109/ICCSI55536. 2022. 9970674.
Fan H,Xiong B,Mangalam K,Li Y,Yan Z,Malik J,Feichtenhofer C,2021. Multiscale vision transformers[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, 6824-6835. DOI:10.1109/ICCV48922. 2021. 00675.
Feichtenhofer C, 2020. X3d:Expanding architectures for efficient video recognition[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA,203-213. DOI:10.1109/CVPR42600. 2020. 00028.
Fuentes A,Yoon S,Park J,Park D S,2020. Deep learning-based hierarchical cattle behavior recognition with spatio-temporal information[J]. Computers and Electronics in Agriculture, 177:105627. DOI:10.1016/j. compag. 2020. 105627.
Girshick R,2015. Fast r-cnn[C]. Proceedings of the IEEE International Conference on Computer Vision, 1440-1448. DOI:10.1109/ICCV. 2015. 169.
Gu Z, Zhang H, He Z, Niu K, 2023. A two-stage recognition method based on deep learning for sheep behavior[J]. Computers and Electronics in Agriculture, 212:108143. DOI:10.1016/j. compag. 2023. 108143.
He K M, Gkioxari G, Dollár P, Girshick R, 2017. Mask r-cnn[C]. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2961-2969. DOI:10.1109/ ICCV. 2017. 322.
He K M,Zhang X Y,Ren S Q,Sun J,2016. Deep residual learning for image recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV,USA,770-778. DOI:10.1109/CVPR. 2016. 90.
He T, Zhang Z, Zhang H, Zhang Z, Xie J, Li M, 2019. Bag of tricks for image classification with convolutional neural networks[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA,558-567. DOI:10.1109/CVPR. 2019. 00065.
IUCN,2022. Panthera tigris[R]. The IUCN Red List of Threatened Species 2022:e. T15955A214862019. DOI:10.2305/IUCN. UK. 2022-1. RLTS. T15955A214862019. en.
Jiang Huilong, 2023. Spatio-temporal overlapping of niche among sika deer, roe deer, and wild boar in Muling Reserve, Heilongjiang Province[D]. Harbin:Northeast Forestry University.DOI:10.27009/d. cnki. gdblu. 2023. 000755. (in Chinese with English abstract)
Li K,Fan D,Wu H,Zhao A,2024. A new dataset for video-based cow behavior recognition[J]. Scientific Reports, 14 (1):18702. DOI:10.1038/s41598-024-65953-x.
Li Y,Wu C Y,Fan H,Mangalam K,Xiong B,Malik J,Feichtenhofer C, 2022. Mvitv2:Improved multiscale vision transformers for classification and detection[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,New Orleans,LA,USA,4804-4814. DOI:10.1109/CVPR52688. 2022. 00476.
Lin T Y,Dollár P,Girshick R,He K,Hariharan B,Belongie S, 2017. Feature pyramid networks for object detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2117-2125. DOI:10.1109/CVPR. 2017. 106.
Lin T Y,Maire M,Belongie S,Hays J,Perona P,Ramanan D, Dollár P,Zitnick C L,2014. Microsoft coco:common objects in context[C]. European Conference on Computer Vision,zurich, Switzerland, 13 (1), 740-755. DOI:10.1007/ 978-3-319-10602-1_48.
National forestry and grassland administration,ministry of agriculture and rural affairs,2021. List of National Key Protected Wild Animals[J]. Chinese Journal of Wildlife,42 (2):605-640. DOI:10.19711/j. cnki. issn2310-1490. 2021. 02. 045.
Qi J Z,Holyoak M,Ning Y,Jiang G S,2020. Ecological thresholds and large carnivores conservation:Implications for the Amur tiger and leopard in China[J]. Global Ecology and Conservation,21:e00837. DOI:10.1016/j. gecco. 2019. e00837.
Qi Miao,Xu Hui,Li Sen,Zhang Yu,Sun Hui,2023. An action recognition method based on two-stream network[J]. Journal of Jilin University Science Edition, 61 (2):347-352. DOI:10.13413/j. cnki. jdxblxb. 2022040. (in Chinese with English abstract)
Tanioka H, Miura T, Matsuura K, Karungaru S, 2023. Development of semi-automatic image annotation using object recognition[C]. Proceedings of the 14th IIAI International Congress on Advanced Applied Informatics. Osaka, Japan, 546-549.DOI:10.1109/IIAI-AAI59060. 2023. 00109.
Tan M, Pang R, Le Q V, 2020. Efficientdet:Scalable and efficient object detection[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 10781-10790. DOI:10.1109/CVPR42600. 2020. 01079.
Wang L,Xiong Y,Wang Z,Qiao Y,Lin D,Tang X,Gool L V, 2016. Temporal segment networks:Towards good practices for deep action recognition[C]. European Conference on Computer Vision, Amsterdam, Netherlands, 20-36. DOI:10.1007/ 978-3-319-46484-8_2.
Wojke N, Bewley A, Paulus D, 2017. Simple online and realtime tracking with a deep association metric[C]. Proceedings of the 2017 IEEE International Conference on Image Processing, Beijing,China,3645-3649. DOI:10.1109/ICIP. 2017. 8296962.
Yang C,Xu Y,Shi J,Dai B,Zhou B,2020. Temporal pyramid network for action recognition[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 591-600. DOI:10.1109/CVPR42600. 2020. 00067.
Zeng R,Huang W,Tan M,Rong Y,Zhao P,Huang J,Gan C, 2019. Graph convolutional networks for temporal action localization[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 7094-7103.DOI:10.1109/ICCV. 2019. 00719.
Zhang H,Wang Y,Dayoub F,Sunderhauf N,2021. Varifocalnet:An iou-aware dense object detector[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 8514-8523. DOI:10.1109/CVPR46437. 2021. 00841.
Zhang Y,Cai J,Xiao D,Li Z,Xiong B,2019. Real-time sow behavior detection based on deep learning[J]. Computers and Electronics in Agriculture, 1-10. DOI:10.1016/j. compag. 2019. 104884.
Zhang Y,Sun P,Jiang Y,Yu D,Weng F,Yuan Z,Luo P,Liu W,Wang X,2022a. Bytetrack:Multi-object tracking by associating every detection box[C]. European Conference on Computer Vision, Tel Aviv, Israel, 1-21. DOI:10.1007/ 978-3-031-20047-2_1.
Zhou D, Fang J, Song X, Guan C, Yin J, Dai Y, Yang R, 2019. Iou loss for 2d/3d object detection[C]. Proceedings of the 2019 International Conference on 3D Vision,Québec City,QC, Canada,85-94. DOI:10.1109/3DV. 2019. 00019.
Zhang Xueying,Zhang Haolin,Han Yingying,Weng Qiang,Yuan Zhengrong,Yao Yuan,2022. Research progress of the wildlife monitoring and identification based on deep learning[J]. Chinese Journal of Wildlife,43 (1):251-258. DOI:10.19711/j. cnki.issn2310-1490. 2022. 01. 032. (in Chinese with English abstract)
Zuerl M, Stoll P, Brehm, I, Raab R, Zanca D, Kabri S, Happold J,Nille H,Prechtel K,Wuensch S,Krause M,Seegerer S,Fersen L,Eskofier B,2022. Automated video-based analysis framework for behavior monitoring of individual animals in zoos using deep learning:a study on polar bears[J]. Animals, 12 (6):692. DOI:10.3390/ani12060692.
乔征磊,韦钦国,2015.东北虎描述性全行为谱的构建[J].黑龙江畜牧兽医,(9):207-209. DOI:10.13881/j. cnki. hljxmsy. 2015. 0717.
齐妙,徐慧,李森,张宇,孙慧. 2023.一种基于双流网络的行为识别方法[J].吉林大学学报 (理学版),61 (2):347-352.DOI:10.13413/j. cnki. jdxblxb. 2022040.
江辉龙,2023.黑龙江穆棱林区梅花鹿、狍和野猪生态位的时空重叠研究[D].哈尔滨:东北林业大学. DOI:10.27009/d. cnki.gdblu. 2023. 000755.
张雪莹,张浩林,韩莹莹,翁强,袁峥嵘,姚远,2022.基于深度学习的野生动物监测与识别研究进展[J].野生动物学报,43(1):251-258. DOI:10.19711/j. cnki. issn2310-1490. 2022. 01. 032.

[1]	Shiyu CHEN, Jin HOU, Dan LIU, Jing LIU, Peng LUO, Bochuan ZHENG, Jindong ZHANG. Progress in the application of deep learning in wildlife image recognition and analysis [J]. ACTA THERIOLOGICA SINICA, 2026, 46(1): 20-38.
[2]	ZHONG Junjie, NIU Bing, CHEN Qin, CHEN Xiang, WANG Yan. Application of deep learning in wildlife conservation [J]. ACTA THERIOLOGICA SINICA, 2023, 43(6): 734-744.
[3]	Yu QI, Han SU, Rong HOU, Peng LIU, Peng CHEN, Hangxing ZANG, Zhihe ZHANG. Giant panda pose estimation method based on high resolution net [J]. ACTA THERIOLOGICA SINICA, 2022, 42(4): 451-460.
[4]	GONG Yinan, TAN Mengyu, WANG Zhen, ZHAO Guojing, JIANG Peilin, JIANG Shiming, ZHANG Dingji, GE Jianping, FENG Limin. AI recognition of infrared camera image of wild animals based on deep learning: Northeast Tiger and Leopard National Park for example [J]. ACTA THERIOLOGICA SINICA, 2019, 39(4): 458-465.