PERTANIKA JOURNAL OF SCIENCE AND TECHNOLOGY

Computer vision (CV), in combination with various sensors and image analysis algorithms, is a frontier direction in diagnosing the state of the environment and its biogenic and abiogenic objects. The work generalizes scientific achievements and identifies scientific and technical problems in this area of research based on the conceptual system of analysis on the time axis: from implemented achievements as part of the past and present to original new solutions—the future. Our work gives an idea of three areas of application of CV in diagnosing the state of the environment: phenotype recognition in digital images, monitoring of living and abiogenic objects, and development of new methods for identifying pollution and its consequences. The advantages of CV, which can be attributed to scientific achievements in this field of research, are shown: an increase in the volume of analyzed samples, simultaneous analysis of several parameters of the object of observation, and leveling of subjective evaluation factors. The main CV problems currently solved are the accuracy of diagnostics and changing quality of the survey, identification of the object of analysis with minimal operator participation, simultaneous monitoring of objects of different quality, and development of software and hardware systems with CV. A promising direction for the future is to combine the capabilities of CV and artificial intelligence. Thus, the review can be useful for specialists in environmental sciences and scientists working in interdisciplinary fields.

• Abou-Donia, M. (Ed.). (2015). Mammalian Toxicology. Wiley.

• Al Duhaiyyim, M., Marzouk, R., Al-Wesabi, F. N., Alrajhi, M., Hamza, M. A., & Zamani, A. S. (2022). An improved evolutionary algorithm for data mining and knowledge discovery. CMC-Computers, Materials & Continua, 71(1), 1233-1247. https://doi.org/10.32604/cmc.2022.021652

• Alam, A., Chauhan, A., Khan, M. T., & Jaffery, Z. A. (2022). Drone-based crop product quality monitoring system: An application of smart agriculture. In R. S, Mor, D. Kumar, & A. Singh (Eds.), Advanced Series in Management (pp. 97-110). Emerald Publishing. https://doi.org/10.1108/S1877-636120220000027007

• Arya, S., Sandhu, K. S., Singh, J., & Kumar, S. (2022). Deep learning: As the new frontier in high-throughput plant phenotyping. Euphytica, 218(4), Article 4. https://doi.org/10.1007/s10681-022-02992-3

• Ashikhmina, T. Ya., Alalykina, N. M., Domracheva, L. I., Shirokikh, I. G., Ogorodnikova, S. Yu., Kantor, G. Ya., Dabakh, E. V., Skugoreva, S. G., Vidyakin, A. I., Pestov, S. V., Golovko, T. K., Kondakova, L. V., Borodina, N. V., Domnina, E. A., Shirokikh, A. A., Tselishcheva, L. G., Kochurova, T. I., Adamovich, T. A., Kolupaev, A. V., Kondrukhova, S. V., Olkova, A. S., Panfilova, I. V., Timonyuk, V. M., Fokina, A. I., & Shulyateva, N. A. (2011). Biological Monitoring of Natural and Man-Made Systems. Syktyvkar: Institute of Biology of the Komi Scientific Center of the Ural Branch of the Russian Academy of Sciences.

• Bierman, A., LaPlumm, T., Cadle-Davidson, L., Gadoury, D., Martinez, D., Sapkota, S., & Rea, M. (2019). A high-throughput phenotyping system using machine vision to quantify severity of grapevine powdery mildew. Plant Phenomics, 2019, Article 9209727. https://doi.org/10.34133/2019/9209727

• Bjerge, K., Nielsen, J. B., Sepstrup, M. V., Helsing-Nielsen, F., & Høye, T. Т. (2021). An automated light trap to monitor moths (Lepidoptera) using computer vision-based tracking and deep learning. Sensors, 21(2), Article 343. https://doi.org/10.3390/s21020343

• Boukouvalas, D. T., Prates, R. A., Leal, C. R. L., & Araújo, S. A. (2019). Automatic segmentation method for CFU counting in single plate-serial dilution. Chemometrics and Intelligent Laboratory Systems, 195, Article 103889. https://doi.org/10.1016/j.chemolab.2019.103889

• Ditria, E. M., Lopez-Marcano, S., Sievers, M., Jinks, E. L., Brown, C. J., & Connolly, R. M. (2020). Automating the analysis of fish abundance using object detection: Optimizing animal ecology with deep learning. Frontiers in Marine Science, 7, Article 429. https://doi.org/10.3389/fmars.2020.00429

• Djerdj, T., Hackenberger, D. K., Hackenberger, D. K., & Hackenberger, B. K. (2020). Observing earthworm behavior using deep learning. Geoderma, 358, Article 113977. https://doi.org/10.1016/j.geoderma.2019.113977

• Dorfling, J., Siewert, S. B., Bruder, S., Aranzazu-Suescún, C., Rocha, K., Landon, P., Bondar, G., Pederson, T., Le, C., Mangar, R., Rawther, C., & Trahms, B. (2022). Satellite, aerial, and ground sensor fusion experiment for the management of elephants, rhinos, and poaching prevention. In AIAA Science and Technology Forum and Exposition (p. 1270). https://doi.org/10.2514/6.2022-1270

• Dudukcu, H. V., Taskiran. M., & Kahraman, N. (2023). UAV sensor data applications with deep neural networks: A comprehensive survey. Engineering Applications of Artificial Intelligence, 123 (Part C), Article 106476. https://doi.org/10.1016/j.engappai.2023.106476

• Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88, 303-338. https://doi.org/10.1007/s11263-009-0275-4

• Fashi, A., Cheraghi, M., Badiee, H., & Zamani, A. (2020). An analytical strategy based on the combination of ultrasound assisted flat membrane liquid phase microextraction and a smartphone reader for trace determination of malondialdehyde. Talanta, 209, Article 120618. https://doi.org/10.1016/j.talanta.2019.120618

• Fischler, M. A., & Firschein, O. (Eds.). (2014). Readings in computer vision: Issues, problem, principles, and paradigms. Elsevier.

• Gad, S. C. (Ed.). (2016). Animal models in toxicology. Routledge.

• Gervasi, A., Cardol, P., & Meyer, P. E. (2022). Automated open-hardware multiwell imaging station for microorganisms observation. Micromachines, 13(6), Article 833. https://doi.org/10.3390/mi13060833

• Henry, J., Rodriguez, A., & Wlodkowic, D. (2019). Impact of digital video analytics on accuracy of chemobehavioural phenotyping in aquatic toxicology. Peer J, 7, Article e7367. https://doi.org/10.7717/peerj.7367

• Høye, T. T., Ärje, J., Bjerge, K., Hansen, O. L. P., Iosifidis, A., Leese, F., Mann, H. M. R., Meissner, K., Melvad, C., & Raitoharju, J. (2021). Deep learning and computer vision will transform entomology. PNAS, 118(2), Article e2002545117. https://doi.org/10.1073/pnas.2002545117

• Hussein, B. R., Malik, O. A., Ong, W. H., & Slik, J. W. F. (2021). Automated extraction of phenotypic leaf traits of individual intact herbarium leaves from herbarium specimen images using deep learning based semantic segmentation. Sensors, 21(13), Article 4549. https://doi.org/10.3390/s21134549

• Johnston, A., Matechou, E., & Dennis, E. B. (2022). Outstanding challenges and future directions for biodiversity monitoring using citizen science data. Methods in Ecology and Evolution, 14(1), 103-116. https://doi.org/10.1111/2041-210X.13834

• Kantor, G. Y., & Ashikhmina, T. Y. (2014). Modeling of spring flood in the floodplain of the Vyatka River on the territory of the Kirovo-Chepetsky Branch Federal State Unitary Enterprise “RоsRАО”. Theoretical and Applied Ecology, 3, 29-35.

• Karl, D. M., & Proctor, L. M. (2007). Foundations of microbial oceanography. Oceanography, 20(2), 16-27.

• Keefe, R. F., Zimbelman, E. G., & Picchi, G. (2022). Use of individual tree and product level data to improve operational forestry. Current Forestry Reports, 8, 148-165. https://doi.org/10.1007/s40725-022-00160-3

• Kuswantori, А., Suesut, T., Tangsrirat, W., & Nunak, N. (2022). Development of object detection and classification with YOLOv4 for similar and structural deformed fish. Eureka: Physics and Engineering, 2, 154-162. https://doi.org/10.21303/2461-4262.2022.002345

• Li, J., Li, Y., Bi, S., Xu, J., Guo, F., Lyu, H., Dong, X., & Cai, X. (2022). Utilization of GOCI data to evaluate the diurnal vertical migration of Microcystis aeruginosa and the underlying driving factors. Journal of Environmental Management, 310, Article 114734. https://doi.org/10.1016/j.jenvman.2022.114734

• Li, X., Li, N., Li, B., Sun, Y., & Gao, E. (2022). AbundanceR: A novel method for estimating wildlife abundance based on distance sampling and species distribution models. Land, 11(5), Article 660. https://doi.org/10.3390/land11050660

• Li, Z., Guo, R., Li, M., Chen, Y., & Li, G. (2020). A review of computer vision technologies for plant phenotyping. Computers and Electronics in Agriculture, 176, Article 105672. https://doi.org/10.1016/j.compag.2020.105672

• Liu, X., Wang, F., Su, J., Zhou, Y., & Ramakrishna, S. (2022). Bio-inspired 3D artificial neuromorphic circuits. Advanced Functional Materials, 32(22), Article 2113050. https://doi.org/10.1002/adfm.202113050

• Lopez-Marcano, S., Brown, C. J., Sievers, M., & Connolly, R. М. (2020). The slow rise of technology: Computer vision techniques in fish population connectivity. Aquatic Conservation: Marine and Freshwater Ecosystems, 31(1), 210-217. https://doi.org/10.1002/aqc.3432

• Lürig, M. D., Donoughe, S., Svensson, E. I., Porto, A., & Tsuboi, M. (2021). Computer vision, machine learning, and the promise of phenomics in ecology and evolutionary biology. Frontiers in Ecology and Evolution, 9, Article 642774. https://doi.org/10.3389/fevo.2021.642774

• Ma, P., Li, C., Rahaman, M., Yao, Y., Zhang, J., Zou, S., Zhao, X., & Grzegorzek, M. (2023). A state-of-the-art survey of object detection techniques in microorganism image analysis: From classical methods to deep learning approaches. Artificial Intelligence Review, 56, 1627-1689. https://doi.org/10.1007/s10462-022-10209-1

• Ma, Y., & Wei, W. (2021). Evaluate the viscosity of small aquaculture water by computer vision technology. Journal of Physics: Conference Series, 2010, Article 012022. https://doi.org/10.1088/1742-6596/2010/1/012022

• Mochida, K., Koda, S., Inoue, K., Hirayama, T., Tanaka, S., Nishii, R., & Melgani, F. (2019). Computer vision-based phenotyping for improvement of plant productivity: A machine learning perspective. GigaScience, 8(1), Article giy153. https://doi.org/10.1093/gigascience/giy153

• Monaldi, A. C., Romero, G. G., Alanís, E. E., & Cabrera, C. M. (2015). Digital holographic microscopy for microalgae biovolume assessment. Optics Communications, 336, 255-261. https://doi.org/10.1016/j.optcom.2014.10.035

• Moraga, J., Duzgun, H. S., Cavur, M., & Soydan, H. (2022). The geothermal artificial intelligence for geothermal exploration. Renewable Energy, 192, 134-149. https://doi.org/10.1016/j.renene.2022.04.113

• Moreno, G., Ascaneo, J. S., Ricardo, J. O., De La Cruz, L. T., Yaumel, A., Strickler, J. R., & Lopes, R. M. (2020). A new focus detection criterion in holograms of planktonic organisms. Pattern Recognition Letters, 138, 497-506. https://doi.org/10.1016/j.patrec.2020.08.004

• Mu, S., Yang, G., Xu, X., Wan, R., & Li, B. (2022). Assessing the inundation dynamics and its impacts on habitat suitability in Poyang Lake based on integrating Landsat and MODIS observations. Science of the Total Environment, 834, Article 154936. https://doi.org/10.1016/j.scitotenv.2022.154936

• Nikitin, O. V., Nasyrova, E. I., Nuriakhmetova, V. R., & Latypova, V. Z. (2018). Using computer vision algorithms to analyze the behavioral activity of daphnia. In Proceedings of the XVI All-Russian Scientific and Practical Conference with International Participation, 99. https://elibrary.ru/item.asp?id=36586088

• Norouzzadeh, M. S., Morris, D., Beery, S., Joshi, N., Jojic, N., & Clune, J. (2020). A deep active learning system for species identification and counting in camera trap images. Methods in Ecology and Evolution, 12(1), 150-161. https://doi.org/10.1111/2041-210X.13504

• Patil, L. P., Ghante, P. H., Thomse, S. R., Navasare, M. G., & Pawar, S. Y. (2021). Robotic applications and management of plant diseases. AgriCos e-Newsletter, 2(6), 36-40.

• Petso, T., Jamisola, R. S., Mpoeleng, D., Bennitt, E., & Mmereki, W. (2021). Automatic animal identification from drone camera based on point pattern analysis of herd behaviour. Ecological Informatics, 66, Article 101485. https://doi.org/10.1016/j.ecoinf.2021.101485

• Qian, W., Huang, Y., Liu, Q., Fan, W., Sun, Z., Dong, H., Wan, F., & Qiao, X. (2020). UAV and a deep convolutional neural network for monitoring invasive alien plants in the wild. Computers and Electronics in Agriculture, 147, Article 105519. https://doi.org/10.1016/j.compag.2020.105519

• Rani, P., Kotwal, S., & Manhas, J. (2022). An intelligent species level deep learning-based framework in automatic classification of microscopic bacteria images. In S. Roy, D. Sinwar, T. Perumal, A. Slowik, & J. M. R. S. Tavares (Eds.), Innovations in Computational Intelligence and Computer Vision. Advances in Intelligent Systems and Computing, (Vol. 1424). Springer. https://doi.org/10.1007/978-981-19-0475-2_52

• Ratnayake, M. N., Dyer, A. G., & Dorin, A. (2021). Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring. PloS One, 16(2), Article e0239504. https://doi.org/10.1371/journal.pone.0239504

• Rauf, S., Tashkandi, N., Filho, J. I. O., Oviedo-Osornio, C. I., Danish, M. S., Hong, P. Y., & Salama, K. N. (2022). Digital E. coli counter: A microfluidics and computer vision-based dnazyme method for the isolation and specific detection of E. coli from water samples. Biosensors, 12(1), Article 34. https://doi.org/10.3390/bios12010034

• Roy, A. M., Bhaduri, J., Kumar, T., & Raj, K. (2023). WilDect-YOLO: An efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detection. Ecological Informatics, 75, Article 101919. https://doi.org/10.1016/j.ecoinf.2022.101919

• Ruiz-Santaquiteria, J., Bueno, G., Deniz, O., Vallez, N., & Cristobal, G. (2020). Semantic versus instance segmentation in microscopic algae detection. Engineering Applications of Artificial Intelligence, 87, Article 103271. https://doi.org/10.1016/j.engappai.2019.103271

• Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., & Fei-Fei, Li. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115, 211-252. https://doi.org/10.1007/s11263-015-0816-y

• Salman, S., Shafait, F., Mian, M., Shortis, M. R., Khurshid, K., Ulges, A., Schwanecke, U. (2020). Automatic fish detection in underwater videos by a deep neuralnetwork-based hybrid motionlearning system. ICES Journal of Marine Science, 77(4), 1295-12307. https://doi.org/10.1093/icesjms/fsz025

• Shapiro, L. G., & Stockman, G. C. (2001). Computer vision. Prentice Hall.

• Sharma, C., Isha, I., & Vashish, V. (2021). Water quality estimation using computer vision in UAV. In 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (pp. 448-453). IEEE Publishing. https://doi.org/10.1109/Confluence51648.2021.9377082

• Sheehan, E. V., Bridger, D., Nancollas, S. J., & Pittman, S. J. (2020). PelagiCam: A novel underwater imaging system with computer vision for semi-automated monitoring of mobile marine fauna at offshore structures. Environmental Monitoring and Assessment, 192, Article 11. https://doi.org/10.1007/s10661-019-7980-4

• Sondhi, Y., Jo, N. J., Alpizar, B., Markee, A., Dansby, H. E., Currea, J. P., Samuel, T. F., Ruiz, C., Barredo, E., Allen, P., DeGennaro, M., Kawahara, A. Y., & Theobald, J. C. (2022). Portable locomotion activity monitor (pLAM): A cost-effective setup for robust activity tracking in small animals. Methods in Ecology and Evolution, 13(4), 805-812. https://doi.org/10.1111/2041-210X.13809

• Stefen, C., Wagner, F., Asztalos, M., Giere, P., Grobe, P., Hiller, M., Hofmann, R., Jähde, M., Lächele, U., Lehmann, T., Ortmann, S., Peters, B., Ruf, I., Schiffmann, C., Thier, N., Unterhitzenberger, G., Vogt, L., Rudolf, M., Wehner, P., & Stuckas, H. (2022). Phenotyping in the era of genomics: MaTrics—A digital character matrix to document mammalian phenotypic traits. Mammalian Biology, 102, 235-249. https://doi.org/10.1007/s42991-021-00192-5

• Torres-Sánchez, J., Escolà, A., de Castro, A. I., López-Granados, F., Rosell-Polo, J. R., Sebé, F., Jiménez-Brenes, F. M., Sanz, R., Gregorio, E., & Peña, J. M. (2023). Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters—Part 2: Comparison for different crops and training systems. Computers and Electronics in Agriculture, 212, Article 108083. https://doi.org/10.1016/j.compag.2023.108083

• Tovstik, E. V., Adamovich, T. A., & Ashikhmina, T. Y. (2019). Identification of sites of mass growth of Heracleum sosnowskyi Manden. using spectral indices according to Sentinel-2 images. Theoretical and Applied Ecology, 3, 34-40. https://doi.org/10.25750/1995-4301-2019-3-034-040

• Vaganov, A.V., Krotova, O.S., & Khvorova, L.A. (2021). Processing and analysis of botanical micro- and macroobjects using computer vision technologies. Journal of Physics: Conference Series, 2142(1), Article 012003. https://doi.org/10.1088/1742-6596/2142/1/012003

• Wägele, J. W., Bodesheim, P., Bourlat, S. J., Denzler, J., Diepenbroek, M., Fonseca, V., Frommolt, K. H., Geiger, M. F., Gemeinholzer, B., Glockner, F. O., Haucke, T., Kirse, A., Kolpin, A., Kostadinov, I., Kuhl, H. S., Kurth, F., Lasseck, M., Liedke, S., Losch, F., Muller, S., Petrovskaya, N., Piotrowski, K., Radig, B., Scherber, C., Schoppmann, L., Schulz, J., Steinhage, V., Tschan, G. F., Vautz, W., Velotto, D., Weigend, M., & Wildermann, S. (2022). Towards a multisensor station for automated biodiversity monitoring. Basic and Applied Ecology, 59, 105-138. https://doi.org/10.1016/j.baae.2022.01.003

• Weinstein, B. G. (2018). A computer vision for animal ecology. Journal of Animal Ecology, 87(3), 533-545. https://doi.org/10.1111/1365-2656.12780

• Wilson, R. J., Siqueira, A. F., Brooks, S. J., Price, B. W., Simon, L. M., Walt, S. J., & Fenberg, P. B. (2021). Applying computer vision to digitised natural history collections for climate change research: Temperature-size responses in British butterflies. Methods in Ecology and Evolution, 14(2), 372-384. https://doi.org/10.1111/2041-210X.13844

• Xia, F., Xie, X., Wang, Z., Jin, S., Yan, K., & Ji, Z. (2022). A novel computational framework for precision diagnosis and subtype discovery of plant with lesion. Frontiers in Plant Science, 12, Article 789630. https://doi.org/10.3389/fpls.2021.789630

• Xiong, Y., Zhu, M., Li, Y., Huang, K., Chen, Y., & Liao, J. (2022). Recognition of geothermal surface manifestations: A comparison of machine learning and deep learning. Energies, 15(8), Article 2913. https://doi.org/10.3390/en15082913

• Yang, B., He, Y., & Yin, H. (2020). Research on data analysis and quality control based on P control chart. In 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) (pp. 1098-1102). IEEE Publishing. https://doi.org/10.1109/CISP-BMEI51763.2020.9263537

• Yang, L., Driscol, J., Sarigai, S., Wu, Q., Lippitt, C. D., & Morgan, M. (2022). Towards synoptic water monitoring systems: A review of AI methods for automating water body detection and water quality monitoring using remote sensing. Sensors, 22(6), Article 2416. https://doi.org/10.3390/s22062416

• Yuan, W., Choi, D., & Bolkas, D. (2022). GNSS-IMU-assisted colored ICP for UAV-LiDAR point cloud registration of peach trees. Computers and Electronics in Agriculture, 197, Article 106966. https://doi.org/10.1016/j.compag.2022.106966

• Zaroudi, F., Nasihatkon, B., Hosseinzadeh, R., Fakhar, A. R., & Seidi, S. (2023). Miniaturized on-chip electromembrane extraction with QR code-based red-green-blue analysis using a customized Android application for copper determination in environmental and food samples. Food Chemistry, 414, Article 135667. https://doi.org/10.1016/j.foodchem.2023.135667

• Zhang, J., Li, C., Rahaman, M., Yao, Y., Ma, P., Zhang, J., Zhao, X., Jiang, T., & Grzegorzek, M. (2022). A comprehensive review of image analysis methods for microorganism counting: From classical image processing to deep learning approach. Artificial Intelligence Review, 55, 2875-2944. https://doi.org/10.1007/s10462-021-10082-4

• Zhang, W., Miao, J., Li, M., & Liu, H. (2022). Research on underwater image registration method based on MSRCR and SIFT. In International Conference on Computer Vision and Pattern Analysis (ICCPA 2021) (Vol. 12158, pp. 74-80). SPIE Publishing. https://doi.org/10.1117/12.2626893

• Zhou, J., Yang, T., Chu, W., & Zhang, W. (2022). Underwater image restoration via backscatter pixel prior and color compensation. Engineering Applications of Artificial Intelligence, 111, Article 104785. https://doi.org/10.1016/j.engappai.2022.104785