Open Access
How to translate text using browser tools
3 October 2022 Automated Size Measurements of Halyomorpha halys (Stål) (Heteroptera: Pentatomidae) with Simple Image-Based Methodology
Amy Tabb, Johanna E. Elsensohn, Tracy C. Leskey
Author Affiliations +
Abstract

Measurements of insect external morphology are important to understanding species ecology, biology, and systematics. Precise measurement of width and length can be laborious and time-consuming. Here, we explored an automated approach to insect measurement using an image-based method with adult Halyomorpha halys (Stål) (Heteroptera: Pentatomidae) as a model. Wild-collected H. halys length and width were measured from their ventral side manually with calipers and compared with automated measurements derived from smartphone images. The comparison between manual methods and our automated approach yielded R2 values of 0.69 and 0.84 for width and length, respectively, indicating that an image-based measurement method has potential for usage in future entomological studies. Automated sizing using this image-based method has the potential to reduce the time and costs of performing insect morphological measurements.

Las mediciones de la morfología externa de los insectos son importantes para comprender la ecología, la biología y la sistemática de las especies. La medición precisa de ancho y largo puede ser laboriosa y llevar mucho tiempo. Aquí, exploramos un enfoque automatizado para la medición de insectos utilizando un método basado en imágenes con adultos de Halyomorpha halys (Stål) (Heteroptera: Pentatomidae) co-mo modelo. Se midieron la longitud y el ancho de H. halys recolectados en la naturaleza desde su lado ventral manualmente con calibradores y se compararon con mediciones automatizadas derivadas de imágenes de teléfonos celulares inteligentes. La comparación entre los métodos manuales y nuestro enfoque automatizado arrojó valores R2 de 0,69 y 0,84 para el ancho y el largo, respectivamente, lo que indica que un método de medición basado en imágenes tiene potencial para su uso en futuros estudios entomológicos. El sistema automatizado de determinar dimensiones basado en imágenes tiene el potencial de reducir el tiempo y los costos de realizar mediciones morfológicas de insectos.

Insect size is affected by genetic and environmental factors such as local resource availability, microclimates, contamination, or even incipient speciation. An important task of many insect morphological studies involves documenting this variability among individuals or populations of a particular species. Preserved museum specimens enable visual and, potentially, genetic studies in perpetuity, and morphological cross-sectional studies provide quantitative and qualitative data from 1 or more time points or locations. However, existing methods for insect measurement often are labor- and time-intensive, and rely on the experience and precision of the individual performing measurements. Additionally, standardized methods for insect measurements often do not exist, such as the case for proboscis length in Lepidoptera (Vajna et al. 2021) leading to difficulty in comparing results among studies. Recently, new techniques have emerged to improve accuracy of size measurements including high resolution digital photography which have been used to measure live and dead Phormia regina (Meigen) (Diptera: Calliphoridae) larvae (Bourne et al. 2019). Moreover, automated image-based approaches have been applied to improve precision and decrease error in agricultural pest management (Miranda et al. 2014; Roosjen et al. 2020), species classification (Martineau et al. 2017), and biomass estimation (Ärje et al. 2020). Image-based measurement methods also have been adapted for plant phenotyping (Pieruschka & Schurr 2019; Feldmann et al. 2020), and pathogen detection (Ngugi et al. 2021).

Given the tremendous improvements to digital camera technology over the last decade and the continued development of computer imaging software, we sought to create an easily accessible image-based tool that could be used to provide real-time size measurements in a fraction of the time of manual measurement methods. We used the brown marmorated stink bug, Halyomorpha halys (Stål) (Heteroptera: Pentatomidae) as a model because of its relevance to the agricultural community (Leskey & Nielsen 2018) and its availability. Here we evaluated a smartphone image-based method for size measurement of H. halys, where the user-acquired, digital images of the insect placed on a surface with a calibration pattern are processed by computer algorithms to obtain size measurements. We compared these estimated sizes to manually obtained measurements.

Adult H. halys were collected from Ailanthus altissima (Mill.) Swingle (Simaroubaceae) trees in Kearneysville, West Virginia, USA. Over 2 d, 166 individuals were captured and held at 0 °C for 20 min to quickly kill them, and then brought back to room temperature. Insects were placed in individual plastic dishes (Falcon, Corning Inc., Durham, North Carolina, USA) and the following data was collected for each specimen: ventral length (mm, tip of maxilla to end of last abdominal segment), and pronotal width (mm, tip to tip of the widest part of the thorax on ventral side; Fig. 1). We used digital calipers (Kobalt #293883; Kobalt, Moresville, North Carolina, USA; Mitutoyo #1002389; Mitutoyo, Takatsu-ku, Kanagawa, Japan) for length measurements. The time to measure and record data from insects was approximately 45 s per insect, with 2 people, 1 to measure, and another to record, for a total time of 2 h per person or 4 h total.

Fig. 1.

Illustration of the image-based measurement method. (A) Image of brown marmorated stink bug used for automated size estimation, shown after CAmera aS Scanner processing. (B) Image of segmented insect, whereby the body is coded as white and the antennae and legs are shown as gray. Gray sections are excluded from analysis. The red line represents the pronotal width measurement, the purple line denotes the ventral length measurement.

img-z1-9_262.jpg

Following these manual measurements, images of H. halys were acquired using an iPhone 8 mobile phone camera (Apple, Inc., Cupertino, California, USA). Each specimen was placed ventral side up on a printed calibration pattern as in Figure 1. This calibration pattern has arUco patterns (Garrido-Jurado et al. 2014), a type of barcode resembling QR codes, at its border. The arUco patterns were 11.76 mm2. CAmera aS Scanner (Tabb et al. 2019; Tabb 2020), a method for calibrating cameras, was used to convert the image from pixels with an unknown physical size to pixels with a per mm scaling, similar to the type of images given by a flatbed scanner (the scaling is 10 pixels per mm for this study). CAmera aS Scanner requires that a calibration pattern is in the scene and camera meta-data is available. After CAmera aS Scanner was applied to the images, the insect was segmented from the image background, morphological operators like dilation and erosion were performed to remove insect antennae and legs from the segmented body image, and length (mm) and width (mm) were computed. The calibration, segmentation, and measurement steps were performed for all 166 specimens sequentially for an entire directory of image samples by a program written in C++. The time to acquire images included 10 s per insect to position each on the calibration pattern; the C++ program runtime was 2.4 s per insect and 6.4 min for all specimens, for a total time of about 33.4 min to measure all specimens using this automated approach.

Manual measurements of insect size were compared with the sizes computed with the image-based method, as shown in Figure 2. Linear regression was computed with R vers. 4.1.1 (R Core Team 2021), stats::lm function, and graphs were generated with Octave vers. 4.2.1 (Eaton et al. 2017). For the pronotum width, manual and image-based measurements averaged 8.05 mm ± 0.06 SEM (Standard Error of the Mean) and 7.60 mm ± 0.05 SEM, respectively. The relationship between these 2 measurements based on the linear regression was reasonably strong with a R2 value of 0.69 (df = 164; P < 0.01). Computing the insect width using image-based methodology occasionally yielded errors in the segmentation step, including: (1) classifying all pixels around the insect's body correctly, or (2) removing the insect's legs from the region to be measured, and these segmentation errors contributed to the difference between manual and image measurements. Possible strategies to reduce errors include experimenting with lighting conditions, as well as changing the segmentation algorithm to use deep learning (He et al. 2017).

The ventral manual measurements yielded average lengths of 14.11 mm ± 0.08 SEM. Using the image-based method, length measurements averaged 14.44 mm ± 0.09 SEM. The comparison of manually measured versus computed insect lengths yielded R2 values of 0.84 (df = 164; P < 0.01). Again, segmentation errors played some role in errors when computing insect length, but in this case, manual and image-based methods yielded more similar measurements.

These results indicate that an image-based approach has potential for automated insect size measurement for large insects such as H. halys. However, we identified several potential constraints in this preliminary study. We used a smartphone camera (see above) because they are widely available and accessible, but the aperture size is quite large (f/1.8) and a different camera and lens combination may provide more robust results for smaller insects. We also used a straightforward background segmentation technique, but are considering alternate approaches such as deep learning (He et al. 2017) given that the segmentation step produced localization and measurement errors. The errors were found by inspecting intermediate stages of the output and visually judging whether the program was classifying pixels as belonging to the insect. These topics are the focus of future work.

Fig. 2.

Relationship between sizes of Halyomorpha halys from manually measured versus image-computed sizes for (A) ventral width and (B) ventral length.

img-z2-8_262.jpg

Manual methods also are prone to user bias and measurement error. Indeed, even our own manually generated results, though standard for entomological studies, may have systematic errors that remain undetected. An imaging tool such as the one described in this paper has the potential to offer better estimations of insect size measurements more quickly and accurately than manual measurement. At this preliminary stage, the tool is currently best suited for insect species that are dorsoventrally flattened, such as Pentatomidae, Cimicidae, or Lepidopteran families. Future work will explore the tool's applicability for other insect body types.

Mention of a concept, idea, trade name, or commercial product in this publication is solely for the purpose of providing specific information and does not imply recommendation or endorsement by the US Department of Agriculture. The USDA is an equal opportunity employer. This work was funded, in part, by USDA-ARS Project 8080-21000-032-00-D.

References Cited

1.

Ärje J, Melvad C, Jeppesen M, Madsen SA, Raitoharju J, Rasmussen MS, Iosifidis A, Tirronen V, Gabbouj M, Meissner K, Høye TT. 2020. Automatic image-based identification and biomass estimation of invertebrates. Methods in Ecology and Evolution 11: 922–931. Google Scholar

2.

Bourne DR, Kyle CJ, Leblanc HN, Beresford D. 2019. Technical note: a rapid, noninvasive method for measuring live or preserved insect specimens using digital image analysis. Forensic Science International: Synergy 1: 140–145. Google Scholar

3.

Eaton JW, Bateman D, Hauberg S, Wehbring R. 2017. GNU Octave version 4.2.1 manual: a high-level interactive language for numerical computations.  https://www.gnu.org/software/octave/doc/v4.2.1/ (last accessed 10 Apr 2022). Google Scholar

4.

Feldmann MJ, Hardigan MA, Famula RA, López CM, Tabb A, Cole GS, Knapp SJ. 2020. Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry. GigaScience 9: giaa030. https://doi.org/10.1093/gigascience/giaa030 Google Scholar

5.

Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47: 2280–2292. Google Scholar

6.

He K, Gkioxari G, Dollár P, Girshick R. 2017. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988. https://doi.org/10.1109/iccv.2017.322 Google Scholar

7.

Leskey TC, Nielsen AL. 2018. The impact of the invasive brown marmorated stink bug in North America and Europe: history, biology, ecology, and management. Annual Review of Entomology 63: 599–618. Google Scholar

8.

Martineau M, Conte D, Raveaux R, Arnault I, Munier D, Venturini G. 2017. A survey on image-based insect classification. Pattern Recognition 65: 273–284. Google Scholar

9.

Miranda JL, Gerardo BD, Tanguilig III BT. 2014. Pest detection and extraction using image processing techniques. International Journal of Computer and Communication Engineering 3: 189–192. Google Scholar

10.

Ngugi LC, Abelwahab M, Abo-Zahhad M. 2021. Recent advances in image processing techniques for automated leaf pest and disease recognition – a review. Information Processing in Agriculture 8: 27–51. Google Scholar

11.

Pieruschka R, Schurr U. 2019. Plant phenotyping: past, present, and future. Plant Phenomics 2019.  https://doi.org/10.34133/2019/7507131  Google Scholar

12.

R Core Team. 2021. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.  https://www.Rproject.org/ (last accessed 10 Apr 2022). Google Scholar

13.

Roosjen PP, Kellenberger B, Kooistra L, Green DR, Fahrentrapp J. 2020. Deep learning for automated detection of Drosophila suzukii: potential for UAV-based monitoring. Pest Management Science 76: 2994–3002. Google Scholar

14.

Tabb A. 2020. Data and code from: using cameras for precise measurement of two-dimensional plant features: CASS. Zenodo. https://doi.org/10.5281/zeno-do.3677473 (last accessed 10 Mar 2022). Google Scholar

15.

Tabb A, Holguín GA, Naegele R. 2019. Using cameras for precise measurement of two-dimensional plant features: CASS. arXiv:1904.13187[cs.CV] doi. org/10.48550/arxiv.1904.13187 (last accessed 10 Apr 2022). Google Scholar

16.

Vajna F, Kis J, Szigeti V. 2021. Measuring proboscis length in Lepidoptera: a review. Zoomorphology 140: 1–15. Google Scholar
Amy Tabb, Johanna E. Elsensohn, and Tracy C. Leskey "Automated Size Measurements of Halyomorpha halys (Stål) (Heteroptera: Pentatomidae) with Simple Image-Based Methodology," Florida Entomologist 105(3), 262-264, (3 October 2022). https://doi.org/10.1653/024.105.0314
Published: 3 October 2022
KEYWORDS
brown marmorated stink bug
chinche apestosa marrón marmorada
computer vision
morfología
morphology
visión por computador
Back to Top