Field estimation of maize plant height at jointing stage using an RGB-D camera

Ruicheng Qiu ,Mn Zhng ,Yong He,*

a College of Biosystem Engineering and Food Science,Zhejiang University,Hangzhou 310058,Zhejiang,China

b Key Laboratory of Modern Precision Agriculture System Integration Research,Ministry of Education,China Agricultural University,Beijing 100083,China

Keywords:Maize plant height Kinect Maize central area Point cloud data Image processing

ABSTRACT Plant height can be used for assessing plant vigor and predicting biomass and yield.Manual measurement of plant height is time-consuming and labor-intensive.We describe a method for measuring maize plant height using an RGB-D camera that captures a color image and depth information of plants under field conditions.The color image was first processed to locate its central area using the S component in HSV color space and the Density-Based Spatial Clustering of Applications with Noise algorithm.Testing showed that the central areas of plants could be accurately located.The point cloud data were then clustered and the plant was extracted based on the located central area.The point cloud data were further processed to generate skeletons,whose end points were detected and used to extract the highest points of the central leaves.Finally,the height differences between the ground and the highest points of the central leaves were calculated to determine plant heights.The coefficients of determination for plant heights manually measured and estimated by the proposed approach were all greater than 0.95.The method can effectively extract the plant from overlapping leaves and estimate its plant height.The proposed method may facilitate maize height measurement and monitoring under field conditions.

1.Introduction

Monitoring of plants can help agronomists and breeders estimate plant growth and conduct field management.Many morphological and physiological traits of plants are monitored during their growth.In particular,plant height is extensively used for assessing field plant vigor,estimating plant biomass,and predicting crop yield.Monitoring plant height during the early stages of growth is beneficial for evaluating the growth rate and determining crop quality.Conventional manual measurement of plant height with a measuring tape is labor-intensive and time-consuming,making it troublesome to acquire plant height on a large scale.To measure plant height with high efficiency and precision,researchers have explored the use of various sensing techniques.In general,the‘‘difference method” of calculating the height differences between the ground and plant canopy is regarded as the most effective heightmeasurement method [1].For this purpose,three-dimensional(3D)has been shown to be superior to two-dimensional(2D)imaging.For plant height measurement,3D sensors or systems that are frequently used are laser sensors,stereo vision systems,ultrasonic sensors,and range cameras [2].

Laser or light detection and ranging (LiDAR) sensors are based mostly on time-of-flight measurement for acquiring 3D information and have been extensively implemented in agriculture.The sensors can be installed on ground-based or unmanned aerial vehicle(UAV)platforms to collect the 3D structure of plants and extract their heights.A line laser scanner was tested to scan rice in the field and estimate plant height by extracting plant height changes[3].A 2D LiDAR was installed on a tractor driven along rows to acquire a dense 3D model of cotton plants and extract their heights[4].The system achieved average coefficient of determination (R2)values of 0.98 for cotton height estimation.

The installed heights and angles of LiDAR affect plant height measurement [5].When laser sensors are installed on UAV platforms,it is feasible to acquire a crop surface model(CSM)and digital surface model(DSM)for predicting plant height[6].Some laser sensors can acquire point cloud data(PCD)of plants with high precision.Terrestrial laser scanning acquired the 3D structure of soybean in a large field and extracted the soybean heights to monitor their growth[7].Laser sensors have been reported[8]to be able to measure plant height by collecting multiple data.However,these sensors are costly and PCD processing is challenging,limiting their practical application.RPLiDAR [9] is a low-cost variant of LiDAR sensor.Although it can acquire 3D plant structure,the sensor is difficult to use under field conditions.

A stereo vision system uses a single camera or multiple cameras to collect color images with multiple views and registers these images to generate 3D data.For plant height measurement,depth information of wheat was acquired with a stereo vision system consisting of two cameras and wheat heights at various growth stages were successfully measured [10].Many researchers have applied the structure-from-motion (SfM) algorithm to generate 3D information of plants.In these studies,cameras were mounted on the field-based or UAV platforms to collect a series of overlapping images of plants.A single camera was moved along the row to collect the color images and build a plant row 3D model using the SfM algorithm,and estimate plant heights[11].A UAV platform can acquire plant color images and then generate a DSM.Plant height is calculated by extracting the upper boundary of the canopy and ground [12,13].A stereo vision system can not only collect the color information of plants but also generate a 3D model.But the algorithm is complicated,making it takes much time to acquire 3D information.In addition,the quality of 3D data generated by the SfM algorithm is strongly dependent on image matching.

Several researchers have used ultrasonic sensors to measure plant height.A plant measurement system including three ultrasonic sensors and a real-time kinematics global positioning system was developed to estimate wild blueberry heights and calculate their distributions [14].Ultrasonic sensors were installed on a ground platform to measure maize height,but the results were not satisfactory [15].The measurement accuracy was improved by application of regression kriging to combine the heights measured by ground-based ultrasonic sensor and UAV imagery [16].But data from an ultrasonic sensor are usually noisy and its resolution is low,with measurement accuracy sensitive to leaf motion with wind.

In recent years,a consumer-grade RGB-D camera,Kinect,produced by Microsoft(Microsoft,Redmond,WA,USA),has been used for plant phenotyping.Kinect can simultaneously acquire color and depth information with low cost and simple operation,as an alternative to laser sensors.Kinect has been applied for measurement of plant phenotypes such as height [17],canopy size [18],and 3D structure [19].In indoor environments,the 3D structure of maize crop rows was reconstructed [20] and the height,diameter,and fruit volume of cauliflower were measured[21]with a Kinect camera.The camera was used to measure cotton height in the field[17],with accuracies above 92%.

Current research efforts about plant heights are focused mostly on measuring the plant height of each plot or group.Little attention is paid to the measurement of individual plants,especially when the leaves of different plants overlap.In this case,it is challenging to recognize each plant and extract its height.An optimized watershed algorithm was applied to acquire the plant central areas [22],but the watershed algorithm was sensitive to light conditions.Skeleton extraction and corner detection of plants using binary images have been conducted [23,24] to recognize each maize plant.This strategy is feasible for plants at the emergence and three-leaf stage.However,it is difficult to handle the situation when leaves of adjacent plants cause occlusion or overlapping.Bao et al.[25]used a Kinect camera to collect 3D data of plants from the side view and extracted 3D skeletons from PCD.The stems and leaves were separated by extracting and fitting the skeleton points.Plant heights were calculated based on the fitted stem lines.The R2s for maize height were respectively 0.96 and 0.83 before and after flowering stages.However,they measured the plant heights using side-view depth data,requiring a row spacing of 1.5 m which was much wider than common conditions.Besides,some target leaves might be shielded in the side-view images.The detection accuracy of stem lines was essential for height measurement,but it was affected by leaf orientation.Although the deep learning technique [26] has been applied to accurate classification of overlapping plant leaves,many labeled samples are needed,incurring much labor and cost.Simpler and effective methods for plant height measurement are needed.

In this study,we focused on the measurement of maize plant height at the jointing stage.The jointing stage is a critical period for maize.During this stage,plants grow rapidly with a high nutritional requirement.Plant growth at this stage affects the final yield.At this stage,some leaves from adjacent plants overlap.Maize plant height can be obtained by measuring the canopy height or the whorl height before flowering.Maize central leaf(visible leaf from whorl) height has a strong correlation with maize plant height[25].Using a Kinect camera to acquire color and depth information of maize plants in the field,the objectives were to (1)develop an automatic maize central area localization method to separate the target plant to be measured and(2)extract the central leaf from overlapping leaves and calculate the height of the plant central area.

2.Material and methods

2.1.Data collection

Two common maize cultivars,Jingnongke 728(a compact type)and Nongda 84 (a flat type),were cultivated at the Shangzhuang Station,China Agricultural University (40°14′N,116°19′E).The plant row spacing was 90 cm and the spacing of adjacent plants was 30 cm.

Field experiments were conducted in June of 2018 and 2019.Plants were grown to the jointing stage,occurring on the 30th day after sowing.A Kinect v2 camera was used to collect color images and depth data of plants.The camera can collect color images at 1920 × 1080-pixel resolution and the PCD of plants.A software application was developed using the Kinect application programming interface to save data,including the color image and the registration files linking the Kinect color and depth cameras.The camera was installed on a field platform and pointed vertically towards the plants,as shown in Fig.S1a.Thus,the z axis of the camera was approximatively perpendicular to the ground(Fig.S1b).The distance between the camera and the plants was at least 0.8 m.Kinect is sensitive to environmental conditions;strong light and wind reduce the quality of collected data [17].To acquire adequate data,the experiments were conducted after 18:00.After the platform was positioned manually to place the camera above the plant central area,data were collected under windless conditions.

After data collection,the distances between the ground and the highest points of central leaves(Fig.S1b)were measured manually with a steel ruler.Respectively 52 and 38 samples for Jingnongke 728 and 44 and 43 for Nongda 84 were collected in 2018 and 2019.

2.2.Localization of maize central area

An initial region of interest (ROI) was defined using the positions of plants in the color images (Fig.1A),which contained the whole target plant and leaves of the adjacent plants.The Cg feature of the color image [27] was calculated to generate the gray image of the plant(Fig.1B).The feature is described in detail by Qiu et al.[19].Sequentially,the image was processed using OTSU’s algorithm [28] and a binary image of the plant was generated with the pixel values 1 for plant and 0 for others(Fig.1C).Because parts of some leaves were not wide,they could be separated into two or more small areas after segmentation.The results were further processed by a dilating operation.Then regions whose areas were smaller than a threshold were removed.To reduce computation and interference from the ground,the column pixel values of the generated binary image were calculated to detect the sides of the plant row and further reduce the ranges of ROI.Finally,the plants were precisely extracted (Fig.1D).

Fig.1.Workflow of plant extraction from the color image.

Next,the S component in Hue,Saturation,Value (HSV) color space was selected to generate a gray-level image of the extracted plant for use in recognizing the central area of the plant.The central areas were shaded by surrounding leaves.In the gray image,the pixel values of central areas were higher than those of leaves and ground,as displayed in Fig.2B.A threshold value for segmentation was set to acquire candidate central areas.Testing showed that when the threshold value was set to 0.95,most central areas were accurately segmented.But the resulting binary image also contained some scattered points.An ‘‘open” operation was applied to the segmentation results to remove these nontarget points.The values of candidate central area pixels and other pixels were 1 and 0,respectively.Some pixels of the candidate central areas assigned to shadowed areas of the ground in consequence of the uneven surface.The Cg feature of the plant was calculated to obtain its gray level image (Fig.2C),and binary images were generated using OTSU’s algorithm to extract the plant (Fig.2E).In this manner,only pixels whose values were 1 in both the binary images of the plant (Fig.2E) and candidate central areas (Fig.2D) were retained.

To automatically recognize the central areas of different plants,the retained pixels(blue pixels in Fig.2F)were clustered using the Density-Based Spatial Clustering of Applications with Noise(DBSCAN)algorithm[29].The minimum enclosed rectangle of pixels in a cluster was established to locate the central area of each plant (blue rectangles in Fig.2G).

In general,the measured plant was in the middle position of the image.The minimum enclosed rectangle whose center point was nearest to the center point of the extracted image (Fig.2A) was selected as the central area of the target plant to be measured.The PCD of the central area of the plant was generated based on the located minimum enclosed rectangle.

Fig.2.Workflow of the central area localization of maize plants.

2.3.Maize height measurement

2.3.1.Point cloud data extraction

The raw PCD were generated according to the ranges of the extracted images,but they contained many noise points.The Knearest algorithm [30] was used to remove scattered points.The mean distances d of all points to their k nearest neighbors were calculated.The mean μ and standard deviation σ of nearest-neighbor distances were used to obtain the threshold μ±α×σ.Points with d value above a threshold were removed.In this study,the number k of nearest neighbor points and the value of α were set to 4 and 1,respectively.For the example of Fig.3,the number of PCD decreased from 52,739 to 51,281 after preprocessing.Then the ground and plant were separated using the Random Sample Consensus(RANSAC)algorithm[31]to fit a plane.The maximum point to the plane distance for plane fitting was assigned as 10 cm,considering the uneven surface of the ground.The PCD of the plane and other data were saved as the ground and plant,respectively,as shown in Fig.3A.The PCD of the central area of the plant was also extracted based on the coordinates of the selected minimum enclosed rectangle in the color image.Next,the DBSCAN algorithm was applied to cluster the PCD of the plant into several areas by height differences among leaves (Fig.3B).The minimum number of neighbors for each point and the neighborhood search radius around the point were assigned as 10 and 5 cm,considering the data resolution.If the point cloud contained at least 10 points within 5 cm,they were treated as belonging to the same cluster.The PCD of the plant central area contributed to the accurate extraction of PCD containing central leaves.The main sources of interference for the central leaf extraction arise from its two nearest leaves and the leaves of adjacent plants.The area containing the central leaf had the most PCD of the central area of the target plant generated in section 2.2 and was selected for measurement,as shown in Fig.3C.

2.3.2.Detection of the highest point of the central leaf

To measure the height of the maize central area,it is necessary to identify the highest point of the central leaf.The Laplacianbased method [32,33] was used to generate skeletons of the PCD of the selected area.Following Miao et al.[33],16 nearest neighbors of points were identified.The skeletons were represented by many key points(Fig.4A,B).Some end points of the skeletons corresponded to leaf tips and could be used to identify the central leaf.

The connectivity of each skeleton point was estimated,and the end point was connected with only one skeleton point.Because the central leaf is emerging and other leaves are fully expanded,in general,the leaf tip of the central leaf is closest to the plant central point relative to other leaf tips.Some lines were generated to connect the end points of the central leaf and other leaves with the central point of the plant.The branching angles between the connecting lines and the line that is vertical to the ground plane(namely the z axis of the Kinect camera)were calculated and compared.The branching angle corresponding to the central leaf was the smallest.As shown in Fig.4C,θ1is smaller than θ2and θ3.The point with the lowest height was selected from the PCD of the central area of the plant as the central point of the plant and was combined with the end points of maize skeletons to calculate the branching angles by Eq.(1).

where θiis the branching angle between the end point i and the central point;x0,y0,and z0are the coordinates of the central point;and xi,yi,and ziare the coordinates of the end point i.

The end point with the smallest corresponding branching angle was treated as the leaf tip of the central leaf.But the skeletons show only the basic structure of the plant and do not display the leaves in detail,in particular their margins.To accurately identify the highest point of the central leaf,the selected end point was set as the center and its surrounding points within 5 cm were searched and compared.Among these,the point with the maximum height was finally selected as the highest point of the central leaf.

2.3.3.Performance evaluation

Although the PCD of the ground was fitted using the RANSAC algorithm to generate a plane,the plane is not suitable for plant height measurement because the surface of the ground is uneven.To reduce the measuring errors of plant height,the points surrounding the highest point of the central leaf in the x-y plane of the Kinect camera were searched.A square whose center was the highest point and width was 20 cm was defined.Histogram statistics for all points within the square were performed to characterize their height(z values)distributions.The width of each bin was set to 2 mm.The bin containing the largest number of PCD was selected and the mean of its left and right edge values was assigned as the height of the ground.Finally,the plant height was obtained by calculating the height difference between the highest point of the central leaf and the ground.

The methods described in sections 2.2 and 2.3 were implemented using Matlab 2020b (MathWorks,Natick,MA,America).A desktop computer with a 3.4-GHz AMD R5-2600 CPU,NVIDIA GTX 1070,8 Gb RAM,and Windows 10 64-bit system was used to test the proposed methods.The heights of plant samples were measured to evaluate the performance of the proposed method and the estimated heights were compared with the manual measurements.The statistics R2and root mean squared error (RMSE),as described in the following equations,were calculated as measures of accuracy:

where n is the number of plant samples,tiis the manually measured height,miis the height estimated by the proposed method,andis the mean ti.

3.Results

3.1.Performance of maize central area localization under several conditions

The maize samples collected in 2018 and 2019 were used for testing the performance of the proposed method for plant central area localization.The method showed satisfactory accuracy for both Jingnongke 728 and Nongda 84.The central areas of all target plants were accurately detected.The average processing time of color images required for central area localization was about 0.35 s.Representative samples are shown in Figs.S2 and S3.The samples were collected under stronger light conditions in 2019 than in 2018.First,the plant central areas in the S gray images were more distinct than other areas irrespective of ambient light level.Second,even when the image contained two or more plants,the central area of each plant was accurately detected.Third,the compact or flat structure of the plant had no effect on central area localization.Because the method does not require obtaining the skeletons of plant leaves,the leaves did not affect localization.When the leaves of adjacent plants overlapped,the central area of the plant could also be detected as long as it was not completely covered by other leaves (Figs.S2B and C,S3B and C).

Fig.3.PCD extraction of the central area of maize plant.x,y,and z are the coordinate system of Kinect camera.

Fig.4.Extracted skeletons of a maize plant and illustration of the branching angles for end points.(A)Front view of the skeletons.(B)Side view of the skeletons.(C)Branching angles for end points.z is the coordinate system of Kinect camera.

The PCD of maize central area was generated based on the central area localization results.They were further processed to select the lowest point as the central point of maize.

3.2.Performance of maize height measurement

3.2.1.Classification of the maize PCD and extraction of the central leaf

In a cumulative tale, we find little plot but a lot of rhythm and repetition, such as the gingerbread man s chase and accompanying rhyme as he taunts14 his pursuers

The classification of the PCD can be summarized into several situations,described below.

First,the plant to be measured was isolated and did not intersect with other plants.In this case,the central leaf could be extracted separately or classified with its two nearest leaves or the whole plant (Fig.5A1,B1).

Second,there were intersecting leaves between adjacent maize plants.When the spaces between the intersecting leaves were large,the central leaf was separated from the neighboring plants(Fig.5A2,B2);otherwise it might be grouped with the intersecting leaves (Fig.5A3,B3).

Third,the central areas of plants would sometimes be partially covered or disturbed by the leaves of neighboring plants.If the neighboring leaves were adjacent to the target plant,they and the central leaf would be clustered together (Fig.S4A1,B1,A2,and B2).The leaves that were closest to the central leaf might be clustered into multiple parts owing to the interferences of neighboring plants,so that only portions of the leaves were assigned to one cluster with the central leaf (Fig.S4A3,B3,A4,and B4).

Fig.5.Classification,extraction,and skeletonization results of the PCD of isolated or overlapping maize plants.(A) Color images of plants,(B) Classification of the PCD of plants in the top view,with the PCD in different clusters presented with different colors,(C)Extracted PCD of target plants,with the PCD of plants and central points shown in green and red,respectively,and (D) Skeletons of the extracted PCD of plants in the front view.x,y,and z are the coordinate system of Kinect camera.

Finally,some plant leaves were curled and twisted out of shape(as shown in Figs.S5A1 and A2).Because for these leaves,the connectivity of their PCD was low,they were divided into two or more clusters.Then the portion of the twisted leaf was extracted together with the central leaf,as shown in Figs.S5B1 and B2.

Although the PCD of maize plants were classified into various situations,the PCD of the central leaves were all exactly clustered and extracted (Figs.5C,S4C,and S5C) for further processing for plant height measurement.

The skeletons of plants were calculated using the Laplacianbased method to help extract the highest point of the plant central leaf.The skeletons of the samples in Figs.5,S4,and S5 are shown in Figs.5D,S4D,and S5D,respectively.

The skeleton permitted the highest point extraction.The skeleton extraction based on the Laplacian-based method showed different performances for Jingnongke 728 and Nongda 84.The leaves of Jingnongke 728 are more vertically inclined than those of Nongda 84.In general,the lengths of Jingnongke 728 leaves are also greater.These two morphological characteristics allowed the extracted skeletons of Jingnongke 728 to display more details of the plant shape,especially for the central leaves.There were marked differences in leaves between Nongda 84 and Jingnongke 728.Most of the Nongda 84 leaves were bent down,and the spaces between upper and lower leaves were smaller than those of Jingnongke 728.The PCD density of plants was not high,owing to the limitation of the Kinect camera.Only one skeleton was generated at times when there were two leaves nearby.In particular,the central leaf might be merged with a neighboring leaf,and its skeleton could not be extracted or was too short.The skeletons of central leaves for five plants could not be extracted.When the skeleton of the central leaf could be extracted,the nearest neighbors of points were reduced to facilitate skeleton extraction [33].

3.2.3.Selection of the skeleton points of the central leaf

The end points of the plant skeleton were selected based on the connectivity,and the candidate highest point of the central leaf was then confirmed by calculation of the branching angles for all end points.As described in section 3.2.1,the PCD of the central leaves of some plants were extracted under different situations.In this section,they are used as representative samples to show the performance of the method.The extracted candidate highest points of these samples are shown in Figs.6,S6 and S7.

When the target plant was independent or overlapped with neighboring plants,its skeleton was clear.The highest point of the central leaf was extracted without the influence of other leaves(Fig.6B1,B2).The central leaf might intersect the adjacent leaves of neighboring plants,resulting in an irregular skeleton.But the end point of the skeleton of the central leaf could still be selected using our method(Figs.6B3,S6B1,and B2).The PCD of the central leaf sometimes contained partitioned leaves.In this case,the branching angle determined the selection of the highest point.The end point of the central leaf,which had the smallest corresponding branching angle,could be accurately identified and extracted as the candidate highest point (Figs.S6B3,B4,and S7B).The end points of all central leaves whose skeletons were extracted in section 3.2.2 could be exactly selected.

Testing indicated that the candidate highest points of the central leaves could be accurately obtained using our strategy,which was robust to interferences caused by the irregular distribution of leaves,including the target-plant leaves and the leaves of adjacent plants.

Fig.6.Detection results of the central point,candidate highest point,and surrounding points of isolated or overlapping plants.(A)Color images of plants,(B)Selection results of central point(blue points)and candidate highest point of the central leaf(red points)in top and side views,and(C)Search results for the surrounding points(blue points)that were used to extract the highest point of the central leaf in top and side views.

3.2.4.Detection of the highest point of the central leaf

The skeletons represent merely the basic structures of leaves,and were obtained using a kind of thinning operation,so that they lacked leaf boundaries and tips.The surrounding points of the selected skeleton point were searched,and the results are shown in Figs.6C,S6C,and S7C.The highest point of the skeleton was treated as the highest point of the central leaf.This strategy eliminates localization errors caused by skeleton points,even though the skeletons of the central leaf were short.The two leaves that wrap the central leaf do not influence detection results.The spaces between them and the central leaf made the points of the two leaves within a radius of the candidate highest point lower than that of the central leaf,helping to yield the highest point of the central leaf.

3.2.5.Evaluation of plant height measurement

Comparisons of the plant heights estimated by our method with those from manual measurement are shown in Fig.7.The results showed satisfactory accuracy.For Jiongnongke 728,the R2s and RMSEs between the system-derived height and manually measured height were 0.9568,0.9895 and 1.20 cm,1.07 cm in 2018 and 2019,respectively.For Nongda 84,the R2s and RMSEs between the system-derived height and manually measured height were 0.9667,0.9792 and 1.28 cm,1.13 cm in 2018 and 2019,respectively.The average processing time of PCD for height measurement was about 7.31 s.The plant heights estimated by our method show a strong linear relationship with the manually measured heights.

4.Discussion

The results showed that Kinect has high potential for measuring plant height even if leaves overlap.

A plant central-area localization method based on color imaging was proposed to extract target plants,which yielded better results than similar research [22-24],especially when plant leaves overlapped.By comparison,our method can reduce the influences of light conditions and leaves,weather and light did not affect central area localization.The DBSCAN algorithm effectively and automatically recognized and classified the central areas of the plant.In addition,plant central areas can be located by processing the PCD or stereo images of plants [20,34].These studies used depth information of maize to acquire maize central points,but their methods are not suitable for maize at stages beyond V4 (4th leaf).

Fig.7.Regression of manually measured on estimated height.(A) Regression for Jingnongke 728 samples in 2018.(B) Regression for Nongda 84 samples in 2018.(C)Regression for Jingnongke 728 samples in 2019.(D) Regression for Nongda 84 samples in 2019.Red lines are regression lines.Dotted lines are 1:1 lines.

In this study,we obtained estimates of plant height comparable to Bao et al.[25].We used the PCD of plants captured from the top view to realize the height measurement.This sensing strategy can collect plentiful canopy information of plants,is more applicable for common row spacings,and not is affected by leaf orientation.For target plant extraction,plant central area detection using color images is more efficient than stem line detection using PCD.We focused on maize height measurement at the jointing stage.Before flowering,the color characteristics of the central area and the branching angle of the central leaf are almost the same as those in this study,and our approach should still be practicable.After flowering,specific methods based on UAV can be developed to measure maize plant height by detecting tassels [35,36].

We have described a method for maize height measurement based on combining color images with PCD to solve the problem of occlusions caused by leaves from adjacent plants.We applied conventional machine learning and PCD processing methods.The color images were first processed to locate the central area of plants and then the PCD were processed based on the central area and the registration relationship between the color image and the PCD,to extract plant height.As in the study of Miao [30],skeletonization of PCD took most of the time.To streamline this process,supervised learning-based methods can be developed.This study can facilitate the automatic labeling of plant samples and supervised learning-based methods can be explored to directly process the PCD of plants and handle the occlusion problem during plant height measurement.

Some limitations in the proposed method remain to be addressed.First,although the highest points of plant central leaves were extracted from the skeletons of plants,the skeleton points were just the approximate locations of the highest points.The low density of PCD severely weakens the extracted results of skeletons.Although searching the surrounding points within 5 cm of the skeleton point can eliminate or reduce the errors,sometimes small errors are inevitable.Second,fewer ground points surrounding the highest points of the central leaves were collected when the central areas of plants were partly shielded by the leaves of the neighboring maize plants.The positions of plants in the images could lead to different measurements of plant height,owing mainly to the unevenness of the ground,which reduces the accuracy of plant height measurement.Because high accuracy can be achieved when the plant is close to the center of the Kinect depth camera[37],this study measured only plants in the centers of images.

In future,special attention should be given to smoothing,registering,and merging a series of plant color images or PCD to obtain high-quality and high-density PCD of plants in rows.This attention will contribute to the measurement of multiple plant heights and their height distributions.Further methods can be developed to extract the highest point of plants from overlapping leaves and measure the plant height,not just the height of the plant central area.

5.Conclusions

A method for recognizing the central area of a maize plant and measure its height was presented.The S component in the color image of plants was used to extract the candidate central areas of plants.The method accurately located the central areas of plants.The skeletons of plants were extracted to identify the highest point of the plant central area,and the plant height was then calculated.The measurement achieved satisfactory accuracy.The R2s between the estimated heights and manually measured heights of plants were all higher than 0.95 for flat and compact types.We conclude that the method can effectively estimate maize height and assist breeders in monitoring the growth of maize.

CRediT authorship contribution statement

Ruicheng Qiu:Software,Methodology,Writing-original draft.Man Zhang:Writing-review&editing,Funding acquisition.Yong He:Supervision,Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This study was supported by the Key Project of Intergovernmental Collaboration for Science and Technology Innovation under the National Key R&D Plan (2019YFE0103800) and CAU Special Fund to Build World-class University (in disciplines) and Guide Distinctive Development (2021AC006).

Appendix A.Supplementary data

Supplementary data for this article can be found online at https://doi.org/10.1016/j.cj.2022.07.010.