A Study on the Optimal Grasping Angle Algorithm for Plug Seedlings Based on Machine Vision

Abstract

During the replanting operation of a seedling tray, the end-effector needs to repeatedly grab the qualified plug seedlings in the supply tray and release them to the target tray for replanting, and in the process of grasping, the end-effector may cause some mechanical damage to the plug seedlings, thus affecting their quality. Therefore, in order to be able to adjust the position of the hand claw grasping point according to the morphological characteristics of the plug seedlings and select the optimal grasping point, this paper proposes research on the optimal grasping angle algorithm for plug seedlings based on machine vision. Firstly, a rotatable three-jaw end-effector is designed, which uses a three-jaw structure for grasping the burrowing seedlings. The three claws are driven with a telescopic cylinder to carry out clamping and relaxing actions. The rotation of the three claws is controlled with the stepper motor to adjust the optimal grasping position. Secondly, based on the pre-processing of an image of the hole tray seedling, the extraction of feature points in the region of interest, and the calculation of localization, the angle between the angular bisector of the cotyledon leaf blade of the hole tray seedling and the horizontal positive direction is solved. In this paper, two methods are designed to calculate the coordinates of feature points: one is the geometric method and the other is the center-of-mass method. Finally, the optimal grasping angle is calculated by analyzing the angle between the angular bisector of the cotyledon leaf blade and the horizontal positive direction of the cavity seedlings. According to the test, the average calculation error of the proposed algorithm is 3.12 degrees, and the average calculation time is 0.512 sec/sheet, which meet the requirements of the replanting operation.

1. Introduction

Since each seedling is different and receives different light, water, nutrients, etc., during growth, the growth form of each seedling is different, and each seedling in a seedling tray has certain randomness of growth and development. When the end-effector of the replanting machine grabs a plug seedling downstream, squeezing and poking of the seedling’s leaves and the body of the seedling happens from time to time. Damage to seedling leaves and bodies will directly affect the quality and growth of seedling products and bring some economic losses. The structural characteristics of the end-effector directly affect the extraction effect on plug seedlings; therefore, it is important to reduce mechanical damage to seedlings when the end-effector grabs the plug seedlings to improve the quality of replanting operations. Many scholars have researched end transplanting manipulators [1,2,3] and proposed seedling extraction mechanism solutions such as ejector [4], insertion [5,6], ejector clamping [7], and pneumatic types [8].

Lv et al. [9] performed a study on an adjustable floral plug seedling transplanting mechanism. The results of the study showed that the transplanting fingers performed functions including clamping, releasing, and planting seedlings; the operating pressure could function normally as long as it was above 60 psi (pounds per square inch); the transplanting speed was completed at about 675 plants/h; and the success rate of transplanting was above 90%. Zhang et al. [10] designed a new pointer–clamping transplanting jaw mechanism based on pneumatic technology for the end-effector of the transplanting robot. The transplanting claw mainly consisted of a driving cylinder, pointer mounting, and positioning block; a transplanting claw mounting and positioning block; and a spring sleeve, finger cylinder, and clamping pointer. In the transplanting claw four-finger mechanism, the pointer points into the soil at an angle of 10°, the operation of the cylinder drives the pointer along the finger cylinder in a downward movement into the seedling pile when the cylinder is about to fully strike the pointer in the recovery of 3 mm, clamping the seedling pile. Although the pneumatic drive end-effector has many advantages such as easy control and simple and compact structure, the pneumatic drive requires an air source, which increases the cost and the size of the equipment and is not convenient for mobile operation [11]. The compressibility of air also makes its control mode single, and thus it is difficult to achieve accurate position and speed control. Ting et al. [12,13,14] developed an SNS gripper consisting of two tilted pin grippers and near-programmed sensors with transplanting success rates ranging from 50% to 95% for different crop varieties. Choi W C et al. [15] developed a plug seedling transplanter, which had an end-effector consisting mainly of a five-bar mechanism, a pointer driver, and a clamping pointer. The results of their transplanting tests showed that the age of the seedlings, the angle of the clamping pointer, and the speed all affected the success rate of transplanting. For example, the success rate was 97% when the speed was 30 plants/min for 23-day-old plug seedlings. Kang D.H. et al. [16] developed a vegetable transplanting machine. This transplanter mainly consisted of several parts such as the seedling tray transfer mechanism, hole tray seedling clamping mechanism, pot transfer mechanism, and transplanting mechanism. The transplanting test showed that the transplanter was capable of transplanting 2800 pots per hour with a success rate of 99.00%. Zhou and Wang et al. [17] designed a transplanting machine, and its control system was found to be suitable for transplanting plug seedlings. The mechanical structure mainly consisted of a transverse and longitudinal axis and a transplanting manipulator, cavity tray conveying, and other parts, which were driven by a screw. The end-effector of the manipulator used a tilting wedge lever clamping mechanism, and the experimental results showed that the average success rate of transplanting reached 76.11%. Han and Mao et al. [18] designed a light and simple automatic transplanting machine that mainly consisted of several parts, including a transplanting robot arm, seedling picking end-effector, source and destination tray delivery device, and control system. The machine used a pneumatic two-finger, four-pin clamp-type end-effector to pick up seedlings. The test showed that for 72-hole plug seedlings, the integrated yield reached 1000 seedlings per hour, and the success rate of plug seedling transplanting reached 90.70% on average.

Han et al. [19] designed a dual-cylinder-driven swing arm three-position control structure to achieve accurate stopping of the swing arm at the center position for seedling pick-up and release, fast movement at the beginning of the seedling clamping stroke, and clamping of the corresponding position of the plug seedlings with appropriate clamping force when the swing arm swings to the seedling clamping position. They calculated that the clamping force was not more than 23 N, which was less than the maximum squeezing force limit (25 N) for the stem of the plug seedlings when using the drive structure with an l value of 130 mm and an h value of 130 mm. Thus, the seedlings could be picked up and not damaged. Jiang et al. [20] combined the advantages of conventional plugging and clamping end-effectors and designed a single cylinder-driven plugging and clamping end-effector, which continuously completed the insertion and clamping of the seedling substrate within one thrust of the cylinder. The experimental results showed that the end-effector was able to transplant 270 plug seedlings, the transplanting success rate was 100% and the substrate damage rate was 17%, and there was no substrate block damage in 8 out of 27 groups of agronomic factor combinations. Furthermore, at the average transplanting operation rate of 2.8 sec/plant, the end-effector could improve the transplanting success rate and reduce the injury rate. Kang et al. [21,22] proposed a full deep learning neural network and a novel end-to-end geometric perception network A3N based on a full deep learning neural network and a novel end-to-end geometric perception network, respectively, to implement fruit recognition and grasping estimation methods. The experimental results showed that these two methods can accurately perform visual perception and grasping estimation, and they exhibited good performance in terms of accuracy, robustness, and operation speed. Hu et al. [23] designed a machine vision system for real-time measurement of seedling leaf area in each hole that was used to determine whether it was suitable for transplanting operations and the suitable grasping positions for transplanted seedlings.

In summary, the research on low-damage grasping technology for plug seedlings consists of two main aspects [24,25]. The first is to reduce damage to the plug seedling substrate during grasping by optimizing the design of the mechanical structure of the end-effector. The second is to reduce the damage caused by the end-effector to the plug seedling body using certain technical methods in the grasping process. There are several studies that used end-effectors to reduce damage to substrates during plug seedling grasping [26,27,28,29,30,31,32], but there are relatively few studies related to reducing end-effector damage to plug seedling bodies. Adaptive adjustment of the gripping angle of the end-effector according to the growth of the plug seedlings is one of the effective ways to reduce damage during transplanting. Among them, accurate and real-time calculation of the optimal grasping angle for each plug seedling is one of the key aspects. Therefore, this study designs a rotatable three-claw end-effector and applies image processing methods including image segmentation, binarization, filtering, and feature point extraction of regions of interest to design an optimal grasping angle algorithm for plug seedlings. Based on this algorithm, the end-effector can adjust the position of the gripping point of the hand claw according to the morphological characteristics of the plug seedlings and select the optimal gripping point for gripping, directly avoiding damage to the body of plug seedlings caused by the mechanical structure.

2. Materials and Methods

2.1. Experimental Materials and Equipment

The CPU of the computer used in this experiment is an 8th intel core i7 processor, the graphics card is GTX1060, and the algorithm program for testing is performed in Matlab2018b. The test sample was capsicum. Capsicum is an annual or limited perennial herb belonging to the genus Capsicum in the family Solanaceae. The early stage of growth and development of pepper seedlings is the germination stage, and after germination and sowing seeds, they usually emerge from the soil in about 5–8 days. They then grow the first true leaves in about 15 days, after which the flower buds are revealed, which is called the seedling stage. In this study, 20 sample images of pepper seedlings cultivated in a greenhouse were randomly selected for testing after 20 days of cultivation in seedling trays, which had entered the two-leaf-one-heart stage. The name of the pepper variety was the Chinese large pepper, which was cultivated by Beijing Zhongnong Futong Horticultural Co., Ltd., Beijing, China.

The end-effector is based on a rotatable three-jaw end-effector that is able to adjust the position of the hand claw gripping point according to the morphological characteristics of the plug seedlings and to select the optimal point for gripping. The three-dimensional mechanical structure is shown in Figure 1. The mechanical structure of the rotatable three-jaw end-effector mainly includes a hand claw, cylinder, bearing, stepping motor, T-type connecting plate, fixing plate, connecting rod, fixing rod, etc.

(1) Clamping angle design: During the study [33], Tong et al. conducted a test of pressure resistance of a seedling heap under four different clamping angles, namely, 4°, 7°, 10°, and 13°. The test results showed that when the clamping angle was 7° or 10°, the seedling needle horizontal extrusion propulsion of seedling pile pressure resistance value rose gradually and was more stable compared with the angle values of 4° and 13°. In addition, the 7° angle was the most optimal clamping angle when the seedling pile pressure resistance value was the largest. Therefore, in this study, the clamping angle of the end-effector claw was designed as 7°.

(2) Design of the number of hand claws: It is known from study [34] that the use of three-needle and four-needle clamping seedling lumps can achieve a more stable gripping effect, and the seedling lump resistance pressure increases steadily with an increase in the horizontal compression displacement. In comparison, to achieve the same value of seedling heap pressure resistance, the three-pin type needs to increase the horizontal compression displacement. For the end-effector designed in this study, it is necessary to realize low-damage grasping of hole tray seedlings. The three-hand claw design can reduce the grasping point compared to the four-hand claw design, reducing the probability of injury when mechanically grasping of hole tray seedlings. Thus, with comprehensive consideration, this study uses a three-hand claw design.

(3) Hand claw grasping range design: Figure 2 shows the T-type connecting plate and the hand claw with the screw-fixed connection. The T-type connecting plate and cylinder expansion activities occur between the block and the screw connection, but the T-type connecting plate and the relative position of the cylinder expansion activities of the block are adjustable in the range of 0~10 mm. The minimum diameter of the outer circle of the three points of the gripping point of the hand claw is 28 mm, so the diameter of the three claws can be adjusted within the range of 28~48 mm, to apply to different specifications of the hole tray during hole tray seedling gripping.

(4) Horizontal clamping displacement design: From the study [35], it is known that the clamping force required to remove hole tray seedlings along the vertical hole direction is about 1.94 N. From the study [33], it is known that when the horizontal squeezing stroke of the finger pin on the seedling pile is 4 mm, a clamping force of more than 2.6 N can be obtained. This clamping force can overcome the adsorption force between the seedling pile and the hole tray as well as the gravitational force of the seedling pile. Accordingly, the horizontal clamping displacement was set to 4 mm in this study to ensure sufficient clamping force to remove a seedling from the hole tray, and the cylinder stroke was thus determined.

In this case, three hand jaws of the same size are evenly distributed under the end-effector and are connected to the cylinder telescopic movable block with a T-shaped connecting plate. The stepper motor is fixedly connected to the connecting plate with a sleeve, and the cylinder is also fixedly connected to the connecting plate with screws. Bearings are provided between the connecting plates to reduce frictional resistance. The stepper motor output shaft rotation drives the cylinder and hand claw to rotate according to a specified angle, thus adjusting the position of the hand claw grasping point. The fixed rod is fixedly connected to the replanting moving mechanism, and the end-effector is driven to reciprocate with the replanting moving mechanism.

The end-effector grasps the plug seedlings as follows: First, the end-effector moves to a position above the target cavity compartment of the cavity tray, as shown in Figure 3a. Second, according to the optimal gripping angle information from the controller, the stepper motor in the end-effector drives the cylinder and the hand claw to rotate and reach the optimal gripping point position, as shown in Figure 3b. Again, the end-effector moves downward as a whole until the hand claw is inserted into the plug seedling substrate at a certain depth, as shown in Figure 3c. Then, the end-effector is contracted with the control cylinder, which drives the hand claw to clamp the seedling, as shown in Figure 3d. Finally, the end-effector and the plug seedling move together upward to the specified height, as shown in Figure 3e, and this completes the plug seedling grasping action.

Combined with the characteristics of the end-effector mechanism in this study, the optimal location of the gripping point should be as follows. One of the three gripping points of the end-effector should be on the angular bisector of the cotyledon leaves of the cotyledon seedling, located on the side where the angle of the cotyledon leaves is less than 180 degrees. This will avoid damage to the cotyledon seedlings caused by the end-effector gripping point to the maximum extent possible, as shown in Figure 4, where the yellow dots are the three gripping points of the end-effector gripping points.

2.2. Image Pre-Processing

The images of seedling trays with pepper seedlings were uniformly divided according to seedling tray size (50-hole, 72-hole, 105-hole, etc.) by the number of cavities to obtain a uniform size image of the seedling tray containing plug seedlings. An example of cavity trays containing different forms of plug seedlings is shown in Figure 5.

From the sample analysis of the cavity grid image, it can be seen that the image mainly consists of three parts: the edge of the cavity tray, the culture substrate, and the plug seedlings.

To achieve plug seedling segmentation, this study first converts the cavity grid RGB image into a Lab image, obtains the a and b component values for each pixel point in the image, and finally uses a K-means clustering algorithm to perform 3-part clustering segmentation.

(1)

Transformation of cavity grid RGB images into Lab space

Converting cavity grid images from RGB into Lab space requires XYZ as an intermediate mode, where RGB and XYZ space conversions are shown in Equation (1).

$$[X,Y,Z]=M*[R,G,B]$$

(1)

where:

$$M=\left[\begin{array}{ccc}0.4125& 0.3576& 0.1805\\ 0.2126& 0.7152& 0.0722\\ 0.0193& 0.1192& 0.9505\end{array}\right]$$

The transformation of XYZ space into Lab space is shown in Equation (2).

$$\begin{array}{l}L=116f(Y/Y_n)-16\\ a=500\left[f(X/X_n)-f(Y/Y_n)\right]\\ b=200\left[f(Y/Y_n)-f(Z/Z_n)\right]\end{array}$$

(2)

where:

$$f(t)=\left\{\begin{array}{c}{t}^{1/3}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}if\hspace{0.17em}t>{(\frac{6}{29})}^3\\ \frac{1}{3}{(\frac{29}{6})}^{2}t+\frac{4}{29}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}otherwise\end{array}\right.$$

(3)

$$X_n=0.95045,Y_n=1.0,Z_n=1.08892$$

(2)

Image segmentation using the K-means clustering algorithm

The K-means clustering algorithm was used to segment the four cavity grid images in Figure 5, and the obtained image processing results are shown in Figure 6. The clustering results using the K-means algorithm are shown in the first column from the left, as shown in Figure 6a. The seedling tray edge segmentation image is shown in the second column from the left, as shown in Figure 6b. The culture substrate segmentation image is shown in the third column from the left, as shown in Figure 6c. The plug seedlings segmentation image is shown in the last column from the left, as shown in Figure 6d.

The effect of segmentation using the K-means clustering algorithm for the first seedling cavity grid image on the left in Figure 5 is shown in Figure 7. In this figure, three different shapes of graphs are used to represent the three parts of clustering after image segmentation. The three parts of clustering are good, and there is almost no crossover of data points, which can realize image segmentation of cavity tray plastic edge, substrate, and plug seedlings. The horizontal and vertical coordinates represent the a and b component values of each pixel point in the image in Lab space, respectively.

(3)

Binarization of the segmented plug seedlings images

This study uses the maximum interclass variance method to find a suitable threshold TH of the image. The segmented plug seedlings image is binarized by this threshold, and the calculation formula is shown in Equation (5). The binarized image of the processed plug seedlings is shown in Figure 8.

$$\left\{\begin{array}{c}I(x,y)=255 B(x,y)>TH\\ I(x,y)=0 B(x,y)<TH\end{array}\right.$$

(4)

(4)

Image noise removal using median filtering

A tiny amount of noise is present in the images, and the interference is caused by a few impurities on the seedling substrate and the leaves of the plug seedlings. Median filtering is used to remove the noise.

(5)

Morphological segmentation of cotyledon leaves of plug seedlings

By analyzing the growth morphology of the plug seedlings, it was found that the cotyledons and true leaves occupied a large image area. The connecting rods of the cotyledons and true leaves were relatively thin, so a morphological method is used to segment them and thus obtain a segmented image of the cotyledon leaves and seedling cores. And the fine weed area in the corm seedlings is eliminated using this method.

The open operation in the morphological method is to erode and then expand the image. The effect of the open operation is to make the contours in the image smooth and break narrow connections and eliminate fine burrs. The open operation is performed on A using the structural element S and is denoted as $A\circ S,$ which is expressed as:

$$A\hspace{0.17em}\circ \hspace{0.17em}S=(A\hspace{0.17em}\mathsf{\Theta }\hspace{0.17em}S)\oplus S$$

(5)

The choice of structure elements in the morphological method open operation affects the results of image processing. When the structure element is too large, it will cut the effective area contour in the image, and when the structure element is too small, it will not be able to segment the connections effectively. Since the connecting rod does not have a fixed orientation feature, it is appropriate to use a square matrix as the structure element. Therefore, this study uses the all-one matrices of (6,6), (7,7), and (8,8) as structure elements to perform the open operation on the image, and the running results are shown in Figure 9.

From the results, it can be seen that the open operation with the structure of (6,6) sometimes cannot split the joints, as shown in Figure 9a, and the open operation with the structure of (8,8) can weaken the contour of the seedling core, as shown in Figure 9c. The open operation with the structure of (7,7) ensures that the cotyledon leaf outline is not weakened, and it also divides the cotyledon and the seedling core and has the effect of eliminating the tiny weeds in the plug seedlings, as shown in Figure 9b.

2.3. Feature Point Extraction and Localization Calculation in the Region of Interest

In this study, the angle between the angular bisector of the cotyledon leaf blade of plug seedlings and the horizontal positive direction is solved by extracting the feature points in the region of interest and the localization calculation. In this paper, two methods are designed to calculate the coordinates of feature points: one is the geometric method and the other is the center-of-mass method.

Method 1: Geometric method

The three characteristic points determined with the geometric method are the center points A and B of the minimum external rectangle of the cotyledon leaf blade and the intersection point C of the midline of the two rectangles.

First, according to the statistical analysis of the plug seedlings image, the two largest connected areas in the image were identified as the cotyledon leaf area, and its outer minimum rectangle was made, as shown in Figure 10. Second, after determining the direction of the long and short sides of the outer minimum rectangle of the cotyledons, the specific orientation of the leaves is determined. According to the minimum external rectangle, the endpoints calculate the rectangle center point A and B coordinates. Finally, according to the equation of the center line of the smallest external rectangle, the coordinates of its intersection point C is determined.

(1) The first large connected area outside the smallest rectangle outside the four endpoints is numbered clockwise. The highest point is marked K1, followed by the markers K2, K3, and K4. Similarly, the second largest connected region outside the four endpoints of the outer rectangle are numbered as K5, K6, K7, and K8, as shown in Figure 11.

(2) The next step is to detect the two side lengths L(K1,K2) and L(K1,K4) of the smallest rectangle outside the first large connected region to determine the direction of the cotyledon leaf blades.

Suppose M1 and M2 are the two midpoints of the short side of the outer rectangle of the cotyledon blade, and M3 and M4 are the two midpoints of the short side of the outer rectangle of the other cotyledon blade.

If L(K1,K2) > L(K1,K4), then the M1 and M2 coordinates are calculated as shown in Equations (9) and (10).

$$X_{M1}=\frac{X_{K1}+X_{K4}}{2},Y_{M1}=\frac{Y_{K1}+Y_{K4}}{2}$$

(6)

$$X_{M2}=\frac{X_{K2}+X_{K3}}{2},Y_{M2}=\frac{Y_{K2}+Y_{K3}}{2}$$

(7)

If L(K1,K2) < L(K1,K4), then the M1 and M2 coordinates are calculated as shown in Equations (11) and (12).

$$X_{M1}=\frac{X_{K1}+X_{K2}}{2},Y_{M1}=\frac{Y_{K1}+Y_{K2}}{2}$$

(8)

$$X_{M2}=\frac{X_{K3}+X_{K4}}{2},Y_{M2}=\frac{Y_{K3}+Y_{K4}}{2}$$

(9)

The coordinates of M3 and M4 are calculated in the same way as above, and thus the coordinates of M1, M2, M3, and M4 are calculated.

(3) Next, based on the coordinates of the four points M1, M2, M3, and M4, find the coordinates of the midpoints A and B of the two rectangles and the coordinates of the intersection C of the bisector of the two rectangles.

The coordinates of A (X_A, Y_A) are calculated as shown in Equation (13).

$$X_A=\frac{X_{M1}+X_{M2}}{2},Y_A=\frac{Y_{M1}+Y_{M2}}{2}$$

(10)

The coordinates of B (X_B, Y_B) are calculated as shown in Equation (14).

$$X_B=\frac{X_{M3}+X_{M4}}{2},Y_B=\frac{Y_{M3}+Y_{M4}}{2}$$

(11)

The coordinates of C are calculated by combining the equations of the line established by M1 and M2 and the equations of the line established by M3 and M4 to find the coordinates of the intersection C (X_C, Y_C) as shown in Equations (15) and (16).

$$\frac{y-Y_{M1}}{x-X_{M1}}=\frac{Y_{M2}-Y_{M1}}{X_{M2}-X_{M1}}$$

(12)

$$\frac{y-Y_{M3}}{x-X_{M3}}=\frac{Y_{M4}-Y_{M3}}{X_{M4}-X_{M3}}$$

(13)

Method 2: Mass center method

Based on the growth characteristics of the plug seedlings, the three largest connected areas in the image were found and identified as the two cotyledon leaves and the plug seedling’s leaf core. The coordinates of the three connected areas are determined by finding the coordinates of the center-of-mass of the three points A, B, and C, as shown in Figure 12.

The center-of-mass coordinates are calculated as follows:

$$Center-of-mass coordinates=\frac{Total coordinates of the connected area and}{Total coordinate points}$$

(14)

2.4. Calculation of the Optimal Gripping Angle for the Plug Seedlings

The initial gripping point position of the end-effector hand claws is designed so that one of the hand claws is horizontally oriented, where the yellow dots are the three gripping points of the end-effector gripping points, as shown in Figure 13a. To avoid damaging the plug seedlings with the hand claw, it is rotated according to the calculated optimal gripping angle to avoid the plug seedlings’ blades. The gripping point in the horizontal positive direction of the end-effector is adjusted so that it is rotated diagonally between the central axes of the two blades, as shown in Figure 13b, which is the optimal gripping angle of the end-effector.

From the previous analysis, it is clear that the angle between the angular bisector of the cotyledon leaves and the horizontal positive angle of the cotyledon is the optimal gripping angle for plug seedlings.

Assume that the included angles between vectors $\overrightarrow{CA}$ and $\overrightarrow{CB}$ and the positive horizontal direction are $\alpha 1$ and $\alpha 2$, respectively. The included angle between the angular bisector $\overrightarrow{CD}$ formed by vectors $\overrightarrow{CA}$ and $\overrightarrow{CB}$ and the horizontal positive direction is $\beta$.

The distribution of the three feature points A, B, and C and $\alpha 1$, $\alpha 2,$ and $\beta$ in the plane coordinate system is shown in Figure 14.

The vector $\overrightarrow{CA}$ coordinates are (X_A − X_C, Y_A − Y_C), and the joint cube equation is as follows:

$$\cos \alpha 1=\frac{x_A-x_c}{\sqrt{{(x_A-x_C)}^2+{(y_A-y_C)}^2}}$$

(15)

$$\sin \alpha 1=\frac{y_A-y_c}{\sqrt{{(x_A-x_C)}^2+{(y_A-y_C)}^2}}$$

(16)

Next, find $\alpha 1$ according to the joint cube equation. Similarly, calculate the included angle $\alpha 2$ between $\overrightarrow{CB}$ and the horizontal positive direction.

Let the angle between the angle bisector $\overrightarrow{CD}$ and the horizontal positive direction be $\beta$, and discuss it in two cases:

Case 1, when $\left|\alpha 2-\alpha 1\right|<180$

$$\beta =\frac{\alpha 1+\alpha 2}{2}$$

(17)

Case 2, when $\left|\alpha 2-\alpha 1\right|>180$

$$\beta =\frac{\alpha 1+\alpha 2}{2}+180$$

(18)

Thus, the calculation for the optimal grasping angle $\beta$ of the plug seedlings is completed.

The three gripping points in the end-effector are spaced at 120 degrees. To achieve fast grasping of plug seedlings, it is considered that the grasping angle sometimes exceeds 120 degrees. However, in practice, if the rotation angle exceeds 120 degrees, the orientation after rotation is the same as the residual result of dividing the rotation angle by 120 degrees. Therefore, the following end-effector rotation control strategy is defined.

Assuming that the optimal grasping angle of the plug seedlings is $\beta$ and the rotation angle of the end actuator is $\gamma$, then:

$$\gamma =MOD(\frac{\beta }{120})$$

(19)

3. Results and Analysis

Manual measurement method: The position of the seedling core in the image of the plug seedlings was manually marked as the corner point, and the midline of the two cotyledon leaves of the plug seedlings was manually marked, as shown in Figure 15a. The angle between the two midlines and the horizontal line was measured manually using the software Picpick, and the angle between the angular bisector of the cotyledons and the horizontal line was calculated. Angle A in the figure is the optimal gripping angle obtained using manual measurement.

Angle B was measured using the geometric method, and Angle C was measured using the center-of-mass method based on the optimal grasping angle algorithm for plug seedlings designed in this paper, as shown in Figure 15b,c.

The test samples were tested for the optimal gripping angle of plug seedlings using manual measurement, the geometric method, and the center-of-mass method. The test data are shown in

Table 1. Test results for the optimal grasping angle algorithm for plug seedlings.

Test Serial Number	Manual Measurement Angle A (Degrees)	Geometric Calculation of Angle B (Degrees)	Angle of Mass Method C (Degrees)	Angle Absolute Difference (Degrees) |a − b|	Absolute Difference of Angle (Degrees) |a − c|	Geometric Calculation Time (Seconds)	Mass Center Method Calculation Time (Sec)
1	89.37	87.57	83.49	1.80	5.88	0.451	0.657
2	137.95	140.78	133.62	2.83	4.08	0.502	0.632
3	207.32	204.15	203.81	3.17	3.51	0.516	0.603
4	214.33	210.68	205.01	3.65	9.32	0.578	0.559
5	108.84	106.20	104.77	2.64	4.07	0.523	0.556
6	36.91	34.32	32.29	2.59	4.62	0.572	0.611
7	94.83	98.18	87.31	3.35	7.52	0.490	0.594
8	107.35	109.70	102.62	2.35	4.73	0.528	0.635
9	220.39	223.56	214.29	3.17	6.10	0.541	0.571
10	142.85	145.97	134.14	3.12	8.71	0.479	0.550
11	180.25	184.39	187.92	4.14	7.67	0.502	0.574
12	96.48	233.02	359.92	F (136.54)	F (263.44)	0.481	0.590
13	77.61	75.12	82.50	2.49	4.89	0.471	0.583
14	93.23	90.45	88.04	2.78	5.19	0.499	0.604
15	52.30	55.72	59.68	3.42	7.38	0.528	0.612
16	333.49	331.93	312.18	1.56	F (21.31)	0.515	0.581
17	34.66	31.57	28.49	3.09	6.17	0.484	0.599
18	205.13	210.27	213.75	5.14	8.62	0.547	0.642
19	172.79	176.36	177.02	3.57	4.23	0.503	0.659
20	88.37	92.90	94.26	4.53	5.89	0.537	0.613
Average	N/A	N/A	N/A	3.12	6.03	0.512	0.601

Note: It is assumed that a calculation failure occurred when the difference between the calculated angle and the manually measured angle was greater than 15 degrees, which is recorded as F.

.

Based on the test data obtained with the optimal grasping angle algorithm, Table 1 shows that the average error calculated using the geometric method is 3.12 degrees, and the average running time is 0.512 s. The average error calculated using the center-of-mass method is 6.03 degrees, and the average running time is 0.601 s. The comparison shows that the geometric method is more accurate and has a faster running time compared with the center-of-mass method, so the geometric method is used for angle calculation for the optimal grasping angle algorithm for plug seedlings designed in this paper.

Considering the failed calculations for samples 12 and 16, it can be seen that the optimal grasping angle algorithm designed in this paper may fail when the cotyledon leaves of the plug seedlings are not fully visible or when there are special samples invaded by the adjacent plug seedlings. Schematic diagrams showing the calculation of the two algorithms for samples 12 and 16 are shown in Figure 16. In Figure 16a, both the geometric method and the center-of-mass method fail, and in Figure 16b, the geometric method is valid, but the center-of-mass method fails.

Further, the algorithm validation for the special case sample revealed that for the special case sample where the plug seedlings invasion is small or the plug seedling leaves are incomplete but the main body is intact, as shown in Figure 17a, the results of the geometric method calculation are valid, and the results of the center-of-mass method calculation may fail. The geometric method can determine the cotyledon leaf blade based on the two largest connected areas and thus calculate the optimal gripping angle. The center-of-mass method is subject to computational errors due to interference from intruding leaves when determining the center-of-mass of the leaf core region. For the special case samples where the invasion of the plug seedlings is large or the plug seedlings have incomplete leaves and the main body is not fully displayed, as shown in Figure 17b, both the geometric and center-of-mass methods failed to calculate the results. The geometric and center-of-mass methods failed due to their inability to obtain the exact position of the leaves of the plug seedlings.

4. Conclusions

This paper presents a study of an optimal grasping angle algorithm for plug seedlings. First, a damage analysis is performed on the end-effector to grasp the plug seedlings. Second, we propose an optimal grasping angle algorithm for plug seedlings by segmenting the region of interest using image pre-processing, calculating the localization coordinates of the feature points in the region of interest, and then calculating the angle between the angular bisector of the cotyledons and the horizontal direction of the plug seedlings. Finally, the optimal gripping angle algorithm for plug seedlings was tested and validated. The experimental results show that the average calculation error is 3.12 degrees, and the average calculation time is 0.512 s per sheet, which meets the actual replanting requirements. Our next research direction is to find an optimal detection method based on the image analysis of the overall image of the seedling tray for all the plug seedlings to avoid the loss of plug seedlings information in some special cases, due to the image analysis of individual cavity frames, and thus reduce the angle calculation failure rate.