An Overview of End Effectors in Agricultural Robotic Harvesting Systems

Abstract

In recent years, the agricultural sector has turned to robotic automation to deal with the growing demand for food. Harvesting fruits and vegetables is the most labor-intensive and time-consuming among the main agricultural tasks. However, seasonal labor shortage of experienced workers results in low efficiency of harvesting, food losses, and quality deterioration. Therefore, research efforts focus on the automation of manual harvesting operations. Robotic manipulation of delicate products in unstructured environments is challenging. The development of suitable end effectors that meet manipulation requirements is necessary. To that end, this work reviews the state-of-the-art robotic end effectors for harvesting applications. Detachment methods, types of end effectors, and additional sensors are discussed. Performance measures are included to evaluate technologies and determine optimal end effectors for specific crops. Challenges and potential future trends of end effectors in agricultural robotic systems are reported. Research has shown that contact-grasping grippers for fruit holding are the most common type of end effectors. Furthermore, most research is concerned with tomato, apple, and sweet pepper harvesting applications. This work can be used as a guide for up-to-date technology for the selection of suitable end effectors for harvesting robots.

1. Introduction

As the human population increases, moving towards sustainable agricultural management becomes ever harder. In the next decades, human population is expected to grow by 40% to 9.7 billion by the year 2050 [1], calling for double fruit production. The worldwide agricultural area is expected to double as well. Nevertheless, agriculture employment is expected to decrease in half by the year 2050, resulting in a shortage of 5 million harvesters. Hence, more than 10% of fruits worldwide cannot be harvested; the latter is equal to the European Union’s annual consumption [2].

Harvesting is a seasonal, low-paid, repetitive, and labor-intensive work with meagre professional prospects. Experienced harvesters are retiring, whereas younger people are not interested in replacing them. Labor shortages result in harvest delays. However, a fruit harvested with several days of delay deteriorates in quality and may lose up to 80% of its market value. As a result, worldwide growers lose up to an estimated USD 30 billion a year in potential sales from fruits that cannot be harvested [3].

For this reason, crop management has significantly changed over the last decades. More specifically, ground and aerial robots have been introduced in agriculture, demonstrating their potential to meet the rising food demand, by automating traditionally manual agricultural operations [4], including harvest. Thus, robotic systems have been designed to cover labor shortage, to increase the speed of harvesting, and to improve the efficiency of harvesting.

In traditional manual harvesting, the harvester uses his/her hands to remove leaves or branches, to grasp the fruit, and to detach it from the plant by pulling it away, sometimes using a cutting tool. Efficient manual harvesting requires experienced skills; an inexperienced harvester may unintentionally harm the plants. However, the kinematics of human hands and body, sense of touch, and muscular strength give inherent grasping abilities to humans and a major degree of quick adaptation to different crop shapes and textures for applying a suitable detachment force. However, human skills are only limited by tiredness. On the other hand, a robotic system can harvest continuously, accurately, and tirelessly with consistency. For this reason, researchers try to emulate human harvesting skills, resulting in kinematic models for the movement of robotic arms and the design of sophisticated end effectors with appropriate sensors for crops’ manipulation [5,6].

An end effector is a peripheral device attached to a robot’s wrist, enabling interaction during a task. For harvesting robots, the end effector is considered the contact point between the robot and the product to be harvested. If not designed effectively, an end effector could damage the crop and deteriorate the overall performance of the harvesting system. The diversity of the crops requires that each harvesting system possess an end effector according to the specifications of the harvested crop. Toward the latter end, this work provides a detailed summary of state-of-the-art robotic end effectors for harvesting applications. Requirements for the detachment of harvesting products, different types of end effectors, and sensory control strategies are also discussed.

Many excellent review papers have been published recently, focusing on robotic harvesting end effectors [5,7,8,9,10,11,12]. More specifically, in [5], the authors reviewed the evolution of robotic end effectors for harvesting apples, tomatoes, sweet peppers, and cucumbers. In [7], Davidson et al. examined recent developments in manipulator and end effector technologies for robotic harvesting of various crops. Navas et al. [8] presented the latest advances in the design and implementation of soft grippers for crop harvesting. In [9], Mohiuddin et al. provided a survey on aerial manipulation, classifying unmanned aerial vehicle (UAV) grippers and manipulators based on their design and characteristics. Seven harvesting manipulators are reviewed and analyzed in [10] for five different products, namely, strawberries, tomatoes, apples, sweet peppers, and iceberg lettuce. Feng, in [11], provided a review of existing end effectors for fresh-market fruit picking and harvesting. Zhang et al. [12] reviewed robotic grippers, grasping and sensor-based control strategies, and their applications in the agriculture and food industry.

This work reviews the state-of-the-art robotic end effectors for both ground and aerial harvesting systems, reported between the years 2016 and 2022, that have been tested and evaluated in actual harvesting applications. The main contributions of this work compared with recent review articles [5,7,8,9,10,11,12] on the same subject are:

• The examination of recent works from 2016 up to date. In [7], reported research spans from 1987 to 2017, including only three reviewed research papers after 2016.
• The examination of end effectors developed for ground harvesting applications and for UAV harvesting systems. UAV manipulation systems were revised in [9]; however, the emphasis was on grasping and picking tasks rather than on mounted end effectors targeted for aerial harvesting operations.
• The examination of various types of manipulators for automatic harvesting is not limited only to grippers [12] and/or soft grippers [8].
• The examination of end effectors for harvesting many types of crops. In [5,10], the review is limited to end effectors for harvesting four and five different crops, respectively. Τhis work reviews harvesting end effectors for over 15 different kinds of crops.
• The evaluation of end effectors in real-world harvesting applications. In [11], the focus is on fresh market fruit picking rather than on harvesting end effectors, and consequently, no in-field evaluation performance of the revised end effectors is provided.

This review aims to serve as a complete guide for up-to-date technology for designers or researchers who need a complete picture of the current status of harvesting end effectors and their implementation and evaluation in real-world agricultural applications. Agricultural end effectors could be a very broad topic, and it may regard flowers, vegetables, trees, fruits, and others, thus implying different handling and harvesting processes. This work focuses on harvesting consumer agricultural products, such as fruits and vegetables. This review emphasizes (1) end effectors’ structure in terms of different end-effector types and sensors, (2) end-effector operation in terms of detachment methods and operating requirements, and (3) basic end-effector development principals that could serve as general guidelines for end-effector designers. All of the above are examined in real-world harvesting applications, concentrating on end effectors already used in agriculture toward end effectors’ evaluation, followed by the suggestion of effective alternative designs for specific crops.

The rest of the paper is structured as follows. Materials and methods are presented in Section 2. Section 3 summarizes detachment methods, types of end effectors, additional sensors, operating requirements, and basic end-effector development principles. Applications of end effectors in agricultural robotic harvesting systems are shown in Section 4. End effectors are examined for both ground and aerial harvesting systems. Section 5 summarizes and discusses the contribution of the work. Finally, Section 6 concludes the paper.

2. Materials and Methods

The adopted research method consisted of four main steps:

• Extensive literature research. It has been suggested that in all areas, Google Scholar citation data are a superset of Web of Science and Scopus, with significant extra coverage [13]. Therefore, the database of Google Scholar was selected as the source of the literature for the scope of this review article. Combinations of keywords, such as “end-effector”, “harvesting robot”, and “manipulation”, were applied for the initial research.
• All extracted papers were reviewed for their relevance to the subject. The papers that did not provide information regarding the in-field used end effectors were excluded.
• The third step of the method included examination of the references in the papers of step 2, towards a more thorough review.
• In the final step, all research papers dated up to 2015 were excluded from the research, keeping the recent literature from 2016 to date.

The final set of papers regarding end effectors in robotic harvesting included 41 research articles. The distribution of the 41 articles per year is shown in Figure 1.

3. End Effectors in Agricultural Robotic Harvesting Systems

Once a fruit is detected and located, the challenge is to pick it without damaging the fruit or the crop. Typically, fruits are difficult to reach due to many unstructured obstacles that interfere with the harvesting system, such as branches and leaves. An effective harvesting system must balance between speed and efficiency.

Bulk harvesting approaches, such as limp or branch shaking systems, are used in harvesting [14] and report a lower harvesting cycle, that is, harvesting time per fruit. However, such techniques are not appropriate for soft or for sensitive fruits. To ensure superior quality and high market value, most fruits need individualized harvesting mechanisms. A key component of any harvesting mechanism is a harvesting end effector that detaches a fruit from a tree. It is common for end effectors to be customized based on the harvested crop. However, the design can be adapted to other crops of the same fruit size or can be adjusted to different fruits with minor readjustments.

In all cases, a manipulation system in harvesting applications has to fulfill specific requirements. The most important of these requirements include: (1) operational characteristics: adaptation to various shapes, not causing damage to the harvesting product, consistency, and high precision of operations, and (2) technical characteristics: high activation speed, low maintenance, low weight, and low energy consumption [15]. The design of the end effector needs to comply with the aforementioned requirements, and it is an essential aspect for effective harvesting. Finally, a specialized end-effector control unit is used to provide a standardized software and hardware interface.

In what follows, first, fruit detachment methods are reviewed to underline the requirements of an effective end effector. Second, the most common types of end effectors that meet the harvesting requirements are summarized. Third, additional sensors for agricultural end effectors are discussed, as well as operating requirements. Basic agricultural end effector development principles are also proposed.

3.1. Detachment Methods

The most common detachment method is grasping and pulling [16]. Simply grasping and pulling a fruit away from the crop results in breaking the stem and cutting it free from the plant. However, pulling away places extreme force on the fruit, which may damage it and cause plant shaking. Shaking will cause all fruits on the tree to swing. The latter may cause three additional problems: (1) movement of the fruits that had been detected by the vision system from their original position and hence failure of the manipulation system to grasp them, but the latter can be resolved by recalculating the location of the fruits when a fruit is removed; (2) detachment or brushing of nearby fruits due to shaking of the tree; and (3) possible detachment of the stem from the plant, instead of the fruit alone. The part of the remaining stem can damage (e.g., pierce/scratch) the rest of the fruits during their collection, storage, and/or transport.

In order to deal with fruit damage due to unnecessary detachment pulling force, grasping from the stem and pulling has been applied [17]. Stem grasping is an intact alternative method, which does not require the fruit to be grasped directly; therefore, it does not typically damage the harvested fruit. The end effector grasps the stem of the fruit and detaches it from the tree. However, the shaking of the tree remains. To overcome this problem, end effectors able to cut the fruit stem were designed. Therefore, it could be either direct contact grasping of the fruit and simultaneous cutting of the stem [18] or grasping from the stem and cutting it [19]. The second may be a fully intact method, yet it is more challenging, since the detection of a small stem is more demanding, due to dense foliage and occlusions in in-field applications, than the detection of an entire fruit.

One solution to overcome this difficulty and still reduce the exerted force from the stem is to rotate the fruit before pulling it from the tree [20]. Rotation before pulling is a traditional technique of manual harvesting. The stem folds at the fruit–stem interface, which facilitates the breaking of the stem at its contact point with the fruit.

Suction detachment methods are also common. Suction, when applied alone, is similar to grasping. However, due to the applied suction force, it could wound the fruit at the point of contact and pierce or injure the fruit skin. Therefore, vacuum is used auxiliary to another detachment method, usually with grasping [21]. Vacuum pressure can facilitate the fruit to expose and approach the grasping end effector by removing and releasing it from the foliage.

In Section 4, applications of end effectors in agricultural robotic harvesting systems are reviewed. It turns out that grasping in general could be applied with a gripper of a vacuum. In this work, grasping refers to the direct manipulation of the fruit, where the end effector is adapted to the fruit’s shape. Grasping from the stem is an alternative strategy. Vacuum refers to grasping with pressure force. Rotating refers to the rotation motion applied to the end effector to break peduncles. Finally, cutting refers to the cutting of the stem with a scissorlike device, blade, saw, thermal device, or any other cutting tool.

3.2. End-Effector Types

End effectors consist of a gripper or a tool. Grippers provide temporary direct contact with the product to be grasped, from either the fruit body or the stem. Vacuums, which are properly constructed suction devices, are considered types of grippers; however, no actual grasping takes place (indirect harvesting). In this work, suction devices will be considered separately, while grippers will exclusively be considered devices that make actual direct grasping.

Therefore, the end effectors in this work are grouped into the following four categories according to detachment method:

• Contact-grasping grippers, which can be either fruit holding (Figure 2a) or stem holding (Figure 2b);
• Rotation mechanisms (Figure 2c);
• Scissors/sawlike tools (Figure 2d); and
• Suction devices (Figure 2e).

The identified four end-effector types in this work are illustrated in Figure 3. End effectors of the above categories can be combined. Therefore, an end effector mounted on a harvesting system may detach the fruit by combining grasping and rotation, grasping and suction, grasping and cutting, suction and cutting, and so on.

Contact-grasping grippers can grasp the fruit securely. However, they are prone to interference from branches, leaves, or other fruits due to their wide workspace dimensions. Moreover, fruit-holding grippers are more likely to bruise sensitive crop. Stem-holding grippers have the advantage of not touching and, thus, of not damaging the fruit body. However, dynamic environmental conditions, occlusions, illumination, and so on may obstruct the vision system from accurately detecting the grasping point on the delicate stem and may complicate access to it. Suction cups are common gripping tools and mechanically simple and only need access to a part of the fruit that is exposed. Rotation mechanisms are usually combined with grasping grippers. Rotation motion is effective for breaking peduncles and is employed for harvesting crops that have stiff stems not easily detached by pulling.

For some crops, however, rotation is not enough to detach it from the plant. Crops such as sweet peppers need to be cut off the plant; hence, an additional harvesting tool other than grasping is required. In these cases, scissors/sawlike tools are used as harvesting end effectors. Grasping and pulling/rotating can be performed easier compared with scissor tools. A scissor calls for estimation accuracy of stem detection so that the blades of the scissor are placed accurately around the stem and avoid damaging the fruit or the crop.

In order to minimize fruit damage from grasping and suction cups, soft gripping fingers and soft vacuum nozzles have been developed [25]. Soft robotic components can protect from collisions and minimize damages to crops and hardware components through soft interaction with the crop. Soft robotics have already demonstrated good results in constructing effective grippers for robotic crop harvesting [8]. The disadvantage of soft robotic fingers is that they are difficult to implement and require further refinement before being used in practical applications [8].

Contact-grasping grippers can be further classified into several categories based on different classification strategies, such as number of fingers, actuation types, gripping modes, mechanism types, and physical gripping principles [12]. A popular classification of grippers is the one introduced by Monkman et al. [26], based on their physical principle of operation, in four major groups, namely, impactive, ingressive, astrictive, and contigutive. Based on their actuation control, grippers are also classified as: magnetic [27], vacuum [27], hydraulic [27], pneumatic [28], and electric [29]. Furthermore, based on their number of fingers, four major categories are proposed for robotic grippers [12]: two-finger grippers (Figure 4a), three-finger grippers (Figure 4b), four-finger grippers (Figure 4c), and anthropomorphic hand (more than four fingers) (Figure 4d). In all cases, the selection of the right actuator is strongly related to the application, and it presupposes a clear understanding of the end-effector requirements and working environment conditions [30].

3.3. Additional Sensors

Human grasping integrates inherent sensory abilities, enabling stable, flexible, and adaptive grasping. Robotic end effectors attempt to imitate the skills of human hands by using tactile and visual sensors, so as to insert touch and visual perception to the designed end effectors. When sensors interfere between an end effector and the harvested product, useful information can be acquired toward optimizing the functionality of the end effector regarding the specific harvested product. Sensors can add intelligence to an end effector [31]. End-effector sensors related to grasping operations can be divided into four broader categories [12]: (1) switching sensors, (2) tactile sensors, (3) visual sensors, and (4) measuring sensors, as detailed in the following.

Switching sensors supply a trigger signal (binary on/off) for the mechanical system when a particular status for the gripper is monitored (open/close), and they have been used extensively with mechanical grippers [31]. Proximity [34], reed [35], touch [36], and Hall effect switches [37] belong in this category.

Tactile sensors [38] are able to detect the properties of objects through physical contact. Mechanical properties can therefore be measured, such as pressure, grasping force, torque, slipping, vibration, humidity, and temperature. In harvesting end effectors, the focus is on contact force and torques. Tactile sensors placed on the fingertips of grippers can detect applied forces and torques so as to monitor deformations on the structure of the end effector while grasping. The most common forms of tactile sensors are piezoresistive, capacitive, and optical [38]. Contact with an object returns a digital imprint that can be interpreted as information regarding contour characteristics, dimensions, position, or orientation of the grasped fruit.

Visual sensors are the most important contactless sensors for robotic manipulating systems. Visual sensors can detect obstacles, sense structures, determine grasping points, locate objects, and so on. Monocular and stereoscopic cameras have been reported in the literature, mounted on robotic end effectors to provide visual feedback to the manipulation system [39]. More specifically, stereoscopic cameras provide 3D information of the surroundings and, therefore, are considered more appropriate for harvesting applications in order to recognize and accurately locate the harvesting application points.

Measuring sensors are used to measure the distance between the end-effector tool and the target object, the dimensions of the grasped object, or the status of the end effector in terms of speed, acceleration, force, torque, and so on. For distance measurements, ultrasonic, microwave, and laser triangulation sensors are used [40]. For status measurements are employed Hall effect sensors, acceleration sensors, force–torque sensors, and so on [41].

Multiple sensors can be fused at hardware and/or software levels so as to provide a set of possible outcomes. In all cases, since the act of grasping is a function of time, sensory data acquisition and fusion need to be temporally dynamic [42]. The integration of multiple sensors could result in more dexterous handling of objects through the analysis of acquired information between the end effector and the object, which could support real-time decision making. Therefore, sensors combined with the respective control strategies could make an end effector more “intelligent”. End-effector functions, which can be monitored or be controlled, are the gripping force and movement, speed, position, and orientation of the object, forces, and movement during manipulation, identification of contact point, and so on. Extensive details regarding end-effector control strategies can be found in [12,26,43].

3.4. Operating Requirements for Agricultural End Effectors

End effectors in robotic systems used in agricultural harvesting may be (a) commercially available multifinger grippers suitable for harvesting a particular agricultural product, (b) integrated devices that have a control mechanism and the required elements (e.g., fingers) for harvesting, or a combination of mechanism elements and a simultaneous or two-step cutting system.

In all those cases, depending on the features of the agricultural product (fruit or vegetable), such as its dimensions, weight, difficulty of detachment, and degree of sensitivity, the end effectors should be properly designed by considering a number of different features always combined with the robotic system (e.g., robotic arm) on which they are mounted [7]. Those features include:

• The maximum load they can lift. This load ranges from a few tens of grams up to a few kilograms depending on the weight of the fruit. It should be clarified that it is not enough to consider only the end effector’s load capacity, but also the robotic system’s load capacity on which the end effector is mounted on. In addition, depending on how the product is detached (e.g., suction, rotation, or cut), this maximum load should be increased so that the additional forces required for the fruit’s detachment could be applied for the final detachment of the harvested product from a branch on which it is usually connected [44].
• The power exerted by the mechanism. To hold the product before it is removed from the branch, an appropriate force should be applied, which should not deform the product. In this case, depending on the product, the end effector may have sensors that control the applied force or have properly configured fingers with soft interior surfaces for holding smoothly agricultural products. In the former case, continuous control of the applied forces with the help of appropriate algorithms is required, whereas in the latter case, no sensors are often required, and end effectors have smaller control problems [8,45].
• The geometry and dimensions of the end effectors directly related to the geometry and dimensions of the products collected each time. Depending on the product, fingers with different geometries and dimensions can be adapted to the same gripper to serve this purpose [46].
• The type of movement to perform a particular task. In most cases of end effectors for the harvesting of agricultural products, the movement of the gripper is limited to opening and closing operations. However, the trajectory of the robotic system should be properly planned so that the end effector can approach, catch, and hold the product and remove it from the branch, for example, by moving towards a specific direction and orientation and by employing a rotational movement until the product is detached or cut off with the help of an appropriate cutting mechanism [47].
• The type of actuators required. Most end effectors are based on electric actuators that permit accurate control. The required power is small (some watts or a few dozens of watts), so it could be provided by the moving robotic system on which they are mounted. However, in recent years, implementations with soft actuators have appeared that allow the handling of sensitive products with much higher safety and less manufacturing costs. These actuators mainly operate with air and only limited by the accuracy of their movement and control [48].
• The time of action completing a movement to harvest a product. The end effector, depending on its complexity (e.g., use of sensors, control software, etc.) requires some time to complete a processing cycle of holding and detaching the harvested fruit from the crop. This picking time concerns: (a) the time to detect the fruit with the help of a detection system, which in most cases is a vision system with one camera (monocular), stereoscopic camera, or 3D camera; (b) the design of the trajectory that the robotic system should follow toward the desired goal; (c) the navigation to the desired goal; (d) the time required to grasp the product; (e) the detachment of the fruit from the branch; and (f) the transfer of the harvested product to a predetermined collection location. The grasping time of an end effector may be in the range of a few tenths of a second to a few seconds [7].
• The characteristics of contact with the product. Depending on the type of product, as mentioned above, the final configuration of the fingers of the end effector should be carefully determined. If the product is very sensitive, such as a strawberry, then the fingers should have soft surfaces and additional force or pressure sensors to regulate the holding force. If the product has medium hardness, such as an apple, then the use of only soft surfaces on the fingers (and control of the power with the current in the actuator–electric grippers) is generally sufficient to hold without damaging the product. To hold a product from the stem and detach it with an appropriate stem-cutting mechanism, such as in grape harvest, no sensors or control approaches are required [10].
• The tolerances and accuracy of the system. The end effector and the robotic navigation system towards the desired goal should at all times have the required precision so that the target could be detected and approached accurately. If the robotic system does not have the required accuracy by construction, then accuracy improvement techniques should be developed and applied as the end effector approaches the final target, for example, with the sensory feedback of a vision system mounted on the end effector and a visual servoing application [49].

3.5. Basic Agricultural End-Effector Development Principles

Identifying principles for an end-effector design is expected to lead to more effective prototypes. In what follows, basic principles, from conceptual design to the final prototype manufacturing, are proposed in four basic steps, including (1) research and requirements, (2) design, (3) prototype development, and (4) testing.

3.5.1. Research and Requirements

Research and requirements include a literature review on traditional harvesting and selective harvesting techniques on already-existing automation, extraction of the functional and nonfunctional requirements, and evaluation of the system based on these requirements.

• Bibliography. Research in the bibliography is the first step toward effective end-effector development. First, traditional manual harvesting and selective harvesting methods need to be reviewed. Selective harvesting is the segmented picking of a fruit at harvest based on different yield or quality criteria in order to exploit any observed variations [50]. By studying the human hand patterns, imitation detachment techniques and appropriate tools could be developed. Research on automation and end effectors used can also provide useful insights; a detailed design and evaluation of end-effector systems could be used as a guide for system development, modification, or improvement toward enhancing performance.
• Functional and nonfunctional requirements. According to the study of the bibliography, the system requirements need to be extracted. System requirements that need to be addressed, summarized in Section 3.4, include the maximum payload, grip force, geometry, and dimensions of the end effector in relation to the harvesting product, type of harvesting movement, type of actuators, picking time, detachment method, definition of product contact surface, material selection, tolerance, and accuracy of the system.
• Evaluation based on requirements. Evaluation of the development process and cost estimation is essential after specifying the operational requirements so as to ensure that the proposed design idea is feasible to fabricate and cost-effective.

3.5.2. Design

Based on the identified requirements, an end-effector system design, in terms of both hardware and software, follows.

• Hardware design. The study and selection of all hardware components and materials takes place, such as plastics, metals, motors, cutters, and all the remaining necessary mechanical components, sensors, batteries, and so on. The design of necessary components with the help of known computer-aided design (CAD) software applications (AutoCAD, SolidWorks, FreeCAD, etc.) follows. The way that the hardware will be connected, installed, and assembled and the design of appropriate driving and control electronic circuits for all devices (schematic and printed circuit board (PCB) design) are also included. Energy and payload requirements also need to be considered.
• Software design. For effective control of the end-effector system, driving algorithms, navigation, and control strategies based on software engineering principles need to be developed. Unified Modeling Language (UML) diagrams could be used for the visual representation of the system design. Effective data—as well as knowledge—representations should also be considered.
• System simulation. End-effector manufacturers try hard to realize true system performance until it is too late in the design process; mechanical and electrical subsystems need to be validated against the identified requirements. However, testing and validation of the entire system is usually delayed, leading to potential redesign or changes to the initial design of the end effector, which is costly, time-consuming, and risky. In order to improve engineering efficiency and reduce product development challenges, early system design validation is considered necessary, enabled by simulation. In this phase, if the appropriate software tools exist (for example, Robot Operating System (ROS), Gazebo simulation suite), a virtual model of the end-effector system can be constructed (e.g., a Unified Robot Description Format (URDF) file), and the operation of the end effector can be evaluated in a simulation environment. In ROS, with the help of the Gazebo simulation environment, an important number of parameters can be tested. For example, in the simulated environment, simulated sensors’ values can be read, and simulated actuators can change their state depending on sensor values. Contacts, forces implemented, torques, pressures, light variation, and so on, as responses of sensors, change the state as well as the behavior of the end effector. However, if an extensive analysis of the end-effector operation is required, more specific simulation applications are adopted. For example, Adams and Simulia multibody dynamic simulation environments can evaluate and manage the complex interactions of a system, including motion, structures, actuation, and controls, to better optimize product designs for performance and safety.

3.5.3. Prototype Development

Prototype device development includes the detailed development of a simple first prototype device hardware as well as the development of the appropriate software application system, as a result of the previous analysis and design.

3.5.4. Testing

After the end-effector prototype development and system integration, the whole system testing follows:

• Testing of the prototype system. The evaluation of fabrication time and estimation of the final cost of the prototype is required. The testing of the effectiveness of the prototype through an experimental procedure; recording of material behavior, applied forces, payload, pressure control, energy requirements, sensory feedback, control algorithm performance, and so on; and measurement of selected performance metrics (damage rate, picking time, etc.).
• Optimization and fine-tuning. Optimization of the design of the end effector, re-experimentation and measurement of performance metrics, decision on an acceptable minimum/maximum performance for the prototype so as to be considered potentially viable are suggested. If the required changes are significant, then a new full cycle is repeated (design, prototype development, and testing).
• Alternative designs and comparison. Prototype development, evaluation, and optimization of alternative end-effector designs toward comparison and final selection.

4. Applications of End Effectors in Agricultural Robotic Harvesting Systems

Several end-effector configurations have been designed and employed for autonomous harvesting robotic applications. The selection of the appropriate end effector depends mainly on the physiology of the harvesting fruit. In what follows, end effectors used in recent agricultural applications are reviewed, for a wide range of fruits, from year 2016 to date. The reviewed applications are divided into two broader categories: end effectors for ground harvesting systems and end effectors for aerial harvesting systems. The results of the review are summarized in

Table 1. Ground robotic manipulation systems for harvesting. Evaluation metrics of harvesting cycle time (time) and detachment success rate (accuracy) are included. Grasping from the stem is marked with “✓s”; all other cases refer to direct contact grasping of the fruit.

			Manipulator		Detachment Method				End Effector		Evaluation
Ref.	Year	Crop	DOF	Type	Grasp	Vacuum	Rotate	Cut	DOF	Type	Time	Accuracy
Heavy Crops
[51]	2018	Pumpkin, cabbage	5	RAVebots-1	✓	-	-	✓	2	5-finger gripper/cutter [53]	-	-
[52]	2019	Pumpkin, watermelon	5	RAVeBots-1	✓	-	-	✓	2	5-finger gripper/cutter [53]	33 s	Up to 2.06 nm
[33]	2020	Pumpkin	5	RAVeBots-1	✓	-	-	✓	2	5-finger gripper	-	Up to 92%
Tomatoes
[16]	2017	Cherry tomato	3	Custom	✓	-	-	-	1	2-finger gripper	30 s	100%
[54]	2018	Cherry tomato	6	Denso VS-6556G	✓s	-	-	✓	1	2-finger gripper/cutter	8 s	83%
[18]	2019	Cherry tomato	-	TakoBot arm	✓	-	-	✓	1	Semispherical gripper/cutter	-	-
[55]	2016	Tomato	6	UR5	✓	-	✓	-	2	3-finger gripper	23 s	60%
[56]	2017	Tomato	4	Custom	✓	-	-	✓	1	Shear-type gripper	15 s	86%
[57]	2016	Tomato	3	Custom (×2)	-	✓	-	✓	1	Saw-type cutter	-	-
									1	Suction device
[58]	2019	Tomato	3	Custom (×2)	-	✓	-	✓	1	Cutting gripper	30 s	87.5%
									1	Suction cup
[21]	2020	Tomato	-	-	✓	✓	-	-	1	4-finger gripper/suction	-	-
[24]	2021	Tomato	6	UR3	-	✓	-	✓	1	Suction/cutter	5.9 s	-
Strawberries
[59]	2019	Strawberry	5	RV-2AJ	✓	-	-	✓	1	6-finger gripper/cutter	7.5 s	96.8%
[60]	2020	Strawberry	3	Rail multiarm (×2)	✓	-	-	-	1	3-clamp gripper	4.6 s	Up to 97.1%
[61]	2019	Strawberry	3	Custom	✓	-	✓	-	1	Soft finger gripper	4s	-
Apples
[22]	2017	Apple	6	Custom	✓	-	-	-	1	3-finger gripper	6 s	84%
[32]	2017	Apple	6	Custom	✓	-	-	-	1	3-finger gripper	1.5 s	-
									1	Catching device
[63]	2019	Apple	5	Custom	✓	-	-	-	1	3 soft-robotic pneumatic actuators	7.3 s	67%
[64]	2019	Apple	6	UR3	✓	-	✓	-	2	4-finger gripper	16 s	90%
[65]	2020	Apple	-	-	✓	-	✓	-	1	Bowl-shaped gripper	-	-
Sweet peppers
[19]	2017	Sweet pepper	9	Custom	✓	-	-	✓	1	4-finger gripper/cutter	94 s	29%
[66]	2017	Sweet pepper	6	UR10	-	✓	-	✓	1	Suction/cutter	40 s	92%
[67]	2020	Sweet pepper	6	UR10	-	✓	-	✓	1	Suction/cutter	40 s	76.5%
[68]	2019	Sweet pepper	3	Custom	-	-	-	✓	1	Pose-control/cutter	51.1 s	70%
[69]	2020	Sweet pepper	6	Fanuc LR Mate 200iD	-	-	-	✓	1	Stem-fix device/cutter	15–24 s	Up to 61%
Kiwifruit
[20]	2019	Kiwifruit	3	Custom	✓	-	✓	-	2	Soft 2-finger gripper	5.5 s	51%
[70]	2020	Kiwifruit	3	CF3-3	✓	-	-	-	1	Soft bionic fingers	4–5 s	94.2%
Other
[17]	2017	Sugar pea pods	5	WidowX Mark II	✓s	-	-	-	1	2-finger gripper	15 s	-
[23]	2021	Grape	7	Jaco 2 Kinova	✓s	-	-	✓	1	2-finger gripper/cutter	-	-
[73]	2016	Orange	6	ARC Mate	✓	✓	✓	-	2	4-finger gripper/suction	-	Up to 95%
[72]	2019	Citrus	6	AUBO-i5	✓	-	-	✓	1	Bite-mode scissors	-	78%
[74]	2020	Aubergine	6	MICO Kinova (x2)	✓	-	-	-	1	3-finger gripper (KG-3 Kinova) (×2)	26 s	91.67%
[75]	2016	Coconut	-	-	-	-	-	✓	3	Arm/cutter	-	-
[76]	2021	Plum	6	UR5 CB3	✓	-	-	-	1	Soft 4-finger pneumatic gripper	-	42%
[77]	2020	Pineapple	3	Custom (×2)	✓	-	-	✓	1	2-finger gripper/cutter	12 s	95.56%
[78]	2019	Iceberg	6	UR10	✓	-	-	✓	1	Soft gripper/cutter	31.7 s	88%
[79]	2022	Cotton	3	Custom	✓	-	-	-	1	3-finger gripper	4–18 s	66–85%

and

Table 2. Aerial robotic manipulation systems for harvesting. Evaluation metrics of harvesting cycle time (time) and detachment success rate (accuracy) are included. Grasping from the stem is marked with “✓s”; all other cases refer to direct contact grasping of the fruit.

			Manipulator		Detachment Method				End Effector		Evaluation
Ref.	Year	Crop	DOF	Type	Grasp	Vacuum	Rotate	Cut	DOF	Type	Time	Accuracy
[88]	2019	Apple	3	Solid arm	✓	-	-	-	1	Cup-shaped gripper	-	-
	2021									3-finger gripper
[89]	2020	Pomegranates	3	Solid arm	✓	-	-	✓	1	3-finger gripper/cutter	-	-
[90]	2021	Coconut	-	-	-	-	-	✓	1	Cutting blade slider crank mechanism	-	-
[91]	2017	-	2	Custom	✓	✓	✓	-	1	2-finger gripper cups/suction	-	-

for ground and aerial systems, respectively.

4.1. Ground Harvesting End Effectors

Ground harvesting systems consist of three main parts: (1) a recognition system for fruit identification and location, (2) a picking system of a robotic arm with the appropriate end effector, and (3) a mobile platform, which is the ground robot that moves inside the fields during a harvesting. End effectors of ground robotic systems are reviewed in the upcoming section. Τhe categorization of the bibliography, in this work, was based on the harvested product. Alternatively, a categorization could be pursued based on the characteristics of the end effector, for example, based on its type or the way it detaches the fruit and so on. The characteristics of all reviewed ground harvesting end effectors are summarized in Table 1.

4.1.1. Heavy Crops

Special end effectors have been proposed to harvest heavy crops, such as pumpkins, cabbages, watermelons, and melons. In [51], a robotic arm design was proposed for harvesting heavy crops, such as pumpkin and cabbage. The authors presented the kinematics of the robotic arm; however, details regarding the end effector, practical application, and evaluation of the system were not presented. The same authors in [52] performed minor modifications in their initial harvesting system design and measured its experimental performance. In both studies, the authors used the same end effector, described in [53]. It was a 2 DOF robotic end effector with five fingers based on the physical properties of heavy crops, such as pumpkins. Each finger was equipped with two digital touch sensors to detect the heavy crop, two micro switches for emergency situations, and two blades for cutting the crop’s stem. The harvesting method consisted of grasping, lifting, stem cutting, and transporting. The same research team, in [33], presented a detailed design of the heavy crop harvesting end effector, a five-finger anthropomorphic hand with an electric drive and internal impactive gripping mode, probably the same as that used in their previous works, with minor improvements. The fingers had a combined mechanism to support various sizes of pumpkins, watermelons, and other heavy fruits. Rubber covers were placed on each finger, microswitches to control their motion, and capacitive sensors to sense the grasping. Moreover, sharp blades connected to both sides of each finger were embedded to cut the stem of the crop. From a technical point of view, results on the evaluation of the specific end effector for heavy crops indicated that the fingers had sufficient capability for the maximum payload of the system. Moreover, results showed that the control unit and supporting algorithm could control the system properly.

In all cases, a five-finger gripper with a cutter was used, inserting minimum updates to the initial end-effector design. The latter is due to the large volume and weight of heavy crops. A multifinger gripper could hold such fruits firmly around their perimeter with greater safety and could support their large weight when removing them. Moreover, the stem of such fruit is rigid and thick; therefore, a cutter is considered necessary to detach the fruit from the stem. In conclusion, grasping force control with respect to the size of heavy crops is considered necessary for the design of heavy-crop end effectors for maximum volume of harvested fruits, with a greater contact force with a balanced distribution in all fingers to be applied. The latter could reduce the damage rate and increase the harvesting success rate.

4.1.2. Tomatoes

Tomato harvesting end effectors have also been proposed in the literature. In [16], a cherry tomato harvesting robot was presented. The harvesting system included a two-finger gripper as a harvesting end effector. The robot could grasp a tomato and pick it without damage. The latter was accomplished by measuring the force between the gripper and the tomato by using a sensor (Interlink Electronics 1.5″ Square 20N FSR) to measure force of up to 20 N. Additionally, the sensor included a polymer thick film device to decrease resistance when the applied pressure on the tomato was increasing. Cherry tomato harvesting was also implemented in [54]. The end effector was a double cutter to cut the stalk and a gripper fixed to the cutters so as to hold/release the stalk. The driving cylinder could trigger (pull/push) the active cutter to rotate. Reported technical problems due to the end effector included failures of holding the stem due to collisions with the main stem near the tomato bunch and failures to support the tomato bunch and hold it reliably after being removed. A cherry tomato end effector was proposed in [18] (Figure 5a). It regarded a gripper, with a semispherical shape, able to grasp spherical objects, such as tomatoes. On the edge of both cups, blades were added so as to cut the tomato stem. By proposing that simplified mechanism of passively cutting the stem, the researchers improved the system’s reported harvesting time since no stem recognition process was taking place. The development of an autonomous tomato harvesting robot is described in [55] (Figure 5b). The end effector was a rotational plucking gripper. More specifically, the authors employed a three-finger gripper and an ever rotational joint to deliver a safe plucking mechanism. However, the proposed design needed improvements since the gripper could grasp multiple fruits in case the clusters were cluttered, and the calyx could be damaged when the angle of the stem was deeper from the rotation axis. A tomato harvesting end effector was developed in [56]. The end effector was a share-type gripper, capable of grasping, cutting, and detaching tomatoes. The gripper was composed of a telescopic cylinder, an air pump, a magnetic valve, a relay, and a shear. Since tomatoes are soft and can be damaged easily, special attention was paid to the control system, toward increasing precision regarding tomato detection and movements. Failures of the end-effector system were reported and were attributed to overlapping of fruits, leaves, and stems. However, the overall system reported efficient environmental adaptability and high success rates. In [57], Zhao et al. reported a tomato harvester with dual-arm manipulators, equipped with two different types of end effectors, namely, a cutting device and a vacuum cup for grasping, which work cooperatively as follows. First, the suction end effector would approach the tomato center to stabilize the tomato, and second, the cutter would cut it free from the stem. The cutting of the stem was made repetitively until the suction end effector gripped the tomato. The cooperation of two different types of end effectors was proven to considerably improve the efficiency of harvesting. An evaluation performance of the aforementioned harvester was provided in a future work of the authors [58], with minor improvements of the overall system architecture and algorithms. Vu et al. in [21] designed a four-finger gripper end effector for tomato harvesting. The end effector combined the gripper with a vacuum suction nozzle able to move simultaneously with the fingers. The fingertips were equipped with pressure sensors and rubber pads. When the end effector would move toward a fruit, the fingers would open, and the vacuum nozzle would extend farther. The novelty was in the movement of the vacuum nozzle; a linear motion drive was used to move the fingers and the suction nozzle synchronously with the aim of a rack and pinion. The end effector was not tested in operating conditions. In [24], Jun et al. developed an end effector for tomato harvesting. It comprised a grasping module and a cutting module. Grasping was based on a suction gripper, whereas cutting was based on a scissors-shaped module. The proposed soft material suction gripper developed a pressure difference between inner and outer surfaces, thus enabling the grasping of tomatoes. The cutting module was composed of a pair of scissors with a supporting rotating scissor blade. The soft suction pad had the advantage of adapting with flexibility to unstructured objects. However, the mechanism would require structural changes for heavier tomatoes that would need to be harvested/lifted. Indicative end effectors for tomato harvesting applications are illustrated in Figure 5.

As a general observation from the referenced literature, it should be noted that in tomatoes, a combination of grasping and cutting was proven to be the quickest and most reliable harvesting method. Tomatoes’ grasping can be easily supported by a two- or three-finger gripper due to the fruit’s size and weight. Tomatoes are not easily detached by a simple grasp and pull. Therefore, a stem detachment method needs to be considered, by either cutting or rotating the tomato stem. A vacuum can also be applied as a supporting tool; however, the skin of tomatoes although firm, is thin and can be easily pierced. The latter could be resolved with a suction pad, which is soft and adaptable to the round surface of tomatoes. In the case of cherry tomatoes being in bunches, the 3D information of both plant stems and fruit stalks’ posture is necessary toward developing conflict-free end effectors. A grasp state estimation of the gripper could also improve the harvesting rate so as to ensure that the fruit is successfully grasped before being removed from the plant. Moreover, the latter could prevent tomatoes from being cut with their sepal, which may damage the tomato skin during storing and transportation.

4.1.3. Strawberries

Xiong et al. [59] designed a harvesting end effector for strawberries. The end effector consisted of three active fingers and three passive cover fingers combined with a cutting mechanism. The end effector could pick, transmit, send, and store the strawberries. Four tension springs could keep the cover fingers adhering to the active fingers. The cutter, composed of two cutting blades, was activated after the closure of the fingers in order to cut the stem. The target location was identified by three infrared (IR) sensors on the gripper. Most failures of the harvesting system were reported when strawberries were in clusters due to the detection algorithm or to the gripper trying to isolate the berries. The performance of the gripper was evaluated in simplified environments, considering isolated strawberries. The same authors, in [60], improved the design of the end effector. The new version was based on a gripper with three cable-driven clamps. The gripper had a hollow space under the fingers, where a custom-designed punnet was attached. The clamps would open simultaneously by a servo motor and close by a torsion spring. The attachment of the punnet was verified by an IR sensor; furthermore, two additional IR sensors were used at the bottom of the fingers to estimate the weight of harvested strawberries. Moreover, a sponge tongue was placed on the top of the clamps to reduce the impact. In-field testing reported failures due to the limitations of the vision system and the inadequate dexterity of the gripper. De Peter et al. [61] developed a gripper for detaching strawberries, based on a soft, 3D-printed framework. Soft-touch grippers allow for more delicate handling, especially when manipulating soft juicy fruits. The proposed gripper performed a similar movement of the robotic arm to grasp the strawberry, including a rotational motion to detach it easily. A proposed future work included optimization on the picking mechanism so as to reduce picking time and optimization of the gripper’s finger dimensions so as to control the distribution of contact force over the fruit.

Strawberries are soft and sensitive and can be easily damaged. Therefore, although three fingers are maybe enough for their handling, more passive fingers should be inserted in the gripper’s design so as to secure the fruit between them. Suction devices are not recommended for delicate fruits, such as strawberries, while soft-touch fingers are more suitable for their manipulation. In general, isolated strawberries are easy to locate and harvest, as seen from the performances reported in Table 1. However, in nature, strawberry clusters are dense, and therefore, many challenges need to be addressed to avoid gripper separation failures, such as blockage of the gripper opening due to foliage/branches/strawberry clusters. From a technical point of view, the latter could be resolved by considering an additional manipulator to separate the obstacles (put leaves aside). Moreover, the end-effector size needs to be properly studied so as to easily approach and move between delicate small targets, such as strawberries, and to provide a bigger and controlled contact area with the fruit to eliminate potential fruit damage.

4.1.4. Apples

Apples are firmer; however, they can bruise easily if not handled properly. A grasping end effector for apple harvesting was described in [22]. The end effector consisted of a three-finger gripper. The applied force from the fingers to the fruit was measured by an experimental process, where a human operator applied a three-finger grasp on an apple. Force sensors were placed on his fingers. More specifically, an inertial measurement unit (IMU) was installed on the human’s hand to measure the forces at the point of fruit detachment. The end effector was able to produce a spherical power grasp with form closure of the fruit, thus securing an adaptive grasp to variable fruit geometries and orientations. The distal joints of the fingers had passively compliant flexures. Soft polyurethane pads were placed on the palm and fingers to minimize impact while grasping. A mechanical feature incorporated in the proposed design was passive compliance, proven to considerably increase the robustness of grasping during perception errors regarding the position of the fruit. The end effector did not include any sensors, and grappling was achieved by feedforward control in an open-loop manner. Preliminary application results regarding the aforementioned end effector are presented in [62]. The same authors, in the same year, made modifications to their preliminary harvesting end-effector design [32]. In particular, instead of picking and transporting, the new system pursued picking and catching. The picking end effector remained the same; however, an additional end effector for catching the harvested apples was added. The catching end effector was a gravity-fed plastic 3D-printed pipe, lined with flexible baffles, leading the harvested apples into a storage container. The two end effectors worked cooperatively; after the picking end effector approached an apple, the catching end effector prepared its position underneath the apple, assuming a vertical drop. It should be noted that the proposed pick-and-catch method reported half the average picking time compared with pick-and-place. Hohimer et al. [63] designed a soft-robotic end effector for apple harvesting, as illustrated in Figure 6. The prototype consisted of three compliant pneumatic actuators, positioned symmetrically about a circle around a soft, flexible palm. The two upper actuators were shorter to stabilize the apple in the palm, while the third was longer to fully wrap the apple. The actuators were 3D-printed and tested for sufficient grasping strength for the fruit detachment. The soft robotic end effector actuators were not harmed during collision and reported higher grasping speed compared with previous end-effector designs. However, identification failures were reported due to occlusions. In [64], the proposed apple harvester used an end effector with a four-finger gripper. However, details regarding the design of the end effector were not presented. The robotic hand would grasp the apple and detach it by rotating it four times to twist its peduncle. Robotic apple harvesting was also investigated by Bu et al. [65], who also developed an appropriate end effector. Their research concentrated on picking patterns and dynamic simulations to identify the optimal picking model. The end effector combined grasping and peduncle twisting by using a 3D-printed bowl-shaped gripper with an eccentric connector. Results indicated that the optimal detachment can be achieved by a horizontal pull with a bending and a twisting motion.

Note that, in apple harvesting, only grasping end effectors are sufficient for fruit detachment. This is due to the firm nature of apples that can be easily handled by a three-finger gripper combined with an easy detachment of the fruit from the stem by just either pulling it or rotating it without damaging the tree/branches; stem cutters are not considered at all in the referenced bibliography. The latter reduces both design costs and the complexity of the system. Suction is also not applied since apples can be easily reached, usually hanging out of the foliage, and are relatively heavy to be vacuumed; it would require greater suction forces that would risk damaging the fruit skin. In order not to bruise their skin, since pulling applies forces, soft grippers are highly recommended for apples.

4.1.5. Sweet Peppers

In [19], two different types of end effectors for sweet pepper harvesting were designed and evaluated. The first end effector (Fin Ray) could grasp the fruit and cut the peduncle. It consisted of a four-finger gripper and a pair of scissors mounted above the fingers. A position sensor would decide whether the fruit was grasped or not. The second end effector (Lip-type) could grasp the fruit body with a suction cup and cut the peduncle. It used a vacuum sensor to detect the grasp success. Both lips of the end effector would close to cut the peduncle with a knife spanned across all the length of the upper lip. The system reported failures in successful harvesting related to the end-effector system: grasp failure due to misplacement or weak grasp/suction force and cut failure due to partial or missed cut of the stem. Lehnert et al. [66] designed a harvesting robot for sweet peppers, namely, “Harvey”. The harvesting end effector was able to grip the fruit with a suction cup and then cut it from the peduncle with an oscillating blade. A pressure sensor on the vacuum was used to confirm the correct attachment of the suction cup. A decoupling mechanism was applied to allow gripping, followed by cutting sequentially and independently. The same research team later presented an improvement of Harvey, reporting higher evaluation performance; however the same end effector was used [67] (Figure 7a). The main advantage of the proposed end effector was the passive decoupling mechanism, which did not require additional actuators and allowed for higher harvest success rates. In [68], Lee et al. developed a pose-control device for sweet pepper harvesting. The end effector propped the fruit from underneath to position its peduncle to the cutting pneumatic cylinder through pose control. The cutting was implemented vertically to the peduncle (Figure 7b). The end effector was specially designed based on the characteristics of a cultivar of South Korea. Therefore, the dimensions of the end effector were according to the sweet pepper’s average length and diameter. Most harvesting failures were observed due to the diameter of the stem being bigger than the entrance of the cutting segment, while only peppers of specific size were considered. Arad et al. [69] designed a plant stem fixation mechanism with a vibrating knife to harvest sweet peppers with a robot (Figure 7c). The end effector was able to cut the pepper peduncle. When cutting was starting, the end effector was moving downwards, and the fixation mechanism was lifting so as to give room to the knife to cut downwards through the pepper’s stem. A camera and LED lighting were also mounted on the end effector, which was driven by an electrical motor. A catching device of six metal fingers was attached to the end effector and placed underneath the cutter so as to hold the harvested peppers. The fingers were coated with soft plastic and could bend backwards in case of hitting an obstacle. Failures were reported to the end-effector system mainly due to the knife slipping off the stem in longer and more vertical stems as those of the variety “Sardinero”. Indicative end effectors for sweet pepper harvesting applications are illustrated in Figure 7.

Sweet peppers have thick stems and cannot be detached from the tree by just pulling or rotating. Moreover, the crop branches are delicate, and extreme pulling could result in damaging the plant. Therefore, for sweet pepper harvesting end effectors, cutting devices are necessary. Cutters can directly cut the stem without the need to grasp the fruit body. In order to stabilize the fruit prior to grasping and cutting the stem, instead of grippers, suction devices can be employed. Peppers are light in weight and have a firm body; therefore, suction could easily guide the fruit and hold it steadily to be cut from its stem without being injured. However, it should be noted that different cultivars have different characteristics. For instance, the “Waltz” sweet pepper cultivar has less fruit clusters than the “Stayer” cultivar, as well as a different stem length. The latter can influence the end-effector performance when the (same) system is applied to different cultivars.

4.1.6. Kiwifruits

Robotic harvesting of kiwifruit was implemented in [20] by Williams et al. The proposed end effector was a grasping mechanism with a soft two-finger gripper. The mechanism grasped the kiwi and then rotated it to detach it from the stem. An asymmetrical four bar linkage was used for the rotation. The gripper mechanism was made of food-grade silicone molded around 3D-printed fingers. Silicon sections contained air pockets to better adjust to the shape of the fruit. A pneumatic cylinder activated both the gripper and the rotation of the end effector. Failures to grip were noticed when the fruit could not detach from the plant, due to obstructions, positioning errors of the end effector, or absence of friction between the end effector and the fruit. Absence of friction occurred due to the silicon skins that were added to the silicon grippers for the end effector to slip easily into clusters of fruits without damaging them. A prototype end effector was assembled in [70] for kiwifruit harvesting, comprising a fiber sensor, a Hall effect position sensor, a pressure sensor, a stepper motor, and two soft bionic fingers. The end effector would approach a kiwifruit, the fiber sensor would detect the fruit between the fingers, and then the motor would initiate a downward movement to detach it. The design of the proposed end effector was previously analyzed in [71]. Performance evaluation revealed that the combination of fiber sensors, pressure sensors and adjusting spring for fruit size and shape tolerance could ensure a nondestructive accurate end effector harvesting system. Failures were reported in cases where the end-effector fingers were crowded to separate a bunch of fruits and thus not able to accurately reach the target.

Kiwifruits are stone-hard when harvested, and their stem is thin. Therefore, a two-finger gripper can easily grasp the fruit without harming its surface and remove it from the tree by just pulling or rotating. Soft grippers could be used to eliminate altogether the risk of fruit injury. However, the uneven pulling force of the two fingers may result in failures, and therefore, more fingers should be considered. Moreover, the width of the fingers needs to be adjustable so as to easily envelope different sizes of kiwifruits.

4.1.7. Other Agricultural Products

Wang et al. in [72] described the design of an end effector for citrus harvesting. The proposed end effectors resembled a mouth and a pair of scissors; they could harvest with a biting mode, comprising two symmetrical jaws with blades, an upper and a lower. Biting mode could ensure a large-enough opening angle of the jaws to envelop the fruit. Results indicated that the end-effector design combined with harvesting posture optimization could lead to promising results. A proof-of-concept harvesting robot for sugar pea pods was developed in [17]. However, the interest was mainly in the correct position of the cutting action rather than in the design of the end effector. All experiments were conducted with a two-finger gripper instead of a cutter, and the pregrasping position of the fingers around the stem was evaluated. Thus, an infrared proximity breakout sensor (Vishay VCNL4000, SparkFun Electronics) was mounted underneath the fingers to measure the distance between the nearest detected part of the pod and the cutting point. A grape harvesting end effector was presented in [23]. The end effector consisted of a 3D-printed robotic two-finger gripper and a mounted cutting tool. A detailed design of the end effector is not yet available since it is patent pending. A generic fruit picking end effector applied in orange harvesting was introduced in [73]. It was a hybrid multigripping device able to grasp with a four-finger gripper covered with polychloroprene and secure fruits of medium size by a combination of vacuum and tactile forces. Rotating and pulling the fruits were also employed for effective detachment and reduced peel damages. Circumscribing vacuum pads were adjusted to lateral position based on the irregularities in fruit sizes, enabling firmer attachment on fruit surfaces. The grasping force could be controlled by five electrical pressure regulators. An aubergine harvesting robot was presented by Sepulveda et al. in [74] based on dual-arm manipulation able to reproduce complex human movements. If an aubergine was occluded by leaves, an arm was engaged to move leaves aside, while the other one would pick the exposed aubergine with a commercial three-finger gripper (Kinova Gripper KG-3). A prototype robot for coconut harvesting was presented in [75]. The end effector of the robot was an arm of 3 degrees of freedom (DOF) made of a hollow cylindrical steel pipe since the end effector had to be light but hard enough to resist to pressure while cutting the hard stem of the coconut. The cutting device, that is, a saw teeth blade, was based on top of the cylindrical part, driven by a motor at high speed. Brown and Sukkahier [76] tested two gripper designs for plum harvesting: a tendon-driven parallel gripper actuated by a servomotor and a four-silicone pneumatic finger end effector. The 3D-printed base palm enclosed a wide-angle camera and was covered with protective padding. The use of soft materials was selected to reduce collision impact and change the harvesting performance. The best harvesting rate was reported when a soft gripper was combined with complex motion. Automated pineapple harvesting was investigated in [77]. The end effector of the harvesting system was activated by pneumatic actuators and equipped with a cage-shaped two-finger gripper to capture the pineapple inside it and a saw-disc cutting device under the cage to detach it from the stalk.

Birrell et al. developed an iceberg lettuce harvesting system [78]. The final end effector used two pneumatic actuators, one for grasping (a soft moving gripper and a fixed gripper lined with foam) and one for cutting (a moving cutting knife with a timing belt system to transfer the motion to both sides of the blade for smoother movements). The developed end-effector system could achieve damage-free harvesting, with consistency and repeatability, due to a control method utilizing force feedback to detect the ground. The design of a cotton harvesting end effector was presented in [79] composed of a three-finger, moving pinned belt, underactuated end effector. Table 1 summarizes details regarding the referenced literature for ground robotic manipulation systems for harvesting.

As a general guideline, the selection of an appropriate end-effector tool or a combination of tools depends on the harvested crop. For tree structures that are sturdy and can withstand pulling without a high risk of removing branches along with the fruit, simple grasping grippers are preferable. Detachment can be accomplished by grasp and pull or rotate for greater convenience. Suction is applied to firm fruits, which are not relatively heavy, so as to facilitate grasping or cutting. Cutting from the stem should ideally be considered in all cases; however, it is not always easy; therefore, it is not applied if not necessary. Inserting an additional stem cutting device to the end-effector design would add complexity to the design, coordination, and control of the end effector. Moreover, stem detection from a vision system is challenging due to illumination/shadowing and other environmental noises, and it is not always visible due to dense crop foliage.

4.2. Aerial Harvesting End Effectors

Aerial agricultural applications are common in the literature. Drones are used extensively to determine crop characteristics [80] in farm-scale, for surveillance [81], monitoring of irrigation [82], temperature [83], soil condition [84], ripeness [85] and yield estimation [86], disease detection [85], and spraying operations [87]. The next major step, to utilize drones further in the crop management cycle, is their employment in fruit-picking and harvesting operations. Small drones could cause less damage in the crop rows compared with a robotic heavyweight vehicle.

Theoretically, drones could be effectively employed for harvesting hard fruits on trees, such as apples, pears, and oranges. Lower-growing and softer fruits, such as strawberries, require gentler handling so as not to be damaged and, additionally, grow at a prohibitive altitude for the drone to reach. Heavy crops are not compatible with the low payload requirement of a drone, while leafy vegetables need gentle lifting from the ground and could be easily torn if harvested from above by a drone. To this end, aerial harvesting is challenging and therefore scarce; however, it is underway.

A cutting-edge company, Tevel, in 2019, introduced autonomous flying fruit-picking robots (Figure 8) [88]. The first prototype’s end effector was a cup-shaped two-finger gripper. In 2021, the same team introduced a full harvesting system. The system comprised a wheeled base vehicle and four quadcopters electrically tethered to the base (Figure 9). The drones used an integrated grasper arm with an end effector. The proposed end effector for apple picking was a three-finger gripper; however, no further detail or evaluation was provided. The system was applied to apples, as well as to other similar-sized fruits, such as oranges and mangoes.

In [89], a hexacopter with a robotic arm for orchards fruit harvesting was presented and tested on pomegranates. The robotic arm with the gripper had to be inferenced with the hexacopter to pluck and hold the fruit. A flight sensor (VL531X) was used to identify the distance between the fruit and the robotic arm. The robotic arm was solid, allowing for horizontal harvesting, with the end effector at one end. The end effector was a three-finger gripper designed to hold the fruit firmly. The gripper was attached with two gears to enable opening/closing. Blades were mounted on the top of each finger for the detachment of the fruit from the stem.

A coconut harvesting drone was developed in [90]. The system was manually operated. The end effector was a cutter with stainless steel blades. A slider–crank mechanism was used to move the blade. Due to the rotary movement of the crank, the cutter could reciprocate back and front and cut the stalk of the fruit. When the drone reached for the coconut, the cutter end effector was activated manually by the controller to initiate harvest. It should be noted that aerial coconut harvesting is challenging due to the height of the tree, the weight of the fruit, and the forces needed to cut the fruit stem.

In [91], a quadcopter drone end effector was designed for picking fruits from an elevated position. The end effector was able to grasp and rotate and was equipped with suction cups to hold the payload firmly. However, an increase in the drone’s payload resulted in displacement in the drone’s 3D-printed arm.

Table 2 summarizes details regarding the referenced literature for ground robotic manipulation systems for harvesting. In aerial operation, grasping from the stem is not applied, since this is a detailed job that requires stability, which can only be provided by ground robotic manipulation systems. The evaluation performance of the referenced systems is not yet available. It is obvious that the development and deployment of UAVs to perform harvesting and fruit picking is at its early stages and still has a long way to go. It could be concluded that UAV-based harvesting can only be applied to fruits hanging from trees, which are easy to detect and reach from the air. Moreover, the fruits must be relatively heavy so that the air stirred by the drone’s propellers would not move the fruits erratically around, thus obstructing their grasping.

5. Discussion

From the referenced literature of the previous sections, it is obvious that many harvesting end effectors have been successfully designed, applied, and evaluated under real-world conditions, yet on a limited scale. In this work, 41 robotic harvesting applications on 17 different kinds of crops are reviewed from 2016 to 2022. The distribution of the referenced literature per crop types is illustrated in Figure 10.

Most applications focus on tomatoes (22%), followed by apples (17%) and sweet peppers (12%), which are high-demand agricultural products globally. This can be attributed to the fact that all three fruits are firm, have no extreme variations in size and weight, are relatively hard so they are not easily injured from mechanical manipulation, and thrive in all countries. Moreover, all fruits hang from the crop, and thus, they are easy to detect due to their color and shape. All of the above make those fruits ideal for automating their harvest. However, different characteristics can be detected between different cultivars of the same fruit. Research finding indicated that for the same kind of fruits, characteristics such as fruit dimensions, peduncle length, and diameter influenced the harvesting success and posed problems to the adaptation of end effectors between different cultivars of the same fruit. Future work could, therefore, focus on identifying breed cultivars that would be ideally suitable for robotic harvesting.

The distribution of end-effector types used in the referenced literature is shown in Figure 11. The four types are: (1) contact-grasping grippers (fruit-holding and stem-holding), (2) suction devices, (3) rotation mechanisms, and (4) scissors/sawlike tools. When a combination of different types is employed, all involved mechanisms are considered separately. As can be seen from Figure 11, the most popular end effectors are the grasping multifinger grippers (contact-grasping grippers). Almost in half of the referenced literature (~46%), a gripper was used, either alone or combined with an additional mechanism. The second most popular end effectors are the scissor/sawlike tools (31.4%), then the suction devices (11.4%), and finally, the rotation mechanisms (10%). Review findings revealed that the most effective way to harvest a fruit is by grasping (either the fruit body or the stem) and detaching it from the stem (by pulling or cutting). Moreover, keeping the end-effector design simple results in less complex control strategies and, thus, better performance and quicker harvest time.

Since grippers are the most commonly used end effectors, Figure 12 indicates the types of used grippers in the referenced literature based on their finger number. The most used types are the two-finger and three-finger grippers. It is worth mentioning, that two-finger grippers are the most common gripper type in demand from manufacturers’ catalogs [92]. Soft robotics [93] proposes a four-variable list to determine the appropriate number of fingers for each application requirement: size, shape, structure, and mass.

In general, the more the fingers, the better the manipulation. Multi-fingered grippers provide a more secure grasp; however, they call for more complex control strategies. However, the number of fingers needs to comply with the size of the product; small products require just two fingers, whether bigger ones require four or five fingers. The shape of the product plays a vital role. Square or circular products, such as round-shaped fruits, require between two and five fingers for effective grasping. The structure of the product is also important; simple structures require fewer fingers compared with complex structures, which require more. Finally, the mass of the product is also a considerable factor; as the mass of a product increases, more fingers are needed to support the product. Heavy-crop referenced applications in this work, for instance, use five-finger grippers, while for tomatoes and apples, in most cases, three-finger grippers are selected, as shown in Table 1. In all cases, the width of the fingers needs to be adjustable so as to easily envelope the same fruits of different sizes.

The detachment methods include grasping, vacuum, rotation, and cutting. Different detachment strategies have been employed alone or in combination. In Figure 13, all combinations in the referenced literature are presented. Seven combinations are reported: (1) grasp-cut, (2) grasp, (3) grasp-rotate, (4) cut, (5) vacuum-grasp-rotate, (6) vacuum-cut, and (7) vacuum-grasp. As can be seen from Figure 13, the most common method of detachment is the combination of grasping and cutting. Grasping and cutting can ensure that both the harvested fruit and the crop are secure and not damaged by excessive pulling. The latter combination is also the most efficient in terms of evaluated accuracies, as shown in Table 1. Grasp-cut methods, compared with vacuum-cut, have reported lower harvesting cycle times and higher detachment success rate in all cases. Grasp-cut from the stem was reported in only 3 out of the 13 reviewed articles, although it is considered advantageous due to the minimum contact with the fruit body, which reduces the risk of damage. This is attributed to the encountered difficulties mainly of the vision system to properly detect and locate the grasping/cutting points on the stem, as already mentioned. End-effector systems that engage rotation do not consider the stem detection; therefore, they are more robust to estimation errors. Grasp-rotate combination comes third in the referenced literature. However, by just grasping and pulling the fruit, the harvesting system is quicker and less complex; therefore, grasping is the most popular detachment method for fruits, yet only for fruits that can be detached easily from the plant without cutting their stem. Additionally, as shown in Table 1 and Table 2, most of the end-effector systems are of 1-DOF. Fewer DOFs also minimize the possibility of damaging the fruit/plant since they imply a reduced contact with the fruit.

Although the level of commercial robots for fruit harvesting has not yet been high, reported results are encouraging and indicate future perspectives. Unfortunately, not all referenced harvesting applications were tested and numerically evaluated. The lack of performance evaluation for standard testing conditions makes it difficult to compare several different systems and to identify the optimal harvesting strategy. At this point, it should be noted that any evaluation metric shown in Table 1 and Table 2, for example, time and accuracy, involves all aforementioned aspects; in other words, they do not refer solely to the end-effector performance, but rather, they refer to the overall system. These metrics are included in the tables as a more general information of interest, without being able to be used directly for comparisons between the various systems even if they are intended for the same kind of crops. Therefore, the outcomes of this review highlight that inconsistent performance indicators, that is, picking times, are reported; some studies report picking time as the time included for the robotic arm to reach, pick, and store the fruit [54]; others also include the time of fruit recognition by the vision system [16]; others consider the time of just reaching and removing the fruit from a predetermined position [76], and so on. Thus, it is not easy to determine best practices. Moreover, neither formal design nor functional requirements are reported in most of the referenced literature toward evaluating comparatively the design process of each system. For aerial harvesting systems, design details of end effectors and application results are absent altogether.

Drones are used extensively in harvesting applications mainly for path planning and mapping of the crop field [94]. It is clear that actual drone harvesting operations are still scarce, and they require a sophisticated design and algorithms. However, the view of up-to-date drone harvesting applications indicates that an emerging tool with multiple opportunities for sustainable farming is at our service.

So far, significant research work has been reported worldwide, and further effort is ongoing to resolve particular harvest-oriented problems and develop cutting-edge robotic technologies for sustainable agriculture. The main difficulties of robotic fruit harvesting research are low harvesting speed, insufficient fruit recognition in variable environmental conditions, low harvesting success, difficulties in robotic arm kinematics, insufficient manipulator controls, difficulties in autonomous navigation, difficulties in the coordination of detect-grasp-cut actions, and high manufacturing costs. On top of those, the different physiologies of harvesting fruits introduce additional constraints that need to be considered.

To overcome those obstacles and develop an overall robotic harvesting system, three main issues that need to be addressed by priority are robotic navigation, fruit detection and localization, and fruit removal from the plant. Even if extremely reliable perception systems were developed, as well as recognition systems that could locate accurately obstructed fruits in variable environments, still, this would not be enough if appropriate end effectors were not available for grasping and detaching the fruit. Effective grasping and cutting systems are essential for harvesting the detected fruits without causing any physical damage to either the fruit/crop or the harvesting system. The end effector needs to be able to cope with soft, delicate, juicy fruits with respect to their various shapes and sizes at a higher possible speed and precision. Modifications to current pruning practices are also encouraged for minimizing obstacles, such as leaves and branches, for both the detection system and the movement of the end effector.

Moreover, sensor incorporation to the end-effector design would be able to provide useful feedback to the system to identify and avoid failures, therefore improving harvesting efficiency and system robustness. Research finding indicates that force sensing on the end effector could give valuable feedback on the grasping status during harvesting. Additionally, smart sensors need to be developed for intelligent end-effector systems. Abilities of self-learning, adaptation to dynamically changing conditions, and on-site decision making need to be considered in related future research of intelligent end-effector systems. Real-time decision making is critical for variable selective harvesting. More specifically, fruits, for example, apples, grapes, and tomatoes, ripen heterogeneously; therefore, instead of harvesting them massively, a selective harvesting of fruits of the same maturity degree is required. Selective harvesting is difficult to automate, and therefore, it is typically carried out by humans [50]. The design of end effectors for selective harvesting is even more demanding since end effectors need to operate safely so as to leave the leaves, the plant, and the remaining fruits intact. Future trends highlight the use of soft robotics. Soft robotics could minimize damage risks, increase the contact surface of the gripper with the product, and thus provide a firmer grasp. A soft touch, controllable grasping, and a gentle manipulation of the fruits are essential in order to avoid reducing the value of the crop for the current year and for the following harvesting years. Grasping or suction forces, orientation of the stem or fruit body, and so on could be determined by smart sensory feedback. End effectors inspired by the biological properties of human hands may also be considered. None of the referenced end-effector systems employed overall dexterous manipulation of the plant/fruit during harvesting. Dexterous multitasking and multisensory mechanical hands able to imitate the hand movements of an experienced harvester, enabled with multiple DOFs and sophisticated control strategies, are challenging and may result in complex systems; however, they should be investigated in future research since they could provide reliable solutions for robust robotic harvesting well fitted to multiple types of fruits.

In general, an end-effector design is multifaceted and could be extended in many ways. Future works could focus on variable selective harvesting end effectors or could provide information regarding harvesting speed, ways to improve control algorithms and reduce damage, and system robustness or review the overall costs of end effectors, suggesting feasible and effective harvesting solutions. A real-world cost-effective robotic fruit harvester has still a long way to go until a fully autonomous prototype is commercially available. Based on the above, it is obvious that there is room for future research for numerous innovations.

Overall, the development of robotic end effectors for agricultural harvest seems to follow certain patterns and to operate optimally under certain conditions. For example, an end effector is typically structured, and it also operates in a structured environment. The detachment of a fruit calls for a harmless stability of the operation location and for a reasonable end-effector dexterity typically pursued by an adequate number of fairly flexible fingers mounted on the end effector. Personalized fruit treatment is preferable especially when fruits occur in clusters. Furthermore, different types of end effectors may be combined collaboratively, for example, a suction end effector and a cutter end effector. Sometimes an additional manipulator is required to move aside obstacles, such as leaves. An end effector of interest should be supported by intelligence and/or knowledge that considers disparate sensory signals fed back, including images, 3D structures, forces, and other. “Educated guesses” might also be necessary, for example, regarding unseen objects, such as hidden stems.

Regarded as a product of evolution, the human hands are a living proof of successful end effectors for agricultural harvest. Likewise, effective robotic end effectors are expected to bear substantial similarities to human hands. Note that a human hand alone cannot harvest everything; for example, it cannot harvest thorny fruits. Hence, a human uses specialized tools. Likewise, a robotic end effector for agricultural harvest is advisable to have the capacity to use specialized tools, including gloves. In other words, instead of developing a specialized end effector per cultivar, it may be preferable to have the same end effector, for example, a mechanical hand, handling a specialized tool per cultivar in a learnable pattern. Between end effectors of comparable capacity, the simplest one is expected to be preferable due to the lower cost expected.

Robots, including their components, are driven by computer software, which implements certain mathematical models typically developed in the (nonstructured) Euclidean space R^N. The abovementioned needs, namely, regarding agricultural robotic end effectors, call for intelligent modeling that may also deal with structures. In the latter context, the lattice computing (LC) information processing paradigm [95] emerges to be promising [96,97] because it lends itself to mathematically rigorous analysis and design, as it will be demonstrated in a future work.

6. Conclusions

While the shortage of labor is driving the demand for automation in harvesting operations, the emerging capabilities for harvesting with high consistency and speed motivates research for ground and aerial robotic harvesting solutions. Moreover, the introduction of new modern technological trends in the agricultural sector could potentially attract a new generation of farmers, as older generations of farmers retire. This work aims to review and discuss the latest developments in the implementation of end effectors in real-world robotic harvesting operations. A detailed summary is provided regarding end-effector types, detachment methods, and sensory control strategies. Research indicated grasp-and-cut as the most effective detachment method, while contact-grasping grippers with two and three fingers were the most commonly used in real-world applications. Harvesting automation mainly involved tomatoes and apples. This can be attributed to the fact that both are common crops worldwide, but also to the fact that both harvested products are without extreme variations in size and weight, relatively hard so they are not easily injured, and easy to detect due to their position in the crop, distinct color, and shape. Moreover, their circular structure is simple, and for this reason, a finite number of fingers (two or three) are sufficient for their grasping. All of the above facilitate the automation of the harvest.

This review aims to serve as a guide for up-to-date technology for designers or researchers who need a complete picture of the current status of harvesting end effectors and their implementation and evaluation in real-world harvesting applications for both ground and aerial robotic systems. In the future, for more effective end-effector designs, more sophisticated sensors, alternative materials, and intelligent control methods will be constantly required.