Back to search

FFL-JA-Forskningsmidlene for jordbruk og matindustri

Strawberry Harvester for Polytunnels and Open Fields

Alternative title: Robotisert innhøstingssystem for jordbær

Awarded: NOK 9.7 mill.

The objective of the SHAPE project has been to develop a fully autonomous harvester for strawberry production in polytunnels and open fields. Labour shortage being one of the major concerns for strawberry growers around the globe, there is a need for new technology to help growers automate this process. The project has been built on the already developed Thorvald robot at the Norwegian University of Life Sciences, and know-how within advanced image analysis of fruits at the University of Lincoln and at the University of Minnesota. The consortium had developed a novel and patent-pending gripper that has been used in the project. One of the largest challenges to reach autonomous harvest is the identification and picking of the strawberry using a robotic arm. The berries need to be accurately localized in the field. In the SHAPE project, an improved and more reliable harvest of berries have been achieved. The introduction of a novel method for tracking and picking berries in dynamic clusters, and the work of enhancing the obstacle separation technique have boosted the picking precision. Implementation and tests of these new algorithms at the polytunnel at Norwegian University of Life Sciences (NMBU) have shown that the strawberry harvester is efficient and reliable, and capable of operating continuously for over 80 minutes, which is a significant breakthrough. The project also aimed for the development of methods for determining the ripeness prior to picking the berry. The gripper was equipped with internal sensors that can estimate the ripeness of the berries and evaluate the quality in an enclosed and controlled environment before harvesting. Advanced machine learning has been used to give the robot the ability to reason and make decisions in the field, substantially improving the current state-of-the-art by inspecting each and every berry before picking. This will guarantee that diseased berries are not picked and ensure online quality control and grading of the berries. The consortium has also created and tested a new and universal non supervised segmenter for fruit. Although encouraging results have been obtained, the state of the art supervised algorithms still outperform this new and universal method. Parallel to the work on the robot gripper, three have been some work on developing an infrastructure for data collection in the field. Whilst the robot traverses the strawberry tunnels, it automatically collects data from multiple cameras and stores the images on a remote server where it can easily be downloaded by the research community.

The project has pushed the state of the art of localization and identification of strawberries in a real operating environment. The accuracy of localization and the accuracy of ripeness detection as well as size estimates has been improved. These are important results in themselves, but also the backbone of future strawberry harvesting robots. Identification of berries allows for better mapping and potentially yield detection analysis. Yield detection has the potential to drastically reduce yield loss which will have a huge environmental impact.

The SHAPE project will develop a fully autonomous harvester for strawberry production in polytunnels and open fields. Labour shortage is one of the major concerns for strawberry growers around the globe. There is thus a need for new technology to help growers automate this process. The SHAPE project builds on existing developments within the consortium that puts us in a unique position to close the technological gaps needed to make these systems useful for farmers around the world. The first challenge that needs to be solved is robust and reliable mobility in the field, which is an unstructured environment. The consortium has already developed the Thorvald robot, which is capable of moving autonomously both in polytunnels and in open fields. The second large challenge is the identification and picking of the strawberry using a robotic arm. The fruits need to be accurately localized in the field. We find the fruits using a deep neural network and find the location in the three-dimensional space using 3D camera. We will also develop methods for determining the ripeness prior to picking the berry. The consortium has also developed a novel and patent-pending gripper that will be used in the project. The gripper picks the berries by surrounding them entirely before picking. This makes us able to pick berries in the presence of uncertainties and moving berries. We will also equip this gripper with internal sensors that can estimate the ripeness of the berries and evaluate the quality in a enclosed and controlled environment before harvest. Advanced machine learning will be used to give the robot the ability to reason and make decision in the field, substantially improving the current state-of-the-art by inspecting each and every berry before picking. This will guarantee that diseased berries are not picked and ensure online quality control and grading of the berries. Finally the full on-robot and off-robot logistics of the harvested berries will be developed.

Funding scheme:

FFL-JA-Forskningsmidlene for jordbruk og matindustri