MarinRobotics's Vision is development of novel concepts for flexible automated sorting and handling of marine raw materials. These concepts will enable robots to better 'see' and 'understand' when performing sorting and handling of objects automatically in 'sense-think-act' manner. The primary objective in this project is development of an active perception approach for robots, based on the RGB-D sensor mounted to a robot arm, a concept to be used for automatic 3D object reconstruction. During robotic grasping and manipulation of compliant object of biological origin, there is a need for an accurate understanding of 3D model and geometry of the objects prior to grasping and manipulatio. Often, the objects to be manipulated have occluded areas and therefore it is difficult to reconstruct their accurate 3D model in order to do estimation of object pose and grasping pose. In this project is developed an active vision methodology for automated scanning and 3D reconstruction of the objects based on a RGB-D sensor mounted on a robot arm. The method is based on high dimensional point-cloud space and optimization of a specific cost function for viewpoint selection in order to achieve the active vision by visual servoing of a robot arm. The method is real-time and achieves a 3D reconstruction model of an object with a high accuracy and sufficient number of point clouds.
Dette prosjektet har bidratt til at vi har fått en veldig bra samarbeid mellom SINTEF Ocean og INRIA innenfor robotikk. INRIA miljøet regnes som blant de beste i verden i robotikkforskning. Som resultatet av dette samarbeidet ble det sendt en søknad (CUTMAN) til Bionær programmet (september fristen 2018) med INRIA som internasjonal partner. I tilleg så har Dr. Ekrem Misimi, som følge av arbeidet sammen men forskere fra INRIA i løpet av utenlandsopppholdet hevet kompetansen sin i robotikkforskning og spesielt i visual servoing og har utviklet en metode som skal publiseres i løpet av 2019.
Når det gjelder effekter som forskningen i dette prosjektet vil resultere i, regnes det at den nye metoden kan brukes til utvikling av nye fleksible og robot-baserte konsepter for sortering og håndtering av marint råstoff. Dette vil bety bedre håndtering og sortering, mindre svinn og avfall, og bedre utnyttelse av råstoff som vil bidra til en større bærekraft i matproduksjon i Norge.
One of the greatest challenges in the Norwegian marine industry is the lack of technological solutions for an automated sorting and handling of marine raw materials. Many harvesting and processing tasks in the marine industry are still manual based such is the example of roe, milt, and liver sorting in whitefish industry, sorting and post-trimming operations in both whitefish, pelagic and other sectors. As a result, only a portion of marine resources are efficiently used. To enable a higher harvesting sustaniability, resource efficiency and monitoring there is a need for automated solutions for sorting and handling based on 3D robot vision.
Development of novel concepts for robotic automation in marine industry such as sorting and manipulation involves a robot being able to interact (e.g. by grasping ) with compliant objects, such as fish, fillets, or fractions such as heads, roe, milt and similar. These objects typically move or deform when interaction/manipulation is initiated. Given a 3D image by a depth camera, one can compute the interaction vector (e.g. for the grasp), for the object of interest. Once the interaction has begun the object may become partially occluded and begin to move and deform. For this interaction phase an active perception with robot by means of 3D visual servoing is necessary to have a complete overview of the 3D structure of the object and to be able to handle occlusion and deformation. Research activity in MarinRobotics project is therefore aimed at giving a robot the kind of interaction intelligence that human workers have and to enable a robot to 'see' better so that robots can work in a 'sense-think-action' manner in the same way skilled human operators do when they perform sorting/manipulation of marine raw materials.
This leads us to the following hypothesis that will be investigated:
-Active perception by means of 3D visual servoing improves object's 3D exploration and correct 3D localization during sorting/manipulation.