Processing

Please wait...

Settings

Settings

Goto Application

1. WO2020195416 - DETECTION, TRACKING AND 3D MODELING OF OBJECTS WITH SPARSE RGB-D SLAM AND INTERACTIVE PERCEPTION

Publication Number WO/2020/195416
Publication Date 01.10.2020
International Application No. PCT/JP2020/006950
International Filing Date 14.02.2020
IPC
G06T 7/55 2017.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
7Image analysis
50Depth or shape recovery
55from multiple images
G06K 9/00 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
G06T 7/33 2017.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
7Image analysis
30Determination of transform parameters for the alignment of images, i.e. image registration
33using feature-based methods
CPC
B25J 9/1697
BPERFORMING OPERATIONS; TRANSPORTING
25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
9Programme-controlled manipulators
16Programme controls
1694characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
1697Vision controlled systems
G06K 9/00201
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
00201Recognising three-dimensional objects, e.g. using range or tactile information
G06K 9/00664
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
00664Recognising scenes such as could be captured by a camera operated by a pedestrian or robot, including objects at substantially different ranges from the camera
G06K 9/3241
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
20Image acquisition
32Aligning or centering of the image pick-up or image-field
3233Determination of region of interest
3241Recognising objects as potential recognition candidates based on visual cues, e.g. shape
G06K 9/4671
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
46Extraction of features or characteristics of the image
4671Extracting features based on salient regional features, e.g. Scale Invariant Feature Transform [SIFT] keypoints
G06K 9/6211
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6201Matching; Proximity measures
6202Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
6203Shifting or otherwise transforming the patterns to accommodate for positional errors
6211Matching configurations of points or features, e.g. constellation matching
Applicants
  • MITSUBISHI ELECTRIC CORPORATION [JP]/[JP]
Inventors
  • CORCODEL, Radu, Ioan
  • DE ALMEIDA, Diogo, Rodrigues, Marcal
Agents
  • FUKAMI PATENT OFFICE, P.C.
Priority Data
16/365,83127.03.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) DETECTION, TRACKING AND 3D MODELING OF OBJECTS WITH SPARSE RGB-D SLAM AND INTERACTIVE PERCEPTION
(FR) DÉTECTION, SUIVI ET MODÉLISATION 3D D'OBJETS AVEC UNE LOCALISATION ET UNE CARTOGRAPHIE SIMULTANÉES (SLAM) RGB-D CLAIRSEMÉ ET UNE PERCEPTION INTERACTIVE
Abstract
(EN)
An object detection system for detecting and manipulating objects on a workspace includes a three dimensional (3D) sensor configured to acquire and transmit point clouds of a scene, each point cloud including one or more objects in the workspace, manipulator configured to move or grip each of the one or more objects, a memory to store the images and a computer executable program including an object detection-localization program, a segmentation program, a gripping-moving program and a geometry reconstruction program, a processor to perform the computer executable program using the images in connection with the 3D sensor, the memory and the manipulator. The computer executable program is configured to cause the processor to perform steps of generating frames including the point clouds and measurements, generating an initial map from a first frame, moving part of the one or more objects by using the manipulator using the first image, acquiring, after the moving step, a second frame, generating a first dynamic map and a static map from the initial map using the second frame, moving another part of the one or more objects based on the first dynamic map, splitting at least one dynamic map among the first dynamic map and the second dynamic maps when the at least one dynamic map includes a segment being outlier with respect to the one or more objects, reconstructing a 3D geometry of the one or more objects from the split map, and determining when the one or more objects are separated from each other.
(FR)
L’invention concerne un système de détection d'objet destiné à détecter et à manipuler des objets sur un espace de travail comprend un capteur tridimensionnel (3D) configuré pour acquérir et transmettre des nuages de points d'une scène, chaque nuage de points comprenant un ou plusieurs objets dans l'espace de travail, un manipulateur configuré pour déplacer ou saisir l'objet ou chacun des objets, une mémoire pour stocker les images et un programme exécutable par ordinateur comprenant un programme de localisation de détection d'objet, un programme de segmentation, un programme de préhension-déplacement et un programme de reconstruction de géométrie, un processeur pour exécuter le programme exécutable par ordinateur à l'aide des images en connexion avec le capteur 3D, la mémoire et le manipulateur. Le programme exécutable par ordinateur est configuré de façon à amener le processeur à effectuer des étapes consistant à générer des trames comprenant les nuages de points et des mesures, à générer une carte initiale à partir d'un premier cadre, à déplacer une partie du ou des objets en utilisant le manipulateur à l'aide de la première image, à acquérir, après l'étape de déplacement, une seconde trame, à générer une première carte dynamique et une carte statique à partir de la carte initiale à l'aide de la seconde trame, à déplacer une autre partie du ou des objets sur la base de la première carte dynamique, à diviser au moins une carte dynamique parmi la première carte dynamique et les secondes cartes dynamiques lorsque la ou les cartes dynamiques comprennent un segment qui est aberrant par rapport au ou aux objets, à reconstruire une géométrie 3D du ou des objets à partir de la carte divisée, et à déterminer le moment où le ou les objets sont séparés les uns des autres.
Also published as
Latest bibliographic data on file with the International Bureau