Processing

Please wait...

Settings

Settings

Goto Application

1. WO2020117657 - ENHANCING PERFORMANCE CAPTURE WITH REAL-TIME NEURAL RENDERING

Publication Number WO/2020/117657
Publication Date 11.06.2020
International Application No. PCT/US2019/063969
International Filing Date 02.12.2019
IPC
G06T 5/00 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
5Image enhancement or restoration
G06T 5/50 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
5Image enhancement or restoration
50by the use of more than one image, e.g. averaging, subtraction
CPC
G06T 2207/10016
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
10Image acquisition modality
10016Video; Image sequence
G06T 2207/10024
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
10Image acquisition modality
10024Color image
G06T 2207/10028
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
10Image acquisition modality
10028Range image; Depth image; 3D point clouds
G06T 2207/20081
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
20Special algorithmic details
20081Training; Learning
G06T 2207/20084
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
20Special algorithmic details
20084Artificial neural networks [ANN]
G06T 2207/30196
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
30Subject of image; Context of image processing
30196Human being; Person
Applicants
  • GOOGLE LLC [US]/[US]
Inventors
  • PANDEY, Rohit
  • TAYLOR, Jonathan
  • BRUALLA, Ricardo Martin
  • YANG, Shuoran
  • PIDLYPENSKYI, Pavlo
  • GOLDMAN, Daniel
  • FANELLO, Sean Ryan Francesco
Agents
  • GUENTHER, Brett
  • BENNETT, Daniel M.
  • BELLERMANN, Mark R.W.
  • BRAKE, R. Edward
  • FORD, Timothy D.
  • HUGHES, William G.
  • KEY, Joseph F.
  • GROVER, Melanie
  • SODERBERG, J. Richard
Priority Data
62/774,66203.12.2018US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) ENHANCING PERFORMANCE CAPTURE WITH REAL-TIME NEURAL RENDERING
(FR) AMÉLIORATION DE LA CAPTURE DE PERFORMANCE AVEC RENDU NEURONAL EN TEMPS RÉEL
Abstract
(EN)
Three-dimensional (3D) performance capture and machine learning can be used to re-render high quality novel viewpoints of a captured scene. A textured 3D reconstruction is first rendered to a novel viewpoint. Due to imperfections in geometry and low-resolution texture, the 2D rendered image contains artifacts and is low quality. Accordingly, a deep learning technique is disclosed that takes these images as input and generates more visually enhanced re-rendering. The system is specifically designed for VR and AR headsets, and accounts for consistency between two stereo views.
(FR)
L'invention concerne une capture de performance tridimensionnelle (3D) et un apprentissage machine qui peuvent être utilisés pour restituer à nouveau de nouveaux points de vue de haute qualité d'une scène capturée. Une reconstruction 3D texturée est d'abord rendue à un nouveau point de vue. En raison des imperfections de la géométrie et de la texture à faible résolution, l'image rendue 2D contient des artéfacts et est de faible qualité. Par conséquent, l'invention concerne une technique d'apprentissage profond qui prend lesdites images en tant qu'entrée et génère un nouveau rendu visuellement amélioré. Le système est spécifiquement conçu pour des casques VR et AR, et tient compte de la cohérence entre deux vues stéréo.
Latest bibliographic data on file with the International Bureau