Search International and National Patent Collections

1. (WO2018102700) PHOTOREALISTIC FACIAL TEXTURE INFERENCE USING DEEP NEURAL NETWORKS

Pub. No.:    WO/2018/102700    International Application No.:    PCT/US2017/064239
Publication Date: Fri Jun 08 01:59:59 CEST 2018 International Filing Date: Sat Dec 02 00:59:59 CET 2017
IPC: G06K 9/00
G06N 3/02
Applicants: PINSCREEN, INC.
Inventors: SAITO, Shunsuke
WEI, Cosimo
HU, Liwen
LI, Hao
Title: PHOTOREALISTIC FACIAL TEXTURE INFERENCE USING DEEP NEURAL NETWORKS
Abstract:
A method for generating three-dimensional facial models and photorealistic textures from inferences using deep neural networks relies upon generating a low frequency and a high frequency albedo map of the full and partial face, respectively. Then, the high frequency albedo map may be used for comparison with correlation matrices generated by a neural network trained by a large scale, high-resolution facial dataset with simulated partial visibility. The corresponding correlation matrices of the complete facial textures can then be retrieved. Finally, a full facial texture map may be synthesized, using convex combinations of the correlation matrices. A photorealistic facial texture for the three-dimensional face rendering can be obtained through optimization using the deep neural network and a loss function that incorporates the blended target correlation matrices.