Processing

Please wait...

Settings

Settings

Goto Application

1. WO2021062105 - TRAINING NEURAL NETWORKS TO GENERATE STRUCTURED EMBEDDINGS

Publication Number WO/2021/062105
Publication Date 01.04.2021
International Application No. PCT/US2020/052649
International Filing Date 25.09.2020
IPC
G06N 3/04 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
G06N 3/08 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
CPC
G06N 20/00
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
20Machine learning
G06N 3/0454
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0454using a combination of multiple neural nets
G06N 3/0481
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0481Non-linear activation functions, e.g. sigmoids, thresholds
G06N 3/084
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
084Back-propagation
Applicants
  • GOOGLE LLC [US]/[US]
Inventors
  • CLARK, Robert Andrew, James
  • CHAN, Chun-an
  • WAN, Vincent Ping, Leung
Agents
  • KRUEGER, Brett, A.
Priority Data
16/586,22327.09.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) TRAINING NEURAL NETWORKS TO GENERATE STRUCTURED EMBEDDINGS
(FR) FORMATION DE RÉSEAUX NEURONAUX POUR GÉNÉRER DES INTÉGRATIONS STRUCTURÉES
Abstract
(EN)
A method (200) for training a machine learning model (108) to generate embeddings (106) of inputs to the machine learning model, the machine learning model having an encoder (110) that generates the embeddings from the inputs and a decoder (120) that generates outputs from the generated embeddings, wherein the embedding is partitioned into a sequence of embedding partitions that each includes one or more dimensions of the embedding. The method includes: for a first embedding partition (106 A) in the sequence of embedding partitions: performing initial training to train the encoder and a decoder replica (122) corresponding to the first embedding partition, for each particular embedding partition that is after the first embedding partition in the sequence of embedding partitions: performing incremental training to train the encoder and a decoder replica corresponding to the particular partition.
(FR)
L'invention concerne un procédé (200) destiné à apprendre à un modèle d'apprentissage machine (108) pour générer des incorporations (106) d'entrées au modèle d'apprentissage machine, le modèle d'apprentissage machine ayant un codeur (110) qui génère les incorporations à partir des entrées et un décodeur (120) qui génère des sorties à partir des incorporations générées, l'incorporation étant partitionnée en une séquence de partitions d'incorporation qui comprennent chacune une ou plusieurs dimensions de l'incorporation. Le procédé consiste à : pour une première partition d'incorporation (106 A) dans la séquence de partitions d'incorporation, effectuer un apprentissage initial permettant l’apprentissage du codeur et d'une réplique de décodeur (122) correspondant à la première partition d'incorporation : pour chaque partition d'incorporation particulière qui est ultérieure à la première partition d'incorporation dans la séquence de partitions d'incorporation : effectuer un apprentissage incrémentiel permettant l'apprentissage du codeur et d'une réplique de décodeur correspondant à la partition particulière.
Also published as
Latest bibliographic data on file with the International Bureau