Processing

Please wait...

Settings

Settings

Goto Application

1. WO2022004057 - AUTOMATED CONSTRUCTION OF NEURAL NETWORK ARCHITECTURE WITH BAYESIAN GRAPH EXPLORATION

Publication Number WO/2022/004057
Publication Date 06.01.2022
International Application No. PCT/JP2021/008847
International Filing Date 26.02.2021
IPC
G06N 3/04 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
G06N 3/08 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
G06N 7/00 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
7Computer systems based on specific mathematical models
CPC
G06N 3/04
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
G06N 3/0454
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0454using a combination of multiple neural nets
G06N 3/0472
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0472using probabilistic elements, e.g. p-rams, stochastic processors
G06N 3/082
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
082modifying the architecture, e.g. adding or deleting nodes or connections, pruning
G06N 7/005
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
7Computer systems based on specific mathematical models
005Probabilistic networks
Applicants
  • 三菱電機株式会社 MITSUBISHI ELECTRIC CORPORATION [JP]/[JP]
Inventors
  • AKINO, Toshiaki
  • WANG, Ye
  • DEMIR, Andac
  • ERDOGMUS, Deniz
Agents
  • FUKAMI PATENT OFFICE, P.C.
Priority Data
16/919,21102.07.2020US
Publication Language English (en)
Filing Language English (EN)
Designated States
Title
(EN) AUTOMATED CONSTRUCTION OF NEURAL NETWORK ARCHITECTURE WITH BAYESIAN GRAPH EXPLORATION
(FR) CONSTRUCTION AUTOMATISÉE D'ARCHITECTURE DE RÉSEAU NEURONAL AVEC EXPLORATION DE GRAPHE BAYÉSIEN
Abstract
(EN) A system for automated construction of an artificial neural network architecture is provided. The system includes a set of interfaces and data links configured to receive and send signals, wherein the signals include datasets of training data, validation data and testing data, wherein the signals include a set of random number factors in multi-dimensional signals X, wherein part of the random number factors are associated with task labels Y to identify, and nuisance variations S. The system further includes a set of memory banks to store a set of reconfigurable deep neural network (DNN) blocks, hyperparameters, trainable variables, intermediate neuron signals, and temporary computation values including forward-pass signals and backward-pass gradients. The system further includes at least one processor, in connection with the interface and the memory banks, configured to submit the signals and the datasets into the reconfigurable DNN blocks, wherein the at least one processor is configured to execute a Bayesian graph exploration using the Bayes-Ball algorithm to reconfigure the DNN blocks such that redundant links are pruned to be compact by modifying the hyperparameters in the memory banks. The system realizes nuisance-robust variational Bayesian inference to be transferable to new datasets in semi-supervised settings.
(FR) L'invention concerne un système de construction automatisée d'une architecture de réseau neuronal artificiel. Le système comprend un ensemble d'interfaces et de liaisons de données configurées pour recevoir et envoyer des signaux, les signaux comprenant des ensembles de données de données d'apprentissage, de données de validation et de données de test, les signaux comprenant un ensemble de facteurs de nombres aléatoires dans des signaux multidimensionnels X, une partie des facteurs de nombres aléatoires étant associée à des étiquettes de tâche Y à identifier, et des variations de nuisance S. Le système comprend en outre un ensemble de banques de mémoire servant à mémoriser un ensemble de blocs de réseaux neuronaux profonds (DNN) reconfigurables, des hyperparamètres, des variables pouvant être apprises, des signaux de neurones intermédiaires, et des valeurs de calcul temporaires comprenant des signaux de calcul au plus tôt et des gradients de calcul au plus tard. Le système comprend en outre au moins un processeur, connecté à l'interface et aux banques de mémoire, configuré pour soumettre les signaux et les ensembles de données dans les blocs de DNN reconfigurables, ledit processeur étant configuré pour exécuter une exploration de graphe Bayésien à l'aide de l'algorithme de Bayes-Ball pour reconfigurer les blocs de DNN de façon à élaguer des liaisons redondantes pour les compacter en modifiant les hyperparamètres dans les banques de mémoire. Le système réalise une inférence bayésienne variationnelle robuste aux nuisances pouvant être transférée à de nouveaux ensembles de données lors de réglages semi-supervisés.
Related patent documents
Latest bibliographic data on file with the International Bureau