Processing

Please wait...

Settings

Settings

Goto Application

1. WO2021038190 - SKIP PREDICTOR FOR PRE-TRAINED RECURRENT NEURAL NETWORKS

Publication Number WO/2021/038190
Publication Date 04.03.2021
International Application No. PCT/GB2020/051929
International Filing Date 13.08.2020
IPC
G06N 3/04 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
G06N 20/20 2019.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
20Machine learning
20Ensemble learning
G06N 20/00 2019.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
20Machine learning
G06N 7/00 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
7Computer systems based on specific mathematical models
G06N 3/08 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
G06N 5/00 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
5Computer systems using knowledge-based models
CPC
G06F 17/18
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
17Digital computing or data processing equipment or methods, specially adapted for specific functions
10Complex mathematical operations
18for evaluating statistical data ; , e.g. average values, frequency distributions, probability functions, regression analysis
G06K 9/6267
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6267Classification techniques
G06N 20/20
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
20Machine learning
20Ensemble learning
G06N 3/0445
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0445Feedback networks, e.g. hopfield nets, associative networks
G06N 3/0454
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0454using a combination of multiple neural nets
G06N 3/0472
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0472using probabilistic elements, e.g. p-rams, stochastic processors
Applicants
  • ARM LIMITED [GB]/[GB]
Inventors
  • THAKKER, Urmish Ajit
  • TAO, Jin
  • DASIKA, Ganesh Suryanarayan
  • BEU, Jesse Garrett
Agents
  • TLIP LTD
Priority Data
16/855,68122.04.2020US
62/890,99623.08.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) SKIP PREDICTOR FOR PRE-TRAINED RECURRENT NEURAL NETWORKS
(FR) PRÉDICTEUR DE SAUT POUR RÉSEAUX NEURONAUX RÉCURRENTS PRÉ-ENTRAÎNÉS
Abstract
(EN)
The present disclosure advantageously provides a system and a method for skipping recurrent neural network (RNN) state updates using a skip predictor. Sequential input data are received and divided into sequences of input data values, each input data value being associated with a different time step for a pre-trained RNN model. At each time step, the hidden state vector for a prior time step is received from the pre-trained RNN model, and a determination, based on the input data value and the hidden state vector for at least one prior time step, is made whether to provide or not provide the input data value associated with the time step to the pre-trained RNN model for processing. When the input data value is not provided, the pre-trained RNN model does not update its hidden state vector. Importantly, the skip predictor is trained without retraining the pre-trained RNN model.
(FR)
La présente invention concerne avantageusement un système et un procédé permettant de sauter des mises à jour d'état de réseau neuronal récurrent (RNN) à l'aide d'un prédicteur de saut. Des données d'entrée séquentielles sont reçues et divisées en séquences de valeurs de données d'entrée, chaque valeur de données d'entrée étant associée à un pas temporel différent pour un modèle RNN pré-entraîné. À chaque pas temporel, un vecteur d'état caché pour un pas temporel antérieur est reçu du modèle RNN pré-entraîné, et il est déterminé, sur la base de la valeur de données d'entrée et du vecteur d'état caché pour au moins un pas temporel antérieur, s'il faut fournir ou ne pas fournir la valeur de données d'entrée associée au pas temporel au modèle RNN pré-entraîné pour son traitement. Quand la valeur de données d'entrée n'est pas fournie, le modèle RNN pré-entraîné ne met pas à jour son vecteur d'état caché. De manière importante, le prédicteur de saut est entraîné sans ré-entraînement du modèle RNN pré-entraîné.
Also published as
Latest bibliographic data on file with the International Bureau