Search International and National Patent Collections

1. (WO2017201507) MEMORY-EFFICIENT BACKPROPAGATION THROUGH TIME

Pub. No.:    WO/2017/201507    International Application No.:    PCT/US2017/033698
Publication Date: Fri Nov 24 00:59:59 CET 2017 International Filing Date: Sat May 20 01:59:59 CEST 2017
IPC: G06N 3/04
G06N 3/08
Applicants: DEEPMIND TECHNOLOGIES LIMITED
Inventors: LANCTOT, Marc
GRUSLYS, Audrunas
DANIHELKA, Ivo
Title: MEMORY-EFFICIENT BACKPROPAGATION THROUGH TIME
Abstract:
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a recurrent neural network on training sequences using backpropagation through time. In one aspect, a method includes receiving a training sequence including a respective input at each of a number of time steps; obtaining data defining an amount of memory allocated to storing forward propagation information for use during backpropagation; determining, from the number of time steps in the training sequence and from the amount of memory allocated to storing the forward propagation information, a training policy for processing the training sequence, wherein the training policy defines when to store forward propagation information during forward propagation of the training sequence; and training the recurrent neural network on the training sequence in accordance with the training policy.