Search International and National Patent Collections

1. (WO2018022821) MEMORY COMPRESSION IN A DEEP NEURAL NETWORK

Pub. No.:    WO/2018/022821    International Application No.:    PCT/US2017/044065
Publication Date: Fri Feb 02 00:59:59 CET 2018 International Filing Date: Fri Jul 28 01:59:59 CEST 2017
IPC: G06N 3/08
G06N 3/04
Applicants: ARIZONA BOARD OF REGENTS ON BEHALF OF ARIZONA STATE UNIVERSITY
Inventors: SEO, Jae-Sun
KADETOTAD, Deepak
ARUNACHALAM, Sairam
CHAKRABARTI, Chaitali
Title: MEMORY COMPRESSION IN A DEEP NEURAL NETWORK
Abstract:
Aspects disclosed in the detailed description include memory compression in a deep neural network (DNN). To support a DNN application, a fully connected weight matrix associated with a hidden layer(s) of the DNN is divided into a plurality of weight blocks to generate a weight block matrix with a first number of rows and a second number of columns. A selected number of weight blocks are randomly designated as active weight blocks in each of the first number of rows and updated exclusively during DNN training. The weight block matrix is compressed to generate a sparsified weight block matrix including exclusively active weight blocks. The second number of columns is compressed to reduce memory footprint and computation power, while the first number of rows is retained to maintain accuracy of the DNN, thus providing the DNN in an efficient hardware implementation without sacrificing accuracy of the DNN application.