Processing

Please wait...

Settings

Settings

Goto Application

1. CN113168565 - NEURAL NETWORK COMPRESSION METHOD AND APPARATUS

Office
China
Application Number 201880099986.9
Application Date 29.12.2018
Publication Number 113168565
Publication Date 23.07.2021
Publication Kind A
IPC
G06N 3/08
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
CPC
G06N 3/08
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
Applicants HUAWEI TECHNOLOGIES CO., LTD.
华为技术有限公司
Inventors ZHU JIAFENG
朱佳峰
WEI WEI
魏巍
LU HUILI
卢惠莉
Agents 北京同达信恒知识产权代理有限公司 11291
Title
(EN) NEURAL NETWORK COMPRESSION METHOD AND APPARATUS
(ZH) 一种神经网络压缩方法及装置
Abstract
(EN) A neural network compression method and apparatus are used to solve the problem in the prior art of inflexible model compression. The method comprises: according to an initial weight value threshold of an ith layer of an initial neural network model, cropping an initial weight value of the ith layer, and obtaining a cropped neural network model, wherein i is any positive integer from 1 to m, and m is the total number of layers of the neural network model; and performing multiple rounds of training on the cropped neural network model, and during a tth round of training, determining a weight value threshold of the ith layer during the tth round of training according to a weight value threshold of the ith layer of the neural network model obtained from a tth-1 round of training, and cropping a current weight value of the ith layer during the tth round of training according to the weight value threshold of the ith layer during the tth round of training, wherein t is any positive number from 1 to q, and q is the total number of rounds of training. The present invention thus adaptively adjusts the weight value threshold of each cropping, and flexibly compresses the neural network.
(ZH) 一种神经网络压缩方法及装置,用以解决现有技术中模型压缩不灵活的问题。方法包括:根据初始神经网络模型的第i层的初始权值阈值,对所述第i层的初始权值进行裁剪,得到裁剪后的神经网络模型,所述i取遍1至m中的任意一个正整数,所述m为所述神经网络模型的总层数;对所述裁剪后的神经网络模型进行多次训练,在第t次训练过程中,根据第t‑1次训练得到的神经网络模型第i层的权值阈值确定第t次训练时第i层的权值阈值,根据所述第t次训练时第i层的的权值阈值对所述第t次训练时第i层当前的权值进行裁剪;所述t取遍1至q中的任意一个正数,所述q为多次训练的总次数。这样可以自适应地调整每次裁剪的权重阈值,灵活压缩神经网络。
Related patent documents