Search International and National Patent Collections

1. (WO2018099085) NEURAL NETWORK MODEL TRAINING METHOD AND DEVICE, AND CHIP

Pub. No.:    WO/2018/099085    International Application No.:    PCT/CN2017/092092
Publication Date: Fri Jun 08 01:59:59 CEST 2018 International Filing Date: Fri Jul 07 01:59:59 CEST 2017
IPC: G06N 3/04
Applicants: HUAWEI TECHNOLOGIES CO., LTD.
华为技术有限公司
Inventors: BAI, Xiaolong
白小龙
ZHANG, Changzheng
张长征
XIA, Mingzhen
夏命榛
Title: NEURAL NETWORK MODEL TRAINING METHOD AND DEVICE, AND CHIP
Abstract:
A neural network model training method and device, and a chip, which are used for reducing the communication volume between a server module and each working module in a neural network model training process. In the method, a model training mode of each layer is determined according to the estimated data volume in a model parameter set of each layer and the estimated data volume of output data; and when the jth layer is in a model parallel training mode, since second output data is the output data of the (j-1)th-layer training of m working modules, the working modules perform model parameter training according to the second output data so that a global gradient of model parameters be directly obtained. Compared with the solution in the prior art that a global gradient of model parameters is obtained after a working module pushes up a local gradient of the model parameters to a server module and then pulls down a global gradient of the model parameters from the server module, the present invention reduces the communication volume between the working module and the server module.