Processing

Please wait...

PATENTSCOPE will be unavailable a few hours for maintenance reason on Tuesday 27.07.2021 at 12:00 PM CEST
Settings

Settings

Goto Application

1. WO2020142223 - DITHERED QUANTIZATION OF PARAMETERS DURING TRAINING WITH A MACHINE LEARNING TOOL

Publication Number WO/2020/142223
Publication Date 09.07.2020
International Application No. PCT/US2019/067303
International Filing Date 19.12.2019
IPC
G06N 3/063 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
063using electronic means
G06N 3/08 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
G06N 3/04 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
G06N 3/10 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
10Simulation on general purpose computers
CPC
G06F 7/49963
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
7Methods or arrangements for processing data by operating upon the order or content of the data handled
38Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
48using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
499Denomination or exception handling, e.g. rounding, overflow
49942Significance control
49947Rounding
49963Rounding to nearest
G06N 20/00
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
20Machine learning
G06N 3/0481
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0481Non-linear activation functions, e.g. sigmoids, thresholds
G06N 3/063
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
063using electronic means
G06N 3/082
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
082modifying the architecture, e.g. adding or deleting nodes or connections, pruning
G06N 3/084
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
084Back-propagation
Applicants
  • MICROSOFT TECHNOLOGY LICENSING, LLC [US]/[US]
Inventors
  • ANNAU, Thomas, M.
  • ZHU, Haishan
  • LO, Daniel
  • CHUNG, Eric, S.
Agents
  • MINHAS, Sandip, S.
  • ADJEMIAN, Monica
  • BARKER, Doug
  • CHATTERJEE, Aaron, C.
  • CHEN, Wei-chen, Nicholas
  • CHOI, Daniel
  • CHURNA, Timothy
  • DINH, Phong
  • EVANS, Patrick
  • GABRYJELSKI, Henry
  • GOLDSMITH, Micah P.
  • GUPTA, Anand
  • HINOJOSA-SMITH, Brianna L.
  • HWANG, William C.
  • JARDINE, John S.
  • LEE, Sunah
  • LEMMON, Marcus
  • MARQUIS, Thomas
  • MEYERS, Jessica
  • ROPER, Brandon
Priority Data
16/240,51404.01.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) DITHERED QUANTIZATION OF PARAMETERS DURING TRAINING WITH A MACHINE LEARNING TOOL
(FR) QUANTIFICATION DIFFÉRÉE DE PARAMÈTRES PENDANT L'APPRENTISSAGE À L'AIDE D'UN OUTIL D'APPRENTISSAGE MACHINE
Abstract
(EN)
A machine learning tool uses dithered quantization of parameters during training of a machine learning model such as a neural network. The machine learning tool receives training data and initializes certain parameters of the machine learning model (e.g, weights for connections between nodes of a neural network, biases for nodes). The machine learning tool trains the parameters in one or more iterations based on the training data. In particular, in a given iteration, the machine learning tool applies the machine learning model to at least some of the training data and, based at least in part on the results, determines parameter updates to the parameters. The machine learning tool updates the parameters using the parameter updates and a dithered quantizer function, which can add random values before a rounding or truncation operation.
(FR)
La présente invention concerne un outil d'apprentissage automatique qui utilise une quantification différée de paramètres pendant l'apprentissage d'un modèle d'apprentissage machine tel qu'un réseau neuronal. L'outil d'apprentissage automatique reçoit des données d'apprentissage et initialise certains paramètres du modèle d'apprentissage machine (par exemple, des poids pour des connexions entre des nœuds d'un réseau neuronal, des polarisations pour les nœuds). L'outil d'apprentissage machine entraîne les paramètres dans une ou plusieurs itérations sur la base des données d'apprentissage. Plus particulièrement, dans une itération donnée, l'outil d'apprentissage machine applique le modèle d'apprentissage machine à au moins certaines des données d'apprentissage et, sur la base, au moins en partie, des résultats, détermine des mises à jour de paramètres pour les paramètres. L'outil d'apprentissage machine met à jour les paramètres à l'aide des mises à jour de paramètres et d'une fonction de quantificateur différée, qui peut ajouter des valeurs aléatoires avant une opération d'arrondissement ou de troncature.
Also published as
Latest bibliographic data on file with the International Bureau