Processing

Please wait...

PATENTSCOPE will be unavailable a few hours for maintenance reason on Tuesday 25.01.2022 at 9:00 AM CET
Settings

Settings

Goto Application

1. WO2021197659 - UPDATE OF MODEL PARAMETERS IN A PARALLEL PROCESSING SYSTEM

Publication Number WO/2021/197659
Publication Date 07.10.2021
International Application No. PCT/EP2020/087574
International Filing Date 22.12.2020
IPC
G06F 15/173 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
15Digital computers in general; Data processing equipment in general
16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
163Interprocessor communication
173using an interconnection network, e.g. matrix, shuffle, pyramid, star or snowflake
G06N 3/08 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
G06F 9/54 2006.1
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
9Arrangements for program control, e.g. control units
06using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
46Multiprogramming arrangements
54Interprogram communication
CPC
G06F 15/17318
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
15Digital computers in general
16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
163Interprocessor communication
173using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
17306Intercommunication techniques
17318Parallel communications techniques, e.g. gather, scatter, reduce, roadcast, multicast, all to all
G06F 15/17325
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
15Digital computers in general
16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
163Interprocessor communication
173using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
17306Intercommunication techniques
17325Synchronisation; Hardware support therefor
G06F 9/52
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
9Arrangements for program control, e.g. control units
06using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
46Multiprogramming arrangements
52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
G06F 9/522
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
9Arrangements for program control, e.g. control units
06using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
46Multiprogramming arrangements
52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
522Barrier synchronisation
G06F 9/542
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
9Arrangements for program control, e.g. control units
06using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
46Multiprogramming arrangements
54Interprogram communication
542Event management; Broadcasting; Multicasting; Notifications
G06N 3/02
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
Applicants
  • GRAPHCORE LIMITED [GB]/[GB]
Inventors
  • TØRUDBAKKEN, Ola
  • CEVOLANI, Lorenzo
Agents
  • KEELING, THOMAS
Priority Data
16/928,88614.07.2020US
2004857.502.04.2020GB
2008390.304.06.2020GB
Publication Language English (en)
Filing Language English (EN)
Designated States
Title
(EN) UPDATE OF MODEL PARAMETERS IN A PARALLEL PROCESSING SYSTEM
(FR) MISE À JOUR DE PARAMÈTRES DE MODÈLE DANS UN SYSTÈME DE TRAITEMENT PARALLÈLE
Abstract
(EN) A data processing system comprising a plurality of processing nodes that are arranged to update a model in a parallel manner. Each of the processing nodes starts with a different set of updates to model parameters. Each of the processing nodes is configured to perform one or more reduce-scatter collectives so as to exchange and reduce the updates. Having done so, each processing node is configured to apply the reduced set of updates to obtain an updated set of model parameters. The processing nodes then exchange the updated model parameters using an all-gather so that each processing node ends up with the same model parameters at the end of the process.
(FR) Un système de traitement de données comprend une pluralité de nœuds de traitement qui sont agencés pour mettre à jour un modèle de manière parallèle. Chacun des nœuds de traitement commence avec un ensemble différent de mises à jour pour des paramètres de modèle. Chacun des nœuds de traitement est configuré pour effectuer un ou plusieurs collectifs de diffusion de réduction de façon à échanger et à réduire les mises à jour. À cet effet, chaque nœud de traitement est configuré pour appliquer l'ensemble réduit de mises à jour pour obtenir un ensemble mis à jour de paramètres de modèle. Les nœuds de traitement échangent ensuite les paramètres de modèle mis à jour en utilisant un élément all-gather de telle sorte que chaque nœud de traitement se termine avec les mêmes paramètres de modèle à la fin du processus.
Latest bibliographic data on file with the International Bureau