Feedback
Settings

Settings

Goto Application

1. US20140180989 - Parallelizing neural networks during training

PermaLink
Office
United States of America
Application Number 14030938
Application Date 18.09.2013
Publication Number 20140180989
Publication Date 26.06.2014
Grant Number 09811775
Grant Date 07.11.2017
Publication Kind B2
IPC
G06F 15/18
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
15Digital computers in general; Data processing equipment in general
18in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines
G06N 3/04
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
G06K 9/66
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
64using simultaneous comparisons or correlations of the image signals with a plurality of references, e.g. resistor matrix
66references adjustable by an adaptive method, e.g. learning
G06K 9/62
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
G06T 1/20
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
1General purpose image data processing
20Processor architectures; Processor configuration, e.g. pipelining
G06N 3/063
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
063using electronic means
CPC
G06N 3/04
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
G06K 9/4628
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
46Extraction of features or characteristics of the image
4604Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes, intersections
4609by matching or filtering
4619Biologically-inspired filters, e.g. receptive fields
4623with interaction between the responses of different filters
4628Integrating the filters into a hierarchical structure
G06K 9/6256
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6217Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
6256Obtaining sets of training patterns; Bootstrap methods, e.g. bagging, boosting
G06K 9/66
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
64using simultaneous comparisons or correlations of the image signals with a plurality of references, e.g. resistor matrix
66references adjustable by an adaptive method, e.g. learning
G06N 3/0454
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architectures, e.g. interconnection topology
0454using a combination of multiple neural nets
G06N 3/063
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
063using electronic means
Applicants Google Inc.
Inventors Alexander Krizhevsky
Ilya Sutskever
Geoffrey E. Hinton
Agents Fish & Richardson P.C.
Title
(EN) Parallelizing neural networks during training
front page image
Abstract
(EN)

A parallel convolutional neural network is provided. The CNN is implemented by a plurality of convolutional neural networks each on a respective processing node. Each CNN has a plurality of layers. A subset of the layers are interconnected between processing nodes such that activations are fed forward across nodes. The remaining subset is not so interconnected.
Related patent documents
AU2013370514
WO/2014/105865
US20150339571
US20170132514
# -