Search International and National Patent Collections

1. (WO2018085729) DEEP NEURAL NETWORK MODEL FOR PROCESSING DATA THROUGH MULTIPLE LINGUISTIC TASK HIERARCHIES

Pub. No.:    WO/2018/085729    International Application No.:    PCT/US2017/060057
Publication Date: Sat May 12 01:59:59 CEST 2018 International Filing Date: Sat Nov 04 00:59:59 CET 2017
IPC: G06N 3/04
G10L 15/16
G10L 15/18
G10L 25/30
G06F 17/20
Applicants: SALESFORCE.COM, INC.
Inventors: HASHIMOTO, Kazuma
XIONG, Caiming
SOCHER, Richard
Title: DEEP NEURAL NETWORK MODEL FOR PROCESSING DATA THROUGH MULTIPLE LINGUISTIC TASK HIERARCHIES
Abstract:
The technology disclosed provides a so-called "joint many-task neural network model" to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called "successive regularization" technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.