Processing

Please wait...

Settings

Settings

Goto Application

1. WO2017133216 - DISTRIBUTED STORAGE METHOD AND DEVICE

Publication Number WO/2017/133216
Publication Date 10.08.2017
International Application No. PCT/CN2016/095228
International Filing Date 15.08.2016
IPC
G06F 3/06 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
06Digital input from, or digital output to, record carriers
CPC
G06F 11/1435
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
11Error detection; Error correction; Monitoring
07Responding to the occurrence of a fault, e.g. fault tolerance
14Error detection or correction of the data by redundancy in operation
1402Saving, restoring, recovering or retrying
1415at system level
1435using file system or storage system metadata
G06F 16/182
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
16Information retrieval; Database structures therefor; File system structures therefor
10File systems; File servers
18File system types
182Distributed file systems
G06F 3/0604
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
06Digital input from or digital output to record carriers ; , e.g. RAID, emulated record carriers, networked record carriers
0601Dedicated interfaces to storage systems
0602specifically adapted to achieve a particular effect
0604Improving or facilitating administration, e.g. storage management
G06F 3/0608
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
06Digital input from or digital output to record carriers ; , e.g. RAID, emulated record carriers, networked record carriers
0601Dedicated interfaces to storage systems
0602specifically adapted to achieve a particular effect
0608Saving storage space on storage systems
G06F 3/061
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
06Digital input from or digital output to record carriers ; , e.g. RAID, emulated record carriers, networked record carriers
0601Dedicated interfaces to storage systems
0602specifically adapted to achieve a particular effect
061Improving I/O performance
G06F 3/0641
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
06Digital input from or digital output to record carriers ; , e.g. RAID, emulated record carriers, networked record carriers
0601Dedicated interfaces to storage systems
0628making use of a particular technique
0638Organizing or formatting or addressing of data
064Management of blocks
0641De-duplication techniques
Applicants
  • 华为技术有限公司 HUAWEI TECHNOLOGIES CO., LTD. [CN]/[CN]
Inventors
  • 张勇 ZHANG, Yong
  • 蔡艺聪 CAI, Yicong
  • 武鸣 SHETTY S, Ajith Raj
Agents
  • 北京龙双利达知识产权代理有限公司 LONGSUN LEAD IP LTD.
Priority Data
201610083819.806.02.2016CN
Publication Language Chinese (ZH)
Filing Language Chinese (ZH)
Designated States
Title
(EN) DISTRIBUTED STORAGE METHOD AND DEVICE
(FR) PROCÉDÉ ET DISPOSITIF DE STOCKAGE DISTRIBUÉ
(ZH) 分布式存储方法和设备
Abstract
(EN)
A distributed storage method. The method is applied to a distributed storage system. The system comprises a file system, a first thread service and a second thread service. The first thread service combines M small files in the file system into one first file in an off-line manner according to a first rule, wherein the small files are less than pre-set bytes, and M is an integer greater than 1 (S110). In the distributed storage method and device, by means of a first thread service and a second thread service, when a small file is stored in a file system, the memory overheads of a name node and the handle overheads of a data node can be significantly reduced. Moreover, the calculation efficiency of mapping reduction can be improved, and thus the efficiency of storing a small file can be improved.
(FR)
L’invention concerne un procédé de stockage distribué. Le procédé s’applique à un système de stockage distribué. Le système comprend un système de fichier, un premier service de fil d’exécution et un second service de fil d’exécution. Le premier service de fil d’exécution combine M petits fichiers dans le système de fichier en un premier fichier d’une manière hors ligne selon une première règle, les petits fichiers représentant moins que des octets préréglés, et M étant un nombre entier supérieur à 1 (S110). Dans le procédé et le dispositif de stockage distribué, au moyen d’un premier service de fil d’exécution et d’un second service de fil d’exécution, lorsqu’un petit fichier est stocké dans un système de fichier, les surdébits de mémoire d’un nœud de nom et les surdébits de gestion d’un nœud de données peuvent être significativement réduits. En outre, l’efficacité de calcul d’une réduction de mappage peut être améliorée, et, ainsi, l’efficacité de stockage d’un petit fichier peut être améliorée.
(ZH)
一种分布式存储方法,该方法应用于分布式存储系统,系统包括文件系统、第一线程服务和第二线程服务,第一线程服务按照第一规则将文件系统中的M个小文件离线合并成一个第一文件,小文件小于预定字节,M为大于1的整数(S110)。分布式存储方法和设备,通过第一线程服务和第二线程服务,在文件系统上存储小文件时,能够大幅降低名字节点的内存开销和数据节点的句柄开销,并且能够提高映射归约的计算效率,从而能够提高存储小文件的效率。
Also published as
Latest bibliographic data on file with the International Bureau