Traitement en cours

Veuillez attendre...

Paramétrages

Paramétrages

Aller à Demande

1. WO2001077841 - PROCEDE ET APPAREIL DE DISTRIBUTION FIABLE ET ECHELONNABLE DE FICHIERS DE DONNEES DANS DES RESEAUX DISTRIBUES

Note: Texte fondé sur des processus automatiques de reconnaissance optique de caractères. Seule la version PDF a une valeur juridique

[ EN ]

What is claimed is:

1. A system for distributing information to a plurality of group members connected via a communication network, comprising:
a content control manager (CCM) for processing said information into at least one assignment for a distribution job, and for managing said distribution of said at least one assignment to said plurality of group members;
a set of group leaders, each having a corresponding set of group members and each communicating with said CCM, for forwarding said assignments to other group leaders in said set of group leaders according to commands from said CCM, and for forwarding said assignments to said corresponding set of group members belonging to a set of destinations; each of said group members being associated with a command interpreter for carrying out said assignments on each group member,
whereby said information is communicated to each of said group members within said set of destinations.

2. The system of claim 1 further comprising a database including:
a copy of said information to be distributed;
a set of content groups each comprising a set of destinations;
a list of group members currently included in at least one of said set of
destinations; and
a set of user-specified policies relating to distribution of said information, including a set of scheduling parameters.

3. The system of claim 2 in which said database is further accessible from a workstation on said network, said workstation having a graphical user interface adapted for interactive operation of said database by an operator.

4. The system of claim 1 in which said distribution server further comprises
a scheduler for scheduling a distribution job according to events scheduled in a database,
a communication interface to said network for communicating with at least one of said set of group leaders; and
a graphical user interface to a database.
5. The system of claim 1 in which said distribution server further comprises
a first version of a source file;
a second version of a source file; and
a file delta generator for comparing said first version with said second version to generate a set of changes between said first and second version;
whereby a set of information for distribution comprises said set of changes.

6. The system of claim 1 in which said at least one assignment is further comprised of: a set of commands for at least one of said command interpreters; and
if said assignment is for distribution of information, further including a set of said information for distribution.

7. The system of claim 6 in which said assignment further comprises
a header having:
an assignment identifier,
an assignment type selected from a set of assignment types, and
an error detection hash for said set of information for distribution.

8. The system of claim 1 in which said content control manager further comprises
a distribution manager for managing distribution of said at least one assignment including
a list of group leaders corresponding to said set of destinations;
a transmission process for sending a copy of said at least one assignment to each of said group leaders on said list as a set of data packets using a reliable transport protocol;
a completion process for determining when a copy of said assignment has been successfully sent to each of said group leaders on said list; and
a verification process for collecting reports from each group leader and for determining which destination of said set of destinations has received said assignment and successfully carried out said assignment.

9. The system of claim 1 in which each of said group leaders further comprises
a first reliable transport protocol for communicating with said CCM and for communicating with other group leaders;
a second reliable transport protocol for communicating with each of said group members corresponding to said group leader;
a store-and-forward process for receiving an assignment from said CCM, for receiving an address of another group leader of said set of group leaders from said CCM, for sending a copy of said assignment to said other group leader if so commanded by said CCM, and for requesting additional group leader addresses from said CCM until said CCM signals that each of said set of group leaders has received a copy of said assignment; a distribution process for forwarding a copy of said assignment to each of said corresponding group members and for verifying that each of said group members has successfully handled any commands and information of said assignment, and for generating and sending a report to said CCM according to said handling by each of said group members.

10. The system of claim 9 in which
each of said corresponding group members is connected to said group leader by way of a network supporting a multicast protocol; and
said second reliable transport protocol is a multicast protocol,
whereby each group member in said set of destinations receives a copy of said assignment at substantially the same time as each other group member in said set of destinations.

11. system of claim 1 in which each of said set of group leaders communicates with its said corresponding set of group members using a reliable multicast protocol, whereby each group member in said set of destinations receives a copy of said assignment at
substantially the same time as each other group member in said set of destinations.

12. The system of claim 1 in which each of said group members further comprises
a reliable multicast transport protocol for receiving a copy of said assignment from said corresponding group leader;
a reliable transport protocol for communicating with said corresponding group leader; and
a reporting mechanism for generating and sending a status report to said group leader after receiving and processing said assignment.

13. The system of claim 1 in which at least one of said set of group leaders communicates with said CCM and with other group leaders of said set via TCP/IP over the Internet and at least one of said group members receives said assignment from said corresponding group leader using a reliable multicast protocol over a local area network.

14. A method of distributing information via a communication network, comprising the steps of:
determining a content change in a source file;
determining a set of destination servers for receiving an update;
generating a sequence of update commands for said destination servers;
obtaining a list of clusters for delivery of said sequence to said set of destination servers;
communicating said sequence to each of said clusters on said list;
forwarding said sequence to each destination server within said cluster; and
executing said sequence on at least some of said destination servers.

15. The method of claim 14 further comprising the steps of:
monitoring within each said cluster a status indication of which destination servers properly execute said sequence of commands; and
reporting said monitored status to a designated monitoring center.

16. The method of claim 14 in which said step of determining a content change further comprises the steps of:
generating a list of filenames that occur in both said current version and said present version;
comparing each file, corresponding to each filename in said list, between said prior version and said current version to build a table of file changes according to each filename;
generating a set of directory names and filenames that occur in either said prior version or said current version, but not both; and
reporting said set and said table as a content update.

17. The method of claim 14 in which said step of generating a sequence of update commands further comprises the steps of:
selecting a command set according to a type of destination server for each destination;
identifying any data change in said determination of content change that requires transmission of file data; and
making a list of commands corresponding to each said determined content change, along with associated references to file data for each identified data change;
whereby said list of commands and said associated references are sequenced in a data structure for transmission to each destination server.

18. The method of claim 17 in which said step of making a list of commands further comprises the steps of:
evaluating configuration input to determine which file data must be encrypted or compressed, or both, prior to transmission;
carrying out such encryption or compression, or both, as indicated for said associated file references; and
inserting corresponding decryption or decompression commands, or both, into said list of commands.

19. The method of claim 14 further comprising the steps of:
administratively dividing said set of destination servers into a set of content groups;
determining a cluster identifier corresponding to each destination server; and
nominating a group leader as a communication channel for each said cluster identifier;
whereby a list of clusters is created for each content group and a nominated group leader represents each cluster.

20. The method of claim 14 in which said step of communicating said sequence further comprises the steps of:
(A) nominating a group leader for each said cluster;
(B) ordering said list of group leaders for carrying out a store-and-forward distribution;
(C) sending a notification to a target group leader to obtain a copy of said sequence from a named source;
(D) receiving said notification at said target group leader and requesting said sequence from said named source;
(E) receiving said sequence from said named source and reporting completion of said receiving;
(F) responding to said report by notifying a next target leader determined from said ordered list to obtain a copy of said sequence from a named source; and
(G) repeating steps (C) through (G) for each group leader until said list of group leaders is exhausted.

21. The method of claim 14 in which said step of determining a set of destination servers for receiving an update further comprises the step of identifying destination servers having stale content, wherein any of said set of destination servers were omitted from one or more earlier updates.

22. The method of claim 14 in which said step of determining a set of destination servers for receiving an update further comprises the steps of:
identifying a set of network resources for which load must be reduced; and
notifying a resource manager of said set of network resources to be taken out of service during an update.

23. The method of claim 14 in which said step of determining a set of destination servers for receiving an update further comprises the step of:
managing a load factor on a set of network resources by communicating with a resource manager to determine which of said network resources should be added or removed from service during a particular update;
whereby selected network resources can be taken out of service to receive an update, thereby reducing the load on said network resources.

24. The method of claim 23 in which said network resources are selected from the set of group leader, destination server, and group of destination servers.

25. A method for selecting a group leader among servers in a multicast network segment comprising the steps of:
configuring a set of said servers to participate in electing a leader, each said server having a corresponding voting priority;
determining when a new leader is needed; and
electing one server of said set to become said new leader.

26. The method of claim 25 in which said step of configuring a set of servers further comprises the steps of:
measuring a set of leader selection parameters in each participant in said set; and calculating the corresponding voting priority according to said measurements.

27. The method of claim 25 in which said step of determining when a new leader is needed further comprises the steps of:
configuring each server that is not currently the group leader to listen for periodic messages from said group leader;
adapting each server to send said periodic messages only if said server is currently the group leader;
waiting a configurable period after no periodic messages are heard; and
multicasting said voting priority to each participant.

28. The method of claim 27 in which said periodic messages are multicast on a predetermined network channel comprised of an IP multicast address and a port number.

29. The method of claim 27 in which said multicasting is addressed to a preconfigured IP multicast IP address and port combination for each server of said set of participating servers.

30. The method of claim 25 in which said step of electing further comprises the steps of: sending a claim of leadership containing a sent voting priority;
listening for other servers to claim leadership;
comparing a received priority in any other claims to leadership with said sent voting priority; and
determining said new leader according to the server having claimed leadership with the highest voting priority.

31. The method of claim 30 in which said step of sending a claim of leadership is implemented using a multicast message on said multicast network segment.

32. A system for determining a group leader among a group of servers comprising:
a set of participant servers including at least some servers capable of participating in electing a group leader;
a communication channel from each participant to each other participant;
a monitor process in each participant to determine which server is the current group leader; and
an election process in each participant to calculate a voting priority of said participant and to select a new group leader according to said voting priority, said election process triggered by said monitor process.

33. The system of claim 32 in which said monitor process further comprises:
a listener in each participant for determining how long since a group leader alive message has been heard on said communication channel;
a transmitter in each participant, operable in an elected group leader, that periodically signals each other participant who the current group leader is; and
a trigger adapted to detect that a group leader has not been heard from for a time longer than a threshold time, according to the period of said periodic signal.

34. The system of claim 33 in which said threshold time is configured such that a trigger will occur no less than five seconds after the last group leader alive message was received by said monitor process.

35. The system of claim 32 in which said election process further comprises a state machine adapted to transition from a temporary state of group leader unknown to a stable state of either group leader lαiown or group leader is me, according to the following steps: in said state of group leader unknown, if one or more group leader claim messages (LC) are received in which a received voting priority is greater than said calculated voting priority, then transition to a concession state and wait for a group leader alive message (LA); and if no LC is received before a period determined by said calculated voting priority, or no received voting priority is greater than said calculated voting priority, then transmit an LC including said calculated voting priority, and transition to a voting open state;
in said concession state, when an LA is received, transition to said group leader known state;
in said voting open state, if no LC or LA is received for a predetermined time interval, then transition to said group leader is me state and transmit an LA; and if one or more LC is received prior to said predetermined voting time interval, then transition to said group leader unknown state and transmit an LC; and if an LA is received, then transition to said group leader known state;
in said group leader is me state, periodically send an LA until an LC or LA is received, and then transition to said voting open state and send an LC; and
in said group leader known state, upon a trigger from said monitor process, transition to said group leader unknown state and transmit an LC;
whereby a participant having the highest calculated voting priority is elected group leader.

36. The system of claim 33 in which said predetermined voting time interval is no greater than 15 seconds.

37. The system of claim 32 in which said voting priority for each participant is determined dynamically according to at least one parameter selected from the set of: how recently was said participant the group leader, the number of servers known to said participant, the amount of resources available to such participant, reliability of the participant, the amount of recent information content the participant has acquired, and a user-specified priority factor.

38. A method for determining registration of members of a cluster of servers on a network segment comprising the steps of:
(A) designating a group leader on said network segment;
(B) each member sending a registration message to said group leader;
(C) said group leader multicasting a registration report including an identifier corresponding to each registered member;
(D) sending another registration message from any member receiving said registration report in which said member's corresponding identifier is missing;
(E) repeating steps (C) and (D) until each said member receives a registration report including its own corresponding identifier as a registered member.

39. The method of claim 38 in which said step of designating said group leader is carried out among a set of servers of said cluster according to a voting priority determined by at least some of said members from a set of dynamic parameters measured within
themselves.

40. The method of claim 38 in which said step of multicasting said registration report occurs after a configurable interval has expired since the most recent registration request was received by said group leader.

41. The method of claim 38 in which said step (D) of sending another registration message occurs after a preconfigured interval, after receipt of a registration report or a registration request, comprised of a fixed interval and a random interval, said random interval being up to one second.

42. The method of claim 38 in which said registration requests include at least some identification information selected from the set of: a server's IP address, a server's name, a server's port number, and a secret key.

43. The method of claim 38 further comprising the steps of:
processing said registration reports in said group leader to create a cluster report; and
transmitting said cluster report to a network distribution server, whereby said cluster reports are dynamically collected from all clusters in said network.

44. The method of claim 43 in which said cluster report further includes at least some group leader information selected from the set of: a list of registered members, identification information from at least some of said registered members, a network segment identifier, an LP address of said group leader, and a location parameter for said group leader.

45. A system for distributing information to a set of destination nodes connected via a communication network comprising:
a reporting process in each destination node for generating and transmitting a report to a distribution manager, said report containing an identification of said destination node and corresponding destination node parameters, whereby said destination node offers to become a participant in a distribution job;
said distribution manager connected to said network and configured to receive said reports from said destination nodes and to create a prioritized list of destination nodes selected as participants in a distribution job according to said destination node parameters, and having a management process for sending information to each participant;
said management process adapted to send each participant instructions to obtain a copy of said information either from said distribution manager or from another identified participant, until each participant has received a copy of said information;
each participant having a store-and-forward process configured to receive instructions from a prior participant or from said distribution manager and to request a copy of said information from said prior participant or from said distribution manager, and to thereafter request further distribution instructions from said distribution manager until instructed that no other participants require said information;
whereby each participant obtains a copy of the information and the distribution manager obtains confirmation that each destination node has obtained said information.

46. The system of claim 45 in which the destination node parameters, by which said prioritized list is created, are selected from the set of parameters consisting of location, destination node performance, distance from other destination nodes, transmission speed of said destination node's network connection, and reliability of said network connection.

47. A method of distributing information to a set of destination nodes connected via a communication network comprising the steps of:
obtaining a list of destination nodes desiring to participate in a distribution;
prioritizing said list according to parameters associated with each destination node on said list;
issuing instructions to each destination node according to the prioritized list order, said instructions including the identification of a source for obtaining said information and an identification of the next destination node on the prioritized list;
distributing said infoπnation according to said instructions; and
notifying each destination node when the prioritized list is exhausted.

48. The method of claim 47 in which said step of prioritizing further comprises the steps of:
obtaining a set of parameters corresponding to said list of destination nodes; and sorting said list according to at least some of said set of parameters;
whereby the sorted list provides a prioritized ordering of destination nodes to receive said information being distributed.

49. The method of claim 47 in which said steps of issuing instructions and distributing said information further comprise the steps of:
(A) obtaining an address of a target node on said prioritized list;
(B) sending a notification message to said target node, said notification containing the address of a source node having an information file to distribute;
(C) receiving said notification message at said first destination node and requesting a copy of said information from said identified source node;
(D) reporting when said target node has successfully received said copy of said information;
(E) responding to said report by selecting a next target node address on said prioritized list and repeating steps (A) through (E) until said prioritized list is exhausted; whereby said copy of said information is sent to each target node on said list in an order determined according to the order of the list for each successful copy.

50. The method of claim 49 in which said steps of sending and receiving are carried out using a reliable transport protocol across said communication network.

51. The method of claim 47 in which said step of obtaining a list of destination nodes further comprises the steps of:
electing a group leader among a set of destination nodes on a network segment;
sending reports of the respective address and capabilities of each destination node to said group leader;
collecting and processing said reports in said group leader;
generating a network segment discovery report in said group leader;
sending said network segment discovery report to a distribution manager; and arranging said list of destination nodes according to at least one of said network segment discovery reports.

52. The method of claim 51 in which said distribution server further comprises a database for storing said reports and a graphical user interface for providing an interface for system administration personnel.

53. The method of claim 51 in which said step of arranging said list further comprises the steps of:
building a database of reports collected from said group leaders;
collecting distribution policies;
administratively dividing said destination nodes into content groups;
collecting information about the network communication delays;
creating a list of group leaders corresponding to each content group; and
sorting each said list of group leaders according to at least one of said reported destination node capabilities, distribution policies, and information about the network communication delays.

54. A system for distributing information to a set of destination nodes connected via a communication network comprising:
a discovery process for discovering destination nodes in a network;
a distribution server for receiving the results of said discovery process, comprising at least one network group comprised of a set of destination nodes;
a list of destination nodes comprising at least one user-defined content group;
a user interface for obtaining distribution policies; and
a distribution process for managing the transmission of said information to each of said network groups.

55. The system of claim 54 in which said discovery process further comprises:
a group leader for collecting reports from at least one destination node in a network segment and for processing said reports into a network segment report and for sending said report to said distribution server.

56. The system of claim 54 in which said network group is created through the process of receiving reports from a set of group leaders, in which at least some reports identify destination nodes under the control of the corresponding group leader.

57. The system of claim 54 in which said distribution process further comprises a mechanism for:
arranging a list of network groups according to at least one parameter selected from the set of: physical location, transmission delay for reaching said network group from said distribution server, and a measure of destination node capabilities of at least one destination node within each network group; and
transmitting said information to each network group according to said arranged list.

58. The system of claim 57 in which said system further comprises:
a report generator for generating a report from any destination node that has changed its availability status or system capabilities, or has successfully received an information transmission from said distribution process; and
a report transmitter, in a group leader for each cluster of destination nodes, for collecting reports from any destination node in the corresponding cluster, and for processing said reports and transmitting a consolidated report to the distribution server.

59. The system of claim 54 in which said discovery process further comprises:
at least one network cluster comprised of destination node connected to each other with a multicast communication medium;
an election mechanism for dynamically selecting one destination node of each said at least one network cluster to be group leader; and
a reporting process in which each said group leader solicits reports from each destination node in the corresponding cluster, creates a profile report for the cluster, and transmits it to said distribution server.

60. The system of claim 59 in which said reporting process is triggered by an event selected from the set of: a timer, a group leader receiving a registration request from a destination node in said network cluster, a change of group leader, a user input, an information transfer from said distribution server, and an instruction from said distribution server.

61. The system of claim 54 in which said distribution process further comprises:
a list of destinations, each corresponding to a network group, said list adapted to represent a transmission tree structure in which each destination on said list becomes an information source for any later destination occurring later in said list;
a point-to-point transmission process for sending information from one of said information sources to any of said later destinations;
a distribution manager containing an information source, and for obtaining said list of destinations and requesting a point-to-point transmission of said information to each destination according to said list, and for determining when transmission has been completed for each destination.

62. The system of claim 61 in which said distribution manager further comprises:
a process for creating an information notification message for transmission, said notification including an earlier destination address from said list and an information identifier, and for triggering transmission to said next destination;
whereby said next destination receives a notification that information
corresponding to said information identifier is to be obtained from one of said information sources.

63. The system of claim 61 in which said list is ordered according to parameters selected from the set of: the CPU power of the destination, the network location of each
destination, and user inputs.

64. The system according to claim 54 in which each network group further comprises:
a group leader for receiving information from an information source and for instructing other group leaders to request infoπnation from an information source;
a local distribution mechanism in which a group leader transmits said information to each destination node in its corresponding network group using a multicast protocol; and
an interpreter in each said destination node for executing a set of commands in said information, and for reporting completion status of said execution.

65. The system according to claim 54 in which said distribution process is responsive to said distribution policies and identifies which network groups correspond to any destination made in said content group.

66. A method for determining the completion status of processing by individual nodes of a group of destination nodes on a multicast data channel, said group having a group leader, comprising the steps of:
notifying an assignment processor in at least one of said destination nodes that an assignment has been received for processing;
determining completion status of said processing by at least one of said destination nodes; and
notifying said group leader of said completion status by a message from said at least one destination node such that the group leader acquires a collective indication of status from nodes within said group.

67. The method of claim 66 further comprising the steps of:
determining whether the number of non-reporting nodes in the group exceeds a configured number or proportion of nodes;
and transmitting a multicast request from said group leader (GL) to nodes in said group where said request is fonnatted according to said number of non-reporting nodes, and otherwise transmitting a request to each non-reporting node; and
repeating said determination of the number of non-reporting nodes and transmitting a multicast or unicast request until either each node has reported its status or a preconfigured number of requests has been transmitted.

68. The method of claim 67 further comprising the steps of:
when the number of non-reporting nodes is less than the number of reporting nodes, listing said non-reporting nodes in said multicast request;
when the number of non-reporting nodes is not less than the number of reporting nodes , listing said reporting nodes in said multicast request;
interpreting said multicast request in each node to determine whether any given node in the group should transmit a missing report to said GL; and
transmitting said missing report according to said determination, after a time delay configurable for each node.