Pesquisa nas coleções internacionais e nacionais de patentes
Algum conteúdo deste aplicativo está indisponível no momento.
Se esta situação persistir, por favor entre em contato conoscoFale conosco & Contato
1. (US20170004185) METHOD AND SYSTEM FOR IMPLEMENTING COLLECTION-WISE PROCESSING IN A LOG ANALYTICS SYSTEM
Nota: O texto foi obtido por processos automáticos de reconhecimento ótico de caracteres.
Para fins jurídicos, favor utilizar a versão PDF.

Claims

1. A method implemented with a processor for implementing collection-wise log data processing, comprising:
maintaining a first datastore that holds log data that corresponds to a first time period, the first datastore organized as an indexed datastore where the log data is partitioned into collections by both collection size and temporal range, the first datastore organized into collections by:
receiving the log data for processing:
placing the log data into a currently open collection;
checking if the currently open collection reaches a size threshold;
closing the currently open collection to form a closed collection when the size threshold is reached, and associating a starting and ending time metadata with the closed collection;
creating a new collection when closing the currently open collection;
maintaining an index that identifies a given collection based upon the starting and ending time metadata;
maintaining a second datastore holds the log data that corresponds to a second time period, the second time period being longer than the first time period such that the second datastore forms a historical record of the log data; and
processing a query for the log data, wherein a determination is made whether the first datastore or the second datastore is accessed to process the query by determining whether a temporal range of the query matches the first time period for the first datastore or the second time period for the second datastore.
2. The method of claim 1, wherein the size threshold corresponds to a count of entries within the currently open collection.
3. The method of claim 1, wherein the closed collection comprises a buffer storage portion to hold late arriving log data.
4. The method of claim 1, wherein the query is processed by:
performing a lookup within metadata associated with the collections;
identifying the collections correlate to the query;
processing on an individual basis the collections correlate to the query; and
returning query results on a collection basis.
5. The method of claim 1, further comprising displaying query results on a display device, wherein lazy loading of the query results is performed to display the query results by display query results for a first collection before query processing has completed for a second collection.
6. The method of claim 5, further comprising:
monitoring for user interaction on a user interface during the lazy loading of the query results;
determining whether the user interaction indicates an instruction to stop query processing; and
ending the query processing after the query results for the first collection have been displayed but before the query processing has completed for the second collection
7. The method of claim 1, wherein one or more additional containers are created for the closed collection to hold additional log data that would otherwise cause the closed collection to exceed the size threshold, wherein the starting and ending metadata of the one or more additional containers are the same as the starting and ending metadata of the closed collection.
8. The method of claim 1, wherein the log data is formed by parsing raw log data and storing a plurality of different data fields from within the raw log data, the raw data being stored into a third datastore.
9. The method of claim 1, wherein both the first datastore and the second datastore are accessed to process the query.
10. A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for configuring log monitoring to be performed in a system that implements collection-wise log data processing, the method comprising:
maintaining a first datastore that holds log data that corresponds to a first time period, the first datastore organized as an indexed datastore where the log data is partitioned into collections by both collection size and temporal range, the first datastore organized into collections by:
receiving the log data for processing:
placing the log data into a currently open collection;
checking if the currently open collection reaches a size threshold;
closing the currently open collection to form a closed collection when the size threshold is reached, and associating a starting and ending time metadata with the closed collection;
creating a new collection when closing the currently open collection;
maintaining an index that identifies a given collection based upon the starting and ending time metadata;
maintaining a second datastore holds the log data that corresponds to a second time period, the second time period being longer than the first time period such that the second datastore forms a historical record of the log data; and
processing a query for the log data, wherein a determination is made whether the first datastore or the second datastore is accessed to process the query by determining whether a temporal range of the query matches the first time period for the first datastore or the second time period for the second datastore.
11. The computer readable medium of claim 10, wherein the size threshold corresponds to a count of entries within the currently open collection.
12. The computer readable medium of claim 10, wherein the closed collection comprises a buffer storage portion to hold late arriving log data.
13. The computer readable medium of claim 10, wherein the query is processed by:
performing a lookup within metadata associated with the collections;
identifying the collections correlate to the query;
processing on an individual basis the collections correlate to the query; and
returning query results on a collection basis.
14. The computer readable medium of claim 10, wherein the sequence of instructions, when executed by the processor, further causes the processor to execute displaying query results on a display device, wherein lazy loading of the query results is performed to display the query results by display query results for a first collection before query processing has completed for a second collection.
15. The computer readable medium of claim 14, wherein the sequence of instructions, when executed by the processor, further causes the processor to execute:
monitoring for user interaction on a user interface during the lazy loading of the query results;
determining whether the user interaction indicates an instruction to stop query processing; and
ending the query processing after the query results for the first collection have been displayed but before the query processing has completed for the second collection
16. The computer readable medium of claim 10, wherein one or more additional containers are created for the closed collection to hold additional log data that would otherwise cause the closed collection to exceed the size threshold, wherein the starting and ending metadata of the one or more additional containers are the same as the starting and ending metadata of the closed collection.
17. A system, comprising:
a processor;
a memory having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute operations comprising: (1) maintaining a first datastore that holds log data that corresponds to a first time period, the first datastore organized as an indexed datastore where the log data is partitioned into collections by both collection size and temporal range, the first datastore organized into collections by: (a) receiving the log data for processing; (b) placing the log data into a currently open collection; (c) checking if the currently open collection reaches a size threshold; (d) closing the currently open collection to form a closed collection when the size threshold is reached, and associating a starting and ending time metadata with the closed collection; (e) creating a new collection when closing the currently open collection; (f) maintaining an index that identifies a given collection based upon the starting and ending time metadata; (2) maintaining a second datastore holds the log data that corresponds to a second time period, the second time period being longer than the first time period such that the second datastore forms a historical record of the log data; and ( 3) processing a query for the log data, wherein a determination is made whether the first datastore or the second datastore is accessed to process the query by determining whether a temporal range of the query matches the first time period for the first datastore or the second time period for the second datastore.
18. The system of claim 17, wherein the size threshold corresponds to a count of entries within the currently open collection.
19. The system of claim 17, wherein the closed collection comprises a buffer storage portion to hold late arriving log data.
20. The system of claim 17, wherein the sequence of instructions, when executed by the processor, further causes the processor to process a query for the log data by:
performing a lookup within metadata associated with the collections; identifying the collections correlate to the query; processing on an individual basis the collections correlate to the query; and returning query results on a collection basis.
21. The system of claim 17, wherein the sequence of instructions, when executed by the processor, further causes the processor to display query results on a display device, wherein lazy loading of the query results is performed to display the query results by display query results for a first collection before query processing has completed for a second collection.