Processing

Please wait...

Settings

Settings

Goto Application

1. WO2021067443 - SEQUENCE EXTRACTION USING SCREENSHOT IMAGES

Publication Number WO/2021/067443
Publication Date 08.04.2021
International Application No. PCT/US2020/053556
International Filing Date 30.09.2020
IPC
G05B 19/04 2006.01
GPHYSICS
05CONTROLLING; REGULATING
BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
19Programme-control systems
02electric
04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
G06T 1/00 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
1General purpose image data processing
G06Q 10/06 2012.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
QDATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
10Administration; Management
06Resources, workflows, human or project management, e.g. organising, planning, scheduling or allocating time, human or machine resources; Enterprise planning; Organisational models
G06K 9/20 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
20Image acquisition
G06F 3/0484 2013.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
3Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
01Input arrangements or combined input and output arrangements for interaction between user and computer
048Interaction techniques based on graphical user interfaces
0484for the control of specific functions or operations, e.g. selecting or manipulating an object or an image, setting a parameter value or selecting a range
G06F 9/44 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
FELECTRIC DIGITAL DATA PROCESSING
9Arrangements for program control, e.g. control units
06using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
44Arrangements for executing specific programs
CPC
B25J 9/163
BPERFORMING OPERATIONS; TRANSPORTING
25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
9Programme-controlled manipulators
16Programme controls
1628characterised by the control loop
163learning, adaptive, model based, rule based expert control
B25J 9/1656
BPERFORMING OPERATIONS; TRANSPORTING
25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
9Programme-controlled manipulators
16Programme controls
1656characterised by programming, planning systems for manipulators
G06K 9/344
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
20Image acquisition
34Segmentation of touching or overlapping patterns in the image field
344using recognition of characters or words
G06K 9/6218
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6217Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
6218Clustering techniques
Applicants
  • UIPATH, INC. [US]/[US]
Inventors
  • BERG, Christian
  • DISFANI, Shirin Feiz
Agents
  • NORTON, Todd A.
Priority Data
16/591,16102.10.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) SEQUENCE EXTRACTION USING SCREENSHOT IMAGES
(FR) EXTRACTION DE SÉQUENCE AU MOYEN D’IMAGES DE CAPTURES D’ÉCRAN
Abstract
(EN)
A system and method for sequence extraction using screenshot images to generate a robotic process automation workflow is disclosed. The system and method include capturing a plurality of screenshots of steps performed by a user on an application using a processor, storing the screenshots in memory, determining action clusters from the captured screenshots by randomly clustering actions into an arbitrary predefined number of clusters, wherein screenshots of different variations of a same action is labeled in the clusters, extracting a sequence from the clusters, and discarding consequent events on the screen from the clusters, and generating an automated workflow based on the extracted sequences.
(FR)
L’invention concerne un système et un procédé pour l’extraction de séquence au moyen d’images de captures d’écran pour générer un flux de travail d’automatisation de processus robotique. Le système et le procédé consistent à capturer une pluralité de captures d’écran d’étapes mises en œuvre par un utilisateur sur une application utilisant un processeur, à sauvegarder les captures d’écran dans une mémoire, à déterminer des groupes d’actions à partir des captures d’écran en regroupant aléatoirement des actions dans un nombre de groupes prédéfini arbitrairement, des captures d’écran de différentes variations d’une même action étant étiquetées dans les groupes, à extraire une séquence des groupes, à rejeter des événements conséquents sur l’écran depuis les groupes, et à générer un flux de travail automatisé sur la base des séquences extraites.
Also published as
Latest bibliographic data on file with the International Bureau