Processing

Please wait...

Settings

Settings

1. WO2020006142 - AUTOMATIC DYNAMIC DIAGNOSIS GUIDE WITH AUGMENTED REALITY

Note: Text based on automatic Optical Character Recognition processes. Please use the PDF version for legal matters

CLAIMS

What is claimed is:

1. A system for diagnosing and repairing a problem with a machine, said system comprising:

a machine controller in communication with a machine, said controller including a processor and memory and being configured with machine control software for controlling operation of the machine and recording trouble codes for the machine; and an augmented reality (AR) device, said AR device having one or more cameras, position tracking sensors and a display, said AR device further including a processor and memory configured to run a diagnosis guide software application;

where the diagnosis guide software application provides features including;

providing data from the machine controller to the AR device including data indicating a problem with the machine;

establishing and continuously tracking a position and orientation of the AR device relative to a work cell coordinate frame; creating a diagnosis tree including diagnosis steps and decision points which will diagnose a root cause of the problem; and providing a diagnosis guide which traverses the diagnosis tree by interacting with a user at the diagnosis steps and the decision points until the root cause of the problem is determined and resolved.

2. The system according to Claim 1 wherein the AR device is a headset apparatus worn by a user.

3. The system according to Claim 2 wherein a portion of the diagnosis tree including the pictures, videos and descriptions are displayed in a user dialog window which is positioned at a fixed location in space for viewing by the user.

4. The system according to Claim 1 wherein the AR device is a smart phone or a tablet computing device held by a user, and an AR display and a user dialog window are displayed in a split screen mode.

5. The system according to Claim 1 wherein establishing the position and orientation of the AR device relative to the work cell coordinate frame includes analyzing images of a visual marker having a known design and placed at a known location in the work cell coordinate frame.

6. The system according to Claim 1 wherein providing data from the machine controller includes establishing communications between the AR device and the controller and downloading data about the machine including alarm history, robot joint loads and torques, servo motor status, motor currents, and trend data.

7. The system according to Claim 1 wherein providing data from the machine controller includes generating and displaying a two-dimensional barcode (QR code) by the machine controller and reading the QR code by the AR device, where the QR code identifies the trouble code.

8. The system according to Claim 1 wherein creating a diagnosis tree includes selecting a diagnosis tree previously created for the problem from a diagnosis tree library and, when a diagnosis tree for the problem does not exist in the diagnosis tree library, constructing the diagnosis tree as a chain of actions including replacing machine components in order of likelihood of resolving the problem.

9. The system according to Claim 1 wherein creating a diagnosis tree includes pruning the tree by automatically answering questions at one or more decisions points using the data from the machine controller.

10. The system according to Claim 1 wherein the user guide presents the diagnosis steps with pictures, videos, descriptions and AR data designed to assist the user in completing the diagnosis steps, asks the user to answer a question at each of the decision points, and follows a branch of the tree based on the answer.

11. The system according to Claim 1 wherein the position tracking sensors include one or more gyroscopes and one or more accelerometers, where the sensors provide signals to the processor in the AR device enabling continuous computation of changes in position and orientation of the AR device.

12. The system according to Claim 11 wherein the AR data includes virtual features superimposed on camera images of the machine, where the virtual features are designed to provide visual assistance to the user in completing the diagnosis steps, and display of the virtual features is continuously updated based on the changes in position and orientation of the AR device.

13. The system according to Claim 1 further comprising a web server including a knowledge database, where the AR device communicates with the web server and downloads diagnosis trees and AR data from the knowledge database, and uploads root cause information to the knowledge database.

14. The system according to Claim 1 wherein the machine is an industrial robot.

15. An apparatus for diagnosing and repairing a problem with an industrial robot, said apparatus comprising an augmented reality (AR) device in communication with a robot controller, said AR device having one or more cameras, inertial sensors and a display, said AR device further including a processor and memory configured to run a diagnosis guide software application, where the diagnosis guide software application is configured to download data from the robot controller including data indicating a problem with a robot, create a diagnosis tree including diagnosis steps and decision points which will diagnose a root cause of the problem, and provide a diagnosis guide which traverses the diagnosis tree by interacting with a user at the diagnosis steps and the decision points until the root cause is resolved.

16. A method for diagnosing and repairing a problem with a machine, said method comprising:

providing a machine and a machine controller in communication with the machine,

providing an augmented reality (AR) device, said AR device having one or more cameras, position tracking sensors and a display, said AR device further including a processor and memory configured to run a diagnosis guide software application;

establishing and continuously tracking a position and orientation of the AR device relative to a work cell coordinate frame; providing machine diagnostic data from the machine controller to the AR device, including data identifying a problem with the machine;

determining a diagnosis tree based on the diagnostic data, where the diagnosis tree includes diagnosis steps and decision points which will diagnose a root cause of the problem;

presenting a diagnosis guide to a user of the AR device, where the diagnosis guide follows the diagnosis tree; and

following the diagnosis guide, by the user, until the root cause of the problem is identified and corrected, including performing testing and component replacement actions at the diagnosis steps and answering questions at the decision points.

17. The method according to Claim 16 wherein presenting the diagnosis guide to the user includes providing AR data on the AR device, said AR data depicting at least one testing or component replacement activity which is included in the diagnosis tree.

18. The method according to Claim 17 wherein the AR device is a headset apparatus worn by the user, where the AR data is superimposed on real-time images of the machine and its surroundings, and where a portion of the diagnosis tree including pictures, videos and descriptions are displayed in a user dialog window which is positioned at a fixed location in space for viewing by the user.

19. The method according to Claim 18 wherein the AR data includes virtual features superimposed on the real-time images of the machine, where the virtual features are designed to provide visual assistance to the user in completing the diagnosis steps, and display of the virtual features is continuously updated based on changes in position and orientation of the AR device.

20. The method according to Claim 16 wherein determining a diagnosis tree includes selecting a diagnosis tree previously created for the problem from a diagnosis tree library or, when a diagnosis tree for the problem does not exist in the diagnosis tree library, constructing the diagnosis tree as a chain of actions including replacing machine components in order of likelihood of resolving the problem.

21. The method according to Claim 16 wherein determining a diagnosis tree includes pruning the tree by automatically answering questions at one or more decisions points using the diagnostic data from the machine controller.

22. The method according to Claim 16 further comprising communicating with a web server including a knowledge database, by the AR device, including downloading diagnosis trees and AR data from the knowledge database, and uploading root cause information to the knowledge database.

23. The method according to Claim 16 wherein the machine is an industrial robot.

24. The method according to Claim 16 wherein the data identifying a problem with the machine is an alarm code or a trouble code.