Processing

Please wait...

Settings

Settings

1. WO2017136833 - SYSTEMS AND METHODS FOR AUGMENTED REALITY

Note: Text based on automatic Optical Character Recognition processes. Please use the PDF version for legal matters

CLAIMS

1. An augmented reality (AR) display system, comprising:

an electromagnetic field emitter to emit a known magnetic field in a known coordinate system;

an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field;

a depth sensor to measure a distance in the known coordinate system;

a controller to determine pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor; and

a display system to display virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter.

2. The AR display system of claim 1, wherein the depth sensor is a passive stereo depth sensor.

3. The AR display system of claim 1, wherein the depth sensor is an active depth sensor.

4. The AR display system of claim 3, wherein the depth sensor is a texture projection stereo depth sensor.

5. The AR display system of claim 3, wherein the depth sensor is a structured light projection stereo depth sensor.

6. The AR display system of claim 3, wherein the depth sensor is a time of flight depth sensor

7. The AR display system of claim 3, wherein the depth sensor is a LIDAR depth sensor.

8. The AR display system of claim 3, wherein the depth sensor is a modulated emission depth sensor.

9. The AR display system of claim 1, wherein the depth sensor comprises a depth camera having a first field of view (FOV).

10. The AR display system of claim 9, further comprising a world capture camera,

wherein the world capture camera has a second FOV at least partially overlapping with the first FOV.

11. The AR display system of claim 10, further comprising a picture camera,

wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV.

12. The AR display system of claim 11, wherein the depth camera, the world capture camera, and the picture camera have respective different first, second, and third resolutions.

13. The AR display system of claim 12, wherein the first resolution of the depth camera is sub- VGA, the second resolution of the world capture camera is 720p, and the third resolution of the picture camera is 2 megapixels.

14. The AR display system of claim 11, wherein the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images.

15. The AR display system of claim 14, wherein the controller is programmed to segment the second and third images.

16. The AR display system of claim 15, wherein the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.

17. The AR display system of claim 16, wherein measuring a distance in the known coordinate system comprises:

generating a hypothetical distance by analyzing the first image from the depth camera; and

generating the distance by analyzing the hypothetical distance and the fused image.

18. The AR display system of claim 11 , wherein the depth camera, the world capture camera, and the picture camera form a single integrated sensor.

19. The AR display system of claim 1, further comprising an additional localization resource to provide additional information, wherein the pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system is determined based at least in part on the parameter related to the magnetic flux measured by the

electromagnetic sensor, the distance measured by the depth sensor, and the additional information provided by the additional localization resource.

20. The AR display system of claim 19, wherein the additional localization resource comprises a WiFi transceiver.

21. The AR display system of claim 19, wherein the additional localization resource comprises an additional electromagnetic emitter.

22. The AR display system of claim 19, wherein the additional localization resource comprises an additional electromagnetic sensor.

23. The AR display system of claim 19, wherein the additional localization resource comprises a beacon.

24. The AR display system of claim 23, wherein the beacon emits radiation.

25. The AR display system of claim 24, wherein the radiation is infrared radiation, and wherein the beacon comprises an infrared LED.

26. The AR display system of claim 19, wherein the additional localization resource comprises a reflector.

27. The AR display system of claim 26, wherein the reflector reflects radiation.

28. The AR display system of claim 19, wherein the additional localization resource comprises a cellular network transceiver.

29. The AR display system of claim 19, wherein the additional localization resource comprises a RADAR emitter.

30. The AR display system of claim 19, wherein the additional localization resource comprises a RADAR detector.

31. The AR display system of claim 19, wherein the additional localization resource comprises a LIDAR emitter.

32. The AR display system of claim 19, wherein the additional localization resource comprises a LIDAR detector.

33. The AR display system of claim 19, wherein the additional localization resource comprises a GPS transceiver.

34. The AR display system of claim 19, wherein the additional localization resource comprises a poster having a known detectable pattern.

35. The AR display system of claim 19, wherein the additional localization resource comprises a marker having a known detectable pattern.

36. The AR display system of claim 19, wherein the additional localization resource comprises an inertial measurement unit.

37. The AR display system of claim 19, wherein the additional localization resource comprises a strain gauge.

38. The AR display system of claim 1, wherein the electromagnetic field emitter is coupled to a mobile component of the AR display system.

39. The AR display system of claim 38, wherein the mobile component is a hand-held component.

40. The AR display system of claim 39, wherein the mobile component is a totem.

41. The AR display system of claim 38, wherein the mobile component is a head-mounted component that houses the display system.

42. The AR display system of claim 38, wherein the mobile component is a torso-worn component.

43. The AR display system of claim 42, wherein the torso- worn component is a belt-pack.

44. The AR display system of claim 1, wherein the electromagnetic field emitter is coupled to an object in the known coordinate system, such that the electromagnetic field emitter has a known position and a known orientation.

45. The AR display system of claim 44, wherein the electromagnetic sensor is coupled to a mobile component of the AR display system.

46. The AR display system of claim 45, wherein the mobile component is a hand-held component.

47. The AR display system of claim 46, wherein the mobile component is a totem.

48. The AR display system of claim 45, wherein the mobile component is a head-mounted component that houses the display system.

49. The AR display system of claim 45, wherein the mobile component is a torso-worn component.

50. The AR display system of claim 49, wherein the torso-worn component is a belt-pack.

51. The AR display system of claim 1, wherein the pose information comprises a position and an orientation of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system.

52. The AR display system of claim 1 , wherein the controller analyzes the pose information to determine a position and an orientation of the electromagnetic sensor in the known coordinate system.

53. A method for displaying augmented reality, the method comprising:

emitting, using an electromagnetic field emitter, a known magnetic field in a known coordinate system;

measuring, using an electromagnetic sensor, a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field;

measuring, using a depth sensor, a distance in the known coordinate system;

determining pose information of the electromagnetic sensor relative to the

electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured using the electromagnetic sensor and the distance measured using the depth sensor; and

displaying virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter.

54. The method of claim 53, wherein the depth sensor is a passive stereo depth sensor.

55. The method of claim 53, wherein the depth sensor is an active depth sensor.

56. The method of claim 55, wherein the depth sensor is a texture projection stereo depth sensor.

57. The method of claim 55, wherein the depth sensor is a structured light projection stereo depth sensor.

58. The method of claim 55, wherein the depth sensor is a time of flight depth sensor

59. The method of claim 55, wherein the depth sensor is a LIDAR depth sensor.

60. The method of claim 55, wherein the depth sensor is a modulated emission depth sensor.

61. The method of claim 53, wherein the depth sensor comprises a depth camera having a first field of view (FOV).

62. The method of claim 61, wherein the depth sensor further comprises a world capture camera,

wherein the world capture camera has a second FOV at least partially overlapping with the first FOV.

63. The method of claim 62, wherein the depth sensor further comprises a picture camera,

wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV.

64. The method of claim 63, wherein the depth camera, the world capture camera, and the picture camera have respective different first, second, and third resolutions.

65. The method of claim 64, wherein the first resolution of the depth camera is sub-VGA, the second resolution of the world capture camera is 720p, and the third resolution of the picture camera is 2 megapixels.

66. The method of claim 63, further comprising capturing first, second, and third images using respective depth camera, world capture camera, and picture camera.

67. The method of claim 66, further comprising segmenting the second and third images.

68. The method of claim 67, further comprising fusing the second and third images after segmenting the second and third images to generate a fused image.

69. The method of claim 68, wherein measuring a distance in the known coordinate system comprises:

generating a hypothetical distance by analyzing the first image from the depth camera; and

generating the distance by analyzing the hypothetical distance and the fused image.

70. The method of claim 63, wherein the depth camera, the world capture camera, and the picture camera form a single integrated sensor.

71. The method of claim 53, further comprising, determining the pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured using the electromagnetic sensor, the distance measured using the depth sensor, and additional information provided by an additional localization resource.

72. The method of claim 71, wherein the additional localization resource comprises a WiFi transceiver.

73. The method of claim 71, wherein the additional localization resource comprises an additional electromagnetic emitter.

74. The method of claim 71, wherein the additional localization resource comprises an additional electromagnetic sensor.

75. The method of claim 71, wherein the additional localization resource comprises a beacon.

76. The method of claim 75, further comprising the beacon emitting radiation.

77. The method of claim 76, wherein the radiation is infrared radiation, and wherein the beacon comprises an infrared LED.

78. The method of claim 71, wherein the additional localization resource comprises a reflector.

79. The method of claim 78, further comprising the reflector reflecting radiation.

80. The method of claim 71, wherein the additional localization resource comprises a cellular network transceiver.

81. The method of claim 71 , wherein the additional localization resource comprises a RADAR emitter.

82. The method of claim 71, wherein the additional localization resource comprises a RADAR detector.

83. The method of claim 71, wherein the additional localization resource comprises a LIDAR emitter.

84. The method of claim 71, wherein the additional localization resource comprises a LIDAR detector.

85. The method of claim 71, wherein the additional localization resource comprises a GPS transceiver.

86. The method of claim 71, wherein the additional localization resource comprises a poster having a known detectable pattern.

87. The method of claim 71, wherein the additional localization resource comprises a marker having a known detectable pattern.

88. The method of claim 71, wherein the additional localization resource comprises an inertial measurement unit.

89. The method of claim 71, wherein the additional localization resource comprises a strain gauge.

90. The method of claim 53, wherein the electromagnetic field emitter is coupled to a mobile component of an AR display system.

91. The method of claim 90, wherein the mobile component is a hand-held component.

92. The method of claim 91, wherein the mobile component is a totem.

93. The method of claim 90, wherein the mobile component is a head-mounted component that houses a display system.

94. The method of claim 90, wherein the mobile component is a torso-worn component.

95. The method of claim 94, wherein the torso- worn component is a belt-pack.

96. The method of claim 53, wherein the electromagnetic field emitter is coupled to an object in the known coordinate system, such that the electromagnetic field emitter has a known position and a known orientation.

97. The method of claim 96, wherein the electromagnetic sensor is coupled to a mobile component of an AR display system.

98. The method of claim 97, wherein the mobile component is a hand-held component.

99. The method of claim 98, wherein the mobile component is a totem.

100. The method of claim 97, wherein the mobile component is a head-mounted component that houses a display system.

101. The method of claim 97, wherein the mobile component is a torso-worn component.

102. The method of claim 101, wherein the torso- worn component is a belt-pack.

103. The method of claim 53, wherein the pose information comprises a position and an orientation of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system.

104. The method of claim 53, further comprising analyzing the pose information to determine a position and an orientation of the electromagnetic sensor in the known coordinate system.

105. An augmented reality display system, comprising:

a hand-held component coupled to an electromagnetic field emitter, the electromagnetic field emitter emitting a magnetic field;

a head-mounted component having a display system that displays virtual content to a user, the head mounted component coupled to an electromagnetic sensor measuring a parameter related to a magnetic flux at the electromagnetic sensor resulting from the magnetic field, wherein a head pose of the head-mounted component in a known coordinate system is known;

a depth sensor measuring a distance in the known coordinate system; and

a controller communicatively coupled to the hand-held component, the head-mounted component, and the depth sensor, the controller receiving the parameter related to the magnetic flux at the electromagnetic sensor from the head mounted component and receiving the distance from the depth sensor,

wherein the controller determines a hand pose of the hand-held component based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor,

wherein the system modifies the virtual content displayed to the user based at least in part on the hand pose.

106. The AR display system of claim 105, wherein the depth sensor is a passive stereo depth sensor.

107. The AR display system of claim 105, wherein the depth sensor is an active depth sensor.

108. The AR display system of claim 107, wherein the depth sensor is a texture projection stereo depth sensor.

109. The AR display system of claim 107, wherein the depth sensor is a structured light projection stereo depth sensor.

110. The AR display system of claim 107, wherein the depth sensor is a time of flight depth sensor

111. The AR display system of claim 107, wherein the depth sensor is a LIDAR depth sensor.

112. The AR display system of claim 107, wherein the depth sensor is a modulated emission depth sensor.

113. The AR display system of claim 105, wherein the depth sensor comprises a depth camera having a first field of view (FOV).

114. The AR display system of claim 113, further comprising a world capture camera,

wherein the world capture camera has a second FOV at least partially overlapping with the first FOV.

115. The AR display system of claim 114, further comprising a picture camera,

wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV.

116. The AR display system of claim 115, wherein the depth camera, the world capture camera, and the picture camera have respective different first, second, and third resolutions.

117. The AR display system of claim 116, wherein the first resolution of the depth camera is sub- VGA, the second resolution of the world capture camera is 720p, and the third resolution of the picture camera is 2 megapixels.

118. The AR display system of claim 115, wherein the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images.

119. The AR display system of claim 118, wherein the controller is programmed to segment the second and third images.

120. The AR display system of claim 119, wherein the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.

121. The AR display system of claim 120, wherein measuring a distance in the known coordinate system comprises:

generating a hypothetical distance by analyzing the first image from the depth camera; and

generating the distance by analyzing the hypothetical distance and the fused image.

122. The AR display system of claim 115, wherein the depth camera, the world capture camera, and the picture camera form a single integrated sensor.

123. The AR display system of claim 105, further comprising an additional localization resource to provide additional information, wherein the controller determines the hand pose of the hand-held component based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor, the distance measured by the depth sensor, and the additional information provided by the additional localization resource.

124. The AR display system of claim 123, wherein the additional localization resource comprises a WiFi transceiver.

125. The AR display system of claim 123, wherein the additional localization resource comprises an additional electromagnetic emitter.

126. The AR display system of claim 123, wherein the additional localization resource comprises an additional electromagnetic sensor.

127. The AR display system of claim 123, wherein the additional localization resource comprises a beacon.

128. The AR display system of claim 127, wherein the beacon emits radiation.

129. The AR display system of claim 128, wherein the radiation is infrared radiation, and wherein the beacon comprises an infrared LED.

130. The AR display system of claim 123, wherein the additional localization resource comprises a reflector.

131. The AR display system of claim 130, wherein the reflector reflects radiation.

132. The AR display system of claim 123, wherein the additional localization resource comprises a cellular network transceiver.

133. The AR display system of claim 123, wherein the additional localization resource comprises a RADAR emitter.

134. The AR display system of claim 123, wherein the additional localization resource comprises a RADAR detector.

135. The AR display system of claim 123, wherein the additional localization resource comprises a LIDAR emitter.

136. The AR display system of claim 123, wherein the additional localization resource comprises a LIDAR detector.

137. The AR display system of claim 123, wherein the additional localization resource comprises a GPS transceiver.

138. The AR display system of claim 123, wherein the additional localization resource comprises a poster having a known detectable pattern.

139. The AR display system of claim 123, wherein the additional localization resource comprises a marker having a known detectable pattern.

140. The AR display system of claim 123, wherein the additional localization resource comprises an inertial measurement unit.

141. The AR display system of claim 123, wherein the additional localization resource comprises a strain gauge.

142. The AR display system of claim 105, wherein the hand-held component is a totem.

143. The AR display system of claim 105, wherein the hand pose comprises a position and an orientation of the hand-held component in the known coordinate system.