Traitement en cours

Veuillez attendre...

Paramétrages

Paramétrages

Aller à Demande

1. WO2020108834 - SYSTÈME ET PROCÉDÉ D’ANALYSE DE LA DÉMARCHE D’UN ÊTRE HUMAIN

Note: Texte fondé sur des processus automatiques de reconnaissance optique de caractères. Seule la version PDF a une valeur juridique

[ EN ]

Claims

1. A computer-implemented method (1000) for human gait analysis based on a video stream (202) obtained from a monocular camera device (201), the video stream comprising a plurality of frames reflecting the walk of a human individual (10), the method comprising:

inferring (1100), from the obtained video stream, three-dimensional gait information wherein the three-dimensional gait information includes estimates of the individual's joint locations including at least the individual's foot locations on each frame, the estimates being derived by matching for each frame two-dimensional joint coordinates of the respective frame with respective three-dimensional model information of the individual's body;

and

determining (1200) one or more gait parameters of the individual based on the individual's foot locations in local extrema frames showing local extrema of the distance between one foot location of the individual and a corresponding reference joint location.

2. The method of claim 1, wherein inferring (1100) further comprises:

deriving (1110) heat-maps and location-maps for joint location estimation by using a convolutional neural network trained on three dimensional human pose datasets, wherein a particular heat-map describes, for a corresponding frame, probability values that a particular joint is associated with respective pixels of the corresponding frame, and wherein a particular set of location-maps includes a plurality of location-maps, with each of the location maps describing the distance of a particular joint to a root location for the corresponding frame in a respective spatial dimension;

receiving (1120) a selection of at least a frame sequence of the video stream during which the joints of the individual (10) move smoothly over time;

estimating (1130) a skeleton model of the individual (10) by determining, for each frame of the selected sequence, a loss for each joint of a default skeleton model in each spatial coordinate, and adjusting the default skeleton model to compensate the determined losses to provide an adjusted skeleton model; and

performing (1140) kinematic skeleton fitting per video frame using the adjusted skeleton model to determine a plurality of joint locations including at least the foot locations of the individual's feet on each frame.

3. The method of claim 2, wherein deriving (1110) further comprises:

identifying (1111) a plurality of heat-map pixels in a particular heat-map wherein each of the identified pixels is associated with the same maximum probability value for the respective joint associated with the particular heat-map;

for each identified pixel, integrating (1113) heat-map values in a predefined area adjacent to the identified pixel; and

selecting (1115) the identified pixel with the highest integral value as the pixel associated with the respective joint location.

4. The method of any of claims 1 to 3, wherein the one or more gait parameters comprise a step length gait parameter, and wherein the corresponding reference joint location is the location of the individual's second foot in a respective local maximum frame (flma*), the method further comprising:

computing (1210) a mean value (si) of the distances (si*) between the individual's foot locations in the determined local maximum frames (flma*) as the step length gait parameter.

5. The method of claim 4, wherein the distance between the individual's foot locations is measured directly in a respective frame based on the corresponding pixel coordinates.

6. The method of claim 4, wherein the distance between the individual's foot locations is measured indirectly for a respective local maximum frame based on the distances of the individual's feet from the monocular camera device.

7. The method of any of the claims 4 to 6, wherein the one or more gait parameters further comprise a walking speed gait parameter, the method further comprising:

computing (1220) a walking distance of the individual based on the determined step length and the number of the local maximum frames identified in the video stream, and dividing

the walking distance by the time interval associated with the sequence of frames associated with the walking distance to obtain (1230) the walking speed.

8. The method of any of the previous claims, wherein the one or more gait parameters further comprise a step height gait parameter, the method further comprising:

extrapolating (1240) a walking floor plane (WFP) from floor contact points (cp*) of the individual's feet identified from local maximum frames (flma*) wherein the corresponding reference joint location is the location of the individual's second foot in the respective local maximum frame;

determining (1250) further local maximum frames (flma*') with further local maxima (Ima*') of the distances of the individual's foot locations from the walking floor plane (WFP), and computing (1260) a mean value (sh) of the further local maximum distances as the step height.

9. The method of any of the claims 1 to 7, wherein the one or more gait parameters further comprise a step height gait parameter, and wherein the corresponding reference joint location is the location of the head (318) or pelvis (319) of the adjusted skeleton model in a respective local minimum frame, the method further comprising

determining (1270) local minimum frames with local minima of the distances between the individual's foot locations and the corresponding reference joint location; and

computing (1280) a mean value of the distances between the individual's two feet in local minimum frames as the step height.

10. The method of any of the previous claims, wherein at least one of the determined gait parameters is associated with a score characterizing a risk of fall for the individual.

11. The method of any of the previous claims, wherein, for determining (1200) the one or more gait parameters, clusters of honest local extrema frames are determined where the frames of a particular cluster contribute to the computation of the individual's step length in that an average distance value is computed based on all frames of the respective cluster so that the honest local extrema frames of the particular cluster include such frames which collectively reflect the distance between the individual's feet for a particular step of the individual.

12. A computer program product comprising instructions that, when loaded into a memory of a computing device and executed by at least one processor of the computing device, execute the method steps of the computer implemented method according to any one of the previous claims.

13. A computer system (100) for human gait analysis comprising:

an interface (110) configured to obtain a video stream (202) from a monocular camera device (201), the video stream (202) comprising a plurality of frames reflecting the walk of a human individual (10);

a skeletal motion extractor (120) configured to extract from the video stream (202) a skeletal motion associated with the individual (10), by inferring, from the obtained video stream, three-dimensional gait information wherein the three-dimensional gait information includes estimates of the individual's joint locations including at least the individual's foot locations on each frame, the estimates being derived by matching for each frame two-dimensional joint coordinates of the respective frame with respective three-dimensional model information of the individual's body; and

a gait parameter determining module (130) configured to determine one or more gait parameters of the individual based on the individual's foot locations (311, 312) in local extrema frames showing local extrema of the distance between one foot location of the individual and a corresponding reference joint location.

14. The system of claim 13, wherein the extractor module further comprises:

a two dimensional joint coordinate estimator module configured to provide, for each frame, estimates of two dimensional joint coordinates of each joint, together with a confidence value for each joint;

and

a three dimensional fitting module configured to fit a three dimensional human body model to the two dimensional joint coordinates.

15. The system of claim 13 or 14, wherein the one or more gait parameters comprise a step length gait parameter, and wherein the corresponding reference joint location is the location of the individual's second foot in a respective local maximum frame, the gait parameter determining module further configured to:

compute a total distance traveled by the individual by applying the pinhole projection model to determine distances between the camera and a reference point being characteristic of the individual's current location for at least a first and a second frame, the first and second frame defining a video stream sequence to be used for step length analysis, the traveled distance derived from the determined distances;

determining the number of steps in the defined video stream sequence as the number of local maximum frames with local maxima of the distance between the individual's feet; and

dividing the computed traveled distance by the number of local maximum frames to obtain the average step length.

16. The system of any of the claims 13 to 15, wherein the one or more gait parameters further comprise a step height gait parameter, the gait parameter determining module further configured: to determine local minimum frames with local minima of the distances between the individual's foot locations and the head or pelvis of the adjusted skeleton model; and

to compute a mean value of the distances between the two feet in local minimum frames as the step height.

17. The system of any of the claims 13 to 16, wherein the gait parameter determining module (130) is further configured to determine clusters of honest local extrema frames where the frames of a particular cluster contribute to the computation of the individual's step length in that an average distance value is computed based on all frames of the respective cluster so that the honest local extrema frames of the particular cluster include such frames which collectively reflect the distance between the individual's feet for a particular step of the individual.

18. The system of any of the claims 13 to 17, wherein at least one of the determined gait parameters is associated with a score characterizing a risk of fall for the individual.