In block-based motion or depth estimation, a block is assigned a motion or depth value as a result of minimizing the matching error over a limited set (45) of candidate values. This set (45) usually comprises one random motion or depth value, to prevent the algorithm from stacking in local minimums. The method and system according to the invention instead add to this set (45) one motion or depth value based on properties of the image. A history of temporal changes for chosen motion or depth values is recorded, a histogram is made and a plurality of values is created in proportion to the histogram. The one added value is chosen from this plurality. Additionally, information about the direction of the changes is recorded to prevent oscillation. An apparatus for adapting a video signal (40) uses the chosen candidate values to create an enhanced version of the video signal (40).