A system and method for distributing candidate motion vectors divides a picture frame (110) into a plurality of segments (120), where each segment (120) includes a plurality of pixel blocks (130). The local motion complexity of each segment may then be measured by determining a Sum-of-Absolute Differences (SAD) between pixel blocks (130) of the frame (110a) and pixel blocks (130)of an adacent frame (110b) and summing the SAD values for the pixel blocks (130) within each segment (120). The maximum, minimum and average SAD value for the segments may be used to define a distribution function that distributes candidate vectors to conform with a desired maximum, minimum and average number of candidate vectors per block. This distribution function is then used to determine the number of candidate vectors to assign to pixel blocks (130) based on the measured SAD value of the associated segment (120).