A head mounted display (HMD) comprises an eye tracking system configured to enable eye tracking using light. The eye tracking system comprises two or more illumination sources positioned relative to one another and an optical detector for each eye. The optical detector is configured to capture images of the illumination sources using reflections from the eye. The system gathers data as the user views various objects in the HMD. The system determines a shape of the eye by determining one or more of a pupil shape, a foveal offset, and a corneal radius in three-dimensional space. The shape of the eye is used determine a three-dimensional position of the user's eye as well as a gaze direction.