Disclosed are methods and apparatus for inspecting a semiconductor sample. This system comprises an illumination optics subsystem for generating and directing an incident beam towards a defect on a surface of a wafer. The illumination optics subsystem includes a light source for generating the incident beam and one or more polarization components for adjusting a ratio and/or a phase difference for the incident beam's electric field components. The system further includes a collection optics subsystem for collecting scattered light from the defect and/or surface in response to the incident beam, and the collection optics subsystem comprises an adjustable aperture at the pupil plane, followed by a rotatable waveplate for adjusting a phase difference of electric field components of the collected scattered light, followed by a rotatable analyzer. The system also includes a controller that is configured for (i) selecting a polarization of the incident beam, (ii) obtaining a defect scattering map, (iii) obtaining a surface scattering map, and (iv) determining a configuration of the one or more polarization components, aperture mask, and rotatable ¼ waveplate, and analyzer based on analysis of the defect and surface scattering map so as to maximize a defect signal to noise ratio.