An apparatus and method for delivering a spatially enhanced live streaming experience for virtual reality or 360 degree live streaming of video is disclosed. A live streaming video signal is encoded into multiple streams at varying resolutions. A portion of the high resolution video stream, corresponding to a field of view within the entire 360 degree view, is merged with a low resolution video stream. The resulting video stream is referred to as a spatial adaptive video stream. Multiple spatial adaptive video streams are generated to provide a high resolution field of view across the entire 360 degrees. As the viewer looks in different directions, the video player plays back one of the spatial adaptive video streams according to the direction in which the viewer is looking.