To overcome the limitations of compute capability and bandwidth in UAV (Unmanned Aerial Vehicle) video, this paper proposes a novel video compression algorithm based on hierarchical description. The video is split into background layer and object layer. Background layer is encoded with global motion estimation, while object layer is encoded with local block matching. The experimental results demonstrate that, when bandwidth is narrow e.g. 50 kbit/s, using background layer encoding can achieve higher PSNR than H.264, and the subjective quality is much more acceptable; in most cases, combining background and object layer can give more details than H.264 with the same bitrate; the consuming time is only 18% of H.264's. Additionally, alternative bitstream can be extracted for different bandwidth. Consequently, the proposed algorithm is suitable for low latency and complexity applications in UAV video due to its good performance, high efficiency and flexibility.