Deep Networks for Image Super-Resolution with Sparse Prior

Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, Thomas Huang


Deep learning techniques have been successfully applied in many areas of computer vision, including low-level image restoration problems. For image super-resolution, several models based on deep neural networks have been recently proposed and attained superior performance that overshadows all previous handcrafted models. The question then arises whether large-capacity and data-driven models have become the dominant solution to the ill-posed super-resolution problem. In this paper, we argue that domain expertise represented by the conventional sparse coding model is still valuable, and it can be combined with the key ingredients of deep learning to achieve further improved results. We show that a sparse coding model particularly designed for super-resolution can be incarnated as a neural network, and trained in a cascaded structure from end to end. The interpretation of the network based on sparse coding leads to much more efficient and effective training, as well as a reduced model size. Our model is evaluated on a wide range of images, and shows clear advantage over existing state-of-the-art methods in terms of both restoration accuracy and human subjective quality.

Network Architeture

network architecture


results table

Super-resolved images from various datasets:


Test code in python: [Download]

Reimplementation in Matlab: [GitHub]


Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han and Thomas S. Huang, Deep Networks for Image Super-Resolution with Sparse Prior. Proceedings of the IEEE International Conference on Computer Vision, 2015. [pdf][bib]

Ding Liu, Zhaowen Wang, Bihan Wen, Jianchao Yang, Wei Han and Thomas S. Huang, Robust Single Image Super-Resolution via Deep Networks with Sparse Prior. IEEE Transcations on Image Processing, 2016. [pdf][bib]

Related Work

Ding Liu, Zhaowen Wang, Nasser Nasrabadi and Thomas S. Huang, Learning a Mixture of Deep Networks for Single Image Super-Resolution. Asian Conference on Computer Vision, 2016. [project webpage][pdf][bib]