In this paper, we present a novel and effective framework, named 4K-NeRF, to pursue high fidelity view synthesis on the challenging scenarios of ultra high resolutions, building on the methodology of neural radiance fields (NeRF).
The rendering procedure of NeRF-based methods typically relies on a pixel-wise manner in which rays (or pixels) are treated independently on both training and inference phases, limiting its representational ability on describing subtle details, especially when lifting to a extremely high resolution. We address the issue by exploring ray correlation to enhance high-frequency details recovery. Particularly, we use the 3D-aware encoder to model geometric information effectively in a lower resolution space and recover fine details through the 3D-aware decoder, conditioned on ray features and depths estimated by the encoder. Joint training with patch-based sampling further facilitates our method incorporating the supervision from perception oriented regularization beyond pixel-wise loss.
Benefiting from the use of geometry-aware local context, our method can significantly boost rendering quality on high-frequency details compared with modern NeRF methods, and achieve the state-of-the-art visual quality on 4K ultra-high-resolution scenarios.
|
|
|
|
|
|
The following shows how our method compares in detail to all kinds of comparison methods in representative scenario.
@article{wang20224k,
title={4K-NeRF: High Fidelity Neural Radiance Fields at Ultra High Resolutions},
author={Wang, Zhongshu and Li, Lingzhi and Shen, Zhen and Shen, Li and Bo, Liefeng},
journal={arXiv preprint arXiv:2212.04701},
year={2022}
}