Recently, Dr. Yaosen Chen's team at the Sobey Media Intelligence Lab successfully published their latest research findings in the field, "UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene," in the international computer graphics and visualization journal, IEEE Transactions on Visualization and Computer Graphics (TVCG). This research not only brings a new research perspective to the academic community but also provides an achievable engineering path to solve the problem of realistic stylization defects in 3D scenes and the consistency of video rendering from different perspectives, opening a new chapter in the combination of realistic style transfer and neural radiance field rendering technology.
About IEEE Transactions on Visualization and Computer Graphics (TVCG): TVCG is a CCF Class-A and Q1 journal in the fields of computer graphics, visualization, and virtual reality research, with an impact factor of 5.226. It has long been regarded as one of the most authoritative publishing platforms in these fields.
This paper presents a 3D scene realism style transfer algorithm that can seamlessly transfer the color style of a given image to a 3D model, achieving fine-grained color style control and video consistency across continuous viewpoints, making the rendered scene more lifelike after style transfer. This algorithm overcomes the limitations of traditional style transfer methods in handling complex 3D scenes, such as inconsistent style representation and loss of detail, thus achieving 3D realistic style transfer for images of any style, surpassing existing methods in visual quality and consistency. This has significant practical value for multiple fields such as video production, virtual reality, and game modeling.
In this algorithm framework, 3D scene realism style transfer is divided into two stages: the first stage is geometric training for a single scene, and the second stage is style training.
Comparison of style transfer using the Sobey algorithm (second column) and traditional algorithms, and an example of style transfer using the Sobey algorithm alone
Since its establishment in 2016, Sobey Media Intelligence Lab has combined audiovisual application scenarios with a commitment to breakthroughs in core algorithms, conducting in-depth research in AI fields such as natural language processing, computer vision, and machine learning. At the academic level, the lab has not only published dozens of SCI papers but also obtained numerous patents and software copyrights, demonstrating its strong capabilities in media intelligence research. At the engineering level, Sobey has successfully applied its research results to the development of multiple platform-level products, such as VIDA, intelligent media assets, and the Mingmou language model.
In the future, Sobey will continue to dedicate itself to technological research and product development in key areas of artificial intelligence, further promote the integration of science and art, and unlock the development code and provide innovative impetus for the industry.
