Fraunhofer HHI presents the future of immersive imaging technologies at NAB
The Fraunhofer Heinrich Hertz Institute will present current innovations in the field of immersive imaging technologies at the NAB in Las Vegas (Fraunhofer booth SU4916). This is the first time that the Video Codec of the next generation will be presented to the public, along with a demonstrator for continuous live video streaming with a resolution greater than 4K for Virtual Reality (VR). In addition, the current developments of 3D Human Body Reconstruction for Virtual Reality and Augmented Reality will be shown.
Future Video Coding beyond HEVC
Compressed video data are growing at a faster rate than ever before. Already today, video data make up by far the highest percentage of bits on the Internet and in mobile traffic. This demonstrates the need for even more efficient compression, which goes beyond the current state of the art High Efficiency Video Coding standard (HEVC). In order to master this demanding challenge, the ITU Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) have already started working together in the Joint Video Experts Team (JVET).
Last October, JVET issued a call for proposals for video coding technology beyond HEVC. Fraunhofer HHI has responded to the call by submitting cutting edge coding technology to be included in the final standard by 2020. At NAB 2018, Fraunhofer HHI will display its codec to the public for the first time. The Fraunhofer HHI codec already shows significant coding efficiency improvements over HEVC for content ranging from standard High Definition (HD) to High Dynamic Range Ultra-HD content.
Live End-to-End Streaming of VR360 10K Video with MPEG-OMAF and HEVC Tiles
At NAB, Fraunhofer HHI presents the world’s first demonstrator for continuous live video streaming of VR360 videos with a resolution beyond 4K. This includes high-resolution VR360 10K video capturing and live rendering from the Fraunhofer HHI Omnicam-360, tile-based live encoding with the Fraunhofer HHI HEVC encoder, packaging according to the MPEG-OMAF Viewport-Dependent Media Profile and high-quality playback on VR glasses and TV screens.
• Capturing: Fraunhofer HHI’s OmniCam-360 is a worldwide unique system for recording high-resolution VR360 video. Thanks to the real-time solution (RTSE) of Fraunhofer HHI, live events can be captured with a resolution up to 10k x 4k.
• Encoding: The latest generation of Fraunhofer HHI‘s H.265/MPEG HEVC software encoding technology is integrating tile-based HEVC live encoding for distribution of VR360 content. Before encoding, the content from the Omnicam-360 is spatially segmented and the resulting tiles are independently encoded at different resolutions.
• Streaming: The HEVC tile streams are packaged with the MPEG-OMAF standard using the Viewport-Dependent Media Profile. Using these tiles, the devices – VR glasses and TV screen – can assemble the required image, with optimal resolution. In the user‘s line of vision, the image is therefore of high definition and behind him, the resolution is low. MPEG-OMAF and HEVC tiles enable efficient delivery of live ultra-high-resolution VR360 video to fixed and mobile devices.
3D Human Body Reconstruction for Virtual and Augmented Reality
Fraunhofer Heinrich Hertz Institute has developed a novel and uniquely integrated 360-degree multi-camera capture and lighting system for the creation of highly realistic Volumetric Video content of moving persons. A set of 16 stereo cameras create 3D information from all different viewpoints around the person. This 3D information is then fused and transformed into a consistent, natural and dynamic 3D representation of the person. The automatically computed sequence of meshes can be integrated in VR and AR applications.