Virtual 3D Symphony Concert

Volumetric video is a brand new 3D video production technology that uses hundreds of high-resolution cameras to capture portraits and actions from multiple angles and restore scenes in three-dimensional space. This technology can be used to make VR movies, or to create AR and MR content.

Recently, a shopping mall in Guangzhou is planning to hold a large-scale virtual concert. Customers only need to take out their mobile phones and aim at the stage area, and they can see the 3D dynamic live performers performing in the real space from different angles. Musicians cannot actually touch them. They recorded them in advance by volume capture, and then holographically placed them in the mall through location-based AR. People can hear symphonic stereo concerts, and performers do not need to travel to Guangzhou's shopping malls across cities or even internationally. This solves the inconveniences of cross-regional isolation during the epidemic and has also gathered a lot of popularity for the malls.

People take out their mobile phones and turn on the cameras in the open space, and bring AR glasses prepared in advance by the mall to watch the concert immersively. It supports adjustment of different distances and angles. The audience wears masks to keep the distance without affecting the viewing. Experience, it can be said that each location is equivalent to a VIP ticket for a concert.

The image technology that this concert relies on is volumetric video technology that was only put into commercial use in 2015. At present, there are only more than 10 volumetric video capture program producers in the world. Due to the high technical threshold, most of them are Microsoft, Google, and some technology-based startups in the United States and Western Europe. The technical support for this event comes from Shenzhen Prometheus Vision Technology Co., Ltd. (hereinafter referred to as Prometheus Vision), a domestic manufacturer that provides second-generation volumetric video solutions. The company was established in Shenzhen in 2018. With lightweight volumetric video technology, on the one hand, it provides customers with a complete volumetric video capture solution, including dynamic 3D shooting scheme construction, shooting software, and after-sales service, API and SDK embedded services; on the other hand, it also provides time-based shooting For the service, users only need to go to the designated location to shoot, and the processed 3D volume video can be taken on site, and can be easily embedded into their own products through the provided API and SDK.

The core advantage of this second-generation volumetric video solution from Prometheus Vision is to use algorithms such as deep learning to make up for the lack of precision of consumer-grade cameras. It only uses detachable mobile studios and lightweight consumer-grade cameras to shoot. Can achieve the first-generation production effect. At the same time, by optimizing the process to increase the processing speed, real-time volumetric video production can be achieved, and the content of one-minute shooting can be produced into 15-20 frames of video within one minute, so that "what you shoot is what you get.

In the founding team of Prometheus Vision, 3 of the 4 partners are of technical background. They have worked in Lenovo Hong Kong Research Institute and OPPO's vision algorithm department. They have expertise in high-performance computing, computer vision, computer graphics, etc. There is more in-depth research and practice. CEO Zhang Yu is a PhD in Computer Science at Tsinghua University, and his laboratory is a cooperative member of Microsoft Research Asia. At the beginning of its establishment, the team won the second prize in the finals of the Deep Innovation Competition, and then successively won the top 100 companies in the budding list of the 2019 China Entrepreneurship Martial Arts Conference, the 2019 Golden V Award Best Application Enterprise Award and other honors.

Volumetric video technology has been developed for several years since its appearance in 2015. The first generation of technology was led by large companies such as Google and Microsoft. Most manufacturers would build a "photo studio" the size of a factory building when shooting, using a large number of The camera records all the light and shadow changes and transmits them to the computer room, and finally makes the video through the computer. This method can reflect the world more realistically, but requires extremely high costs and professional shooting and editing techniques. Therefore, the fees have always been relatively high. Microsoft's volumetric video capture equipment in the United States costs US$20,000 per day. Another bigger problem is that the production time is also longer, and it takes 6-8 hours to process a 1-minute volume video. Therefore, the video application party usually complains about the shooting content on the first day, and only discovers the problem on the second day, which causes a waste of cost and time.

On the basis of the first generation, the second-generation volumetric video technology uses lighter equipment, realizes real-time video production, and has a lower cost, which greatly improves the cost-effectiveness of the solution.

With the development of economic and technological level, consumers have an increasing demand for cultural and entertainment innovative experiences, and they are increasingly eager to reduce time and money costs and obtain a richer sensory experience. This is a challenge to business and innovation capabilities for industries such as commercial real estate and cultural activities. With the emergence of the second-generation volumetric video, it is believed that in the future, there will be more innovation and inspiration collisions and collisions in customer flow and enhanced scene experience. Interactive form.

More about“ virtual 3D”