＂Leslie Cheung Concert＂ Repair the beginning and end： Tencent Cloud Multimedia Lab waiting for an opportunity

　

On April 1, Leslie Cheung’s last concert, which was also considered a "classic", "Passion" Ultra HD Repair, premiered globally on TME live, QQ Music, WeChat Channels, Tencent Video and other platforms.

The ultra-clear repair version of the concert by Tencent Music Entertainment Group and Universal Music’s record label Polygram jointly launched, Tencent Cloud Multimedia Lab undertook the specific repair work of the concert video.

　

Tencent Cloud Multimedia Lab is affiliated to Tencent CSIG Business Group (Cloud and Smart Industry Business Group). It has been engaged in video processing since 2013. One of its main responsibilities is to provide fully automated digital repair technology to various internal product departments, featuring low cost and high efficiency. Batch processing of 2D-based massive videos in Tencent Cloud and other systems.

　

According to reports, Tencent Cloud Multimedia Lab is mostly engaged in the repair of movies, TV dramas and other scenes. The repair of concerts is the first time. The difficulty lies in the fact that portraits and scenes can change and switch rapidly in the concert lighting. However, Tencent Cloud Multimedia Lab has previously accumulated less data in this regard.

　

Li Songnan, director of Tencent Cloud Multimedia Lab, told Dongxi Entertainment that this also gave Tencent Cloud Multimedia Lab the opportunity to upgrade the system and algorithm of the repair technology by repairing this concert.

　

Li Songnan said that the goal of this restoration is to ensure that the repair is as old as the old, but also to preserve the artistic effect of the concert. Due to the many scenes of the concert screen and the complicated changes caused by frequent lighting and switching, Tencent Cloud Multimedia Lab has adopted the measure of "algorithmic classification processing for different scenes" to find the optimal solution in the continuous trial and error, and at the same time add manual intervention to optimize the algorithm parameters.

　

The restoration of Leslie Cheung’s concert has further enriched the restoration category of Tencent Cloud Multimedia Lab. In view of the strong demand for the restoration of old films, Tencent Cloud Multimedia Lab plans to further connect its own technology with various demand parties, and use technology to achieve the goal of helping the inheritance of classic culture.

　

First attempt at concert repair

Automated digital repair + manual intervention combined

　

Tencent Cloud Multimedia Lab expert researcher Xia Zhen told Dongxi Entertainment that the cooperation between Tencent Cloud Multimedia Lab and Polygram began in early March.

"At that time, Tencent Music Entertainment Group contacted Polygold Records, and the two sides had their own needs, and the cooperation was reached very quickly."

　

Xia Zhen introduced that since the establishment of Tencent Cloud Multimedia Lab in 2013, one of the main technical directions has been fully automatic digital repair, but previously it was mainly engaged in the repair of movies, TV series, etc., and it was the first time to engage in the repair of concerts.

　

"Concert scenes are not as stable as movies and TV dramas, which are more uniform in terms of color and artistic style, and are more suitable for automatic repair through algorithms. The picture, color and scene of concerts are constantly changing and switching. Previously, the algorithm model of Tencent Cloud Lab did not process much data on concert scenes, so it would be difficult to automate repair based on algorithms," Xia Zhen said frankly.

　

In this regard, Tencent Cloud Multimedia Lab is based on a set of automated processing engines.According to the different scenes of the concert screen, the algorithm classification processing is carried out to find the optimal solution in the continuous trial and error, and the manual intervention optimization is completed.

Xia Zhen gave an example. For example, for a large scene in a concert, Tencent Cloud Multimedia Lab used GAN superfractions to repair it. Before the repair, such large scenes were generally blurred, but after GAN superfractions, the clarity and texture were greatly optimized. For example, if there is a lot of noise in a concert, the technical team will also reduce the noise for this scene.

　

It is understood that pure algorithm repair usually takes only two days, but this time it is a concert, so more time is spent trying different algorithms to ensure that there is a balance between the results of the algorithm and the final effect.

This is also related to the overall goal set by Tencent Cloud Lab for this fix:That is to repair the old as the old, while ensuring the improvement of image quality, it can also retain the original artistic effect of the whole concert.

　

However, Xia Zhen also said that there is no standard in the industry to define the degree of repair, "I can only say that this is the goal or purpose, and then try to adjust the algorithm towards this goal, and then try to judge according to the degree visible to the naked eye."

　

"For example, the resolution of the master tape we got at that time was only 720 × 480, which was not high. But after the repair, the relevant image quality has indeed been greatly improved. We have achieved a resolution close to 4K, and the resolution has been increased by about 6 times. The smoothness of the whole picture has also been improved. We have also made some optimizations in the frame rate, including some details of Leslie Cheung’s face, as well as his eyes, eyes, and the long fake hair he wore throughout the concert. The hair is much better than before." Xia Zhen said.

　

　

Borrow the concert to repair

Realize the dual iteration of the original technical system + algorithm

　

Li Songnan said that by repairing Leslie Cheung’s concert, Tencent Cloud Multimedia Lab has also undergone a relatively large number of technical iterations, mainly including system-level iterations and algorithm-level iterations.

　

The system-level iteration is mainly on the iteration of the analysis module.According to the introduction, in terms of the repair process, it is generally divided into four steps. One is intelligent analysis, which mainly analyzes the quality of the video, including the type and degree of distortion; the other is repair, which is mainly used to repair various distortions in the video, such as the distortion that often occurs in old movies, such as noise, interlaced distortion, etc.; the third is enhancement, the goal is to make the image content look clearer and richer; the fourth is coding, that is, to restore the content of the video at as little bit rate as possible.

　

The iteration of the analysis module is the first step in the repair workflow. Li Songnan said that since the algorithm of Tencent Cloud Lab has basically not touched the type of concerts before, the video content of various industries varies greatly, especially the type and intensity of video distortion in different generations are very different. "This requires the adaptability of the iterative analysis module, so that under the principle of minimizing manual intervention, the analysis module can adaptively handle different types of videos."

　

Algorithmic iteration mainly refers to video super-resolution and face repair.

　

Li Songnan said that the original work of Tencent Cloud Multimedia Lab was mainly video noise reduction and background enhancement, based on traditional algorithms.

　

"After that, many of the models on Tencent Cloud were based on AI and deep learning, but they were originally based on convolutional neural networks, and now they are expanded to use generative networks because generative networks can generate a lot of details," Li Songnan said.

　

Another major iteration is face repair.According to reports, the previous face repair in the industry was mainly based on pictures. The face repair in the video of Tencent Cloud Multimedia Lab was considered an innovative attempt in the industry. At the same time, with this concert starting to be based on video repair, the lab also began to make some artificial algorithm parameter adjustments based on the algorithm.

　

Li Songnan introduced: "During this research and development process, I tried some technical methods, accumulated some good results, and became a good experience in the laboratory. Of course, there is still room for progress in the future."

　

The market demand for old film restoration is increasing day by day

Technology empowered culture

　

According to official information, the Tencent Cloud Multimedia Lab is a technology-oriented laboratory under the Tencent CSIG business group, and its work mainly consists of two parts. First, it will participate in the formulation of international and domestic multimedia-related standards; second, it will provide the daily accumulated technology to Tencent Cloud, Tencent PCG (Tencent Platform and Content Business Group) and WXG (Tencent WeChat Business Group), and Tencent Music Entertainment Group and other departments internally, and export the technology to customers externally.

　

Li Songnan introduced,At present, the common "old film repair" in the market generally has two categories: physical repair and digital repair. Physical repair refers to the repair of film, including removing impurities on it and repairing deformation; while digital repair is divided into automatic repair and manual repair.

　

"At present, Tencent Cloud Multimedia Lab mainly focuses on fully automated repair based on machine learning and deep learning. The goal is to use algorithmic automation to reduce the cost of subsequent manual repair and improve related efficiency through less human intervention," Li Songnan said.

　

This path choice comes from the needs faced by Tencent Cloud Multimedia Lab. "Since the user videos served by Tencent Cloud Multimedia Lab are very large, it is impossible to do fine offline processing for each video, so it is necessary to continuously improve the adaptive ability of the algorithm to achieve the purpose of saving time and increasing efficiency."

　

Li Songnan, for example, for a two-hour concert, traditional manual repair may take one or two months. But from the perspective of algorithm repair, the result can be run in two days.

　

It is understood that the market demand for the restoration of old films is increasing, and many film studios and TV stations have accumulated a large number of old videos. Some of them plan to be re-released or need to be adapted to the current 4K or 8K TV. In addition, some entertainers also need to restore earlier concerts.

　

"The restoration of old films is a relatively long-term thing, and it can be done all the time," Mr. Li said. "The reason is that the quality of the videos that the audience sees now is not bad, and the quality may not be good in three or five years. At the same time, the viewing equipment is also being upgraded. For example, VR monitors and immersive displays will be used more and more in the future, and the update of the display method will accelerate the diversification of the demand for old film restoration technology."

　

Li Songnan stressed that the current video processing is mainly based on 2D video repair and enhancement, and in terms of resolution and frame rate, try to improve the user’s immersion in 2D conditions."But in terms of the future direction, it may also develop in a holographic way.3DAlso, including3Freedom, and finally6Degrees of freedom, the lab has invested in this. "

　

However, Li Songnan also pointed out that some voices have raised the question of whether the current video repair can be used in more social fields, such as optimizing surveillance video and cracking down on crimes. "Although the current technology can achieve face repair and face clearing, it is still difficult to apply in the actual security field. It is impossible to judge whether it is that person from the video. After all, it is generated by an algorithm, and it is not suitable to use it as evidence.

　

Li Songnan said that at present, as a technical department of the laboratory, more business goals are to cooperate with more partners with the ability of the laboratory’s ultra-clear immersion repair engine, while helping the development of the audio & video industry, empowering the classic culture through technology.

　

　

EW-Entertainment

Recent topics · Related content