Video coding technology pays tribute to the unique charm of Film Grain

2022-06-13 0 By

“The graininess of film imaging is a unique charm that digital images cannot replicate.”Christopher Nolan said at the release of Creed.A fan of celluloid films, Nolan has shot every film on celluloid since his debut, including “Interstellar” and “Inception” and other popular works around the world.The “Grain of Film imaging”, as director Nolan calls it, is the Film Grain to be discussed in this paper.Why, as a kind of noise, can Film Grain be loved by a large number of directors and audiences?In fact, in some specific nonlinear systems, the presence of noise can enhance the detection ability of weak signals, which is called stochastic resonance.In the image system, the image visual quality can achieve better results under certain noise intensity (such as Film Grain).Therefore, as part of the creative intention of Film and television, it is very important to retain Film Grain effectively when coding the video.However, Film Grain, as an image noise, has irregular distribution and size and is random.The randomness of Film Grain makes it difficult for traditional coding techniques to compress effectively, and also makes prediction very difficult, and the accuracy of motion estimation will be reduced.Film Grain encoding technology based on the Aurora AV1 microframe technology has been successfully applied to a well-known PGC video platform and has been well received by customers and platform users.Film Grain can be retained and synthesized more efficiently with AV1.Since Film Grain is not included in the traditional video standard when it is formulated, it is easy to cause problems such as uneven Grain distribution and incomplete reservation by using the traditional encoding method to encode video.In AV1, the Film Grain processing tool has been added to the standard, which has higher support for Film Grain encoding and decoding.Aurora AV1 has the advantage of faster and more efficient encoding than Libaom, and supports 10bit Film Grain denoising.Also, keeping Grain on the open source Libaom encoder is prone to obvious banding, and aura-AV1’s adaptive quantization mode can better avoid this situation.For different types of grain, The Aura-AV1 video encoder was adopted by Microframe Technology in three directions:Denoising +Synthesis (Grain Synthesis), Direct Preservation (Grain Preservation), Preservation+Synthesis (Preservation+Synthesis).Grain Synthesis is denoising to remove the Grain from the video and estimate the Grain parameters, which are sent to the decoder along with the compressed video stream. After decoding, the Grain is synthesized and added to the reconstructed video frame.This method helps preserve the grain look of the encoded video and maintains a lower bit rate than encoding grain directly.The downside of Grain Synthesis is that the first step in the Synthesis process requires a 64×64 template to be generated, followed by a random 32×32 block from the template.The x, Y, and x coordinates need to be less than 32. As a result, adjacent 32×32 blocks will overlap and face the same direction. Flat areas in a grain-strong sequence are prone to pattern and obvious artifacts.The solution to this problem is to select the seed of Grain and compose only the seeds with less obvious pattern.Left: source video;If the Film Grain in the source video is uncompressed, you can use the AV1 Film Grain tool to reconstruct it.However, it is difficult to reconstruct the video that has been transcoded and compressed.After the compression of grain, low frequency components will appear and certain patterns will be generated, which is difficult to be fully represented by AR model.Therefore, the grain pattern with obvious low-frequency components needs to be reserved by direct coding.Additionally, for sequences where grain is thin, reserving grain directly allows for more even retention of grain, as well as better clarity, resulting in better grain videos without a Grain artifact.It is worth noting that the Aurora AV1 microframe is used to encode grain directly, which can suppress grain inconsistency and inconsistent resolution between different frames, which is currently not possible with ordinary AV1 encoders.During the process of coding, we will also encounter the situation that the grain distribution in the same frame is very uneven, such as the upper part of the grain and the lower part of the grain. In this case, we cannot rely on the method of grain Synthesis.Because the standard full drawing is the same grain model.Grain reservation may be incomplete and inconsistent in direct coding.For such complex grain, the method of microframing is as follows: The grain is decomposed, and for the grain that is difficult to generate with the Grain Synthesis standard, the encoder is directly used to encode the grain.If the Grain Synthesis standard is relatively easy to generate, use the standard to generate Grain. For example, high frequency Gaussian noise can be synthesized using the Grain Synthesis standard.The method of Preservation+Synthesis can solve the limitation of Grain Synthesis and reproduce Grain with complex pattern.At the same time, the added grain composition method can save more grain than the direct coding method.Left: Keep grain directly;Right: retention + composition