The 3.6 billion-dollar "Nezha" Took Five Years to produce. How Were the Difficulties in Animation Creation Solved?

Since its release on July 26, "Nezha: The Devil Child Comes into the World" has received rave reviews and has been soaring at the box office, currently exceeding 3.6 billion. Behind this popular animation is the meticulous polishing of the main creative team over a period of 5 years. During this long production process, artificial intelligence technology has promoted the progress of multiple links, allowing "Nezha" to be presented to us earlier.

Nezha breaks the record again!

Since its release on July 26, the movie "Nezha: The Devil Child Comes into the World" has been breaking the box office record for animated films. So far, the total online box office has exceeded**3.6 billion,More than 1,000 viewers100 million!**It became the first animated film to have over 100 million viewers.

Behind the crazy box office growth, its Douban score also remains at**8.6 points,**Many people began to sigh: A milestone in Chinese animation has arrived.

How could a movie adapted from a mythological story become a dark horse in the Chinese animation industry and achieve such a high box office?

In addition to the subversive character settings and brand new story settings, which bring surprises to the audience,**Exquisite visual effects,**These cool visual effects make the sad and naughty "demonized" Nezha the most "handsome" boy this summer.

The animation production that almost caused Nezha to fail

When the audience is deeply attracted by this most "handsome" boy, they would never have thought that he almost suffered a difficult labor.

Behind every hit movie, there is the careful polishing of the main creative team, and the cumbersome procedures often make the production time very long, and some works may even fail to make it and die halfway.

The production process of Nezha was also not easy, but fortunately, it took a long time.Five yearsAfter a long time, it was finally "fought through".

This 110-minute animated film has a total of66 timesScript modification, yes20Multiple professional outsourcing teams,1600Many animation production staff participated in the production, and the whole film has1400Multiple special effects shots.

It can be seen that an excellent animation is destined to be a huge project. Many practitioners have complained that animation production is a **"Hard work."**The director’s interview also revealed that due to heavy workload, the turnover rate of the outsourced team has increased sharply.

The production of a large number of characters, storyboard animation, and special effects is a time-consuming and labor-intensive process. A short video of just over ten seconds may require several teams to work for several months.

Despite the team's extensive efforts,**But there are still many problems that cannot be solved by just fighting hard.**For example, when the spirit pearl and the magic pill combine in the film, the director wanted to create a visual effect of time going backwards and everything returning to one. However, after several months of attempts, the shot was eventually abandoned because it did not meet expectations.

Production difficulties like this are a major factor limiting the development of animation. If the production process can be more efficient, perhaps more high-quality movies will emerge.Artificial intelligence technology can bring new opportunities to the animation industry.

Liberating the original artist: AI line drawing coloring

In the process of animation production, the basic link is to complete the drawing of the drawings, and coloring these drawings is a huge project.

For a normal 12-frame animation, 25 minutes long, 18,000 images are needed. For a team of 10, about2 monthsTo be completed.

However, AI-based tools can greatly improve efficiency. For example, the most advanced technology available today only requires2.5 hours,The efficiency improvement reached2000 timesAs many as.

Recently, the number of stars on GitHub has exceeded 10,000**Style2paints,**It is such a magical drawing tool. Through it, production staff can quickly complete the complex coloring process.

Style2paints was created by researchers from Soochow University and the Chinese University of Hong Kong, and its latest version has been upgraded to V4. It has also been rated as the best AI tool for line drawing coloring.

Style2paints is based onUnsupervised deep learning,passStyle TransferandGAN Technology, turning the original line drawing into a rich color picture.

The whole process is divided into two stages. The first stage renders the sketch into a rough color drawing. To perfect the picture, the second stage identifies its errors and refines it to obtain the final result.

It is also very simple to use. The artist only needs to complete the line drawing and then click the mouse to get a full color drawing. For painters with high requirements, some fine-tuning may be required.

Liberating Animators: Neural Network Automatically Fills in Frames

In the production of animated films, there are key frames and in-between frames. In-between frames are the pictures strung between two key frames, which serve to connect and make the pictures smooth, but their production is a very time-consuming process in animation.

If we give AI a series of adjacent key frames in a video and ask it to fill in the middle of the picture, the workload will be greatly reduced.

The generative model released by Google AI some time ago follows this idea and solves the problem to a certain extent.

The AI system they released includes a**Fully Convolutional Model****,**It consists of a 2D convolutional image decoder, a 3D convolutional latent representation generator, and a video generator.

By decoding the image, in addition to decoding the input video information, the target video information is also mapped to the latent space. The latent representation generator merges the two types of information, and finally the video generator decodes to obtain the predicted intermediate frame.

The AI-generated video frame sequence is stylistically consistent with the given start and end frames, and looks smooth overall. Surprisingly, this method can achieve video generation over a longer period of time.

In their study, some videos produced promising results, but some complex videos had strange images that still needed to be improved.

Liberating Directors: Text-Generated Animation

Of course, the most surprising feature is that AI can generate animations directly from the script. Those scene effects that cannot be produced may be solved in the hands of AI.

Some time ago, scientists from Disney and Rutgers University published a paper introducing an AI model that generates animated scenes using text descriptions.

In order for AI to achieve the generation of videos from text, it is necessary for it to "understand" the text and then generate the corresponding animation. To do this, they used a neural network with multiple module components.

This model consists of three parts:Script parsing module, automatically parses out the scenes in the script text, and thenNatural Language Processing Module, can extract the main description sentence and refine the action representation, and finally aGenerate Module, converting action instructions into animation sequences.

The researchers collected thousands of freely available movie scripts and selected the996, compiled a scene description corpus.

This corpus consists of525,708The description consists of 1,402,864 sentences, of which 920,817 contain at least one action.

A mapping between description and video is established, so that simple animation clips can be generated by inputting the script.The rationality is 68%.

Although the system still relies on a corpus and cannot completely complete the generation of text to video, it brings new directions in animation production.

How far is AI from subverting animation?

The popularity of "Nezha" has once again made people pay attention to the potential of animated films, and this tradition-breaking film is still on its way to setting new records.

Although there have been some good attempts at using AI technology in animation production, these technologies can only be truly utilized by filmmakers when they become more perfect and mature.

We look forward to the development of technology in the future so that we don’t have to wait for five long years to see more classics like “Nezha”.

-- over--

Click to read the original article