FACE ANIMATION SOFTWARE

Neural networks are used in a wide variety of fields, including the creation of realistic animation of human faces in computer graphics. It is a complex, costly and time-consuming data transformation process that can be greatly simplified through the integration of artificial intelligence (AI).

The technology is gradually trained and reproduces various facial expressions as close as possible to human ones, which makes it one of the most promising in the gaming and entertainment industry.

Neural networks are capable of solving many intellectual tasks: recognizing visual images, detecting objects, postures and movements, as well as creating animation. They are a set of algorithms that work on the model of the human nervous system: they learn from a database, interpret information and make decisions based on experience so that they do not make mistakes in the future. Each element of the network is responsible for recognizing a certain parameter, for example, size, color, volume, shape, etc.

One of the key methods of artificial intelligence is machine learning, which helps the system independently find solutions to various problems. The machines independently perform analytical work and identify patterns using special algorithms and downloaded data.

The creation of facial animation is based on computer vision and multi-view video, which are used by neural networks after appropriate training to generate a three-dimensional model. One of the most important factors in this is the input data on which the artificial intelligence will be trained.

There are six categories for describing a person’s basic emotions based on facial expressions: anger, disgust, fear, happiness, sadness, and surprise. They are mainly associated with negative feelings, but “surprise” can be caused by both positive and negative emotions, therefore it is the most ambiguous for artificial intelligence to reproduce. Machine learning is widely used to correctly recognize emotions.

The process of training a neural network to animate faces consists of four main stages:

• Range of motion
  In order for the system to realistically reproduce facial expressions, the actor who trains the AI ​​performs a series of exercises, for example, opens his mouth wide, moves his jaw, purses his lips, closes his eyes, opens his eyes, etc.

• Emotions and facial expressions
  If at the initial stage the machine studies the exaggerated facial expressions of the actor, then after that it will have to learn and reproduce natural emotions, including joy or disgust, winking, squinting, etc.
• Pangrams (short texts using all or almost all of the letters of the alphabet)
  The actor pronounces pangrams, which contain as many different phonemes as possible in different emotional tones. At this time, the machine reads out a set of possible facial movements during normal speech for a given language.
• Character material
  At the last stage, material is selected, including a preliminary version of the script for the character created using neural networks, taking into account all its features.

As an example, consider the Microsoft project, in the development of which they used a deeply trained neural network (DNN) for an HMM-based lip movement synthesizer. Nvidia also presented a study that described technology learning from 3-5 minute 3D animation data and audio. A similar model (VOCA) was proposed by the Max Planck Institute for Intelligent Systems.

This approach can significantly save resources, for example, in the development of modern video games, where digital twins of actors are used.

The AVADA MEDIA team is engaged in the design of software based on artificial intelligence and neural networks, including innovative technologies for creating realistic animation of human faces.

The potential of AI is very great – it quickly masters new mechanics, learns and creates complex complex animations that look as natural as possible. Thanks to the development of machine vision technology and the integration of neural networks into programs, their functions and capabilities are significantly expanded.

Realistic facial animation is used for many products, such as computer games, animated films, teleconferencing, and so on. Traditional methods of capturing facial expressions have already made huge strides in showing a high level of realism. However, face capture technologies, including sensors or markers, remain expensive and time consuming to use. The alternative are systems using artificial intelligence and machine learning.

Our specialists combine several face animation technologies that reveal emotions and micromimics, and also combine them into one system to create the most vivid anthropomorphic model. For example, first, a head model with a neutral facial expression is designed, after which, for each expression, its own facial mask is created, which has a specific purpose and weight. The method of mixing such masks is used for the final animation of facial expressions.

We offer a complete system combining face, eye, and head motion algorithms, controlled via audio, that works with a wide range of face models. When you input an audio file of recorded or generated speech, the technology automatically outputs a realistic animation of a 3D character. It interacts correctly with a variety of speech sources, languages ​​and 3D face models, and also has additional control options to change emotions, speech style, and head position during animation.