This AI one click photo talking model is called SadTalker, which was proposed by researchers from Xi'an Jiaotong University. It can make the characters in the photos move with the audio input, and the head movement and facial expression are more realistic. The following figure is the official rendering. What are you waiting for? Have it now!
You can synthesize a video of facial speech based on a picture and an audio. Pictures need real people or close to real people. At present, the project has supported stable diffusion webui, which can be combined with an audio to synthesize facial speech video after SD mapping
Download: https://pan.quark.cn/s/4fc0e7d8e52d
Original address: https://github.com/OpenTalker/SadTalker
1. Start software
Right click the downloaded SadTalker.zip and select "Extract to SadTalker Folder" to extract it
Open the extracted SadTalker folder, find WEBUI run.bat, open it, and a black window will pop up. Wait a moment, and two lines of text will appear in the window door, as shown below
Put the URL in the first line of the black window (that is http://127 …1:7860 This) After copying, you can only select it, press Ctrl+C to copy, right click to display the menu, and then paste Ctrl+v to the browser address bar to open
Google browser or Microsoft Edge browser is preferred
After opening that URL, the content displayed by the browser is as shown in the following figure
Note: Do not leave the ladder or agent open. The network must be connected directly
2. Use of software
First, upload photos and audio. As shown in the figure below, audio can be produced with text to speech tools or recorded by yourself.
Next, set the options on the right. There are two suggestions
-The first setting scheme-
The digital person generated by this scheme is as large as the original picture, and although the head movement range is small, there will be no separation of the head and body, and the body will not move.
Please set according to the following figure
-The second setting scheme-
This scheme will cut the original image, and only generate the head video without the body. The generated digital human expressions and actions are richer and more vivid than the first scheme.
Please set according to the following figure.
3. Download generated video
After setting, click Generate. After a while, a video will be generated at the bottom left. The progress will be seen in the original black window.
After the video is generated, right-click the generated video to see the download options.
If you can't find the download option, you can directly open the "results" folder in the software directory and then open the folder inside. You will see several mp4 files, some of which are just headers, some of which are ordinary image quality, and some of which are ultra clear image quality. Open them to find a good one.
4. Handling instructions
Description of processing method.
Crop: cut the original image to generate only head video without body. The generated digital human has rich expressions and more vivid actions
Resize: The generated digital human body also moves slightly, but the effect is poor and too fake. It is not recommended to use.
Full: The generated digital person is the same size as the original picture, and the body is still. If "Static Mode" is checked, the head movement range is small, but the head and body will not be separated; If "static mode" is not checked, the head will move more, but sometimes the head and body will separate.
Extcrop, exfull: The functions of extcrop and exfull are the same as those of crop and full, but the cutting range of the head is larger.
Other instructions
Pose style: head shaking amplitude. The higher the value, the more severe the expression shaking.
Expression scale: the range of facial expressions, especially lips, the greater the value, the greater the action.
Batch size in generation: The higher the value, the faster the processing, but the higher the requirements for the graphics card. It is not easy to select 1 for the video card.
Enable the reference video Use Reference Video.
Options below
Pose: The facial pose of the generated digital person is consistent with the reference video.
Blink: The generated blink of the digital person is consistent with the reference video.
Pose+blink: the generated digital person's face and blink are consistent with the reference video all: the generated digital person and the reference video are all consistent, including the sound.
