Skip to main content

🟡 SadTalker: Making Photos Speak

Project URL (including detailed documentation): https://github.com/OpenTalker/SadTalker

First, the effect: Input a photo and an audio clip to make the person in the photo speak.

Installation Method One: Direct Windows Installation​

Zero, Preliminary Work​

Requires git, VPN for internet access, and python3.8+ environment. If you are unfamiliar with these, this method may not be suitable for you.

One, Download ffmpeg​

1. Visit https://ffmpeg.org/download.html, click on Windows, select gyan.dev

step1.png

2. Download the full version of FFmpeg for Windows

step2.png

3. Unzip and rename the extracted folder to FFmpeg

P.S. Unzipping requires installing 7zip, a handy lightweight compression tool.

step3.png

4. Configure PATH environment variable, remember to confirm after adding

step4.png

step5.png

5. Check if the installation was successful

Open 【Command Prompt】, enter ffmpeg, as shown in the image

step6.png

Two, Download SadTalker Code​

git clone https://github.com/Winfredy/SadTalker.git

step7.png

Three, Download SadTalker Model Files​

Refer to the 《Official Documentation》 for three methods, among which Baidu Cloud is accessible without a VPN. The model files involve the following two directories:

Place the downloaded model files under checkpoints and weights directories.

step8.png

step9.png

Four, Launch webUI​

The first run requires downloading dependencies, which can be slow.

step10.png

Five, Run demo in Browser​

Default address is 127.0.0.1:7860, drag in an image and voice, click Generate to synthesize (Note: There are samples in SadTalker/example to try)

step11.png

Common Issues​

  1. Click Generate and the top-right corner shows 【Expecting value: line 1 column 1 (char 0) 】

Close the proxy server, please refer to [Bug]: Something went wrong Expecting value: line 1 column 1 (char 0) #9174

  1. Script fails to pull model files

Solution: Manually download the model files, place them under weights directory. For model file downloads, refer to here。

step12.png

Installation Method Two: Colab​

  1. Visit SadTalker's Colab, copy to your own cloud drive.

  2. Click on different cells, run sequentially

  • Select an image, run, and complete the synthesis

  • Run

  • Preview the results

Common Issues​

  1. pip install prompts python3.8: command not found

Solution: Manually install python3.8

!wget -O mini.sh https://repo.anaconda.com/miniconda/Miniconda3-py38_4.8.2-Linux-x86_64.sh
!chmod +x mini.sh
!bash ./mini.sh -b -f -p /usr/local