๐ข Comprehensive Project Collection
The approach to building digital humans can be divided into three steps:
- Flesh and Bones โ Create the digital human's appearance.
- Senses Creation โ Convert text to audio; combine audio with appearance to make the digital human "speak."
- Soul Infusion โ Input domain knowledge to facilitate intelligent dialogues.
Hence, we have compiled numerous outstanding projects tailored to different domains. There's definitely one suitable for you!
| Domain | Function | Name | Link | Notes | Updated |
|---|---|---|---|---|---|
| Appearance | Input real photos, generate digital human photos | MINISTER AI | https://mst.xyz/home | Free stable | 23/08/07 |
| Appearance | Generate images from commands | Midjourney | https://discord.com/invite/midjourney | Free trial available | 23/08/07 |
| Audio | Voice cloning, e.g., generate cover songs | so-vits-svc | https://github.com/svc-develop-team/so-vits-svc | Open source | 23/08/07 |
| Audio | Text-to-speech with support for music and simple sound effects | bark | https://huggingface.co/spaces/suno/bark | Open source | 23/08/07 |
| Audio | AI cover song voice transformation | DDSP-SVC | https://github.com/yxlllc/DDSP-SVC | Open source, suitable for low-spec computers | 23/08/07 |
| Video | Input images, text/audio to generate digital human speaking videos | DID | https://bittly.cc/studioDI | Free trial available. More info: https://www.learnprompt.pro/docs/Images/start | 23/08/07 |
| Video | 1. Input photo and text to generate digital human videos; 2. Input real video to produce digital human videos | HeyGen | https://app.heygen.com/ | Subscription required | 23/08/07 |
| Video | Input audio and SDR space video to make the person in the original video speak the target content | Video Retalking | https://github.com/OpenTalker/video-retalking | Open source | 23/08/07 |
| Video | Input audio and SDR space video to make the person in the original video speak the target content | Wav2Lip | https://github.com/Rudrabha/Wav2Lip | Open source | 23/08/07 |
| Video | Input audio and image to generate digital human speaking videos | SadTalker | https://github.com/OpenTalker/SadTalker | Open source, also supports direct Windows app installation: https://www.bilibili.com/video/BV1gW4y1o7FC/ | 23/08/07 |
| Video | Input text, select a character template to generate talking head videos | kreadoai | https://www.kreadoai.com/ | Free. The website also supports AI image segmentation and other functions | 23/08/07 |
| Video | Change a photo to a video with face replacement | Roop | https://github.com/s0md3v/roop | Open source, try it on: https://colab.research.google.com/drive/157RluIDQnvjQy9UBFXL8U5Q-UwgZPqAK | 23/08/07 |
| Digital Human Applications | Flexible combinations for different use cases: virtual anchors, live sales, product guides, voice assistants, remote voice assistants, digital human interactions, digital human interviewers and psychological assessments, Jarvis, Her | Fay | https://github.com/TheRamU/Fay | Open source | 23/08/07 |
Next, we will select some projects to demonstrate how to build your own digital human.