mirror of
https://github.com/Project-Babble/ProjectBabble.git
synced 2026-11-20 00:51:16 -08:00
d94bf39e144e634603bee079203924199162b5ea
ProjectBabble
A Mouth tracking system for VR, 100% Opensource!
Currently the architecture is a moddified mobilenetv2 training on blenshapes from ARKit! The model outputs 33 blenshapes of which most cross over with SRanipals blendshapes.
A few things to note
- The current version of the model may not work for your mouth as it has been trained on a not very diverse dataset. Expect this to change in newer versions.
- Not all blendshapes may work as there isnt enough data collected in the dataset.
- No detailed setup guide will be provided until later more stable versions.
- The (ML) model is currently un optimized so expect large amounts of CPU or GPU usage. This will be worked on in later versions
Current setup guide
- Clone the repo
- Install the dependencies in requirements.txt with python 3.8 (or newer).
- run model_loader_no_crop.py
- run the unity build in the unity build folder
- make a netural face and move the camera around your mouth area until the avatar in the unity app makes a netural face too.
- Enjoy mouth tracking!
Credit
Thanks to dfgHiatus#7426 for proviving the unity scripts in the demo!!
Languages
Python
99%
Shell
0.6%
Inno Setup
0.3%