mirror of
https://github.com/Project-Babble/ProjectBabble.git
synced 2026-11-20 00:51:16 -08:00
f282176207585ec941f42a5d13ec2455261cc3aa
ProjectBabble
A Mouth tracking system for VR, 100% Opensource!
Currently the architecture is a moddified mobilenetv2 training on blenshapes from ARKit! The model outputs 33 blenshapes of which most cross over with SRanipals blendshapes.
A few things to note
- The current version of the model may not work for your mouth as it has been trained on a not very diverse dataset. Expect this to change in newer versions.
- Not all blendshapes may work as there isnt enough data collected in the dataset.
- No detailed setup guide will be provided until later more stable versions.
- This model is not a 1 to 1 translation to SRanipal blenshapes. While most blendshapes do cross over with SRanipal cheek and tounge movements are limited.
Warranty
Lol
Current setup guide
- Clone the repo
- Install the dependencies in requirements.txt with python 3.8 (or newer).
- run model_loader_no_Crop.py
- run the unity build in the unity build folder
- make a netural face and move the camera around your mouth area until the avatar in the unity app makes a netural face too.
- Enjoy mouth tracking!
Credit
Thanks to dfgHiatus#7426 for proviving the unity scripts in the demo!!
Copyright (c) 2022 Sameer Suri
Languages
Python
99%
Shell
0.6%
Inno Setup
0.3%