ProjectBabble
A Mouth tracking system for VR, 100% Opensource!
Currently the architecture is a modified efficientnetv2-b0 training on blenshapes from ARKit! The model outputs 45 blenshapes of which most cross over with SRanipals blendshapes.
How to run
Go to the releases tab, download the latest exe. Clone this repo, and then move the calibration file (calib.json) to the same location as the exe. Next get the .onnx model in the repo and move it to the same location as the exe. Run the exe and enjoy!
PLEASE HIT THE TURN OFF CALIBRATION BUTTON UNTILL FURTHER NOTICE
It ruins the model's quality
Warranty
Lol
Credit
Thank you to dfgHiatus#7426 for providing the unity demo as well as the intergrations with other apps!
Thank you to Rames The Generic#3540 who made the dataset, tested and evaluated the performence for the models, and for being awesome!
Copyright (c) 2022 Sameer Suri