Face and Eye Tracking (Avatar)
More actions
This page is(will be) about how the Face and Eye Tracking systems work inside Resonite and how to set it up to work with Avatars and related systems.
What components(And flux?) are used, how they are set up and function.
Setup
Auto setup from: Avatar Creator#Avatar Creator Panel
Avatars set up via this process have the correct components added to the same slot as the body mesh(?)
Avatar needs to be equipped after these components have been added manually.
Components
Most useful
- Component:EyeManager - For bone based eyes, tracked and simulated.
- Component:EyeLinearDriver - For blendshape/shapekey based eyes.
- Component:AvatarExpressionDriver - Drive mouth/face blendshapes directly using tracking inputs
- Component:AvatarRawMouthData - Show/debug incoming mouth data. (Useful to connect to flux?)
Handy to know about
- Category:Components:Users:Common Avatar System:Face (All related)
- Component:MouthTrackingStreamManager (Is this useful?)
- Component:EyeTrackingStreamManager (And this? What do you do with them?)
Avatar Examples
- Default Avatar (Smiles too much. Has no tongue)
- Face Tracking FaceBot Avatar World (Need to test/explore)
- "Flat Face Tracking Avatar" Gearbell: http://redx.adamski2003.lol/parent-link.brson?ownerId=U-GearBell&id=R-a878a956-3df2-49d7-bcce-c556ea4f3afe&v=2&depth=2
Imported Notes
Stuff here came from elsewhere and will be sorted and moved as needed.
-
Once a tracker is connected to Resonite, avatars that support facial tracking will automatically have facial movements. Which driven by the raw data your facial tracker gives. These facial movements are driven by the Avatar Expression Driver Component. You can also use this component to drive a Float Value Field Component rather than shapekeys on a mesh, and use that as an input to ProtoFlux or other components.