Making your avatar’s mouth move and respond to how you are talking is a common avatar feature. In Resonite we use a fairly standard list of Visemes loosely based on similar platforms and products in the space, The Visemes we use can be found below:
| Viseme Name | Phonemes | Example Phrases / Words |
|---|---|---|
| AA | A | Car |
| CH | tS. dZ. S | chair, join, she |
| DD | t, d | tip, doll |
| E | e | bed |
| FF | f. v | fat, vat |
| IH | ih | tip |
| KK | k, g | call, gas |
| NN | n, l | lot, not |
| OH | oh, o | toe |
| OU | ou, u | book |
| PP | p, b, m | put, bat, mat |
| RR | r | red |
| SS | s, z | sir, zeal |
| TH | th | think, that |
| Silence | neutral | Silence |
Viseme Allocation
Resonite supports two options when it comes to setting up Visemes for your avatar:
- Automatic - Requires specific blend shape names.
- Manual - Can use any blendshape names
Automatic Allocation
Automatic allocation, will happen when you create the avatar using the Avatar Creator, ensure you check “Setup Face Tracking”.
Then when you create the avatar, Resonite will try to guess the correct blend shape for each Viseme. Our algorithm, aims to find most common naming schemes including from other platforms and we improve it overtime adding new items into the algorithm to aid users.
To help explain how the automatic allocation works, here’s a table that details what we look for:
| Viseme Name | Accepted Names | Example Names we recognize |
|---|---|---|
| AA | Any segment contains aa, a,ah or あ
|
vrc.a, vrc.ah, vrc.a, a, ah, ah, あ
|
| CH | Any segment contains ch or sh
|
vrc.ch, vrc.sh, ch, sh
|
| DD | Any segment contains d or td or dd
|
vrc.d, vrc.td, d, td
|
| E | Any segment contains e, ee, え
|
vrc.e, vrc.ee, e, e, え
|
| FF | Any segment contains f,ff,fv
|
vrc.f, vrc.ff, vrc.fv
|
| IH | Any segment contains ih, い
|
vrc.ih, ih, い
|
| KK | Any segment contains kk,k
|
vrc.kk, kk, vrc.k, k
|
| NN | Any segment contains nn,n,ん
|
vrc.nn, vrc.n, n, ん
|
| OH | Any segment contains o,oh,oo,uh,お
|
vrc.o, vrc.oh, vrc.uh, uh, oo, uh
|
| OU | Any segment contains ou,u,う
|
vrc.ou, vrc.u u, ou
|
| PP | Any segment contains pp,p
|
vrc.pp, vrc.p, p, u
|
| RR | Any segment contains rr,r
|
vrc.rr, vrc.r r
|
| SS | Any segment contains ss,s
|
vrc.ss, vrc.s s ss
|
| TH | Any segment contains th,h
|
vrc.th, vrc.h, h, t
|
| Silence | Any segment contains silence,sil
|
vrc.sil, silence, vrc.silence
|
When reading the table, keep in mind the following:
- Everything in the table below is case insensitive.
- We split all blend shape names by any non-letter character into segments for searching.
- For example “vrc.ah” gets split into two segments “vrc” and “ah” for searching.
- For example “cheese is a good fruit”, would get split into segments that would make that blend shape get allocated to “AA” below.
- For example “blendshape_monkey_a”, would get split into 3 segments with the last one “a”, matching the AA Viseme above.
vrcprefixes are NOT required, they’re just really common. Do read #Allocation Prioritization though.- You do not need to follow any naming, you can always do manual allocation instead.
Allocation Prioritization
Due to the prevalence of the prefix vrc. on a large amount of commonly found avatars, when a blend shape that contains the prefix vrc. is found, it takes preference over all over blend shapes for that one Viseme.
For example, if you have an avatar with a vrc.aa blend shape and an a blend shape, then the vrc.aa one will be allocated as it takes preference.
Re-running the Automatic Allocation
You can re-run the automatic allocation process at any time, by finding an Avatar’s DirectVisemeDriver and pressing the “Auto-assign targets” button.
Manual Allocation
TODO