when we’re animating a vtuber model we use shapekeys, or blendshapes, or morph targets, the terminology varies between what platform you are working on. im just gonna mostly be refering to them as shapekeys.

A shapekey is essentially an alternate version of a mesh, where you can change vertex positions.

Shapekeys can be binary, only appearing on and off, or they can interpolate.

When working on a VTuber model, we usually use shapekeys for the character expressions instead of using more commonplace bone animation techniques you might see in movies and videogames.

When we work with tracking software, we can detect certain bits of data related to different parts of the face. The data we get depends on our equipment and our software. The default would be using an average desktop webcam that usually can capture basic VRM expressions.

• neutral, the default face
• blink both eyes
• blink left eye and blink right eye individually
• eye look left, right, up, and down
• at least one mouth open expression

depending on your software, and your tracking solution, you can access some additional expressions included in the default VRM standard.

• 4 toggles (joy, sorrow, angry, fun)
  • sometimes detected by cam, usually by keybind toggle, or custom script
• 5 individual mouth shape visemes (A, I, U, E, O)
  • sometimes detected by cam, usually by audio lipsync detection

These tracking values are generally read by a shapekey value, going from 0→100. Its important to note the difference when working in blender versus working in a unity-based program. Blender goes from 0→1.0 while unity-based programs go from 0→100.

In Live2D, we can map multiple poses to a single tracking value. But in 3D, our expression poses are handled with shapekeys, which are singular. Learning how to overcome this limitation is crucial to stop thinking like a novice 3D artist and to instead think like a animator.

Using more than one shape-key per tracking value

The tracker directly modifies a shapekey from 0→100, which might be named something like “jawOpen”. In theory we might think this means have two primary states. The resting state is a closed mouth expression when jawOpen is 0, and then the active state has the mouth open when jawOpen = 100. Theres a few inbetween shapes of course, but we don’t particularly have control over these values. But this is not a good way to work if you’re an animator. You want those in-between poses! You want to be able to use overshoots! Leaving things to the default method of active or inactive, and also limiting yourself to the interpolated inbetweens is not desirable if we want full control of our model.

By default shapekeys are singular. They can store a modified version of the mesh that I’ll refer to as a pose. This pose is then controlled by the shapekey’s value, ranging from 0→100% generally, although it can sometimes go beyond that or under that for negative values, although we wont be discussing these here.