Facial Rigging With Blendshapes
Neysha Castritius shared what she’s learned about facial rigging with blendshapes at CGMA course Character Facial Rigging for Production tutored by Wade Ryer, and talked about rigging her character Emily rendered in Unreal Engine.
Introduction
My name is Neysha Castritius and I am a 3D artist from Peru, currently living in Germany. I started doing some practices in film production companies learning how to work with After Effects and Photoshop. I also visited a 3D school in Peru to learn the basics of modeling, texturing, rigging, and animation. After that, I studied Game Art at the Cologne Game Lab, where I started using Maya instead of 3ds Max and learned how to work with Unity and the Unreal Engine.
Now I am freelancing as a 3D artist focusing on rigging and animation. I had the pleasure to work on different projects, from mobile games to VR, as well as sidescrollers and 3D open worlds games.
During the different projects, I found my passion for character development and I enjoy every step of it, from the first concepts to the final stage of animation.
Course Goals
Raquel Rossetti, Carmen Schneidereit, and I wanted to create a short film (we later called it “Catch It!”) with the Unreal Engine for our bachelor project at the Cologne Game Lab. One of my tasks was to be in charge of the main character, called Emily. We wanted a slight Pixar feeling for our short film and that is why Emily needed a proper facial rig in order to trustworthy express her emotions. I have approached many attempts to make facial rigs in the past, experimented with joints and blendshapes, but nevertheless, the results were rather mediocre. I knew from experience that I needed help to learn how to create a reliable facial rig for Emily and so I decided to enroll in my first online course ever. I found the CGMA Wade Ryer‘s Character Facial Rigging for Production course, which was exactly what I needed.
Character
When we were brainstorming about Emily, we agreed that she should have African roots, be around 10 years old, and look rather cartoony than realistic. I gathered a couple of pictures for inspiration and did the first sketches trying to define our style.
Once we all were happy with the concept, I proceeded to sculpt her body entirely in ZBrush and modeled her hair in Maya. The character was at this stage when the course started:
Before Rigging
When the course started, the instructor recommended us some lectures, which I instantly ordered. I found The Artist’s Complete Guide to Facial Expression by Gary Faigin very helpful. It gave me a solid insight into expressions and emotions.
Before starting to rig, the character needed to have a neutral and relaxed expression. I had to change my first sculpt and I also got some feedback to improve the design so that it looked more appealing and was easier to rig.
Having the correct topology is crucial and knowing to what kind of shapes the face deforms is very helpful. Retopology was part of the course. Surely, I already had some knowledge about this subject, but the course gave me a deeper insight to understand the connections between the different edge loops and the needed amount of subdivisions.
Rigging With Blendshapes
“Catch It!” was rendered in real-time inside of Unreal on PC. In contrast to projects like VR or mobile games, where you really need to be careful about good performance, we had more freedom to work with more complex characters, bigger textures files, and advanced lighting. In addition, after some research, I found out that many game projects in the industry work with blendshapes for their facial rigs. What I really enjoy about the blendshapes is the artistic side of it, because you are sculpting the expressions that will create the final rig.
However, the rig is still partially joint based. Joints are used for every part of the face that moves along arcs, like the jaw, eyelids, eyes, muzzle nose, and nostrils.
The rig of Emily is not a set of different expressions that would blend from “happy” to “sad”, but rather fragmented in around 50 different sculpted mini-expressions that will later connect to many variations and allow a high range of final expressions.
This is an example of the different blendshapes used to rig the lips:
The face has a set of controllers in which its axis positions are linked to the different blendshape sculpts. This connection was done with the Set Driven Key in Maya. In the example of the lips, dragging up the corner lip controller will blend the sculpt of the ‘lip corner up’, and dragging to the side will display the correspondent sculpt ‘lip corner wide’. When two blendshapes are activated at the same time, for example, by dragging the corner lip controller up and to the side, the two linked sculpts would display in an additive way. To avoid this, we use the plugin extractDeltas to add a corrective blendshape, in this example, the ‘smile’ sculpt, which overrides the ‘lip corner up’ and ‘lip corner wide’.
This way all previously sculpted mini-expressions were linked to the controllers. This process needed to be done for the left and the right.
Rigging Eyelids
Emily’s eyelids were rigged using Marco Giordano’s technique for Cartoon Eyelid.?dnt=1&app_id=122963
This amazing technique allows very flexible animations. It needs some Python scripting and it is joint-based. Basically, the joints were placed on the vertices of the upper lid and lower lid and skinned to the topology.
The joints needed an aim locator setup. The locators were constrained to a high-resolution curve for the upper and lower part of the eyelids. A lower resolution curve with five vertex control points was created on top of the original curve to drive it by using the Wire Tool. Then we needed joints to skin them to the vertex controls of the curve and constrain them by the final controller. By moving the controllers, the curve will deform and so the shape of the eye. Finally, we ended up with a nice setup of four controllers and four secondary controllers to open and close the eye and in addition, to alter its shape.
Hair Movements
Emily’s hair is segmented in many curls and almost every curl is skinned to a joint chain. The orientation of the joints is set in a way that by rotating them in the Z-axis it will curl them outwards or inwards and in the X-axis forward or backward. Each joint has a simple FK controller, which is orient and point-constrained.
Her hair is supposed to behave as it would be not so bendable, but springy. So in order to facilitate the animation, I created an extra controller connecting the X- and Z-rotation axis to a floatable value, as well as the Y-position axis. The connections are not directly on the FK curves but on an extra group so that it is possible to use the main hair controller in combination with the individual FK curves.
A simple solution that made the hair animation in the later process quite fast and easy to do.
This is the final test of the rig:
I think a good facial rig should be able to perform at least the five main human expressions: joy, disgust, anger, surprise, sadness, and fear.
Connecting the Facial Rig to the Body
I started the facial rig in a way that I could connect it easily to the rest of the body. Since Emily is wearing a shirt, the entire blendshape system runs only on her face and neck, which is a separate mesh to the torso and limbs. The only thing I had to keep in mind was to have the head at the right altitude. The neck joint is parented to the chest joint and their controllers as well.
Emily was designed to perform the animations inside of a game engine. For game engines, it is necessary to have all the joints of the rig in a single hierarchy.
For the cloth simulation, I modified the collisions inside of Unreal.
With all this done, I finally had a fully rigged character that could be rendered in real time.
Challenges & Feedback
I would say the most challenging part was to transform the facial rig from the course into a rig compatible with Unreal. I had to simplify a couple of things since Unreal does not read through layers of blendshapes. Also, I had to get rid of the corrective blendshapes system, since Unreal wasn’t able to read them, the solution was to add these corrective blendshapes as attributes in the controllers of the rig. I was very happy when I finally could import Emily without problems and display all of her animations in Unreal.
My experience at CGMA exceeded my expectations. There are techniques and methods shown resulting from years of experience, that I wouldn’t have been able to figure out by myself. The know-how of Wyder is amazing and the classes were easy to follow. I have been recommending this course to many of my friends. You definitely make a big step in rigging by taking this course.
