Template-Model Based Modeling and Animation of Human Bodies with Anatomical Structure Xiaomao Wu† Lizhuang Ma† Ke-Sen Huang∗ Yan Gao† Zhihua Chen† †

Shanghai Jiao Tong University {xmwu,ma-lz,gaoyan,zhchen}@cs.sjtu.edu.cn ∗

National Tsing Hua University [email protected]

Abstract Anatomically-based methods can be used for mimic muscle bulges which is very important for creating realistic skin deformations. However, these methods are relatively complex and difficult to use. In this paper, we present a template-model based method, which can simplify and accelerate the process of anatomically-based modeling. The template model consists of individual bones and muscles, which can be modeled by skilled modelers and then stored in our system as a template. Given a new skin mesh, the template model can be semi-automatically reshaped according to the shape of the mesh. And then the skin mesh can be animated by using keyframing or motion capture techniques in an anatomically appropriate way. Experimental results demonstrate the validity and efficiency of our method. Keywords: human body modeling, anatomically based modeling, template model

1 Introduction Creating realistic animation of human bodies remains one of the main difficulties in the research field of computer animation. Anatomicallybased methods provide a good way for producing vivid deformation effects, because it is capable of simulating muscle bulge. However, this method requires users to design the bones, mus-

cles, and sometimes fat tissue, individually. After that, users should also fine-tuning the muscle shapes carefully, in order to fit those underlying structures to the skin mesh. Although some useful tools can help simplifying the process, anatomically based methods remain difficult-touse. In order to take the advantages of anatomically-based methods, and meanwhile, simplify the tedious, difficult modeling and muscle-shape adjusting process, we proposed a template-model based modeling and animation approach. The main idea behind our approach is to construct a template model which contains the basic bone and muscle structures of the human body. This template model need to be constructed only once. After that, the template model can be semi-automatically reshaped according to different new skin shapes. The overview of our approach is illustrated in Figure 1. It consists of four stages. In stage 1, the template model is modeled and several associated parameters are designed. After that, each muscle is resampled to a new representation that is more suitable for animation. The template model can be stored as a standard, morphable model in our system. In the next stage, we obtain the skin mesh needed to be animated, and design a skeleton for it. Then the template model is reshaped according to the skin mesh in stage 3. Finally, we can animate the skin mesh in an anatomical way by using keyframing or mo-

tion capture techniques in stage 4.

mann and Shen [8] formulated skin as cylindrical contours. Schneider and Wilhelms [9] specified underlying structure below an existing skin and used a physically-based method to deform the skin mesh. Simmons et al. [10] proposed a template-model based method to deform a preexisting horse skin mesh. Meanwhile, skinning [11] is a widely used method to deform an existing skin mesh. The development of range scanning technologies has make it possible to scan a whole body in several minutes with satisfactory accuracy. Given a variety of scanned poses of a human body, Allen [12] successfully blended between those poses , which produced smooth and highly realistic skin deformations. Our method falls into the second class. We make our efforts on how to semi-automatically embed anatomical structures into a hand-crafted or range-scanned skin mesh, and how to deform the anatomical structures and the skin.

2 Related work

3 Design of template model

Research work on modeling and animating of skin can be classified into three classes: generating the skin upon underlying components; deforming a pre-existing skin; and deformations by examples. The first class which attempts to extract the skin from underlying components such as bones and muscles, generally use implicit surface to represent the skin. Blinn [1] used implicit surface generated by point skeletons with an exponentially decreasing field function to model his “blobby man”. Bloomenthal [2] modeled a hand with veins with convolution surface. Yoshomito [3] used ellipsoidal metaballs to model the skin of a ballerina. More recently, Scheepers et al. [4] and Wilhelms et al. [5] also used implicit functions to extract the skin from underlying structures. Deforming a pre-existing skin is the second class of human body modeling. Komatsu [6] applied a continuous deformation function to deform skin meshs represented by Bezier patches and Gregory patches. MagnenatThalmann et al. [7] introduced the concept of joint-dependent local deformation or JLD, to deform an existing skin algorithmically. Thal-

The template model is a deformable model which consists of bone and muscle structures of the human body. The model is set to be standing at a muscle-relaxation pose. In such a pose, most muscles are staying in a state of relaxation (Figure 2), which is helpful for simplifying the muscle animation process. We have referred to anatomy books [13, 14] when creating such a template model. Bones and muscles of the template model are modeled individually in 3ds Max 5 as triangular meshes. The final template model consists of 176 bones and 186 skeleton muscles. In order to animate the template model, we should attach a skeleton to it first. A skeleton can be represented by S = {pi , qi }, i = 0, 1, ..., n − 1, where pi ∈ R3 is the offset of the ith joint, qi ∈ S 3 is the orientation of the ith joint, n is the joint number. We have developed a interactive tool for designing a skeleton for the template model. And then, we resample the muscle meshes represented by irregular triangular meshes into another regular shape that is more convenient for animation. We first interactively design parameters Mi = {O, I, Qj , Bo , Bi , Po , Pi } for each

1. Design of template model

3. Morphing of template model

Modeling of bones and muscles

Tranformation of bones

Skeleton design

Tranformation of muscles

Muscle parameter design

Muscle dialation

Resample muscles

Human body with anatomical structure

2. Preparation of skin mesh 4. Animation Obtain skin mesh

Keyframing or Motion capture data

Skin skeleton design

Output animation sequence

Figure 1: Overview of our approach

(a)

(a)

(b)

Figure 2: The template model consisting of bones and skeleton muscles of a human body. (a) Anterior superficial view. (b) Posterior superficial view. muscle, where O is the origin, I is the insertion, Qj s (j ∈ [1, n]) are the n points we use to define the action line. We choose n = 5 in our implementation. However, it can be adjusted by the user. More control points can produces more accurate shape, but will also reduce the performance. Bo is the origin bone which is the bone that the origin attached to. Bi is the insertion bone which is the bone that the insertion attached to. Po and Pi are the origin and insertion of the muscle respectively. When the parameters of each muscle mesh haven been designed, we resample each muscle mesh with the following five steps. 1. Fit Bezier curve to the points on the action line. 2. Shoot rays from the points on the the action line. 3. Calculate intersections between rays and the muscle mesh. 4. Fit ellipse to the intersections. 5. Reconstruct and render the muscle. The muscle resampling process is illustrated in Figure 3.

4 Preparation of skin meshes The skin meshes we want to animate can be obtained by scanning real human bodies via 3D body scanners such as CyberWare WB4 [15], or be crafted by hand in current shape-modeling systems. After the skin mesh has been created, we also need to specify a skeleton for the

(b)

(c)

Figure 3: The muscle resampling process. (a) The original shape of bicep muscle. the origin and insertion are represented by blue and green spheres individually. The middle five points along with the origin and insertion define the action line. (b) Intersections between rays emitted from the middle five point on the action line and the original muscle mesh. (c) The reconstructed muscle shape with Phong Shading. skin mesh, which is the same as the skeletondesigning process stated in Section 3.

5 Morphing of template model In this section, we introduce the algorithm we propose for deforming the template model so that it will be satisfactorily adapted to the shape of the skin mesh. We first transform the bones of the template model so that the they will be transformed and reshaped under the skin mesh. And then, we transform the muscles accordingly, followed by a muscle dilation process to reshape the muscles according to the skin shape. Figure 4 and Figure 5 illustrate this process.

5.1 Transformation of bones The bone scaling process involves the following four steps: 1. Group bones. The bones that lie between two adjacent joints are assigned to the parent joint. 2. Transform bones to their associated local joint coordinate system. 3. Scale bones of each segment individually. Each bone is scaled along its length direction according to its lengths in the skeleton of the template model and in the skeleton of the skin. The scale factor in the width direction can be freely controlled by the user.

4. Change the skeleton of the template model, so that it will have the same pose as the skin.

5.2 Transformation of muscles All muscles should be transformed after the transformation process of bones. The according transformation matrix can be represented by: Mi = Mr · Mt · M0−1 · Ms · M0 · v where: Mi is the according transform matrix of the ith muscle. Mo is the transform matrix that transforms the origin of the muscle to the global origin ,and the origin − insertion vector to the Z axis of the global coordinate system. Ms is the scale matrix, and the scale factor of Z axis is Sz =Ls /L0 , with Sx = Sy = α ∗ Sz . Here, Ls is the length between the associated origin bone point and insertion bone point of the ith muscle in the scaled template model, L0 is the length between the associated origin bone point and insertion bone point of the ith muscle in the original template model. α is the scale factor, which can be adjusted interactively by the user. In our implementation, we choose α = 0.4. M0−1 is the inverse matrix of M0 , which translates the muscle back to its original position from the global origin. Mt is a transform matrix that translates the muscle from its unscaled origin bone point to the scale origin bone point. Mr is the transform matrix that rotates the muscle to their final positions.

5.3 Muscle Dilation The muscles of the template model have been shrunk to be inside the skin mesh. We need to dilate them into proper shapes according to the shape of the skin mesh. This can be implemented by shooting n rays from the centers of each slice of each muscle. The number of rays of each slice is controlled by the user. However, we find that 20 rays is enough to produce satisfactory results. For each ray, we calculate the intersections between the ray and the skin mesh. Because muscles have been transformed to be

inside the skin so that all the muscles are enclosed in the skin mesh, each ray of the slice will have at lease one intersection. If we represent the kth nearest intersection on the jth slice k , the associated point of the ith muscle as Sij k , and the center of the jth on the slice as Eij slice of the ith muscle as Oij , then the scale factor of the ith muscle can be formulated as k − δ|/|O E k |). Where j and fi = min(|Oij Sij ij ij k are integers, j ∈ [1, 5] and k ∈ [0, n − 1], n is the number of rays.

6 Skin animation When the template model has been morphed according to the skin mesh in the previous steps, we can animate the skin by using keyframing or motion capture techniques after an anchoring process. In the stage of anchoring, each vertex in the skin mesh is associated with the closest underlying component (bone or muscle) of the morphed template model. We adopt the method proposed by Wilhems and Gelder [5] for anchoring and animating the final anatomical model.

7 Results The graphics interface was developed in Visual C++ 6.0. Our program ran on a Pentium 4 1.8G computer with 512Mb physical memory and GeForce FX5200 graphics card. Process Stage design of the template model skeleton design for the skin transformation of bones transformation of muscles muscle dilation anchoring animate skin (per frame)

Time Elapsed 3 days 2.1 minutes 1.2 seconds 1.3 seconds 1.2 minutes 28.6 minutes 3 seconds

Table 1: The running time of the main steps of our approach The elapsed time of each step is shown in Table. 1. The most time-consuming step is designing the template model. We spent 3 days for designing the template model. After that, we

save the template model along with its skeleton and muscle parameters to the disk as data files. Those files will be loaded directly into our system as standard template data. In Figure 4, we demonstrate the process of transforming the template model in order to reshape it according to the skin shape of John’s. Figure 5 shows another example, changing the skin mesh to Eric’s. Figure 6 demonstrates the animation sequence of a long-jumping motion after the template model has been reshaped according to John’s skin. Figure 7 demonstrates a walking animation of John’s skin.

Acknowledgements We would like to thank Cyberware for generously providing their 3D whole body samples on their web. We would also like to thank Tomas M¨oller for his ray/triangle intersection code, thank Andrew Fitzgibbon for his ellipse fitting Matlab code. Thanks Brett Allen for his helpful suggestion and advice. We are grateful to the anonymous reviews for their helpful comments. This project was s upported by National Natural Science Fundation of China (grant No. 60373070), 863 Program of China (grant No. 2003AA411310), and Microsoft Research Asia (Project-2004-Image-01).

References [1] J. Blinn. A generalization of algebraic surface drawing. ACM Transactions on Graphics, 1(3):235–256, July 1982. [2] J. Bloomenthal. Hand crafted. Siggraph Course Notes 25, 1993. [3] S. Yoshimito. Ballerinas generated by a personal computer. The Journal of Visualization and Computer Animation, 3:85–90, 1992. [4] F. Scheeppers, R. E. Parent, W. E. Carlson, and S. F. May. Anatomy-based modeling of the human musculature. Computer Graphics (Pro. of SIGGRAPH’97), 31:163 – 172, August 1997. [5] J. Wilhelms and A. Van Gelder. Anatomically based modeling. Computer Graphics

(Proc. of SIGGRAPH’97), pages 173–180, 1997. [6] K. Komatsu. Human skin model capable of natural shape variation. The Visual Computer, 3(5):265–271, March 1988. [7] N. Magnenat-Thalmann, R. Laperriere, and D. Thalmann. Joint-dependent local deformations for hand animation and object grasping. In Proc. of Graphics Interface’88, pages 26–33, 1988. [8] D. Thalmann, J. Shen, and E. Chauvineau. Fast realistic human body deformations for animation and vr applications. In Proc. of Computer Graphics International’96, pages 166–174, June 1996. [9] P. J. Schneider and J. Wilhelms. Hybrid anatomically based modeling of animals. In Proc. of Computer Animation’98, pages 161–169, June 1998. [10] M. Simmons, J. Wilhelms, and A. Van Gelder. Model-based reconstruction for creature animation. In Pro. of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer Animation, pages 139– 146, July 2002. [11] J. Lander. Skin them bones: game programming for the web generation. Game Developer Magazine, pages 11–16, May 1998. [12] B. Allen, B. Curles, and Z. Popvi´c. Articulated body deformation from range can data. ACM Transactions on Graphics, 21(3):612–619, July 2002. [13] E. Goldfinger. Human Anatomy for Artists: The elements of Form. Oxford University Press, 1991. [14] Phillip E. Pack. CliffsQuickReview Anatomy & Physiology. New York Cliffs Notes, 2001. [15] Cyberware whole body color 3d scanner, http://www.cyberware.com/products /wbinfo.html.

(a)

(b)

(c)

(d)

(e)

Figure 4: Given a skin mesh, the template model can be semi-automatically transformed according to the shape of the skin. (a) The skin model of John. (b) The skin model with the template model before transformation. (c) After transformation of bones of the template model. (d) After muscle shrinking process. (e) The final template model after muscle dilation.

(a)

(b)

(c)

(d)

(e)

Figure 5: The same process as in Figure 4, but with difference skin mesh of Eric’s.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6: After the template model has been transformed according to John’s skin, it can be animated by motion capture data. Here shows a long-jumping motion. Notice that muscle contraction and bulging are apparent.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 7: The animation of John’s skin. Here we show a walking motion. The template model underlying the skin mesh is not displayed.