Advertisement

Measuring 3D facial displacement of increasing smile expressions

  • Z. Fishman
    Correspondence
    Corresponding author at: Cari Whyne S-620, 2075 Bayview Ave, Toronto, ON M4N 3M5, Canada.
    Affiliations
    Orthopaedic Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, ON, Canada

    Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
    Search for articles by this author
  • A. Kiss
    Affiliations
    Evaluative Clinical Sciences, Sunnybrook Research Institute, Toronto, ON, Canada
    Search for articles by this author
  • R.M. Zuker
    Affiliations
    Department of Surgery, University of Toronto, Toronto, ON, Canada

    Division of Plastic and Reconstructive Surgery, Hospital for Sick Children, Toronto, ON, Canada
    Search for articles by this author
  • J.A. Fialkov
    Affiliations
    Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada

    Department of Surgery, University of Toronto, Toronto, ON, Canada

    Division of Plastic Surgery, Sunnybrook Health Sciences Center, Toronto, ON, Canada
    Search for articles by this author
  • C.M. Whyne
    Affiliations
    Orthopaedic Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, ON, Canada

    Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada

    Department of Surgery, University of Toronto, Toronto, ON, Canada
    Search for articles by this author
Published:August 21, 2022DOI:https://doi.org/10.1016/j.bjps.2022.08.024

      Summary

      Background

      Following paralysis, facial reanimation surgery can restore movement by nerve and/or muscle transfer within the face. The subtleties of lip and cheek movements during smiling are important aspects in assessing reanimation. This study quantifies average 3D movement vectors of the face during smiling based on the diverse Binghamton University 3D facial expression database to yield normative measures of lip and cheek movement.

      Methods

      The analysis was conducted on 100 subjects with 3D facial scans in a neutral and 4 increasing smile intensities, as well as associated labeled 3D landmark points. Each subject set of 3D scans was rigidly registered to measure average displacement vectors (distance, azimuth, and elevation) between the neutral and happy expressions.

      Results

      The average lip commissure displacement was found to be 9.2, 11.4, 13.5, and 16.0 mm for increasing smile levels 1–4, respectively. Similarly, the average commissure azimuth angle across all 4 smile levels is ∼44 ± 21 degrees, and the average elevation angle across all 4 smile levels is ∼37 ± 15 degrees. The maximum cheek displacement from the neutral expression was 4.5, 5.7, 6.8, and 7.9 mm for the smile levels 1–4, respectively. The average cheek movement azimuth angle is outward (increasing 1–13 degrees), and the elevation angle is upward (increasing 51–59 degrees) from the face.

      Conclusions

      These data quantifying 3D lip and cheek smile displacements improve the understanding of facial movement and may be applicable to future assessment/planning of facial reanimation surgeries.

      Keywords

      Abbreviations:

      3D (three dimensional), 2D (two dimensional), BU-3DFE database (Binghamton University 3D facial expression), P (philtral), LMUL (left mid-upper lip), LC (left commissure), LMLL (left-mid lower lip), LM (lower lip midpoint), HA (happy expression)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Plastic, Reconstructive & Aesthetic Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Manktelow RT
        • Tomat LR
        • Zuker RM
        • Chang M.
        Smile reconstruction in adults with free muscle transfer innervated by the masseter motor nerve: Effectiveness and cerebral adaptation.
        Plast Reconstr Surg. 2006; 118: 885-899
        • Bhama PK
        • Weinberg JS
        • Lindsay RW
        • Hohman MH
        • Cheney ML
        • Hadlock TA.
        Objective outcomes analysis following microvascular gracilis transfer for facial reanimation: A review of 10 years’ experience.
        JAMA Facial Plast Surg. 2019; 16: 85-92
        • Hadlock TA
        • Malo JS
        • Cheney ML
        • Henstrom DK.
        Free gracilis transfer for smile in children.
        Arch Facial Plast Surg. 2011; 13: 190-194
        • Greene JJ
        • Tavares J
        • Mohan S
        • Jowett N
        • Hadlock T.
        Long-term outcomes of free gracilis muscle transfer for smile reanimation in children.
        J Pediatr. 2018; 202 (e2Available from:): 279-284https://doi.org/10.1016/j.jpeds.2018.06.043
        • Lindsay RW
        • Bhama P
        • Weinberg J
        • Hadlock TA.
        The success of free gracilis muscle transfer to restore smile in patients with nonflaccid facial paralysis.
        Ann Plast Surg. 2014; 73: 177-182
        • Hohman MH
        • Hadlock TA.
        Microneurovascular free gracilis transfer for smile reanimation.
        Oper Tech Otolaryngol - Head Neck Surg. 2012; 23 (Available from:): 262-267https://doi.org/10.1016/j.otot.2012.08.002
        • Manktelow RT
        • Zuker RM
        • Tomat LR.
        Facial paralysis measurement with a handheld ruler.
        Plast Reconstr Surg. 2008; 121: 435-442
        • Tomat LR
        • Manktelow RT.
        Evaluation of a new measurement tool for facial paralysis reconstruction.
        Plast Reconstr Surg. 2005; 115: 696-704
        • Gaudin RA
        • Robinson M
        • Banks CA
        • Baiungo J
        • Jowett N
        • Hadlock TA
        Emerging vs time-tested methods of facial grading among patients with facial paralysis.
        JAMA Facial Plast Surg. 2019; 02114: 251-257
        • Fattah AY
        • Gurusinghe ADR
        • Gavilan J
        • Hadlock TA
        • Marcus JR
        • Marres H
        • et al.
        Facial nerve grading instruments: Systematic review of the literature and suggestion for uniformity.
        Plast Reconstr Surg. 2015; 135: 569-579
        • Banks CA
        • Jowett N
        • Azizzadeh B
        • Beurskens C
        • Bhama P
        • Borschel G
        • et al.
        Worldwide testing of the eFACE facial nerve clinician-graded scale.
        Plast Reconstr Surg. 2017; 139: 491e-498e
        • Guarin D
        • Dusseldorp J
        • Hadlock TA
        • Jowett N
        A Machine learning approach for automated facial measurements in facial palsy.
        JAMA Facial Plast Surg. 2019; 20: 2018-2020
        • Kleiss IJ
        • Hohman MH
        • Susarla SM
        • Marres HAM
        • Hadlock TA.
        Health-related quality of life in 794 patients with a peripheral facial palsy using the FaCE scale: A retrospective cohort study.
        Clin Otolaryngol. 2015; 40: 651-656
        • Lowney CJ
        • Hsung TC
        • Morris DO
        • Khambay BS.
        Quantitative dynamic analysis of the nasolabial complex using 3D motion capture: A normative data set.
        J Plast Reconstr Aesthetic Surg. 2018; 71 (Available from:): 1332-1345https://doi.org/10.1016/j.bjps.2018.05.001
        • Manktelow RT
        • Chaban R
        • Paletz JL
        • Manktelow RT
        • Chaban R.
        The shape of a normal smile: Implications for facial paralysis reconstruction.
        Plast Reconstr Surg. 1994; 93 (discussion 790-1. Available from): 784-789
        • Dusseldorp JR
        • Faraway J
        • Razavi L
        • Hadlock TA
        • Trotman CA.
        Nasolabial fold dynamics: Implications for facial paralysis and facial reanimation surgery.
        Orthod Craniofacial Res. 2021; 24: 62-69
        • Yin L
        • Wei X
        • Sun Y
        • Wang J
        • Rosato MJ.
        A 3D facial expression database for facial behavior research.
        in: 7th Int Conf Autom Face Gesture Recognit. 2006: 211-216 ([Internet]2006Available from:)
        • Srivastava R
        • Roy S.
        Utilizing 3D flow of points for facial expression recognition.
        Multimed Tools Appl. 2014; 71: 1953-1974
        • Sandbach G
        • Zafeiriou S
        • Pantic M
        • Yin L.
        Static and dynamic 3D facial expression recognition: A comprehensive survey.
        Image Vis Comput. 2012; 30 (Available from:): 683-697https://doi.org/10.1016/j.imavis.2012.06.005
        • Agianpuye AS
        • Minoi J-L.
        Synthesizing neutral facial expression on 3D faces using active shape models.
        in: 2014 IEEE Reg 10 Symp. 2014: 600-605 ([Internet]Available from:)
      1. Pan G, Han S, Wu Z, Zhang Y. Removal of 3D facial expressions- A learning-based approach.

        • Gilani SZ
        • Mian A.
        Perceptual differences between men and women: A 3D facial morphometric perspective.
        in: Proc - Int Conf Pattern Recognit. 2014: 2413-2418
        • Lee J
        • Teo I
        • Guindani M
        • Reece GP
        • Markey MK
        • Fingeret MC.
        Associations between psychosocial functioning and smiling intensity in patients with head and neck cancer.
        Psychol Heal Med. 2015; 20 (Available from:): 469-476https://doi.org/10.1080/13548506.2014.951371
      2. Amberg B, Romdhani S, Vetter T, Amberg B, Romdhani S, Vetter T. Optimal step nonrigid Icp algorithms.

        • Banks CA
        • Jowett N
        • Hadlock CR
        • Hadlock TA.
        Weighting of facial grading variables to disfigurement in facial palsy.
        JAMA Facial Plast Surg. 2016; 18: 292-298
        • Trotman CA
        • Faraway J
        • Hadlock T
        • Banks C
        • Jowett N
        • Jung HJ.
        Facial soft-tissue mobility: Baseline dynamics of patients with unilateral facial paralysis.
        Plast Reconstr Surg - Glob Open. 2018; 6: e1955
        • Trotman CA
        • Faraway J
        • Hadlock T
        • Banks C
        • Jowett N
        • Regan D.
        Quantifying soft tissue shape and symmetry.
        Plast Reconstr Surg - Glob Open. 2018; 6: e1715
        • Tanikawa C
        • Takada K
        • Van Aalst J
        • Trotman CA
        • van Aalst J
        • Trotman CA.
        Objective 3D assessment of lip form in patients with repaired cleft lip.
        Cleft Palate Craniofac J. 2010; 47 (Available from:): 611-622
        • Trotman C-A
        • Faraway JJ
        • Losken HW
        • van Aalst JA.
        Functional outcomes of cleft lip surgery. Part II: Quantification of nasolabial movement.
        Cleft Palate-Craniofacial J. Nov 2007; 44 (Available from:): 607-616
        • Trotman C-AE
        • Faraway J
        • Soltmann R
        • Hartman T
        • Van Aalst J.
        Facial soft tissue dynamics before and after primary lip repair.
        Cleft Palate-Craniofacial J. May 2013; 50 (Available from:): 315-322
        • Xue Z
        • Wu L
        • Qiu T
        • Li Z
        • Wang X
        • Liu X
        Three-dimensional dynamic analysis of the facial movement symmetry of skeletal class III patients with facial asymmetry.
        J Oral Maxillofac Surg. 2020; 78 (Available from:): 267-274https://doi.org/10.1016/j.joms.2019.09.007
        • Weeden JC
        • Trotman CA
        • Faraway JJ.
        Three dimensional analysis of facial movement in normal adults: Influence of sex and facial shape.
        Angle Orthod. 2001; 71: 132-140
        • Houstis O
        • Kiliaridis S.
        Gender and age differences in facial expressions.
        Eur J Orthod. 2009; 31: 459-466
        • Grover N
        • Kapoor D
        • Verma S
        • Bharadwaj P.
        Smile analysis in different facial patterns and its correlation with underlying hard tissues.
        Prog Orthod. 2015; 16 (Available from:)https://doi.org/10.1186/s40510-015-0099-4
        • Department of Defense Human Factors Engineering Technical Advisory Group Washington
        Human engineering design data digest. US Army Missile Command Redstone Arsenal.
        AL. 2000; 82
        • Tuxen A
        • Bakke M
        • Pinholt EM.
        Comparative data from young men and women on masseter muscle fibres, function and facial morphology.
        Arch Oral Biol. 1999; 44: 509-518
        • Stern CS
        • Schreiber JE
        • Surek CC
        • Garfein ES
        • Jelks EB
        • Jelks GW
        • et al.
        Three-dimensional topographic surface changes in response to compartmental volumization of the medial cheek: Defining a malar augmentation zone.
        Plast Reconstr Surg. 2016; 137: 1401-1408
        • Trotman CA.
        Faces in 4 dimensions: Why do we care, and why the fourth dimension?.
        Am J Orthod Dentofac Orthop. 2011; 140 (Available from:): 895-899https://doi.org/10.1016/j.ajodo.2011.07.014
        • Yin L
        • Chen X
        • Sun Y
        • Worm T
        • Reale M.
        A high-resolution 3D dynamic facial expression database.
        in: 2008 8th IEEE Int Conf Autom Face Gesture Recognition, FG 2008. 2008 (;(1))
        • Zhang X
        • Yin L
        • Cohn JF
        • Canavan S
        • Reale M
        • Horowitz A
        • et al.
        BP4D-spontaneous: A high-resolution spontaneous 3D dynamic facial expression database.
        Image Vis Comput. 2014; 32: 692-706
        • Van Der Geld P
        • Oosterveld P
        • Bergé SJ
        • Kuijpers-Jagtman AM
        Tooth display and lip position during spontaneous and posed smiling in adults.
        Acta Odontol Scand. 2008; 66: 207-213
        • Cvpr A
        • Id P.
        4DFAB: A large scale 4D facial expression database for biometric applications.
        Cvpr. 2018; (Available from)
        • Awiszus M
        • Grabhof S
        • Kuhnke F
        • Ostermann J.
        Unsupervised features for facial expression intensity estimation over time.
        in: IEEE Comput Soc Conf Comput Vis Pattern Recognit Work. 2018: 1199-1207 (2018-June)
        • Canavan S
        • Zhang X
        • Yin L.
        Fitting and tracking 3D/4D facial data using a temporal deformable shape model.
        in: Proc - IEEE Int Conf Multimed Expo. 2013 (January 2014)
        • Batista JC
        • Bellon ORP
        • Silva L.
        Landmark-free smile intensity estimation.
        in: The 29th Conference on Graphics, Patterns and Images - Workshop on Face Processing Applications: Biometrics and Beyond, Sao Jose do Rio Preto, Brazil2016