Dynamic Face Models

Shaogang Gong, Yongmin Li, Jeffrey Ng, Lukasz Zalewski, Queen Mary Vision Group

To be able to recognise faces of moving people not only requires the ability to label novel face images with known identities, but also needs detecting and tracking of faces over time. We are interested in the problem of recognising moving faces captured in image sequences. Compared to the more typical scenarios of face recognition in which a single or a few isolated face images of frontal or near-frontal view are the subjects of interest, it is notoriously more difficult to recognise faces of moving people in natural scenes. This requires not only correct recognition of continuously changing face images of the same person, but also consistent detection and tracking of faces in a given dynamic scene therefore enabling any recognition to take place. In such a scenario, face recognition not only needs to cope with changes in face images caused by variations in illumination, scale, translation and rotation in the image-plane, but also associate face images of the same person from significantly different poses caused by head rotations in depth. Most significantly, faces need to be modelled dynamically in a spatio-temporal context.

Related publications:
  1. S. Gong, S. McKenna and A. Psarrou. Dynamic Vision: From Images to Face Recognition. Imperial College Press, World Scientific Publishing, May 2000.
  2. S. Gong, A. Psarrou and S. Romdhani. Corresponding dynamic appearances. Image and Vision Computing, Vol. 20, No. 4, pages 307-318, 2002.
  3. J. Ng and S. Gong. Composite support vector machines for detection of faces across views and pose estimation. Image and Vision Computing, Vol. 20, No. 5-6, pages 359-368, 2002.
  4. Y. Li, S. Gong and H. Liddell. Constructing facial identity surfaces in a nonlinear discriminating space. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA, December, 2001.
  5. Y. Li, S. Gong, H. Liddell. Recognising trajectories of facial identities using Kernel Discriminate Analysis. In Proc. British Machine Vision Conference, pages 613-622, Manchester, UK, 2001. Best Scientific Paper Award (gzipped pdf).
  6. Y. Li, S. Gong and H. Liddell. Modelling faces dynamically across views and over time. In Proc. IEEE International Conference on Computer Vision, Vancouver, Canada, July 2001 (gzipped pdf).
  7. J. Sherrah, S. Gong and E-J. Ong. Face distribution in similarity space under varying head pose. Image and Vision Computing, Vol.19, No.11, 2001 (gzipped pdf).
  8. Y. Li, S. Gong and H. Liddell. Video-based online face recognition using identity surfaces. In Proc. IEEE ICCV Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems, pp. 40-47, Vancouver, Canada, July 2001. Best Paper Prize (gzipped pdf).
  9. J. Sherrah and S. Gong. Fusion of perceptual cues for robust tracking of head pose and position. Pattern Recognition, Vol.34, No.8, 2001 (gzipped pdf).
  10. Y. Li, S. Gong and H. Liddell. Support vector regression and classification based multi-view face detection and recognition. In Proc. IEEE International Conference on Face and Gesture Recognition, Grenoble, France, March 2000.
  11. S. Romdhani, A. Psarrou and S. Gong. On utilising template and feature-based correspondence in multi-view appearance models. In Proc. European Conference on Computer Vision, Vol. 1, pp. 799-813, Dublin, Ireland, 26 June - 1 July 2000.
  12. S. Romdhani, S. Gong and A. Psarrou. Multi-view nonlinear active shape model using kernel PCA. In Proc. British Machine Vision Conference, Nottingham, England, 13-16 September 1999. Best Scientific Paper Award (gzipped pdf).
  13. J. Sherrah and S. Gong. Fusion of 2D face alignment and 3D head pose estimation for robust and real-time performance. In Proc. IEEE International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, Corfu, Greece, 26-27 September 1999.
  14. S. Gong, Eng-Jon Ong and S. McKenna. Learning to associate faces across views in vector space of similarities to prototypes. In Proc. British Machine Vision Conference, Southampton, England, September 1998 (gzipped pdf).
  15. S. McKenna and S. Gong. Real-time face pose estimation. International Journal on Real Time Imaging, Vol. 4, pp. 333-347, 1998 (gzipped pdf).
  16. F. de la Torre, S. Gong and S. McKenna. View alignment with dynamically updated affine tracking. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14-16 April 1998.
  17. Y. Raja, S. McKenna and S. Gong. Tracking and segmenting people in varying lighting conditions using colour. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14-16 April 1998.
  18. S. McKenna and S. Gong. Recognising moving faces. In Wechsler, Philips, Bruce, Fogelman-Soulie, and Huang (Eds.) Face Recognition: From Theory to Applications, NATO ASI Series F, Springer-Verlag, July 1998.
  19. S. McKenna, S. Gong and Y. Raja. Modelling facial colour and identity with Gaussian mixtures. Pattern Recognition. Vol. 31, No. 12, pp. 1883-1892, 1998 (gzipped pdf).
  20. Y. Raja, S. McKenna and S. Gong. Tracking and segmenting people in varying lighting conditions using colour. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14-16 April 1998.
  21. S. McKenna and S. Gong. Non-intrusive person authentication for access control by visual tracking and face recognition. In Proc. IAPR International Conference on Audio-Video Based Biometric Person Authentication, pp. 177-184, Crans-Montana, Switzerland, March 1997 (gzipped pdf).
  22. S. McKenna, S. Gong, R. Würtz, J. Tanner and D. Banin. Tracking facial motion using Gabor wavelets and flexible shape models. In Proc. IAPR International Conference on Audio-Video Based Biometric Person Authentication, pp. 35-43, Crans-Montana, Switzerland, March 1997 (gzipped pdf).
  23. S. Gong, S. McKenna, and J.J. Collins. An investigation into face pose distributions. Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 265-270, Vermont, USA, October 1996 (gzipped pdf).
  24. S. McKenna and S. Gong. Tracking faces. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 271-277, Vermont, USA, 1996.
  25. S. McKenna and S. Gong. Combined motion and model-based face tracking. In Proc. British Machine Vision Conference, pp. 755-765, Edinburgh, Scotland, 1996.
  26. S. Gong, A. Psarrou, I. Katsoulis and P. Palavouzis. Tracking and recognition of face sequences. In Proc. European Workshop on Combined Real and Synthetic Image Processing for Broadcast and Video Production, pp. 97-112, Hamburg, Germany, November 1994.
Some old mpg movies:
  1. Typical example of a moving person in natural scenes  
  2. Consistent detection and tracking of faces in a changing environment
  3. Face recognition needs to cope with changes in face images caused by variations in illumination, scale, translation and rotation in the image-plane
  4. Learning to associate face images of the same person from significantly different poses caused by head rotations in depth