Expression (including face, gesture)
Return to Major concepts. See visualization, speech, emotion.
Expression in a major facet of animation. Here an image from Piclab.
1. Expression in general
2. Expressions of the work itself
3. Facial and gesture expressions.
3.1. Their generation
3.2. Their capture and perception
1. Expression in general
The term may mean different things, and apply to generation as well as perception.
- a form, e.g. a mathematical expression such as 2x3 ; that may take to abstract considerations about algorithms, see for instance [Cox] and more generally algorithms.
- an output signal corresponding to some state of a system, generally aimed to another system; with the paired issues of generation and recognition, then the more or less formalized language.
The "systems" can be
- animal and human, signaling states of mind and emotions through body and mostly face expressions; this applies to robots as far as they are human-like;
- any machine, as far as it has different states that need to be expressed to other systems and notably to human beings (e.g. spectators).
From a programming standpoint (see form), it is the generation of a raster from a code (for example, drawing a smiling face to translate "happy").
We can also say that a work of art is the expression of an artists impressions, emotions and desire of creation. The work takes the central place into the global posture and discourse of the artist.
2. Expressions of the work itself
Art works are by themselves the expression of artists. But, when they become interactive and reach some level of behavior, they have to show this behavior to spectators.
For [Aziosmanoff], the "expression motor" is a major component of "behavioral art" works. He takes expression in the specific meaning of "the formal mechanism by which the works discourse constitutes itself in reality, in order to be presented to the audience's knowledge". This theoretical formulation refers to (shortly said) the traditional category of art to which the work belongs: series of photos, musical assembly, narration. Expression is complete with the distribution (be it in a museum or to mobile phones).
Expression can be as simple as the switching on of a LED (see actuator) or as complex as the movements and facial expressions of a humanoid robot.
The Artifacial project, by Arthur Elsenaar.
Expressive processing is a formula proposed by Noah Wardrip-Fruin in his book Expressive processing. Digital fictions, computer games and software studies ( MIT Presss, 2009). He gives two meanings to his title:
. authors expressing themselves through processes (for example, the rules which define the behavior of a character in a role game),
- processes expressing something,
One of the problems that Wardrip-Frouin wants to help solving, is the gap between the reality of processes (generated by software) and what actually see the spectators, players and critics through the expression of these processes.
Spectators are taken to build their idea of the real processes, their rules, their limits and the way to control them or at least to dialog with them in compelling way. This idea is frequently deceptive (as notes also [Millington]):
- sometimes they lend to the machine an intelligence and a complexity much higher than the realit (Weizenbaum's Elisa, Pacman),
- sometimes the sophistication of the program remains unperceived by the spectator or player.
3. Face and gesture expression
3.1. Generation of expression
Graphical methods or emotion representation have existed since at least the 19th century [Thiébault]. A recent and practical text, aimed to comic designers [McCloud].
- Motion design, or motion graphic design, according to Wikipedia, is the art of moving graphics and adding typography, videos 3D and sound. See a full file in Créanum, Feb.-March 2011.
As for robots, important-works have been led at MIT by Cynthia Breazeal ( Breazeal].
Mocap, a document from Chrisevans3d
Some other references:
- The website pf Wonsook Lee.
- Artifacial . An artistic project by Arthur Elsenaar.
- Ask Google about Expressive speech animation and facial communication.
< Science Gesture analysis reveals what Obama and Romney care about. Paper by Liat Clark, (Wired, 11/6/2012).
< Dynamic faces. Insights from experiments and computation. Cristobal Curio, Heinrich H. Bülthoff et Martin Giese (eds.). MIT Press 2010.
. Agents conversationnels animés. Greta, une plate-forme d'agent conversationnel expressif. , de J.-C. Martin Hermès/Lavoisier 2010.
. Several chapters in [Magnenat, 2004], Handbook of virtual humans.
. Contrôler par l'expressivité du geste l'environnement sonore et visuel dans les systèmes interactifs multimodaux (an article by par Antonin Camurri, Barbara Mazzarino and Gualtiero Volpe in Interagir avec les technologies numériques (Bruxelles, Contredanse 2004)
. Facial textures [Ebert 2003] p. 395-409.
3.2. Perception and recognition of expression
See perception.
As for generation, an interactive work of art can perceive the attitudes of its audience by very simple means, would it be only a switch or a low cost sensor detecting a presence around. New sensors like cameras and combined sensors like Kinect open new ways. The Laval Virtual presents each year several projects of this type.
Animation film creation makes a wide use of mocap (Motion capture) in order to control the movements (face and body) of characters.
- Markerless capture: more and more used, due to cameras and vision progress.
< Facial motion capture. Some pages in [Magnenat].
- Example of a specialized firm, Mocaplab. The site gives details on different types of mocap.
. Generation and perception of emotions by robots, see Philippe Gaussier.
Some references:
< 3D Face Modeling, Analysis and Recogntion. by M. Daoudi. Wiley 2013.
- Vision-Based Hand-Gesture Applications. A ten pages article in the Communications of the ACM. February 2011. This paper shows that beyond technical issues stand ergonomic problems and the need to define a gesture language.
< Detection of relationship between music and gesture. An article in Physorg 9/11/2010
<
Body gesture recognition and action response, a chapter signed Luc Emering and Bruno Herbelin, in [Magnenat 2004].
< Reading the finger moves on a virtual keyboard SH No 14 (concepts).
< Translate the sign language AH No 134 (concepts).
< Gesture and sign languages in human-computer interaction, I. Wachsmuth and T. Sowa eds. (A conference held in London, 2001)Springer 2002.
< Les nouveaux gestes de la musique. Hughes Genevois and R. Vivo eds. Editions Parenthèses, 1999.
- A text by Pierre Jodlowski about his oeuvre.
DICCAN'S PARTNERS:
Paris ACM Siggraph, the French chapter of ACM Siggraph, worldwide non-profit organization of computer graphics.
