Florent Aziosmanoff: emotional exchange with an Aibo Dog (Picture from the Cube).
Emotion, emotional
This topic was the theme of a Paris ACM Siggraph association, on Dec. 9 2014. See the program
Last revised 21/11/2018. Return to Major
concepts. See psychology, behavior,
robots.
Emotional code in Roxame.
1. From the artists to their public
2. Representation of emotions
3. Modeling of emotion and emotion of machines
Some references
A fascinated young public for the painting software Aaron of Harold Cohen (1982c.)
1. From the artists to their public
1.1 To the spectator, from the artist, through the work
No art without emotion, even if this word takes different meanings along the centuries. The illusion effect for painting in Antiquity. The sublime in the 19th century. The de-statbilisation for contemporary art.
Each technical progress affords artists new ways to stir emotion. The computer and Internet come after the pigment chemistry, the electrical lights and sounds and electronics. In animation cinema, for instance, emotion expression is in constant progress, using high resolution, motion capture and 3D to enhance its effects.
Nicolas Maigret: The Pirate Cinema.
But some emotions are specific to digital art. Emotion of the visitor/spectator, emotion of the artist, "emotion" of the work itself
1.2. The digital as perceptible by the specta(c)tor
What do the spectators feel when they look, listen, touch a digital work, or interact with it? Difficult to say because artists rarely search seriously to know it, critics are more eager to generate papers, and psychologists don't write on this subject. Even the philosophers concentrate on the "judgment" (Kant) than on the feelings. And, since antiquity "De gustibus et coloribus no est disputandum" (Never dispute about tastes and colors). Let's try anyway a first sketch.
First, we have all the works for which the computer is no more than a tool, and the digital aspects are not perceptible. Then, emotions depend on the message content: poetry, eroticism, indignation (for instance about surveillance or ecology). Then generally, the public will not perceive its digital basis.
Olga Kisseleva: Time Value (2012). The digital aspects as part of the spectators emotion.
But there is the case when technical progress in digital techniques are by themselves moving. Sometimes, emotion comes from the technical innovation. That plays mainly in periods of rapid evolution. For instance the realism of Appelle paintings, the regularify of clocks at the end of middle-ages, the higher fineness allowed by oil painting (Van Eyck), see Middle-Ages. In the 1980's, pixelization into large squares was used as an aesthetic feature. and nowadays the 3D video and printing.
In recent years, the White Pixel of Antoine Schmitt, the Horticultural spaces of Darko Fritz or the abstract contraptions of Olga Kisseleva put explicitly digital features in the spectators perception field. Generative works (for instance Jacques Perconte videos, Thomas Israël Metacrane), some advanced light shows in performance arts stress the automation facet of digitization.
Interactive works awake specific emotions. They can be implemented with pre-digital tools (mechanics, electricity), as shown by many works of Julio Le Parc. But of course, electronics and computers brought much more. Until now, interaction remains the poor relation of digital art, in spite of the high hopes it suscited in the 1970's, when some announced the end of the spectator and the birth of the spectactor... See interaction and transmedia.
Lozano-Hemmer
Some artists tend to stir indignation on the negative aspects of digitization in society, for instance the omni-presence of surveillance (See our notice). For example Lozano-Hemmer). But, as we noticed during a show at Gaité-Lyrique, the public did'nt really get the message and was using mainly some graphic features of the work to play with.
The web and it's possible distrurbances may also be given as an explicitly digital spectacle. See several examples at Paris Fall 2013.
Weird science (now on social networks).
On the sex emotional side, be it considered art or not, the fascinating possibliies of 3D graphics and interaction were hot in the 1980's. For instance with Weird Sciences, a game where you can trim the model's measurements to your taste. Japanese digital artists make an extensive use of digital animation, including their trend to not-so-normal sex pleasures (see Hentai in Wikipedia).
- Digital tools allow "special effects", or surpsising tricks like the faces burned into smoke in one of the Lozano-Hemmer works.
- In other cases, emotion surges from a strong visual and auditory stimulation, down to the limits of torture (see our post about Hentschlager (in French).
A very negative emotional intention with the roboticized hospital beds of Bruyere
The positive/negative polarization of the spectators emotion depends on their culture.
In Europe and US, literature is generally technophobic. That goes back to the Greeks and to the Bible, but inspires as well the modern mythologies of StarWars, Lord of the Rings or Harry Potter. Il was impressively illustrated in an expositon at Paris Cité des Sciences (see our post). You can see there dramatically negative images of computer management and robotics, with the hospital beds of Le chemin de Damastès, by Jean-Michel Bruyère, as well an impression of awe in front of gigantic rotocicized verstion of a car, by MacMurtrie.
By contrast, robots in Japan are smiling (see Japan in our geography), for deep cultureal reasons.
Major and minor chords (according to Wikipedia) are digital features mapping emotional moods.
1.3. The digital as a specific emotion in the artist's mind
But artists are not emotion engineers. Of course they use techniques adapted to the kind of works they do. A musician, for instance, to compose a Requiem, will use slow tempos and minor chords and keys, faster ones and major chords for a military anthem. These are digital features of music, but more a frame to inspiration than a set of engineering equations.
Then what imports is the artits's emotion. For many artists, however, the digital technologies are only a tool, like the tempos and chords in music. And the way the term "digital artist" applies to the large teams of illustrators and graphic workers in the games and cinema industry.
But for many of the artists we list in diccan, their creation stems directly out of an emotion borne explicitly from the encounter, sometimes dramatic, with some digital openings. Let's give some examples.
The joys of programming with the first micro-computers (1979).
Sometimes, the digital calling surges in the course of artistic careers yet well engaged. One of the most striking case is Harold Cohen. He was a successful abstract painter,with works in the English museums notably. But, in the early 1970's, he met the computer and never took his brushses again. A similarly digital emotion is told by Anne-Sarah Le Meur. She discovered that a color can be coded as three integers (RBG)... with nearly infinite possibilities and why not negative integers and "negative light". The digital took place more naturally, as an extension of electronics in the dynamic light effects in the sculptures of Alain Le Boucher,who makes it a point of honor to show all the components, processors included, in his generative works.
For others, the digital fascination came earlier, with the fresh mathematic hand calculators and their programming aptitudes. Hugo Verlinde for instance, loved the curves generated by transcendent functions, and made them the heart of his works.
Sand jets by Christoph Guillermet. Emotion for the scenograph, shared with his public.
Conversely, artistic emotion came to computer geeks like Antoine Schmitt, a sorfware developper, who progressively dedicated his life and earned it from art. Similarly, Christoph Guillermet began with light show management and from there went to emotional artistic installations with air and sand.
The move could stem from humanities, psychology and philosophical research. Hantu for example (see our Laval 2014 post about the art sesson, animated by Alain Lioret). My own mixed philosophy/computer culutre took me to write the Roxame painting software. I could add that diccan.com is somehow a work of art, and that, since my first website in 1991 for the Club de l'Hypermonde (still on line, with some minor maintenance), I keep feeling emotions about the aesthetic affordances of the Web, compared to the paper press, be it for its graphics, page setting and perenniality. This last feature has an important aesthetic aspect: you are never obliged to load what you write witt a rewording of the previously written, you have juste to add a link
(E)motion capture for Avatar, by Cameron.
2. Representation of emotions
Animation cinema is the more explicit user of techniques to represent emotions. They can be taken out of a set of samples, like those given for instance by [McCloud]. They can also be translated from an actor facial expressions recorded by Mocap (motion capture). But, for a large part, they remain done by hand.
The Duet film by Glen Keane, used to promote advanced visual and communication technologies by Google, were blatantly presented as a hand work by a genial artist, and not by any automated expression generating software tool.
Great seekers of emotion expression are also the roboticians. Not, in fact, for really artistic purposes, but for entertainment and "pet robots" and, in some cases for a better coexistence of robots and humans in factories. That leads sometimes to artistic research, for example by Zaven Paré.
Modeling the spectator's emotion. The resonance, or "uncanny peak" pattern (image from Wikipedia)
3. Modeling of emotion and "emotion" of machines
Beyond representations are formal models. Emotions can be modeled, but we must keep in mind that these models are always a simplification, a reduction of the real emotions.
Then any formal model can be implemented into a digital system and operate by itself. We call that "simulation" if the model aims te represent a function, or "automation" if it aims to actually perform this function. From a digital standpoint, the gap is not so wide between a flight simulator and an automated pilot.
3.1. Emotions modeling in general
We draw here from the Heudin communication at Laval Vric 2015, where he presented the project Living Joconde. He proposes a three layers model,
- emotion: short-term affects,
- mood: medium term affects, lasting longer than emotions and having a strong influence on cognition, with reference to the Lövheim cube (ses Wikipedia),
- personality: long term affects, specific of an individual and its mental features.
In Joconde, this model will be driven by a neural network system.
A first actuel demonstration of the work is to be presented at Paris Futur en Seine 2015.
In the case of art, we can build models the emotions the spectator (or public), of the artist, or consider the work itself as an independent and emotional actor. That raises of course philosophical and ethical issues, that we generally dodge with the help of quotes, so remaining conscious that the "emotions" of a robot are analogous but not the same as human emotions.
Emotional exchange between Philippe Gaussier ant the Etis robot in his laboratory
3.2. Modeling the spectators emotions
Until now, few artists take care of the spectators emotions. Interactive works detect their presence, their position, some of their moves. Anne-Sarah Le Meur (for Outre-Ronde) and Hugo Verlinde, for Boréal (if our memory is right) go a step farther, inferring from the moves the fact that the spectator is spending some time and doing some efforts to enter in relation with the work, and triggering only in this case a retributing behavior from the work.
The future will probably bring progress on this axis. The marketing of Google or Amazon does a lot to understand what the customer is looking for. And many automated tools come on the market to recognize stress, for instance.
Spectators emotions can be simulated by robots, as does Philippe Gaussier
at Etis labs wihin the frame of their research on robots behaviors. They have
gone up to simulate an art critic, nicknamed Berenson, and tested in real
environment.
Application to art: he robot critic Berenson (Etis lab, Gaussier) visiting the Musée des Arts Premiers.
More globally, the Google's ranking is a modeling of the public at large. Referencing experts detect this kind of algorithms and adapt to them the websites. Transmedia brings back metadata, which are not directly emotional, but very far from it. "To build a site accessible to the robots which explore the Web is as important as to design the site for the internauts..." writes Danny Sullivan, chief editor of Search Engine Land). In more and more cases, machines are better emotion detectors that we human beings.
A variant is the modeling of tastes through algorithmic aesthetic. We have tried a synthesis in Aesthetics and Algorithms : Around the Uncanny peak. In 2012, an art critic robot, Berenson, was presented in Musée des Arts Premiers (Paris).
For interactive works, a major issue is to actually perceive these emotions or at least behaviors. See our note on perception.
3.3. Modeling the artist's emotions
Effects of the emotional states on Roxame's drawing. From left to right: cold, sensitive, frenzied.
That may be considered a part of generative art, but is not explicitly modeled by artists as far as we know. We have made an attempt to such a modeling in out Roxame software with, for instance, a way of drawing lines that differs according to the emotional state ot Roxame considered an artist. Emotion influences for instance the way of drawing lines, from strictly straight to quite erratic. It can also play on the color scales, for instance using low saturation (grays) for coldness and high saturation for excitement.
Effects of the mood on the processing of a picture by Roxame. Same series as above, but the mood impacts also on line density and color saturation.
3.4. Modeling the works "emotions"
We reach here the most disputable... and perhaps most promising issue: "emotions" in artifacts, machines and notably works of art. Emotion can be seen as a parameter in behavior. Then we meet "behavioral art" or "living art", according to {Aziosmanoff] who says that a "living art" works, must have three motors: expression, behavior, perception. We are fully in the emotional realm.
The field is explored through diferent ways: robotics, artificial life, artificial intelligence, convergence of biology and computing. Generally not with artistic aims, but more practical applications.
In robotics, for instance, emotions can be a way for robots to adapt their behavior to the environment. Hunger was perhaps the first "emotion" to be simulated, in the 1970's if not earlier, since it's easy to get from the battery loading level, and to transfrom into an active search for a feeding socket. Fear, as a reduction of expression and visibility, readiness to escape and seek protection, may be appropriate. Expression of emotions, more or less simulated, are used in pet robots.
Artists sharing emotion: Hugo Verlinde and Gregory Lasserre, on a Scenosme interactive work (Gaité Lyique, 2013).
3.5. Machine, humans... let's share our emotions
Digital art takes us to share emotions, from the artists creative impulse through the empathinc resonance of the spectator, through the works "emotions". Interactive artists like Sommerer & Mignonneau, or Scenocosm put the stress on this sharing with vegetals. Artificial Life artists like Jérémie Bordas or François Zajega stress the collective facet of emotions.
But the most important is perhaps the cooperation and sharing of all the "beings", humans or machine, as we could feel it in 2013 at Gaîté Lyrique, when several artist... and the criting shooting the pictures, shared pleasure around a (comparatively) simple but efficient work by Scenocosme.
Some references
- Wikipedia on emotion.
< A Bio-Inspired Emotion Engine in the Living Joconde, by Jean-Claude Heudin. Proceedings of Laval Virtual VRIC 2015.
< The Uncanny Valley. by Angela Tinwell.CRC Press 2015.
- Comutationally Modeling Human Emotion, by Stacy Marsella and Jonathan Gracth. Communications of the ACM, Dec. 2014.
- The Israël company Beyond Verbal provides software to analyse emotions from vocal intonations.
< Techniques and Applications for Sentiment Analysis. by Ronen Feldman. Communications of the ACM, April 2013. More marketing than art.
Matlab GUI prototype for speech signal input and predicted emotion output on smartphones (University of Rochester)
- Emotional intelligence on smartphones, a communiqué by the University of Rochester.
- Software Enables avatar to reproduce our emotions in real time.
- Several notes in
[ Rose], specially the chapter "The emotion engine", pp. 275-288, mainly about the game designer Molyneux.
<
Designing for emotion, by Aaron Walter. A book apart, 2011.
- Why a song can make you cry. Scientists to grasp how the brain translate tempo and rhythm. International Herald Tribune 4/20/2011.
- Emotional gaming an article of New Scientist (4/18/2011).
- Emotional capital: "Term coined by Coca-Cola president Steven J. Heyer to refer to the ways that consumers' emotional investment in media content and brand increases the brand's worth" (according to [Jenkins].
- Maurice Benayoun/Citu: Mécanique des émotions .
- Emotional computer at University of Cambridge.
Emotionsl computer (University of Cambridge).
- Robotic emotions at Siggraph 2008 , AH No 89 (concepts), at Sony CSL.
< Alma, a Layered Model of Affect, by P. Gehrard. 4th Int. Joint. Conference on autonomous Agents and Multiagent systems. 2005.
- Face expressions , AH No 105 (concepts), and [Breazeal].
- A permanent planisphere of violence and motion, par John Klima.
< Robot Emotion: A Functional Perspective. 2005. by Cynthia Breazeal and Rodney Brooks. Online.
- AH No 51 (concepts).
- Colors [Kemp].
- In comic books [McCloud]. History [Thiébault].
< Affective Computing, by R. PIcard. MIT Press 2000.
- The works of Elaine Hatfield about emotional contagion:
< Emotional Contagion (Studies in Emotion and Social Interaction) by Elaine Hatfield, John T. Cacioppo and Richard L. Rapson
< Primitive Emotional Contagion, by E Hatfield, J. Cacioppo and R. L. Rapson. (1992)
< Emotional Contagion: Current Direction in Psychological Science 2. by E. Hatfield, J. Cacioppo and R.L. Rapson. (1993).
- Varia. A post
