As soon as 1977, Myron Krueger proposed advanced interaction concepts (figure from Wardrip-Fruin, p. 380).
Humans and "agents" peer-to-peer
A second look at Laval Virtual 2014
1. Interaction has stagnated since the 1970's. 2. Psychology comes back. 3. No really impressive technology. 4. Where to get bricks for a model. 5. Can art bring new hopes?
We were impressed, at Laval 2014, by the fact that four communications brought psychology in the center (see our first impressions) and the staging of peer-to-peer relations between humans and various kinds of "agents". This evolution contradicted
- the generally low levelapproaches of interactive art
- our pessimistic views on interactive art. Let's try to dig deeper.
1. Interaction art has stagnated since the 1970's
Interaction and high level agents are as old as computing. At least on the conceptual level, ideas were more audacious in the 1960's than today, in spite of computers million times more powerful, interfaces very cheap(cameras, mikes, Kinect) and sofware from open sources. Remember (or read back), for instance:
- Computer Simulation of Personality Frontier of
Psychological Theory, by Silvan Tomkins and Samuel Messick, Wiley and sons,
1963).
- Responsive environments by Myron Krueger 1977
-
or more recently but yet nearly 15 years old, Affective Computing (Picard, 2000).
Even in fiction, the constructive views of Brave New World (Huxley, 1932), I Robot (Asimov, 1950.) or Ecotopia (Callenbach, 1975) are forgotten. The last Suarez (Influx, 2014), staging unrealistic antigravityy and hypernetworking... remains anchored to the hopeless assertion : "human nature remains in the Dark Ages". And SF receded behind fantasy and sorcery.
Why has McCormick chosen an ugly graphics to represent the emotions of a graceful dancer ? ...
Until this Laval session, we were feeling that interaction was not getting momentum in digital art, for several reasons:
- Interaction suits more to games than to art, or more exactly to "fine arts". Painters and sculptors attach much importance to their role of "authors", which would be challenged by an excessive intrusion of the spectators into their work (this pointed is commented, for instance, by [Kwastek].
- Individual artists cannot compete on this field with the games industry, where huge budgets pay for large teams and technical facilities.
- Presently (the 2010's), interaction seems to develop in another way, the "transmedia", with which the spectators use the work as a material, a basis for their own development and communications, typically with their smartphones, taking photographs and sending messages on social networks.
We had commented on this theme about Le Parc's exposition in Paris (2013).
... fortunately, this version shown a Laval (our shot during the session) lets hope more pleasant expessions interaction.
But the Laval 2014 session on art brought new approaches, coming from very diverse and sometimes unexpected environments.
2. Psychology comes back
Interaction in art was presented at Laval by artists/scientists coming not from fine arts but from other specialties and cultures.
Bordas started his career with technical problems (rendering, Midi sequencer...) but dares now to dive directly inyo psychology.
McCormick, represented in Laval team of six members from a university lab, founded "to provide facilities and expertise in motion capture for teaching, research and commercial development", with a strong artistic orientation: 9 over 15 members of the lab are labeled "research artist and PhD candidate).
O.K. Appadoo animation for Weber/Delsaux : standard techniques for 3D animation. But feelings included.
Far from psychology, but the lab calls itsef a studio and its director, Kim Vince, is not the standard motion capture expert, but "a choreographer and interactive dance artist who develops new ways of investigating and creating dance using digital technology." The best of two worlds... More physics than soul, but focusing "on the (automated) agent ability to generate movement from what it has learned and perform in parallel with a human participant".
Haute is a performer as well a teacher. She went on stage, for instance, in the recent Cherchez le texte Conference in Paris, a literature event. She gives expression to his reflections as well in ink drawings as in complex real time events. And the reference of his papers include classics like Barthes and Foucault.
Coming even farther from the highly technical ambient of Laval Virtual, you would never expect to meet there a group of artists like Hantu (Weber and Delsaux). Even less so if you have seen/heard them at their happening show in Clermont-Ferrand staging a psychiatric fiction ( look at their video) . Taking off from such literary and conceptual art, they launched a multifaceted artistic/technical project, Nymphea Alba Ballet. With the help of a 3D professional (O.K. Appadoo), they started with an animation film, went up to simulation and robotics, then to installations and performances, and envisage now a worldwide network of stations and an industrial prototype. Without forgetting elegance and psychological insights!
3. No really impressive technology
This convergence apart, nothing in these presentations was technically impressive.
Perception, or "data fusion", seen by the US Defense. The scheme could be applied to art. And is certainly a very small part of what Defense holds as classified in psychology modeling.
The emotional models of Bordas are quite crude : five factors for personality (Ocean model) , 22 emotions and five steps (0CC), and a transposition of heat dissipation to emotional contagion (Ascribe).
The motion capture, speciality of McCormick's team, uses the cheap Kinect set of sensors. The avatar is smart, learning with neural networks and the SOM technology which dates back to the 1990's.
Weber and Haute use rather standard imaging techniques. Their discourse and practice is psychologically oriented. Any technology will do.
Then the novelty of this session stays mainly in the deliberate choice of peer-to-peer cooperation and similar models for humans and machines in the work of art. This approach could get momentum in the future. Once the taboos broken, budgets and more costly technology could lead to more and more convincing works. But where can art find the necessary resources? Let's try a short survey.
4. Where to get bricks for a model?
The need, clearly, is not mainly of more advanced hardware. What lacks is higher level models of human psychology adapted to computerized agents, and higher level cooperation systems, or languages in the most general sense. And above all the audacity. Alas, psychology is the poor relation of interaction community.
At the low level of programming and physical fabrication, we find typically Programming interactivity (Noble, 2009). It is centered on basic functions with the Processing language and Arduino cardboards. Similarly Press On, Principles of interaction programming ( Thimbleby, 2007) is oriented towards program structures... the word character (let alone emotion or psychology) is absent from the indexes of both books.
Skyrim, a (rare) example of a game with some psychology in its non-player characters.
Defence departments have certainly major resources on the field, as attests for instance a scheme on perception from Information Warfare (Waltz, 1998).
Games are champion in interaction, with $ millions budgets. Unfortunately, there is little to get from game design literature, for instance inThe art of game design (Schell, 2008) and even, surprisingly, in Artificial Intelligence for Games ( Millington,2006). About games, with an accent on dialogue and story telling, Expressive Processing ( Wardrip-Fruin, 2009) concludes also on disappointment. A similar feeling results from Metacreation, Art and Artificial Life ( Whitelaw, 2004).
And, seen from the players side, games don't progress on the psychology side. NPC's don't become smarter and more "affective". A look to video games magazines shows that new versions of games are doing better on graphic quality, realism and rich details. Not on the psychlogy of NPCs or on the call to players intelligence and feelings. There are nevertheless some comparatively advanced behaviors in Skyrim (thanks to Greg B. for his info).
Cybernics, an ambitious fusion of human, machine and information systems". But the project seems now reduced to robotic applications.
More arts oriented , Inventing the Medium (Murray 2012), proposes "principles of interaction design as a cultural practice". Alas, the books remains at low level (useful if you come from artistic or literary cultures, and look for a global introduction to applications of computing in the field). None of the keywords we would expect is even present in the index (emotion, perception, character...).
Robotics take to more general models. This discipline brings do a lot, with structures for behavior, perception and emotions expression with the MIT robot's face Kismet (see notably Breazeal 2002). Japanese do a lot, but it is difficult to get practical documentation. Hiroshi Ishiguro's experiments did not result in practice (in spite of psychological research, see Le jour où les robots mangeront des pommes (Grimaud and Paré, 2011)). Tsukuba University launched a daring blobal project, Cybernics. Its results are summed up in the book Cybernics, Fusion of human, machine and information systems (Sankai, Suzuki and Hasegawa, 2014). But the project seems now focused on the technical and industrial issues of the HAL exoskeleton/robot.
Aesthetics of Interaction in Digital art ( Kwastek, 2011), is more philosophical than practical, and foreign to behavior and psychology.
More useful is the Handbook of Virtual Humans (Thalmann & Thalmann, 2004), which has a full chapter on behavioral animation. But they focuse now on visual problems, not on psychology. It includes a promising chapter on behavioral animation, with architectural considerations, combining a low level (ACE, Agent's Common Environment) and a high level (IVA, Intelligent Virtual Agent). But the hopes stirred in the last paragraph of Thalmann, "What's next", and quoting VHML (Virtual Human Markup Language) and . Unfortunately, the website shows that the project has been sleeping since 2002, as well as AML, a marking language for avatars. One reason may be that marking languages are not sufficienlty programmable for the kind of developments required here.
Florent Aziosmanoff's The little Red riding hood, used robots to represent human psychology. Unfortunately Aibo robots are no supported maintained, and the author has now other aims.
Florent Aziosmanoff proposed a clear and, we think, efficient framework in his book Living Art (2010), sketchind a methodology with three "motor"s: perception, behavior, expression, But it does not go very far beyond drawing a framework (with topic examples), and Aziosmanoff is now busy with other thematics.
The ACM special interest group on computer/human interactoin, Sigchi will hold ICMI 2014 (Interna'tional Conference on Multimodal Interaction) in next November (Istambul, Turkey), with "Modelling Human Communication Patterns" among the topics. But they are far from art.
Remains the IVA (Interactive virtual agents) movement, active since 2006 and an IVA 2014 next August in Boston. Boosted by the Healrh community, its rich programes are quite far from art, and are principallly oriented toward health applicaions. It seems short of a comprehensive model to guide the developments and help artists.
In short, human beings are studied from a large set of standpoints, but not up to the central and global issues. The digigtal world has still to recover from the disappointments about Artificial Intelligence, and specially of the "expert systems" at the end of the 1980's. Since then, in spite of important progress in many fields, in spite of Deep and Deeper Blue on chess and of Watson at Jeopardy, Ai faces constant denial, including by computer geeks.
This sterility, this denial of the central topic of modeling psychology may be due to fear : the growing competitivy of "robots" on the workplace, the disputed annoucements of the "singularity" and, even deeper, and the fact that noboty is presently able to imagine a future where advanced technology would go along with advanced education and democracy. Industral and political leaders of today behave no better than those of yesterday.
Aesthetics of Interaction in Digital Art< em> by Katja Kwastek "(MIT Press 2013 offers a broad review of the topic, with human aspects at the center. Here, on the jacket of her book, Se Mi Sei Vicino, by Sonia Cillari.
5. Can art bring new hopes ?
In this context, it was refreshing to hear in Laval the daring presentations of these artists. But one swallow does not make a summer... Four swallows ? Those are some rather solitary artists/thinkers, coming from very diverse environments They are comparatively unknown, and their short bilbiographies let think that, as said Mark Twain, "They did not know it was impossible so they did it."
Then a full and productive "interactive intelligent art" remains mainly a dream, but once more in 2014, Laval Virtual proved its ability to nurture emergence.
Pierre Berger. 4/19/2014. Revised 4/29/2014
Some more data and and references in our notices : emotion, Interaction, perception, psychology and transmedia (works in progress).
DICCAN'S PARTNERS:
Paris ACM Siggraph, the French chapter of ACM Siggraph, worldwide non-profit organization of computer graphics.
