Form and matter in digital art
An example of play with forms and matter. Roxame generated a set of geometrical forms, then filled them with randomly selected pictures of different matters.
Last revised 9/4/2014. Return to Major concepts. See materialization.
Form and matter is the most traditional frame to conceptualize creation, as shown for instance the first chapters of the Bible, or the Aristotle's philosophy.
A form is perceived at the same time:
- as continuous: a line, a surface; it can be indefinitely cut; but not infinitely, say the continuity definition of topology;
- as creating a discontinuity; it is opposed to its background; a straigtht line or a circle parts a plane in two different surfaces, etc.
Matter is a first perceived as continuous, "blank". But we know that it is made of atoms and particles, with the limit of quanta.
Similar on those points, matter and form have very different philosophical (even theoloogical) and "moral" connotations. Mind must control the matter, that is constrain it to "conform" to the right forms. In particular the body (matter) must serve the soul (immaterial and, for the believers, immortal).
We shall try here that, in the digital world, digitalized matter and digitalized forms are dealt with in the same way by computers and by humans.
This way mixes constantly finite and infinite. Remember that a "real" device, and a computer in particular, is not a Turing machine, since it has hot an infinite memory. It's size in space and time can be so high that we never meet its limit... but in some cases which are of theoretical importance, and sometimes practical (definition, real time). Similarly, our mind is certainly able to conceive the infinite, but in practice operates constantly on finite ways, at least when it works "rationally", that is formally.
And that without entailing metaphysical assumptions about the spiritual nature of man and its irreductibility to matter.
Form
Animals and humans learn early to recognize forms, for feeding and safety reasons, as explained Konrad Lorenz. (Here from the Xenology website.
Form is perhaps the central concept digital art, as the bit is the central concept of the digital world in general. It's a very general and polysemic, but we can concentrate here on a definition sufficiently narrow to accept formalization.
Intuitively, take form of a being as a set of features such that we can intuitively conceive it, perceive it and produce it. Our brain and our primary education are designed to allow these operations, necessary to our life and mere survival. See for instance Lorenz. This education includes evaluation: some forms are dangerous (the predator) other are favorable (the mother). With maturity, comes the aesthetic evaluation of forms. Some forms are generally perceived as beautiful (the circle, symmetric forms) other not (scribblings). More subtly, within a category of forms, some are considered as "ideal", or "canonic".
The scholastic philosophy proposed three criteria, applicable to any being or work of art, but particularly to form:
- integrity: the form must not lack any of its normally constitutive parts (nor have extraneous one, anyway),
- clarity: the form must clearly express what it is supposed to be, "splendor veri", with a probable reference to realism in representations, which was dominant if not exclusive in antique aesthetics,
- harmony: its parts must have regular proportions (a fraction of small integers, or the golden section).
Forms, like classes can be diversified by inheritance. Here, the Character class inherits from Letter and Digit. (From the Staff website).
Formally, let's take form as a class in an OOP (Object Oriented Programming, see Wikipedia) environment. A class is defined by a set of values (variables) and methods (functions), including a creator and a destructor functions. An object is created (instantiated) by the creator function and remains active as long as it is not destroyed. And functions can be written to detect, recognize and specify the characteristics of such a form in a space or flow.
Note that classes can be organized in hierarchies with the inheritance process , quite similar to the hierarchies of genres, species and sub-species in life, for instance, and the specialties and styles in art.
More important, classes also are seen at the same time:
- as continuous, at least when they include continuous functions, such as
real a,b; b = sin(a); and continuity, as defined intopoloy, refers to infinity;
- disjunctive, when they include thresholds or binary conditions, most generally of the form:
real a; if (a>PI) then...
Then, at this formal level, our mind and our computers work in the same manner. As one says : "Tout ce qui est automatisable est formalisable" (If you can formalize it, you can make it automatic).
Some references about forms and shapes
< Practical considerations and Processing source code in Roxame.
< The book of shapes by Alan McCollum Notice published by Michèle Didier (2010). Sold as an artist's book, with luxury binding. Not properly digital. A sort of personal research for a set of basic forms and their combinations.
- Vie des formes, by Henri Focillon. PUF 1943. Interesting if not fascinating. Bu the word form is here too much fuzzy and polysemic to be really useful in digital art (we think).
< Shape Theory. Categorical Methods of Approximation. By J.-M. Cordier and T. Porter. Ellis Horwood 1989. Dover 2008. A very formal and algebraic approach. We don't see how it could be used by artists.
< Calcul de formes par ordinateur. by John Woodmark. Masson 1988.
< Modèles mathématiques de la morphogenèse, by René Thom. Christian Bourgois, Paris, 1980. Beautiful but theoretical approach, difficult to use for art, though the concept of "strange attractor", more or less tied to fractals, has inspired for instance Lioret
Matter
Now in our world generally speaking, the world of nature, forms cannot exist if they are not born by matter. In the external world, they are properties of objects. We perceive them by many means, for instance as the contours of a body. In our mind, neurosciences advance steadily, but we do not know very precisely how a form is stored and activated.
Matter, in art creation, is the pole opposed to spirit. Pure matter, totally unstructured and continuous, is a vue-de-l'esprit. Real matter is nuclear, quantic.
In digital art, matter takes several parts:
- It is the medium organized to let the work exist and operate; it is, somehow, the works "environment".
- It is composed of structured and catalogues materials that artits will use in their project (paper, ink, paste, air... and at the limit, the bodyly matter of the performers).
- Alonc centuries, progress in art (transcendence let out here) consists in the development of these environments, of the available tools and materials catalog. See for instance the Blick catalog for plastic arts, or the Paul Beuscher catalog for music. See also material.
But digital art adds something new: everything can become a material for something else, be it the catalog of all the works digitally downloadable. Digital art is permanent remix.
In the digital world, the materialization of forms can be precisely described. With several levels. And, as for forms, that works along similar lines in our mind and in the computer.
Alain Le Boucher maps time rasters of LED's onto sculptures made of electronic components.
But this materialization cannot be "immediate", since forms bear continuity and infinity, and the digital material objects are finite and discrete. Then the transition between matter and form uses an intermediary conjunction of form and matter, whose fundamental property is that in can go from matter to form and vice-versa without loss. Let's call it the "digital medium".
This medium has two important levels, radically different though there are some intermediary levels, and notably two major ones, that we will call "raster" and "code". Both are stricly material objects. You can copy then on a USB key and deal with them as any other thing, even if that is does not pleases transcendalists and platonist thinkers (All of us are, more or less).
Digital systems are related to matters along two ways:
- their sensors and effectors;
- their hardware itself; each technology features ratios between form and matter, be it the bandwidth of a transmission line or the number of transistors per chip. The "cloud" adds a new, global, expansion to these figures, and "big data" evokes petabytes of data and more. But "peta" is not "infinite"
See :
- Materialization, materials.
- Matter still matters (some artists, on 9/2102)/
- Digital matter and art AH No 27 (concepts).
- Varia about incarnation.
- Sensor, actuator.
- Condensed matter, SH No 17 (concepts) , AH No 16 (manifestations).
If the raster is too coarse, it limits expression to schematic images and short text. Here an image from French color minitel in the 1980's.
Raster
We call here "raster" a set of bits, typically forming a two-dimensional space (e.g. a matrix of pixels for an image, a sequence of amplitudes for a sound record). This mode of form materialization is used by nature (e.g. retina) as well as by machines (memories).
In programming, a 2D blank raster is created by something like
img = createImage(width, height, RGB);
which shows clearly the limited size of the raster, as well as of color values (an RGB color is coded on 24 bits, 8 for each color red, green and blue).
Forms are present in those sets only as values that make them perceptible.
A raster is finite, even though it can be thought of as infinite.
Inside the digital device, the raster is stored or transmitted in forms that fit the device. For instance, a one-dimensional stream for transmission, or a distribution on tracks and sectors on a hard disk. Efficiency takes frequently to complementary compression/decompression operations. These structures are controlled by the operations system.
On the other hand, a raster represents a real phenomenon or is used to generate it. On both ways, the transition is comparatively straightforward: its transitions are "analog" to what is represented (space dimensions of an image, time succession for sound). The coding/decoding is merely an analog to digital (or digital to analog, sampling) conversion, combined with size and energy scales adaptation. These structures are managed by the sensors (camera, mike), effectors (printer, loudspeaker) and I/O control devices with their drivers. These last tools can be quite complex, for instance to use video resources by streaming.
Artists can use the defects of raster compression/decompression algorithms. Here a work by Jacques Perconte.
A lot of artistic talent and time can work at this transition level. Sound capture of qualify requires several mikes, of appropriate characteristics, set at the right places. And the deployment of sound in the real space of an auditorium or a stadium call also for sound engineering if not art activity. It's the case also for cinema, with camera moves and controls. And, at the other end, the projection of images on complex structures (video mapping).
For example, an artist like Alain Le Boucher combines the algorithmic generation of a raster and the latter's materialization through a set of LED's that are embedded in sophisticated sculptures.
By itself, a raster is passive.
A limited set of operations can be done at the raster level:
- disjunction of a form against its background, of of different "channels" for colors (RGB, transparency) or sound (voices, instruments)
-
merging and compositing of images, mixing of sound.
Nearly all artists work today at this level. They use appropriate software to present them the raster on a screen or headphone, and enter new forms through the keyboard and mouse. They don't operate at the coding level, which we will present below.
Code
A code is a set of bits organized as a text, following the rules of a grammar designed to make it readable and executable without ambiguity by humans and by machines, provided they know this "language". Nature uses codes for animal "languages", the DNA and (in ways more and more well known) by the neural system and the brain.
Here, the forms are represented by their names, their definition, the features of their instantiation/materialization. And terms are organized in texts (programs, narrations). This
-
represents an enormous economy in terms of matter,
- allows logical and mathematical operations,
- demand the use of a language (dictionary, grammar) for interpretation as well as generation.
Coding can be more or less "deep". Rasters can be seen as the basic level of coding, with coding only of numbers and characters. Deep coding calls for a high level of abstraction and conventional assignment of codes (words) to concepts.
A code is finite, for the same reasons as a raster. To be stored and used, it must be materialized as a signifier, independently of its meaning. It uses standard methods that permit its writing and reading, mainly using text processors. This includes, when it has to be presented and handled by a human being, a typographic transcription on an image raster.
Code is finite, but calls to infinity. That is due to its recursive abilities. (See recursion in Wikipedia). Since a function can call itself, the loop can last indefinitely. This is allowed by a second, hidden, form of infinity in Turing machines: they are supposed to run indefinitely. A classical issue is that one cannot always say if a given computation will come to an end.
Materially, this infinite operation is geared by the basic digital motor, the clock.
Actually, the infinity will always be limited by the lifespan of the machine... of of the time users are ready to wait.
Code is discrete, but it calls for continuity, as we have seen with the contiunous functions in classes.
Coding is powerful, if not magical, since it can use finite and discrete means to represent and actuate infinite and continuous beings.
But there is a price to pay. Coding breaks the immediatety of our relation to the world. And we regret the time where instinct was sufficient, without the formal contraptions and the social compromises of formalization.
See some more comments about code in the code item.
Now, the function of code, at least in art, is to generate data and notably rasters. And vice-versa. But here the transitions are no longer straightforward, and we reach the most important part of this paper, the uncanny valleys of generation and perception.
From code to raster: generation
From form to raster, the Bresenham's algorithm to draw a straight line (image from Wikipedia).
We can consider the instanciation of a class as similar to the application of a form to matter (for instance casting). But the comparison has its limit.
The generation of a form looks like the instantiation of a class, but is really different. The creation of a form in a raster implies (e.g. in graphics):
- the instanciation of the raster, as seen above
- the deployment of the form onto the raster; the generated form is not properly an object (in OOP language), it's the assignment of values to pixels into the raster; you cannot (unless you provide for it explicitly) get back the form with a pointer; it is "lost" in the raster.
These functions use frequently the LineTo(x,y) procedure, to write a straight line between to points. In principle, this function is continuous. And coninuitty implies infinity (by its topological definition°). In practice, the program has to map it into the raster of pixels, through ad hoc algorithms (for instance Bresenham (Wikipedia)).
That works generally, but we may meet several difficulties:
- if the pixel x,y is outside the raster limits, the line will be clipped
- if the resolution is low, aliases will appera
- if the color chosen is the same as the pre-existing background, nothing will actually be drawn,
- worst of all, if the raster was not "white" but contained yet other forms, the line risks to be damaged or parasitized by other ones.
- any ulterior form generated in the raster risks to deteriorate or delete completely what has been generated.
An example of graphical grammar: some rules from the Agarwal and Cagan grammar for coffee machines design. .
Note also:
- that a finite code can generate an in(de)finite raster or set of data, as long as the machine is running,
- that a given raster can be generated by an in(de)finite number of different codes. The proof is easy: any code generating this raster can be preceded by any other code, provided that the latter ends with the generation of a white page. But of course more interesting variants could be found.
To generate a form onto a raster without taking into account what there is yet in the raster, is a risky business. As says the title of a paper by Harold Cohen " Coloring without seeing: a problem in machine creativity. "
Digital artists have to cope with these issues. Some of them use this "glitch" as a form of art. See for example the videos of Jacques Perconte, who plays on the limits of video compression/decompression algorithms.
Two kinds of tools help this generation process : grammars and models.
A grammar is a set of methods to construct objects of a given class. Language grammars, but also grammar of forms (graphic or material). See Wikipedia . Developed notably by James Gips and George Stiny. Generative grammars are important tools for generative art (from Jean-Pierre Balpe to Miguel Chevalier", for instance).
The word model has a lof of meanings, two of which matter here:
- a structure to generate (and possibly animate) images, characters; used for instance in software like Poser to generate human figures;
- the ideal or canonic set of components and quantitative rations applying of such a structure; for example the Doric canon of Greek temples, or a top model in a fashion show.
A screen from Poser.
From raster to code: perception
Digital artists, in 2014, focus mainly on form generation of all sorts (graphics, music, text), and make little use of form recognition (which conversely is the main topics for medicine and defense).
That's due at the same time to:
- artists psychological trend to use techniques but to rely on their seeing and hearing for perception,
- a comparatively slow development of perception by machines.
We take here perception not as the "captures" providing data, but as the detection, recognition and description of forms out of the data in order to make code, or at least to feed code with meaningful data. Our brain uses an important part of its resources for that function (280 million neurons for V1 only), but until the development of neurosciences, we took vision as quite simple activity.
The raster-to-code transition may be called interpretation, analysis, or research of a meaning.
Smile on a faces. A quite difficult pattern recognition. But many cheap cameras can nowaadays do it.
Read more in our perception notice.
4. Emergence and generative art
From Wikipedia "the arising of novel and coherent structures, patterns and properties during the process of self-organization in complex systems".
according to Jeffrey Goldstein , who adds five characteristics.
(1) radical novelty (features not previously observed in systems);
(2) coherence or correlation (meaning integrated wholes that maintain themselves over some period of time);
(3) A global or macro "level" (i.e. there is some property of "wholeness");
(4) it is the product of a dynamical process (it evolves); and
(5) it is "ostensive" (it can be perceived). For good measure,
This cup of coffee, with its Magritte style, emerged from a random process of Roxame software, using first a random generation of geometrical forms, then applying textures.
These characteristics a trait of art in general, perhaps even the definition of contemporary art. Generative art gives them a programmatic answer. And that opens to evaluation, possibly by autormatic algorithms. They can let open a graduation and possibly evaluation by algorithms.
Let's say it in other words, possibly leading to programming.
A form emerges in a space for a given observer when it is detected in a place where there was nothing of this sort before, and this form is not the logical evolution the observer action of of known trends, other than global. In orher words, the observe sometihing which is not a result, is not a detemrined output of the system as it was.
An type of emergence could be globally expected, but not precisely on this place or with these features.
Emergence evokes some progressivity, wih at first "something" and progressively more features and a possible pattern recognition.
Suppose that we have for instance a white page of pixels, and send it pixels in a more or less random way, but such that groups of pixels tend to form and show some stability and evolution. Then we can hopefully get some features about it: size, color, position, moves... stuctures, patterns, behavior. Then we can regognize it
- abstractedly, as representant of a known class of beings: it's a rectangle, a fish, an Airbus A 320
- concretely, when a form has only one instantiation.
But in other cases, the description fits with no previous known object, or fits only with the features of a general class, but not with any known subclass. Then, there is more properly the emergence of a new category of beings, that the system will characterize with more and more precision. And possibly create a new subclass if other similar object emerge and are of interest.
DICCAN'S PARTNERS:
Paris ACM Siggraph, the French chapter of ACM Siggraph, worldwide non-profit organization of computer graphics.
