Media Arts & Technology Program, UCSB

    Rama C. Hoetzlein  (c) 2007-2008

     Tobias Hollerer
     George Legrady


    1. Artistic Goals
    2. Technical Goals
    3. Qualifying Exams - Reading List
    4. Qualifying Exams - Reference
    5. Additional


What is the relationship between myth and semantics? What types of intelligence reside at each level of the body-mind relationship? What types of inelligence lie outside of language? Can these relationships be understood in the context of all form, whether realistic or imagined (surreal)?

Artists are capable of transferring natural functions - running, jumping, yawning, caressing - to imaginary or mythical creatures in order to give them life through animation.. and they can also imbue them with fictitious emotions and behaviors. Yet what is the status, culturally and scientifically, of these creatures? Can they be categorized? Reproduced by machine? If so, what about action, behavior and form cannot be reproduced.

The artistic portion of this dissertation proposes to explore the nature of intelligence by using surrealism as a bridge between science and mythology. One the one hand, artistic surrealism has a very real basis in the science of light, rendering, perspective and anatomy. Consider Salvador Dali's painting "One Second Before Awakening from a Dream Caused by the Flight of a Bee Around a Pomegranate", from 1944, below.

Human and animal anatomy are both reproduced accurately, as is light and shadow. On the other hand, the scene is fantastical, with floating objects, an elephant with elongated human bones, and unnatural landscapes. These objects, as Levi-Strauss describes, convey a mythical thinking, a kind of 'bricolage' in which the set of precepts (forms) is very broad and not subject to a particular pre-imagined idea. Yet rather, an idea which grows from their sub-conscious combination.

How is it that dreams are both real, and not real?
What is the status of myth in relationship to science?

I propose to explore these questions through an investigation of Surreal Biology, a scientific and experiential study of imaginary forms and creatures. This investigation will be both a scientific one, to sufficiently understand imaginary creatures as a 'class' of forms, yet also an artistic one, to understand the language, the signs and symbols of surrealism as genuinely non-scientific practice.

As Andre Breton says of surrealism, "Above all, we were exclusively preoccupied with a campaign of systematic refusal, exasperated by the conditions under which, in such an age, we were forced to live. But our refusal did not stop there; it was insatiable and knew no bounds... Intellectually, it was vulgar rationalism and chop logic that more than anything else formed the causes of our horror and our destructive impulse."

At the root of surrealism is an all out rejection of logic and reason. Yet, at the same time, the language of form, three-dimensional structure, light and shadow, are used to depict mythical symbols at a time when the rest of the art world was starting down the path of abstraction, minimalism, and conceptualism. What was going on here?

I propose to understand surrealism as both a logical (scientific, structural) and non-logical (artistic, mythic) system. Despite hypertextual post-modernism which expresses a loss of the mythic to logical formalism, my goal is to explore bridges between these two modes of thought which Levi-Strauss outlined long ago.

This artistic work will be both theoretical and practical. The practical work will involve examining surreal and mythological art works, to gather evidence, to sufficiently understand the nature of imaginary creatures. The goal of Surreal Biology is not simply to develop a thoery of post-modern surrealism, but to understand mythical, unnatural, fantastical creatures on a basic level. Successful theories should be able to recreate, even generate, a wide range of mythical creatures with a few fundamental principles. While it seems unrealistic for machines to generate meaning (experience itself), and they can already generate certain forms that convey meaning (artificial life), we do not know to what degree machines can generate archtypal forms.

Technically, I will develop a technical framework (see below) which, when completed, may be used to create autonomous imaginary creatures. Prior to completing this tool, and during development, I will continue to explore these ideas in watercolor and oil painting. Unlike mythical creatures currently produced for film (The Chronicles of Narnia, for example) - which are meticulously crafted through hand animation and motion capture - the completed system should provide a foundation to generate fully autonomous mythical characters.

horace       Horace, Egyptian - Human body /w eagle head
      Sekmhet, Egyptian - Human body /w lions head
argus     Argus, Greek - God with 100 Eyes
      Ravana, Hindu - 100 Heads, 100 Arms
      Chimois Dragon, Chinese
  dore   Caravaggio, Medusa
      Gustave Dore, Jacob and the Angel
  garden   Heronimous Bosch, Garden of Earthly Delights
khnopff   Ferdinand Khnopps, The Caress - Leopard body /w human head
  rops   Jan Toorop, O Grave, Where is Thy Victory?
      Felicien Rops, Pornocrates
brides     Jan Toorop, Three Brides, 1892
  time   Rene Magritte, Time Transfixed
temptation     Salvador Dali, The Temptation of St. Anthony, 1946
  seashade   Salvador Dali, Seashade, 1950
  memory   Salvador Dali, The Persistence of Memory, 1931
  ocampo   Georgio De Chirico, Hector and Andromache
      Octavio Ocampo, Don Quixote
  useless   Frida Khalo
      Remedios Varo, Useless Science Of The Alchemist, 1955
varo     Remedios Varo, The Birds, 1958
roots   Frida Khalo, Roots, 1943
eye   Max Ernst, The Eye of Silence
woman   bride    
    Max Ernst, The Wavering Woman

Max Ernst, The Robing of the Bride
ernst   Max Ernst, L'Ange du Foyeur, 1937
barney       Matthew Barney, Cremaster
  Michael Rees, Putto
    Benoit Polveche
    Jon Beinart, Crustaceous Toddlerpede
  Phillip Toledano, Boobsuit
      H.R. Geiger
    Bruno Torfs, untitled
      Osvaldo Gonzalez, Exodus
    Mark Ryden, The Creatrix
    Martin Oscity, Gordic Knot
    Martion Oscity, Expansion of Love
    Luigi La Speranza, Bird People Woman
      K.D. Matheson, Felino Canino
      Peter Marcek, Dedicstvo
  Heidi Taillefer, Chimeric
  Ray Caesar, Ebb Tide
    Meats Meier
      Tatomir A. Pitariu, Diet Goddess
    Kazuhiko Nakamura
    Dennis Sibeijn, Monument I
      Ray Caesar,
The Pet
  Francesco Mai, Double Coraz de Spessore
  Alessandro Bavari
    Fantasia, Walt Disney, 1940
      Brazil, Terry Gilliam, Universal, 1985
  Brazil, Terry Gilliam, Universal, 1985
  Dark Crystal, Jim Henson, Universal, 1982
      Jan Svankmajer, Alice, 1988
    Jan Svankmajer, Little Otik, 2000
    Pan's Labyrinth, Guillermo del Toro, 2006
    Chronicles of Narnia, Andrew Adamson, Walt Disney, 2005
    Lord of the Rings ("Treebeard"), Peter Jackson, New Line Cinema, 2001
    What Dreams May Come, Vincent Ward, PolyGram, 1998
  Miyazaki Hayao, Spirited Away, 2001
    Karl Sims, Evolving Virtual Creatures, 1994
      Christa Sommerer & Laurent Mignonneau, Genma, 1996
    Scalable City, Sheldon Brown, 2006
    AARON, Harold Cohen, 1983
      Spore (game), Will Wright, Electronic Arts


Regardless of the medium - filmmaking, 3D modeling, game design, hand animation or new media - the animation of characters requires considerable human effort. The is the nature of animation. Static media, such as painting and sculpture support a high degree of expressiveness, and, as they are static, the time to develop surreal, abstract, or imaginative works is reasonable.

To animate imagined worlds, it is necessary to either provide teams of artists (hand animation) or digital tools to facilitate the sheer volume of information needed for film - 30 frames per second. Tools such as Maya and 3D Studio Max, utilize specific semantic structures to enable digital animation. For example, there is a Character tool, and a Fur tool, focused specifically on these tasks. Yet considerable effort is required to create even the simplest creatures. With the exception of short clips, complex animation is generally unavailable to the individual digital artist due to the very high computational demands of rendering. Finally, the ability to create complex, surreal or imaginative characters in real time or for real time performance, while found in interactive games, are currently unavailable to the artist.

Existing tools
generally do not permit the flexibility to fully express the human imagination easily (in real time). Character tools allow for humanoid figures, and Particles for water and smoke - but to transform a human into a fluid is not easily accomplished. Arbitrary mental leaps, such as giving a giraffe a light bulb head, are difficult using digital modeling tools - yet these transformations are immediate in the artist's imagination.


The technical contribution of this doctorate is the development of a graphical language for structure, form and motion, and a dynamic scene graph to allow for real time structural and temporal changes to 3D objects.

One of the most sophisticated structures for rendering complex scenes is the scene graph. However, the development of scene graphs, now several decades old, are a consequence of the performance aspects of hardware. Scene graph systems, such as Inventor and Performer, were originally created to provide efficient culling pipelines for complex scenes. The first goal of this doctoral research is to examine how advanced scene graphs might be implemented to dynamically animate complex systems - where potentially the entire scene is changing dynamically. This advance scene graph would be used not only for culling and performance, but also to provide dynamic communication between parametrically defined primitives (forms). To overcome performance issues, rendering should be performance adaptive - providing as much detail as possible with the given hardware.

I propose to implement a high level renderer with "intelligent primitives" that can dynamically generate any desired level of detail and structure. Finally, a graphical language will be developed to arbitrarily combine and concatenate geometric forms - thus allowing the artist the ability to animate surreal creatures at a high level. The result will be an artistic tool which allows for expressive, real-time creation and performance of imaginary objects and creatures.



- What is the relationship between structure, form and motion?
- Can these relationships be formalized?
- Do existing theories of form & structure provide a suitable model for dynamic object simulation?
- How can we descibe dynamic structures at multiple scales?
- How can we allow for modular changes in structures in real time?
- How does computer rendering of detail relate to the rendering of detail in oil painting?
- What are the formal factors that effect quality and performance of geometric rendering?
- What are dynamic graphics primitives? How do these differ from static primitives?
- Is motion seperable from structure?
- What are the design requirements to model & render with smooth degredation based on available hardware?



One of the key challenges of this proposal is to enable arbitrary changes in structure, form and motion of 3D agents and objects during a live performance (in real time). To support this, it will be necessary to develop a theory of form which allows for efficient, complex transformations. What determines the overall shape of an object? What details can be drawn later based on hardware performance?

A primary contribution of this research is the concept of Primary, Secondary and Tertiary structures. While borrowed from terminology used in protein folding, these concepts are very similar in that they define degrees of complexity in 3D geometric forms.

Primary structure - Geometric shapes having no volume. Primary structure defines the most essential shape of an object. For example: The overall shape of a human is defined by the positions of its bones. Primary structure defines this as infinitely thin lines.

Secondary structure - Geometric surfaces or volumes, built from Primary structure, that give the outer form of an object. For example: Skin is the secondary structure of the human body.

Tertiary structure - Tertiary structure gives details on the surface or interior of an object by combining Secondary structure with another complete object. For example: Hair is a tertiary structure on skin.

The follow table illustrates Primary, Secondary and Tertiary structures for four objects.

While the concept of bones as driving pose has been efficiently applied to character modeling, this is applied only to characters. A similar, but entirely different, set of techniques provide level-of-detail for terrain. A mathematical framework which unifies structure and form across many types of objects does not yet exist. The theory of Primary, Secondary and Tertiary structure presented here allows complex objects to be treated equivalently from a software design perspective. This allows them to be interchangable. For example, this makes it much easier to have a tree walk like a person, or become water. I propose an in-depth investigation of structure to understand how to efficiently compose, transform and animate complex objects.



A useful graphical interface is necessary for any successful software application. This is especially true of artistic tools used by artists. Maya and 3D Studio, for example, require a very technical understanding of mathematics (splines, kinematics) which can be a daunting barrier to many artists. Artist-oriented tools, such as Bryce and Poser, are easier to use but at the cost of constraining the artist in specific ways. To create a natural, flexible interface for digital modeling and animation requires a careful balance between power and ease of use. A succesful interface provides both at a higher degree than existing systems.


This sketch is a softare design for a puzzle-piece interface for modeling complex systems.
Primary, Secondary and Tertiary structures have a specific shape which can plug into other modules. In this design Primary structures are square-ended, and Primary motion is triangle-ended.

This interface shows a human body modeled with puzzle-pieces. A human arm, complete with moving joints, shape, and motion, is attached to the left of the assembly. A bird wing, complete with bones, feathers and flapping motion, is attached at the right. This model allows for arbitrary compositions of structure, shape and motion. For example: The motion of a bird wing could be plugged into the human arm structure, giving a human arm that flaps like a bird.

Finally, as each module is an active object, control variables can be connected to any module. Here, a growth variable is connected to every structure, allowing the above representation of an angel (human /w bird wings) to grow physically.


This design shows how one could build Primary and Secondary structures using fundamental operators, such as loft, extrude and scatter.

In this slide, the Primary structures are curves. Operators, such as lofting, revolving and extruding, generate Secondary structures (surfaces). Finally, these are provided as input to a scatter module, which creates the Tertiary structure of a plant.

The real power of a language of form is the introduction of modules. Many components of a modeling system are analogous to programming language constructs. Each box above is analogous to a class. Each connection is analogous to a pointer. Each parameter (not shown) is analogous to a variable.

While modeling system implement these, they do not provide a most essential construct for flexibility: sub-routines. These are equivalent to modules as show in figure above. Modular graphics objects would allow whole, complex objects to be treated as new components. As shown above, the input curves allow for a much wider range of output by overriding the default curves. Existing modeling systems do not allow this.

Finally, rendering with a Meta-Renderer sensitive to level of detail would allow models of arbitrary complexity to be created and displayed by allowing the renderer to select the detail desired. The user should not have to worry about polygon count, mesh complexity or optimization.

  a2 Yoichiro Kawaguchi, 1982
  a4 Christa Sommerer, 2000
a5   Prezemyslaw Prusinkiewicz, 1993
  a5 Deborah Fowler, 1992
a6   Oliver Deussen, 1998
  Luxo Jr., John Lasseter, 1985
    M. Wein and N. Burtnyk, "Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation", 1976
    Alex Mohr, Luke Tokheim, Michael Gleicher, "Direct Manipulation of Interactive Character Skins", 2003
  Alla Safonova & Jessica Hodgins, "Construction and optimal search of interpolated motion graphs", 2007, SIGGRAPH (left)
  Gregoire Aujay, Franck Hetroy, Francis Lazarus, Chrstine Depraz. "Harmonic Skeletons for realistic character animation", 2007 (right)
    Demetri Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial Fishes: Autonomous Locomotion, Perception, Behavior and Learning in a Simulated Physical World", 1994, Artificial Life, 1(4), p. 327-351


The Art and Practice of 3D Computer-Generated Character Animation Maria Enderton 2003 Macalester College, Capstone Paper
The Science of Images: A Cross-disciplinary Introduction to the Field of Computer Graphics 2003 2003-Hunkins-ScienceOfImages.pdf
A Comparison of 3D Modeling Programs 2000-Durand-ModelingPrograms.pdf
Digital Nature
A Morphological Study of the Form of Nature Yoichiro Kawaguchi 1982 1982-Kawaguchi-MS.pdf
Basic concepts of computer simulation of plant growth M. Jaeger | Ph De Reffye 1992 1992-Jaeger-BCPG.pdf
Modeling and Visualization of Biological Structures Przemyslaw Prusinkiewicz 1993 1993-Prusinkiewicz-MV.pdf
Modeling Complex Systems for Interactive Art Christa Sommerer 2000 2000-Sommerer-ModelingSystems.pdf
Flocks, Herds and Schools: A Distributed Behavioral Model C. Reynolds 1987 Computer Animation
Scenegraphs: Past, Present and Future Avi Bar-Zeev
IRIS Performer: A High Performance Multiprocessing Toolkit for Real-Time 3D Graphics Rohlf & Helman 1994 SIGGRAPH
An Object-Oriented 3D Graphics Toolkit P.S. Strauss & R. Carey 1992 SIGGRAPH
NPSNET: Constructing a 3D Virtual World Zyda et al. 1992 I3D
Behavior-Friendly Graphics K. Russell & B. Blumberg 1999
Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation M. Wein, N. Burtnyk 1976 CACM, 19(10), Oct 1976, p 564-569
Motion Control in Animation, Simulation and Visualization Gerard Hegron, Patrizia Palamidese, Daniel Thalmann 1989
Human Character Animation in 3D-Graphics: The EMOTE System as a Plug-in for Maya Bjoern Hartmann 2002 2002-Hartmann-HC.pdf
Real Time Skin Deformation with Bones Blending 2003 2003-Kavan-RealTimeSkinDeformation.pdf
Direct Manipulation of Interactive Character Skins Alex Mohr, Luke Tokheim, Michael Gleicher 2003
Construction and optimal search of interpolated motion graphs Alla Safonova & Jessica Hodgins 2007 SIGGRAPH
Harmonic skeletons for realistic character animation Gregoire Aujay, Franck Hetroy, Francis Lazarus, Chrstine Depraz 2007
Artificial Fishes: Autonomous Locomotion, Perception, Behavior and Learning in a Simulated Physical World Demetri Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk 1994 Artificial Life, 1(4), p. 327-351
Mythologies Roland Barthes 1957 Jonathan Cape Ltd., eng. 1972
Myth and Reality Mircae Eliade 1963 Harper & Row
Essays on a Science of Mythology C.G. Jung and C. Kerenyi 1949 Bollingen Foundation, NY
Hero With a Thousand Faces Joseph Campbell 1949 Meridian Books
An Open Life Joseph Campbell 1988 New Dimensions Foundation, NY
The Interpretation of Dreams Sigmund Freud 1899
Man and His Symbols C.G.Jung 1964
Understanding Comics Scott McCloud 1993
On the Evolution of Species Darwin
On Growth and Form D'Arc Wentworth Thompson
Art Forms in Nature Earnst Haeckel
Structure in Nature is a Strategy for Design Peter Pearce
Game of Life Joseph Conway
Alice in Wonderland Lewis Carrol 1865
Surrealist Manifesto Andre Breton 1924
Metamorphosis Jon Beinart (editor) 2006
Masters of Deception: Escher, Dali & the Artists of Illusion Al Seckel, Douglas Hofstadter 2004
The Philosophy of Surrealism Ferdinand Alquie 1965 1st ed.
The Age of Intelligence Machines Ray Kurzweil
SHRDLU Terry Winograd MIT AI Technical Report 235, February 1971. Procedures as a Representation for Data in a Computer Program for Understanding Natural Language
"Mechanical consciousness" Samuel Butler 1863
Can machines think? Alan Turing 1950
Mapping Arguments Robert Horn
The Rediscovery of Mind, The Mystery of Consciousness John Searle 1992
Minds, Brains and Programs (Chinese Room argument) John Searle 1980
Physical symbol hypothesis Allan Newell
Why robots will have emotions Aaron Sloman 1981
Artificial Life Steven Levy
Godel, Escher, Bach Douglas Hoffstadter
"What is an Image?"  Harold Cohen 1979 Conference Proceedings, IJCAI 1979
The First Artificial Intelligence Coloring Book, "Off the Shelf," Harold Cohen 1986 The Visual Computer
Can Computers Make Art?  Harold Cohen 1985 Proceedings, NICOGRAPH-85
How to Draw Three People in a Botanical Garden Harold Cohen 1988 Proceedings, AAAI-88
AARON's CODE: Meta-Art, Artificial Intelligence and the Work of Harold Cohen  Pamela McCorduck 1991 The Creative Mind, Myths and Mechanisms (Margaret A. Boden, Weidenfeld & Nicholson, 1990)
Digital Mantras (Steven Holtzman, MIT Press, 1994),  Steven Holtzman 1994 MIT Press
Artificial Life for Computer Graphics Demitri Terzopoulos 1999 Communications of the ACM

"The MAT qualifying examination is a multistep process that follows the hurdle of the masters degree.

First the student assembles a qualifying exam committee. The qualifying exam committee does not have to be the
same as the masters committee or the PhD dissertation committee.

[ Content ]
The content of the exam is negotiated between the student and his/her committee. It usually consists of readings
or more generally an area of mastery that the student proposes to be  tested on and the committee agrees to.
The student should, in general, focus all energy on studying for the qualifying exam for a period, taking
MAT 596 Directed Research. (There is no limit on 596 units post-masters students.) The period of study is usually
between 3 and 6 months.

[ Written Portion ]
The exam usually consists of a list of questions that require long analytical answers.
One set of questions are given in the morning (9-12) and another set of questions is given in the afternoon (1-4).
The exam takes three days, usually a Monday, Wednesday, and Friday.
The location of the written part of the exam is scheduled and  proctored by the Graduate Assistant. A clean computer with no
internet connection is furnished and PDFs are transferred from it daily to the committee members by the Graduate Assistant.

[ Take-Home Portion ]
There is also a takehome assignment portion that is to be completed over the next weekend and turned in by 12 noon the following Monday.
This can be an additional question(s) or a creative or technical assignment, like: "Write a program that ..." or "Compose a piece of music that ..."

[ Oral Exam ]
The next week the oral exam is held with the committee members. This generally involves a discussion of the student's
answers on the written exam.

If all committee members agree, the student passes the oral exam. However, there can also be other outcomes, such as
additional study and work or a retaking of the qualifying exam at a later time. The qualifying exam can only be taken twice.

The next step after the qualifying exam is to prepare a dissertation proposal in the form of a document and
a formal presentation to the student's dissertation committee and the public." (5 / 2007)

Written Exam Take-Home Exam Oral Exam
Media Art & Technology 3 days, ? hrs, Closed book.
9-12 questions mornings, 1-4 questions in afternoons.
Decided /w committee
2 days. Write a program or a musical composition over the weekend. 1-2 hours.
Discussion of written exam.
DX Arts (U. Wash) None 4 days. Do a project. 1 1/2 hours.
Discussion of disseration topic.
Visual Studies (UC Irvine) 3 days, 9 hrs. Open book.
3 parts, 3 hrs/day.
Decided /w committee
Geography (UCSB) 3 days, 24 hrs/day. Open book.
2-3 questions
Decided /w committee
None 2 hrs.
Discussion of dissertation
Psychology (UCSB) 1 day, 4 hrs. Closed book.
4 questions, one from each faculty

1 1/2 to 2 1/2 hrs.
Discussion of dissertation

Mathematics (UCSB) 1 day, 2-3 hrs. Closed book.
6-7 questions. Agreed in writing.
None None
English (UCSB) 1 day, 2 hrs. Closed book.
3 fields, 40 mins each.
Decided /w committee
None 1-2 hrs.
Discussion of dissertation
ECE (UCSB) None Dissertation proposal. No time limit. 30-80 pages. 1 hr.
Presentation + discuss.
Physics (UCSB) None Dissertation proposal. No time limit.

2 hrs.
Presentation + discuss.

Materials (UCSB) None Dissertation proposal. No time limit. 10 pages

2 hrs.
Presentation + discuss.


multi-scale Able to continuously simulate on many levels
kinetic Able to simulation dynamic motion
physical simulation Able to simulate realistic physical motion
agents Able to simulate objects with intelligence (internal rules)
continuous LOD Able to render detail at any desired level
continuous LOS1 Able to render structure at any desired level
continuous LOS2 Able to render scale at any desired level (multi-scale)
tunable rendering Able to continuously render effects (shadows, textures) at any level of detail.
rule-based systems Able to have components with rule-based decision making
higher order behavior Able to support internal frames of thought (2nd order)
selective motion Able to switch dynamically between motion types
   scripted     Supporting scripted motion (data driven)
   dynamic     Supporting dynamic motion driven by physics
   algorithmic     Supporting motion driven by equations (abstract)
evolution Able to generate and evolute populations of agents, and to tune parameters based on fitness
structure-semantic crosstalk Able to support communication between geometric properties (e.g. incident energy area) and semantic properties (e.g. plant health)
procedural modeling Able to create geometric models procedurally from parameters.
functional modeling Able to generate models with iteratively more structural complexity.



Note to Self.. Regardless of their reasoning, and even generative, creative abilities, and explained by emergent systems in modern logical positivism, machines still may never be able to appreciate the beating of their own heart due to our direct human upbringing in nature. That is, not just the sensory fact of its beating, but the experience of mortality that comes from ones own heartbeat. In the distant future, such a mechnical being may even be possible (born?). Yet, we should expect that, if a machine that can experience mortality this way ever occurs, our own relationship to machine will be a much more wholistic and immaterial one.