Crystalline Chlorophyll
Paper, acrylic gesso, PVA, foamcore, two video projectors, two computers, two IR web cams, custom software in Processing. Dimesions variable. Exhibited in Kerfuffle at Bumbershoot 2009, Seattle. Curated by Lele Barnett and Chris Weber.
Video
Summary:

Crystalline Chlorophyll is an interactive digital sculpture. Its surface starts as an icy-crystalline shimmering and is slowly overcome by a verdant-green growth of texture during the course of the exhibition based on the movement of the visitors observing it. By tracking movement in the room it randomly adds a bit of green to the surface using generative pseudo-organic image algorithms.

The sculpture changes from a barren-static surface to a lush-vibrant one based on human presence in the room. If the sculpture is left alone for a while the mossy layer decays revealing the crystal-like surface again, waiting for the next group of visitors.

Crystalline Chlorophyll is sculpture built from a 3D virtual model using cutouts from the models triangles printed on 11X17" cardstock. The surface of the sculpture is painted with white gesso.

Two video projectors project from opposite sides onto the sculptural surface. The projected imagery is generated using real-time imaging software. The images are corrected for distortion by mapping textures to a 3D model of the real-world object, utilizing techniques also found in video game technology.

See also Cube Etude 1.0 and Cube Etude Beta for more about this technique.

Software:

Custom software written in the Processing programming language. The software loads a 3D model of the sculpture, made in Blender, and allows for positioning of the virtual model onto the real model. When the two models align the physical version has a projection-mapped surface that can be treated as though it were a 3D software texture-map.

The software also generates a real-time texture on the objects surface using camera tracking to seed random image generation.

Working Files:

The working files, digital sketches and conceptural renderings for this piece are being "open-sourced" under a Creative Commons license. Below are links to folders containing all files generated in the production and realisation of this piece.

Sketches >
These are the original concept designs. The original object was designed in Blender, it's facets were unfolded using the "unfold" python script. The key to using this script is to break the triangles up in rows and unfold those rows as individual objects. The unfolded meshes are then exported as .svg UV maps. More information on the technique of going from 3D model to papercraft can be found in this tutorial.

Production >
These are the files used for production. Mostly open-source software was used for this piece (albeit running on OS X), however Adobe's Illustrator program was used to convert the .svg files exported from Blender to .pdf files for print. The various facets of the physical sculpture were printed on standard tabloid sized paper (11x17 inches) at a local copy shop.

The individual triangles were cut-out by hand with tabs on the edges that were folded under and glued together, as seen in this image:

Code >
The code is written in Processing and uses the Saito .obj loader library to import the 3D object which was exported from Blender as a .obj file with a .mtl file. Texture maps were also created in Adobe Illustrator and exported in .png format, though other, open-source, programs may have worked just as well (i.e. Inkscape).

The software also allows for rotation, scaling and positioning of the object so that it can be aligned to the real world object. This is a bit of an awkward portion of the tool, but essentially holding x y or z on the keyboard while dragging the mouse changes position. Shift+x y or z changes rotation. w d and h keys change scale. In early versions, and then later added back into the most current version, the l key changes the virtual cameras field of view which is very important for matching the real-world video projectors lens properties to the virtual cameras lens properties.

Camera tracking is done with a simple difference algorithm. Each channel (RGB) of each pixel of the camera image is tracked for the amonut of change from the previous frame. If the number is above a certain threshold a random number is generated to determine if the associated pixel in the real-time texture map should change. A height map generated in Blender partially determines the probability of a pixel changing from the static "ice-crystal" texture to a "mossy" one. Other influences are neighboring pixels values and the amount of change in the camera pixel.

Promotion >
These are promotional materials for the Bumbershoot show.

Images >
Images of the process of constructing the piece and other documentation.

Creative Commons License
Crystalline Chlorophyll by Joseph Gray is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Permissions beyond the scope of this license may be available at http://grauwald.com or by writing josephgray @ grauwald.com.
Photographic Documentation
Conceptual Renderings