Over the past year of generative music experiments in performances and installations, I’ve been chipping away at a homebrew, Ruby-based platform for Ableton Live which I call Loom—named for the textile pattern-generating ancestor of the computer. In hopes of getting more ears on it, I’ve recently distilled it all down to a lean and modular (albeit very alpha) core, and published the source on GitHub, where you’ll also find a slightly more technical introduction than the pontificating, hyperlinking, and screencasting below.

For the uninitiated, you can think of it as computer-aided composition: you describe the music in high-level terms of characteristics and motifs, do your sound design, and let the computer generate the specific MIDI patterns (melodies, rhythms, etc.). You can surface parameters to tweak for a live performance, or you can just let it run for hours (days, weeks…) as an installation or radio station. But, ideally, you’ll always be surprised (maybe even pleasantly!) by the music that emerges.

Loom’s architecture has been emerging from the application of Ruby idioms to generative music. At its core, it’s just a smattering of Ruby APIs hooked into a Max event loop. Writing a Loom module, for example, is just a matter of monkeypatching right in. But these idioms—mixin modularity, the teasing out of randomness from controller logic (kinda inspired by MVC)—have begun to propose a deeper architecture which has just begun to come into focus. Compared to the excellent, more established generative music platforms below, Loom is shaping up to be: more opaque than diagrammatic; more emergent than hierarchical; neither a sonification of a visualized/physical process, nor a live-coding environment. Loom is actually defiantly oblivious to the medium of sight. (I tend to agree with artist Eva Schindling’s criticism of music visualizers; once the visual cortex is engaged, musical listening may suffer.) I’d like to get a little more visual feedback out of those Live knobs, though!

The screencast mainly explains modules and generators, and doesn’t even touch Ruby. (Nor does it include early experiments I have yet to factor into the new alpha architecture, like just intonation or exponential rhythms.) I typically stay in the Live UI while music-making, as dropping into Ruby means engaging a whole abstract-symbol-manipulation brain thing that can stunt my musical thinking. I like this separation: the depth and opacity of code with high-level parameters at the surface. There are many depths to swim at.

But, theory aside, there is something thrilling in the unpredictability of working with Loom. Electronic music made composition more of a textural, quantitative, alchemical process, in which you “mix” amounts of known quantities rather than labor away at the production of each sound. (That said, there’s no substitute for musicianship. I typically turn to physical instruments as tools for thought to work out nascent generative ideas!) Generative music pushes this even further, so that you’re not just mixing volumes or textures, but qualities, characteristics, motifs, as well as their underlying quantities, parameters, gaussian random number distributions. It’s a whole other mode of composition, more like steering than performing any kind of athletic feat, and it brings with it all the attendant dangers of automation. But, so so fun.

Many thanks to Mark Trayle, my advisor for the early life of this project.

In his introduction (posted today in Wired) to the updated edition of West of Eden, a telling of Jobs’ own undoing in ‘85, Frank Rose writes: “Personal computers in the ’80s were exotic beasts, as pregnant with threat as with promise.” This sounds about right. The idea that computers were a life force making designs on humanity enthralled me as a teenager reading Ray Kurzweil, then unnerved me as an undergraduate student of Computer Music. Because, they’re still just a thing we made, right?

Then I came across Derek Powazek’s article about pseudonymity in the place they once called “cyberspace”: “Most communication online [in the mid-'90s] was hidden behind handles, which reinforced the idea that the internet was not ‘real’ in the same way real life was”—finally concluding, “the internet is not a second life anymore, it’s your first one.” Yes, we’ve found much more gratification in online identities as editorializing, rather than fictionalizing, our physical selves.

These two citations from today point to the two “myths of disembodiment” in computing per Jay David Bolter and Diane Gromala, in their excellent Windows and Mirrors, which I draw from in my essay. In the myth of AI (1950s-70s), computers are seen as disembodied minds; in the myth of cyberspace (1980s-90s), computers are viewed as a space for our own disembodied minds. Now, they argue, we live in an era of “embodied computing”.

Since it’s is a fairly rough-hewn text itself, “more about connectivity than a terminus” as my advisor Sara Roberts at CalArts proposed, here are some sparks and teasers on its contents: the eccentric Alan Turing conceiving of computers as “not just a medium but a mind” (Bolter and Gromala) — Sherry Turkle considering why computers are viewed in psychological terms — mid-century structuralist suppositions that human intelligence rests on abstract symbol manipulation — “cyberspace” and computers as a parallel, counter-cultural space — Matthew Kirschenbaum on “programming as world-making” — Donald E. Knuth and “literate programming” — speech act theory and a most-excellent debunking of AI via Terry Winograd and Fernando Flores — Florian Cramer proposing a digital poetry — plus Donald Norman, Nelson Goodman, Adrian Holovaty, and Yukihiro Matsumoto, among others…

Despite the irregular prose and single-sitting writing, I invite you to slog through the unedited “Code: Notes on an expressive, linguistic medium”.

Sound installation for brass slide on guitar strings with computer processing; activated by Arduino, servo, found wood, and a lot of fishing wire.

Shown at the Wave Cave at CalArts, spring 2011.

The “generative” angle came into play in the Arduino code. I felt that letting the brass slide go careening at random times to random positions was quickly exhausting to listen to. So I built a sort of superstructure above the base-level random decisions—a multi-level system which loops random patterns at varying timescales. So, as the slide is sawing along, it will get stuck in little eddies, vortices, in which it repeats a given gesture (be it short or long) a few times before moving on. Then, I think the listener takes it from there, as cognition begins to hear order, making phrases out of the shapes.

UPDATE: new, official video from mono!

The result was SoundAffects, and we’re all quite proud. I was able to bring in my fledgling Ruby-based generative music system for Ableton Live, Loom (still in pre-alpha development at the time), which led to the construction of a larger, more elaborate Live-based system for dynamic input-based composition. And after brainstorming for hours about what effects weather changes and circadian rhythms have on our musical desires, my friend / Tellart employee Jasper Speicher and I had some pretty excellent times fleshing out these ideas on drums and keyboards in the studio.

In fact, you can see that in the behind-the-scenes video, as well as how we explored urban space through amateur surveillance, using cheap USB cameras and San Francisco’s Kearny St as a substitute for 5th Ave.

For more samples of the generated music, the SoundAffects site is still up, and has a really slick archive so you can listen back to highlights from when the experiment was up back in May. All I’ll say about NY last May is, well, I’m glad we factored in precipitation data. !

Screenshot from the beautifully-illustrated “To Have Is to Owe”, from issue 10.

The primary impetus behind the redesign for me is well-expressed in our statement:

Ultimately, we decided to jettison the page entirely, atomizing it into a smaller structural unit, the column. Our hope is to restore some of the sophistication of modern, columnar print layouts while avoiding the pitfalls of slavishly translating that trope to the markedly different environment of the screen. This design will facilitate visual narratives that are seamless, yet allow audiovisual objects to maintain their own internal organization… are multidirectional yet propel the reader along a discrete horizon.

The design is, in and of itself, an “argument for the high-resolution reading experience” online. The editors have done a lovely job with the new tools in Issue #10, and I’m sure it’s only going to get better from here.

The engine I developed to do all that in-browser layout magic is about 500 lines of JavaScript called Horizonize, which I’m pleased to announce I’ve just open-sourced on GitHub. You can set it up, and then feed it standard HTML—with some special cues here and there—and out come side-scrolling layouts. I look forward to seeing what brave organizations find a use for this tool on their own sites!

Water is pumped through the tubing against the metal resonators, where contact mics catch the sound for amplification through the amp; meanwhile, a motor randomly and gingerly plucks two guitar strings, also amplified.

Read it »

The Layout Generator is an automated system to turn blog-like, long-scrolling-column HTML into rich, print-ready PDFs with more a varied visual depth and flow. The actual forms of the layouts—where images appear, how they interact with the running flow of text, etc.—are determined by a combination of some simple heuristics along with markup cues which enable non-designers to create compelling layouts. Here’s a short example article which demonstrates just a few of the typographic and layout possibilities.

e-flux’s booth at Printed Matter’s NY ART BOOK FAIR, October 2009. The poster-sized layouts were designed by hand, but the journals themselves were largely computer-generated.

This project proposed a solid methodology for generative work (be it in design, music, etc.): first take a hand-crafted sample artifact (in this case, Jeff Ramsey’s designs); analyze and articulate the intuitive process behind it; then write that out in systemic code. We had to tease out Jeff’s aesthetic decision-making process in meetings in order to systemize it. Yes, it feels a bit like the Taylorization of intellectual production: you must train the robot that will replace you (or at least train the engineer building it).

In its current state, the system is about 1k lines of Ruby, using Prawn for PDF writing, along with a whole slew of other RubyGems from Hpricot to Sinatra. I’m hoping to grow this as we implement more of Jeff’s designs and possibly concoct a few new ones by accident.

Thanks go to Anton Vidokle and Brian Kuan Wood at e-flux for making this work possible.

It seems like browsers could afford a much richer visual experience. I looked to a proven technology from antiquity for a metaphor: the codex book. The book, like the scroll before it, works because you trace a linear path through information, with the past/read information on one side, and the future/unread information on the other.

I ran with that in a quick prototype (built in jQuery) with zazzy animations for the reader’s progress through information. (I didn’t hang onto any romanticized page-turning metaphor like we did with Triple Canopy; instead, I wanted to keep multiple documents, or at least excerpts from each, visible in a single glance. The more useful context, the better!) In this video demo, footnotes & external links appear consistently in the right column with contextualizing text; the reader clicks a few, then navigates back to a previously-read text by clicking on it in the left column, where it has been shifted.

This way, each link click wouldn’t just wheat-paste over your reading context, but reinforce the greater context of your reading narrative in a unified, designed way. At any moment, you have deeper visibility into what you’ve read (how you got here) and what further related content exists (where you might go). Although we can, practically speaking, only read one document at a time, I think our spatial memory is capable of holding onto a lot more—and our graphics cards are certainly up to it!

This prototype was prepared as part of a unique presentation last month to Scott Sassa and Neeraj Khemlani at the Hearst Corporation, through Louise Sandhaus‘ class Mutant Design: The Future of Publications. Many thanks to all of them for enabling this work. The full PDF of my presentation is available, although I’ve changed some of the wording & thinking since I wrote it.

DOWNLOAD INSTRUCTIONS PDF

These four recordings come from a run-through in Sara Roberts’ Media Theory class, for which the piece was written. Our haphazard 12-piece ensemble was: guitar, banjo, violin, viola, double bass, 2 sopranos, shakuhachi, laptop, melodica, piano, percussion. Each recording began with the (randomly-selected) ‘texture card’ pictured.

This is definitely a take-home game. You can use texture cards like the ones I prepared above or create/google your own. If you do get a group together to play, please share the recordings!!