In the 5th grade, I devised my own language; for I could think of no better or more fascinating challenge. For my Master’s thesis at the Interactive Telecommunications Program at NYU, I could think of no better or more fascinating challenge than to invent language-inventing machines. What resulted was The Dawn Chorus, a group of electronic sculptures that do just this—emergently, and autonomously. Using a suite of custom hardware and a simple neural network for each device, the individuals learn from each other and are thus able to converge on common, conventional “words” for their experiences, such as a flash of light or sudden noise.

Seem them in action:

My final thesis presentation:

All the technical details are in the thesis paper!

This Dawn Chorus has gradually and emergently developed its own song-language, a metastructure that was never designed into any of the Chorus’s individual members.  Each member of the Chorus has the ability to sense the light and sound of its environment and the ability to vocalize—as well as the ability to learn from the actions of its neighbors.  In this way, the Chorus is able to naturally develop its own set of linguistic, musical conventions, entirely independently.

These sculptures contain a compact set of electronics comprising a microcontroller, sound and light sensors, a hybrid digital-analog synthesizer of my own design, a radio module and a speaker.  Each unit uses a simple neural network learning algorithm to associate environmental events and the actions of its milieu with generative sound patterns.  In short, the entities that make up the Dawn Chorus learn to talk to each other.

But what about?  The Dawn Chorus is not meant as an attempt to mimic animal behavior, but rather as an experiment in emergent poetics.  I see the songs of the Chorus as more musical than communicative, as more of a series of conversational poems than a survival strategy.  In time, we humans may even learn their language—but they will continue to enjoy it on their own.

Sketches of possible sculptural forms (these are roughly 8-10″ in height).  The first four are my favorites:

Sound prototyping: A short composition of 4-7 sine wave oscillators harmonizing and disharmonizing:

Two videos of synth design approaches: 8-bit R2R DAC and Schmidt trigger oscillator.

EggBeater uses the intuitive power of rhythm to let anyone control the playback of music. Shaking this small, wireless device in regular patterns can automatically adjust the tempo and timing of loops. Just start playing the EggBeater just as you would a traditional shaker, and listen as the song slows down as you slow down, or speed up as you do!

EggBeater uses an accelerometer coupled with an XBee radio to send your movements to PureData, where they detect your downbeats and rhythmic tempo. The software can then control playback within PureData or send OSC or MIDI messages to other platforms.

Here’s a short video courtesy of Lee-Sean:

BubbleViz is an internet-enabled toy bubble gun that uses an Arduino, XBee and ConnectPort to poll a PHP script and check an IMAP server for new messages, upon which the device showers you with a delightful cascade of bubbles.

View it in action:

More information, pictures and code follow!

As you can see, this project more than meets the “looks like a bomb” quota.

Here’s my relevant PHP code (using my modified Email to DB class):

include_once("class.emailtodb_tedb0t.php");
 
$edb = new EMAIL_TO_DB();
$edb->connect('imap.gmail.com:993', '/imap/ssl/novalidate-cert', 'username@gmail.com', 'password');
echo $edb->getNumNewMessages();
$edb->close();

Here’s the pertinent function from the classfile.  I’m certain there is a better, more logical way to do this.  We can’t use PHP Sessions because Rob’s ConnectPort server doesn’t implement them… maybe I will add that in

function getNumNewMessages(){
	$myFile = "numMessages.txt";
	$fh = fopen($myFile, 'r');
	$lastNum = intval(trim(fgets($fh)));
	$currentNumMessages = $this->num_message();
	fclose($fh);
	$fh = fopen($myFile, 'w+');
	fwrite($fh, $currentNumMessages);
	fclose($fh);
 
	return $currentNumMessages - $lastNum;
}

And the Arduino code:

// BubbleViz
//   Bubble Gun Controller
//   by Ted Hayes <www.liminastudio.com>
 
#include 
 
#define PIN_SWITCH 4
#define PIN_MOTOR 12
#define DUR_MAX 3000
 
Servo myservo;
int lastRead = HIGH;
long i = 0;
int dur = 1000;
int durMult = 500;    // ms per new emails
int curByte;
int curVal;
int curDur;
 
void setup()
{
  Serial.begin(115200);
  pinMode(PIN_SWITCH, INPUT);
  pinMode(PIN_MOTOR, OUTPUT);
  digitalWrite(PIN_SWITCH, HIGH);  // set pullup on pin 2
  digitalWrite(PIN_MOTOR, LOW);    // motor off
 
  //// software serial
  pinMode(rx,INPUT);
  pinMode(tx,OUTPUT);
  digitalWrite(tx,HIGH);
 
  myservo.attach(9);  // attaches the servo on pin 9 to the servo object
  myservo.write(80);  // reset to "closed" position
}
 
void loop()
{
  int switchRead = digitalRead(PIN_SWITCH);
  if(switchRead == LOW && lastRead == HIGH){
    fire(dur);
  }
  lastRead = switchRead;
 
  if (Serial.available() > 0) {
    curByte = Serial.read();
    curVal = curByte - 48;    // "decodes" ASCII number to DEC number
 
    if(curVal > 0){
      curDur = durMult * curVal;    // ms * num of new emails
      if(curDur > DUR_MAX) curDur = DUR_MAX;
      fire(curDur);
    }
  }
 
  // poll web script every n iterations of loop (as opposed to using delay())
  if(i > 300000){
    Serial.print("http://verge.myftp.org:81/Listereen/emailtodb_v0/check_mail.php");
    i=0;
  }
  i++;
}
 
void fire(int duration){
  // trigger gun motor
  digitalWrite(PIN_MOTOR, HIGH);
  delay(300);               // wait for loop to wet
  myservo.write(105);          // pull loop out
  delay(duration);                 // wait for bubbles to blow
  // reset motor and servo
  digitalWrite(PIN_MOTOR, LOW);
  myservo.write(80);
}

“TwelveStep” is the working title for a wireless dodecahedral step-sequencer and MIDI controller.

Here’s a brief overview of my thought process so far for my NetObjx final.I initially was interested in the Telepathic Mesh Society concept:

But I have been in collaboration with Adam Harvey on a blog-data-auralization installation art piece:

Here are some common AT commands I have most frequently while working on XBee projects.  Remember, ALL NUMBER VALUES ARE IN HEX!

• ATID – PAN ID
• ATDH/DL – Destination address high/low: which local address to send ADC data
• ATMY – Local address of this unit
• I/O:
  • ATD0…D8 – digital pins I/O enable
  • ATP0…P1 – PWM output enable
  • ATIU – I/O output enable (UART) – send out through serial
  • ATIA – I/O input address – address to pay attention to for serial output
  • ATIT – ADC Samples before TX
  • ATIR – ADC Sample rate (ms)
  • ATBD – Baud Rate:
    • 3 = 9600
    • 4 = 19200
    • 5 = 38400
    • 6 = 57600
    • 7 = 115200
• ATWR – Write parameter values

Remember to ATWR to save changes to parameters.

Interesting note from the manual: “The maximum sample rate that can be achieved while using one A/D line is 1 sample/ms or 1 KHz (Note that the modem will not be able to keep up with transmission when IR & IT are equal to “1”).”

Using a function generator, a busted speaker and a mic, I recorded this drone piece, “You Were Alive:”

Here’s a making-of video that should illuminate the process:

Here are some sketches for Pixels, Pachyderms and Proselytizing Pixel-Pushers!

Sketch 1: Depicting the “Orbit” watch, Subdermal Display (one day!), and Stopwatch style orbiting rings.

Depicting a choker that changes color according to the time of day, a “Spectrum Clock.”

Details of “Orbit” style wristwatch, which is worn like a loose bangle around the forearm.  A measuring-tape-like ring orbits around your wrist.  SEEKING PATENT!  (Disregard unrelated but also awesome dodecahedron sequencer design in top left)

Cross-section of “Orbit” design.  Very simple!