Group Members:
Vivian Liu
Elena Duran
Andrew Chong
For our cuckoo clock, we built a simple reverse linkage to translate the backwards motion into sending the bird forward. Vivian used the laser-cutter to get the wood in the dimensions we needed, and drilled holes to put in the pivots. We attached it to the box by threading through the hole with chopsticks. We also added chopsticks (and pipe-cleaners) so that the linkage didn’t shift up, down, left or right too much.
We played around with different configurations until we decided to flip the box around and have the door open outwards and up (kind of like a pen where you would keep an animal). We had a lot of difficulty translating the backwards motion into the more diagonal motion of the string, and added little wheel pulleys along the top to help adjust the direction and motion of the string to the right angle.
We also experimented with rubber bands to have the clock “reset” itself automatically, so that our pulling motion would create tension in the rubber band, returning to its original position once we let go.
For my crawler, I originally created a wheel with many “knees” so that only the forward option would bite into the ground, with the backward motion only causing each joint to close. However, it only went around in circles.
To get around that, I tried playing around with the design so that motion would be forward and backwards. I tried making a larger forward leg to see if the uneven legs would cause the crawler to move forward if a peg attached to the motor struck the forward and backward leg.
Then I tried placing a large rubber band across the center and have the motor tense and relax the tension of the rubber band. Unfortunately I did not succeed – the crawler stands motionless.
I liked the immersive aspect, especially when I was able to view my controls or use different kinds of actuation. I also liked when I was able to see parts of myself embodied (such as my hands).
I was hyper-aware that I was in a created space. This was limiting in some way. It was like a copy of reality, which felt like reality but lacked many of its affordances. The ability to navigate the space was limited by the controls, and how the creators had designed the space. It was a little surreal and created a bit of dissonance, like my experience/ability to observe was limited by the creators of the space.
One way to exploit the embodied aspect is to add additional position sensors to your face, hands, and body, and have those map to embodied selves in the 3D context. The environment can have different opportunities to reflect those embodied selves – the view of your hands, reflection in a pool, appearance in a mirror, people’s reactions to you. I think Kafka’s The Metamorphosis and movies like District 9 would be very visceral if individuals could play those characters.
To overcome the sense of being in a created environment, virtual spaces could be constructed either entirely or relying heavily on direct mappings from physical inputs, etc. 3D cameras (though mapping these inputs into vector or 3-D objects might not exist or be advanced enough). Like navigating different real-world environments – in the deep ocean, open grasslands, different architectural landmarks etc.
I was looking around for a thoughtless act and found one right in my backpack. You can see that the notebook, which I take notes in for lectures, has been “thoughtlessly” used to put together in one place papers related to that class and to protect the papers. The practice is not particularly effective as the stack has become rather thick, the edges of the paper are frayed, and the papers don’t look particularly well-organized.
What might an effective solution look like? We see some notebooks with paper pockets for notes, a more extreme solution might be to attach fold-out flaps to the last page or an actual envelope type folder to the back of the notebook to tuck notes into the back. Alternately, to make the notebook significantly larger than A4 would also help protect the edges of the page.
A way to compel the user *not* to commit the thoughtless act might be to make the notebook too small to wedge papers in, forcing the user to bring an extra folder.
For this week, I assembled together a distance sensor with a NeoPixel light display, with the lights extending out the closer the user had his hand to the distance sensor.
I wanted to start working with some equipment/code we used for past projects, in preparation for the final project for TUI.
I wanted to try connecting output through the Serial Monitor to Ableton Live, but couldn’t get that piece to work just yet. I attached the Piezo speaker to make a clicking sound every time an object (etc. my hand) was detected.
/*
HC-SR04 Ping distance sensor]
VCC to arduino 5v GND to arduino GND
Echo to Arduino pin 13 Trig to Arduino pin 12
Red POS to Arduino pin 11
Green POS to Arduino pin 10
560 ohm resistor to both LED NEG and GRD power rail
More info at: http://goo.gl/kJ8Gl
Original code improvements to the Ping sketch sourced from Trollmaker.com
Some code and wiring inspired by http://en.wikiversity.org/wiki/User:Dstaub/robotcar
*/
#include
#define trigPin 13
#define echoPin 12
int speakerPin = 7;
int velocity = 100;//velocity of MIDI notes, must be between 0 and 127
//higher velocity usually makes MIDI instruments louder
int noteON = 144;//144 = 10010000 in binary, note on command
int noteOFF = 128;//128 = 10000000 in binary, note off command
#define PIN 6
// Parameter 1 = number of pixels in strip
// Parameter 2 = pin number (most are valid)
// Parameter 3 = pixel type flags, add together as needed:
// NEO_KHZ800 800 KHz bitstream (most NeoPixel products w/WS2812 LEDs)
// NEO_KHZ400 400 KHz (classic 'v1' (not v2) FLORA pixels, WS2811 drivers)
// NEO_GRB Pixels are wired for GRB bitstream (most NeoPixel products)
// NEO_RGB Pixels are wired for RGB bitstream (v1 FLORA pixels, not v2)
Adafruit_NeoPixel strip = Adafruit_NeoPixel(60, PIN, NEO_GRB + NEO_KHZ800);
void setup() {
Serial.begin (31250);
pinMode(speakerPin, OUTPUT);
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
strip.begin();
strip.show(); // Initialize all pixels to 'off'
}
//send MIDI message
void MIDImessage(int command, int MIDInote, int MIDIvelocity) {
Serial.write(command);//send note on or note off command
Serial.write(MIDInote);//send pitch data
Serial.write(MIDIvelocity);//send velocity data
}
void loop() {
long duration, distance;
digitalWrite(trigPin, LOW); // Added this line
delayMicroseconds(2); // Added this line
digitalWrite(trigPin, HIGH);
// delayMicroseconds(1000); - Removed this line
delayMicroseconds(10); // Added this line
digitalWrite(trigPin, LOW);
duration = pulseIn(echoPin, HIGH);
distance = (duration/2) / 29.1;
delay(100);
if (distance < 43) { // This is where the LED On/Off happens
long k = distance % 7;
long k2 = 6-k;
for( int i=0; i<5; i++ ) { // play it for 50 cycles
digitalWrite(speakerPin, HIGH);
delayMicroseconds(1);
digitalWrite(speakerPin, LOW);
delayMicroseconds(1);
}
int j;
//Serial.println("on stage");
for(j=0;jk-1;j--){
//Serial.print("off: ");
//Serial.println(j);
strip.setPixelColor(j, 0,0,0);
strip.show();
}
}
else {
//Serial.println("external off stage");
int j;
for(j=0;j<7;j++){
//Serial.print("off: ");
//Serial.println(j);
strip.setPixelColor(j, 0,0,0);
strip.show();
}
};
}
Experience 46
Andrew Chong, Vivian Liu & Owen Hsiao
Brief Concept:
Our midterm project proposal is an immersive installation piece exploring the following characteristics:
Altering specific musical traits to intensify their capacity to induce chills/elicit an emotional response
Eliciting agency and involvement of the “audience”, who becomes an active participant in the experience
Minimal “cool” media that is fully engaging but open to perceptual interpretation
After feedback and discussion, we modified our base case to maximize the chances of an engaging interaction, by more actively enrolling the visual element.
Unexpected Concord:
Across the previous research cited in our earlier post, the ability of music to evoke chills or a strong emotional response typically relied on unexpected concord. The time frame, dimensions or network of elements brought into play can all differ. As an analogy, harmony relates to the “vertical” aspect of music, as opposed to melody, which is over time. Unexpected concord can both occur on both dimensions, with composers often enrolling more than one at the same time.
For instance, in Andrew Bird’s piece, the interplay of ambient, “un-music-like” birdsong, heavy strings and a high-pitch whale-like instrument employed what we called “spectral complexity” – utilizing different parts of the spectrum, including both pitch and texture, at each time.
That spectral blending is visible from the spectrogram, with the bird song at the start of the piece as a scattered haze, and a “spectral gap” opening up between the heavy strings and high-pitch melody.
In Samuel Barber’s Adagio for Strings, the unexpected concord reflects itself as the resolution of drawn-out tension, where the time dimension comes much more heavily into play. Dissonance through the first five minutes of the piece builds, with a sense of coming resolution, continually drawn out, until a pause and resolution coming after a last phase of continuous rising dissonance, between 5.30 and 5.54. This gap is also clearly visible on the diagram.
https://www.youtube.com/watch?v=KylMqxLzNGo
Enlisting the visual dimension:
We decided to enlist the visual element as a way to maximize the chances of an engaging interaction for the participant. In our previous examples, unexpected concord occurred only in the audio dimension, though by utilizing both harmonic and melodic elements, and combining the pitch, texture and other elements of the music, composers were able to shape and intensify the experience of unexpected concord. In our installation, we wanted to enlist the participant and his actions as a way to create unexpected concord in both the visual and audio dimensions.
Specifically, where the participant moves to specific locations in the space, the visualization projected on each side of the wall, at first only peripherally related to the music, undergo a marked increase in resonance with the music. The result should both be suggested but somewhat unexpected, in keeping with our interest in using hidden affordances to engender exploration and curiosity for the participant.
We explain the interaction in more detail below. We decided on our base case as a canvas of the key elements we wanted to implement and put in place, that we could then iterate, experiment and improve on by swapping out algorithms, changing light effects, moving actuators, and modifying the participant experience in other ways.
Interaction Walkthrough
The participant enters a dark, enclosed room alone. Faint lights signal hidden affordances. After a lull, Andrew Bird’s Yawny at the Apocalypse begins to play.
Movement towards each general direction induces at first subtle changes in the visualization/music. For instance, the participant’s position would subtly alter the relative loudness of different voices in the piece, as well as the transformations of the music into visualizations on each screen.
This interaction is meant to “coax” each participant towards three specific sites in the room, each of which will induce a different kind of “resonant” visualization with the music. At these specific sites, this “resonant” interaction is activated, with a marked increase in the stimulation and intensity of the visualization.
The entire experience might last several minutes, as the participant moves around the space to explore the specific affordances. While we designed the experience as primarily a solitary experience, it is interesting to explore how participants might interact with more than one person in the room, since actions are “shared” within the installation, with collaborative action required to engender specific effects. Exploration is likewise shared, but where discoveries may not be isolated and remain somewhat mysterious.
In terms of bodily experience – the participant is limited to the constraints of his own dimensions and position – in a smaller installation, he might be able to activate multiple effects alone, without the need to transition or “lose” any aspect of the experience.
As a solitary experience, a sense of tension/dissonance and discovery is accentuated by the “loss” of interactions as they move within the room. As a shared experience, other individuals might then become an avenue for each participant to reclaim that lost experience, with each participant, of course, also playing that role for other individuals in the room.
Such an interaction would be especially interesting to construct in a public space.
Other thoughts:
Yawny’s musical structure is more time-dependent than other pieces, which may detract from the visual interaction we’ve incorporated in the experience. Participants may feel like they’ve missed some aspect of the experience if their exploration of the room does not line up with the specific points in the piece where the composer has set up different forms of resolution.
One way to get around this is to choose pieces with still evocative, “unexpected concord”, but with a far more regular structure. Arvo Part’s Spiegel im Spiegel might be a good choice for this, due to the regularity of its musical structure, which can also be seen in the spectrogram.
I enjoyed the description of de Certeau’s characterization of consumption as “not merely empty or passive”, but “contain[ing] elements of user resistance – nonconformist, adaptive, appropriative, or otherwise transgressive tactics” – that becomes “creative acts of their own fashioning.” “By locating such creativity in the user and beyond the conventional role assumed by the designer, de Certeau opens the possibilities of a design attuned to its use, context, and life rather than only its material quality, prescribed functionality, or formal expression.”
I’m interested in objects that ask or lend themselves to such activities, where the designer incorporates flexible affordances so that user acts of adaptation need not be a form of appropriation or transgression. I found it interesting to try to categorize objects according to where they fell on this spectrum, and whether that affected our initial impressions about such objects. Blauvelt describes de Certeau as shifting our attention to “acts of consumption, or use”, away from the historical preoccupation with the means of production, but it is interesting to consider objects that exhibit these co-opting characteristics both in use AND the means of production, etc. in assembly or manufacture. A product, such as a bicycle, a home appliance or a tool, for example, can designed to be fashioned using proprietary parts that can only be contained by one source, or be built in a way such that spare parts are easy to find, modifications/customizations are easy/encouraged and manufacturing/assembly can be easily transplanted to countries adopting the technology or use.
On a slightly different tangent, I’ve been moving homes recently, at different times requiring the purchase of new cookware/utensils. This reminds me of both de Certeau’s “participatory” consumption and the “familiarity of the strange.” A core set of 4-5 utensils lent themselves to a new variety of uses, allowing themselves to be co-opted in different ways or a different variety of dimensions. In a way, the absence of each utensil constrained me and made me more aware of their “absorption into the familiar” – I did not notice their lack until they were absent, and as I purchased each additional tool, the range of preparations expanded significantly. Some tools were clearly more versatile than others, with a sense of diminishing returns as more specialized tools were purchased.
The most versatile “core” tools were:
a small pot
a frying pan
a chef’s knife
a bamboo cutting board
bamboo spoon
Subsequent tools that expanded the affordances and cooking techniques were
a large cover for a pan or pot
a deep pan that could be used as a large pot or stir-fries
a simple strainer
a rice cooker (etc)
a toaster
It was surprising to me what a large variety of food that could be prepared with the set of “core” tools, which changed as I went from one to two tools, two to three tools, etc, and different ideas that occurred to me as I obtained additional tools (making pasta with vegetables where I could only make instant noodles before, stir-fries from only fried eggs before, making broth soups with a rice cooker), that were not possible before.
The strange was accentuated for me in an interesting way with their absence – the removal of skins and division into cookable pieces with a tool that allowed the focusing of force, containers that allowed the trapping of water or oil for heating that could be used to transform foodstuffs in very specific ways, flat surfaces (a cover) that could trap moisture to allow different kinds of cooking/heating (sunny-side eggs, slow-boiled soups, vegetables that needed more intense heat). The act of toasting seemed a strange act/ritual that opened up a very specific area of food that other products lent themselves to (fruit toasties, sliced bread), as well as the specific affordances of bread you sliced at home, to divide a loaf into flat sections so you could apply intense heat in both directions in a quick manner to produce crispness and caramelization.
My lab project this week consists of a squeezable, glowing orb and a fractal generator. The results are a bit kooky. I reused a soft silicon casing we created for a past project and placed the pressure sensor between the soft casing and the hard plastic kernel.
For my visualization, I played around with the Mandelbrot sketch example in Processing, making it into a sequential, frame-by-frame visualization rather than a static image. I accelerated the fractal growth with greater pressure on the orb, but as that was hard to see, I also changed the background color slightly to show when pressure was applied. The next step might be to try out algorithmic visualizations that are not fractals, as later evolution is hard to see as the fractals get smaller.
Arduino Code:
/*
* Andrew Chong
* Alternating RGB Fade with Potentiometers
* Modified from Clay Shirky <clay.shirky@nyu.edu>
*/
// Output
int redPin = 9; // Red LED, connected to digital pin 9
int greenPin = 10; // Green LED, connected to digital pin 10
int bluePin = 11; // Blue LED, connected to digital pin 11
float red_direction = 1;
float blue_direction = 1;
float green_direction = 1;
float pressure;
// brightness
float blue_brightness = 0;
float red_brightness = 0;
float green_brightness = 0;
// Program variables
float redVal = 255; // Variables to store the values to send to the pins
float greenVal = 255;
float blueVal = 0;
int i = 0; // Loop counter
int j = 0; // display counter
int wait = 500; // 50ms (.05 second) delay; shorten for faster fades
int DEBUG = 1; // DEBUG counter; if set to 1, will write values back via serial
void setup()
{
pinMode(redPin, OUTPUT); // sets the pins as output
pinMode(greenPin, OUTPUT);
pinMode(bluePin, OUTPUT); // If we want to see the pin values for debugging...
Serial.begin(9600); // ...set up the serial ouput on 0004 style
}
// Main program
void loop()
{
//blue_brightness = analogRead(A0)*3/(float)1023;
//red_brightness = analogRead(A5)*3/(float)1023;
//green_brightness = analogRead(A1)*3/(float)1023;
i += 5; // Increment counter
j += 1;
delay(wait);
Serial.println(int(analogRead(A0)/100));
pressure = analogRead(A0);
greenVal = (pressure/1000)*255;
//Serial.println("red:");
//Serial.println("red:");
//Serial.println(greenVal);
analogWrite(greenPin, greenVal);
/*
if (DEBUG) { // If we want to read the output
DEBUG += 1; // Increment the DEBUG counter
if (DEBUG > 20) // Print every 10 loops
{
DEBUG = 1; // Reset the counter
Serial.print(i); // Serial commands in 0004 style
Serial.print("\t"); // Print a tab
Serial.print("R:"); // Indicate that output is red value
Serial.print(redVal); // Print red value
Serial.print("\t"); // Print a tab
Serial.print("G:"); // Repeat for green and blue...
Serial.print(greenVal);
Serial.print("\t");
Serial.print("B:");
Serial.println(blueVal); // println, to end with a carriage return
}
}
*/
//delay(wait); // Pause for 'wait' milliseconds before resuming the loop
}
Processing code:
/**
* The Mandelbrot Set
* by Daniel Shiffman.
*
* Simple rendering of the Mandelbrot set.
*/
import processing.serial.*;
// Change this to the portname your Arduino board
String portname = "/dev/cu.usbmodem1421"; // or "COM5"
Serial port;
String buf="";
int cr = 13; // ASCII return == 13
int lf = 10; // ASCII linefeed == 10
int serialVal = 0;
int currentVal = 10;
int r = 106;
int g = 0;
int b = 209;
void setup() {
size(1200,800);
frameRate(2); // Animate slowly
port = new Serial(this, portname, 9600);
}
void draw() {
// Establish a range of values on the complex plane
// A different range will allow us to "zoom" in or out on the fractal
// It all starts with the width, try higher or lower values
float w = 4;
float h = (w * height) / width;
// Start at negative half the width and height
float xmin = -w/2;
float ymin = -h/2;
// Make sure we can write to the pixels[] array.
// Only need to do this once since we don't do any other drawing.
loadPixels();
currentVal = currentVal + 1 + serialVal;
// Maximum number of iterations for each point on the complex plane
int maxiterations = currentVal ;
// x goes from xmin to xmax
float xmax = xmin + w;
// y goes from ymin to ymax
float ymax = ymin + h;
// Calculate amount we increment x,y for each pixel
float dx = (xmax - xmin)/(width);
float dy = (ymax - ymin)/(height);
// Start y
float y = ymin;
for (int j = 0; j < height; j++) {
// Start x
float x = xmin;
for (int i = 0; i < width; i++) {
// Now we test, as we iterate z = z^2 + cm does z tend towards infinity?
float a = x;
float b = y;
int n = 0;
while (n < currentVal) { float aa = a * a; float bb = b * b; float twoab = 2.0 * a * b; a = aa - bb + x; b = twoab + y; // Infinty in our finite world is simple, let's just consider it 16 if (dist(aa, bb, 0, 0) > 16) {
break; // Bail
}
n++;
}
// We color each pixel based on how long it takes to get to infinity
// If we never got there, let's pick the color black
if (n == maxiterations) {
pixels[i+j*width] = color(0);
} else {
// Gosh, we could make fancy colors here if we wanted
float norm = map(n, 0, maxiterations, 0, 1);
/*
if (r>254) {
r = 0;
}
if (g>254) {
g = 0;
}
if (b>254) {
b = 0;
}
*/
//r = r + 1;
//b = b + 1;
//g = g +1;
pixels[i+j*width] = color(0,g,b);
}
x += dx;
}
y += dy;
}
updatePixels();
}
// called whenever serial data arrives
void serialEvent(Serial p) {
int c = port.read();
if (c != lf && c != cr) {
buf += char(c);
}
if (c == lf) {
serialVal = int(buf);
println("val="+serialVal);
buf = "";
}
if (serialVal>5){
//r = int(random(255));
if (g==0) {
g = 20;
}
else if (g==20){
g = 0;
}
print(g);
//b = int(random(255));
//print(b);
}
}
I’m interested in what kind of information could be unobtrusively provided in an urban environment, without being distracting for drivers/residents, whether as installed at bus stops, on lamp posts or elsewhere in the urban environment.
One idea I would love to see is for a column to light up in a muted fashion whenever a bus arrived, perhaps having different colors for different bus lines. This is especially useful if buses need to be flagged down and do not stop at every stop. I find myself having to look up from my reading/phone, or any kind of work, while waiting for buses, preventing me from focusing on anything in case I end up missing my bus.
Effectively that waiting time becomes dead time (usually twenty minutes or so for me), where I’m unable to focus and constantly distracted by needing to look up to see if my bus is arriving. Even a simple installation that lights up with one light of the same color whenever any bus arrives would allow me to get much more done. Attaching a generic sensor/signal to each bus stop and bus would enable this. Because the column is large enough, I would be able to notice it even without looking directly at it – I can notice it peripherally even as I’m reading my book/talking.
Similarly, I’m often unsure whether I need to run for my bus, esp. since the various apps I use are often inaccurate. The same technology could use a different light (color, half lit up, etc) to indicate buses arriving in 5 minutes (etc). This would signal whether I need to run; for people running errands or deciding whether to go home/get food/run errands, it can unobtrusively provide a signal of what would be most time-efficient for them to do. I might prefer to go home right away then buy take-out (etc) if my bus is arriving soon.
The same technology could be used for the arrival of trains, it is similarly fascinating to think of other information that can be unobtrusively communicated in an urban context that would be useful to commuters, citizens, residents, pedestrians, individuals etc…
Experience 46
Andrew Chong, Vivian Liu & Owen Hsiao
Brief Concept:
Our midterm project proposal is an immersive installation piece exploring the following characteristics:
Altering specific musical traits to intensify their capacity to induce chills/elicit an emotional response
Eliciting agency and involvement of the “audience”, who becomes an active participant in the experience
Minimal “cool” media that is fully engaging but open to perceptual interpretation
One version of the experience is as follows.
The participant enters a dark, enclosed room alone. Faint lights signal hidden affordances. After a lull, Andrew Bird’s Yawny at the Apocalypse begins to play.
Movement towards each general direction induces at first subtle changes in the light/music. For instance, the participant’s position would alter the relative loudness of different voices in the piece. Stepping deeper into the room would intensify the volume (and perhaps clarity) of heavy strings in the piece, with birdsong and light strings fading somewhat, so the participant has the experience of being inside an environmental performance of the music.
Participants can thus “play” with the music by altering their position and exploring hidden affordances within the room.
The minimal conception is as above. Other planned/potential variations include:
Utilizing the full capabilities of music software to create other effects that tie directly (sudden shifts in “volume, timbre or harmonic pattern” [1]), or tangentially (enhancing reverberation, altering beat) to eliciting chills
Using biosensors (etc. empatica watches measuring galvanic skin response, blood volume pulse, heart rate, heart rate variability) to measure effects of different variations/experimental arms, as well as a potential input into the environment
Background:
There has been some work on the specific musical traits that tend to induce a strong emotional response in the listener. One researcher, Martin Guhn, a psychologist who has run experiments with different musical pieces, provides an analysis of these traits. These pieces:
“began softly and then suddenly became loud”
“included the abrupt entrance of a new “voice”, either a new instrument or harmony”
“they often involved an expansion of the frequencies played”
“Finally, all the passages contained unexpected deviations in the melody or the harmony.”
In short, music is “most likely to tingle the spine… when it includes surprises in volume, timbre and harmonic pattern.” [1] We chose Andrew Bird’s Yawny at the Apocalypse as it contains all or most of these traits, but wanted to explore whether some kind of agency on the part of the listener, typically absent in most music and art (which tends to be one-directional), can evoke a more intense emotional experience.
Below is a spectrogram of Andrew Bird’s piece.
Some of the traits described by Guhn are visible here. One way of approaching the analysis is the unexpected resolution of dissonance, or unexpected concord between dissimilar objects, which includes what I describe as “spectral complexity.” Bird’s piece begins with bird-song at a high spectrum, before deep, lower-frequency saturation of sound is introduced around 0:10, until a shrill pitch (rather like whale-song) is introduced the 0.42 mark. A spectral band or gap is clearly visible stretching throughout the song after the high pitch is introduced. Each of these voices are highly dissimilar but produce unexpected concord.
Other possible pieces (with different corresponding actuation) are below. Many of these pieces display similar traits or interesting variations on Bird’s piece.
