What’s your fortune?

Description:
In this one, I tried to create a pulsating effect (as in a fortune-teller’s crystal orb) with a ping-pong ball as a diffuser. I wanted to vary the heart-rate, the baseline intensity and the amplitude of the heartbeat using the potentiometers. I also had to be mindful of how bright my blue LED. One important lesson was to write the code from start to finish even if some of the code can be reused from another source on the internet.

Components
Cotton/wool
1 ping pong ball
3 potentiometers
3 LEDs (Red, Green and Blue)
3 Resistors (220 ohms each)
Arduino UNO
Breadboard and cables

Code

int heartRatePin = A0; // this controls tempo/delay
int heartRate = 0;

int beatWidthPin = A1; // this controls amplitude of the beat
int beatWidth = 0;

int intensityPin = A2; // this controls baseline brightness around which the amplitude is then set
int intensity = 0;

int crest = 0;
int trough = 0;

// initializing the LEDs
int ledPinGreen = 9;
int ledPinBlue = 10;
int ledPinRed = 11;

void setup(){
// declaring pins as OUTPUTS
pinMode(ledPinGreen, OUTPUT);
pinMode(ledPinBlue, OUTPUT);
pinMode(ledPinRed, OUTPUT);

Serial.begin(9600);
}

void driveColor(int pinColor, int c, int t, int hr){
// this will be the cookie-cutter function for starting LEDs and then fading LEDs
int startLED = t;
// int difference = (c-t)/20;
for (startLED = t; startLED <= c; startLED +=30){ analogWrite(pinColor, startLED); delay(90); } delay(hr); for (startLED = c; startLED >= t; startLED -=30){
analogWrite(pinColor, startLED);
delay(90);
}

// analogWrite(pinColor, c); // don't launch this yet.
// Serial.println("LED brightness value (crest): ");
// Serial.println(c);
// Serial.println("Heart Rate: ");
// Serial.println(hr);
// delay(hr);
// analogWrite(pinColor, t);
// Serial.println("LED brightness value (trough): ");
// Serial.println(t);
// Serial.println("\n");
// delay(hr);
}

void loop(){
intensity = analogRead(intensityPin);
intensity = intensity/4; // intensity can now take values from 0 to 255
beatWidth = analogRead(beatWidthPin);
beatWidth = beatWidth/8; // reducing the domain for amplitudes
// beatWidth can now only take values from 0 to ~32

heartRate = analogRead(heartRatePin);
heartRate = 2*heartRate;
// this will be used across the LEDs to inform delay

Serial.println("Intensity: ");
Serial.println(intensity);
Serial.println("Beat width: ");
Serial.println(beatWidth);
// Serial.println("Heart Rate: ");
// Serial.println(heartRate);
// Serial.println("--------------");

crest = intensity + beatWidth;
trough = intensity - beatWidth;

// setting the wave limits
// edge case truth table:
// when both intensity and beat are MAXIMUM,
// crest goes beyond 255, trough is okay.
// when both intensity and beat are MINIMUM,
// crest is okay (extreme value = 0), but trough goes below 0.
// when intensity is HIGH but beat is LOW,
// crest can go beyond 255, trough is still okay
// when intensity is LOW, but beat is HIGH,
// crest is okay, but trough again goes below 0

// Conclusion: handle cases for trough < 0 and crest going above 255 // INDEPENDENTLY. if ((crest >=0) && (trough >=0)){
if (crest < 255 ){ if (trough > 0){
driveColor(ledPinGreen, crest, trough, heartRate);
delay(100);
driveColor(ledPinBlue, crest/2, trough, heartRate);
delay(100);
driveColor(ledPinRed, crest, trough, heartRate);
Serial.println(crest);
Serial.println(trough);
}
else{
driveColor(ledPinGreen, crest, 0, heartRate);
delay(100);
driveColor(ledPinBlue, crest/2, 0, heartRate);
delay(100);
driveColor(ledPinRed, crest, 0, heartRate);
Serial.println(crest);
Serial.println(0);
}
}

else{
// set your crests to 255 explicitly
if (trough > 0){
driveColor(ledPinGreen, 255, trough, heartRate);
delay(100);
driveColor(ledPinBlue, 255/2, trough, heartRate);
delay(100);
driveColor(ledPinRed, 255, trough, heartRate);
Serial.println(255);
Serial.println(trough);
}
else{
driveColor(ledPinGreen, 255, 0, heartRate);
delay(100);
driveColor(ledPinBlue, 255/2, 0, heartRate);
delay(100);
driveColor(ledPinRed, 255, 0, heartRate);
Serial.println(255);
Serial.println(0);
}
}

}

else{

//debugger
Serial.println("Crests and Troughs are now: ");
Serial.println(crest);
Serial.println(trough);
Serial.println("----------------------------");
delay(2000);
}
}

IMG_20160921_000834

The Sixth Sense by Pranav Mistry

Pranav Mistry’s famous Sixth Sense interface that made a huge splash in the tech media after Pattie Maes’ TED Talk in 2009 (video here) was what I used as a frame of reference for TUIs in order to evaluate the taxonomy proposed by Fishkin. Why I regard the Sixth Sense as a strong example of TUI because it breaks conventions of desk-constrained computing and made it so that the interactions are intuitive and self-evident.

In terms of embodiment, the input in Mistry’s prototype is through color coded finger tips that are used to encode gestures which are then captured by a camera. The output is then projected onto surfaces in front of the user which are also interactive completing the augmented reality cycle. This kind of I/O touches various types of embodiments – full when the user is touching a projected button hinting at direct manipulation, nearby when the gesture indicates to scroll information on a projected page and environmental when the gesture takes an image (but doesn’t display the same image in front of you after it’s being captured hinting at the ambient nature of the process).

Metaphorically, Mistry’s prototype is heavily gesture-driven and the inputs to the device are based on how other devices act – hence, Sixth Sense falls into the ‘metaphor as verb’ category. Here’s where I think the taxonomy’s intention to pull all TUIs towards the full embodied and metaphor angle is not very convincing. While the prototype does feature a camera and a projector, a finished unified product does not necessarily have to appear like a camera and a projector – there are plenty of creative angles the designer could take on how it could look like and making it look like a camera and a projector does not seem very innovative. So Fishkin’s taxonomy is not very useful to guide design changes in existing innovations. However, it is a very robust concept as it helps place existing examples of TUI in various parts of the spectrum and in my opinion, by seeing innovations like the Sixth Sense appear in the non-extreme portions of the spectrum, it can also help guide what-if questions.

Colored Smoke Cylinder

Description:
I used dryer sheets, ping pong balls and the cotton-like material that we had used in class to create an impression of a smoke chimney. The diffuser gives a very close approximation of white and the output is in RGB format. Of the things I had trouble with was using only a single input (no carriage return) and changing the color. The blue was also the strongest LED in the set and hence the resultant white had a tinge of blue.

Components:
I used dryer sheets, ping pong balls and the cotton-like material that we had used in class to create an

  • 1 Arduino Uno
  • 1 Breadboard
  • Resistors, Jumper cables
  • Ping pong balls
  • Dryer sheets
  • Cotton-like material for impression of smoke

Code:

/*
Smoke cylinder by Ganesh Iyer
*/

char serInString[100]; // array that will hold the different bytes of the string. 100=100characters;
// -> you must state how long the array will be else it won't work properly
char colorCode;
int colorValRed = 0;
int colorValBlue = 0;
int colorValGreen = 0;

int redPin = 11; // Red LED, connected to digital pin 9
int greenPin = 9; // Green LED, connected to digital pin 10
int bluePin = 10; // Blue LED, connected to digital pin 11

void setup() {
pinMode(redPin, OUTPUT); // sets the pins as output
pinMode(greenPin, OUTPUT);
pinMode(bluePin, OUTPUT);
Serial.begin(9600);
analogWrite(redPin, 0); // set them all to mid brightness
analogWrite(greenPin, 0); // set them all to mid brightness
analogWrite(bluePin, 0); // set them all to mid brightness
Serial.println("Use the r, g and b keys to manipulate and mix colors!");
}

void loop () {
// clear the string
memset(serInString, 0, 100);
//read the serial port and create a string out of what you read
readSerialString(serInString);

colorCode = serInString[0];
if (colorCode == 'r'){
serInString[0] = 0;
if (colorValRed == 255){
colorValRed = 0;
analogWrite(redPin, colorValRed);
}
else{
colorValRed = colorValRed + 20;
}
if (colorValRed < 255){ analogWrite(redPin, colorValRed); } else{ colorValRed = 255; analogWrite(redPin, colorValRed); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } if (colorCode == 'g'){ serInString[0] = 0; if (colorValGreen == 255){ colorValGreen = 0; analogWrite(greenPin, colorValGreen); } else{ colorValGreen = colorValGreen + 20; } if (colorValGreen < 255){ analogWrite(greenPin, colorValGreen); } else{ colorValGreen = 255; analogWrite(greenPin, colorValGreen); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } if (colorCode == 'b'){ serInString[0] = 0; if (colorValBlue == 255){ colorValBlue = 0; analogWrite(bluePin, colorValBlue); } else{ colorValBlue = colorValBlue + 20; } if (colorValBlue < 255){ analogWrite(bluePin, colorValBlue); } else{ colorValBlue = 255; analogWrite(bluePin, colorValBlue); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } // if( colorCode == 'r' || colorCode == 'g' || colorCode == 'b' ) { // colorVal = atoi(serInString+1); // Serial.print("setting color "); // Serial.print(colorCode); // Serial.print(" to "); // Serial.print(colorVal); // Serial.println(); // serInString[0] = 0; // indicates we've used this string // if(colorCode == 'r') // analogWrite(redPin, colorVal); // else if(colorCode == 'g') // analogWrite(greenPin, colorVal); // else if(colorCode == 'b') // analogWrite(bluePin, colorVal); // } delay(100); // wait a bit, for serial data } //read a string from the serial and store it in an array //you must supply the array variable void readSerialString (char *strArray) { int i = 0; if(!Serial.available()) { return; } if (Serial.available() > 0) {
strArray[i] = Serial.read();
}
}

Smoke cylinder

Multimodal Interactions and Passwords

To me, passwords are no longer just an ordered list of characters. My increasing proficiency with a standard US-keyboard has tied in the motion of my hands with the position and combination of the keys on the keyboard to generate the password. The muscle memory in reference here held me in good stead when remembering passwords across various websites and typing them out effortlessly. However, when I enter a need to type in a password to log into a service using my smartphone, the position and the combination of the keys on the touch keyboard are now altered, breaking my flow. What would follow is my visualizing the password as a string of keys and how that often works is reimagining my the hand movements on the standard physical keyboard.

Alluding to McCullough’s point about computers being a tool for the mind and not for the hands, this resonates strongly with me in almost an ironic way as while the keyboard-based computers are accused of restricting what our hands could do and touch-based devices seem to have freed the range of motion for our hands, touch devices in this example force a cognitive responsibility back on the mind and the hands play a subservient role again. This however, as a trend may not last long as technologies like replacing your password with your brainwaves can bypass the hands altogether and workarounds like Slack sending a ‘magic link’ to your inbox to log in are emerging and extant respectively. This is also ideal as typing a password isn’t exactly pleasurable, but it illustrates the importance that hands have in letting the mind not be burdened by mundane tasks.

In so far as an experience goes in terms of reutilizing the notion of touch and movements of hands, Virtual Reality apps like BowSlinger in The Lab (played on an HTC Vive) and Google’s TiltBrush are on the right track to revive the usage of hands in a more generative way. The former uses a multimodal interaction coupling the pull of the arrow on the bow with the sound of the string stretching to give you an illusion of tension in the string; and it certainly engages the hands more actively than writing out passwords. While the technology may be far too advanced to do away with the Vive hand controllers which hold the hand in a semi-closed position and include non-intuitive motions like having a trackpad near the thumb, we might be able to witness devices which provide feedback to various positions on our palms without it having to be in one type of position and also create resistance if needed. For example, would you be able to feel the weight of a heavy object if you tried to lift it in virtual reality? Or would the sensors on your palms recreate the feel of petting a dog?

Lab 01 – Disco Lights

I tried to play around with the lights and more than a loop, I thought I could make it groovy and hence, chose to sync the lights with the legendary disco number Stayin’ Alive by the BeeGees (click for video; may be too loud though).

To get the different accents and beats, I set three connections in parallel and I placed the green and the yellow LEDs in serial to indicate a stronger third beat. I added the tempo variable in the code itself so if we change the song to, say a slower tempo in 4/4 time, this can be adjusted too! A slightly more varied drumbeat can also be incorporated!

Code:


int ledPin = 13; // goes to yellow and green
int ledPinTwo = 12; // goes to red
int ledPinThree = 11; // goes to blue

void setup()
{
pinMode(ledPin, OUTPUT);
pinMode(ledPinTwo, OUTPUT);
pinMode(ledPinThree, OUTPUT);
}
void loop()
{

// Drumbeat pattern (each column is one beat)
// YG: 1100 0000 0000 0000
// R: 1100 1100 1100 1100
// B: 1100 0000 1100 0000
// Tempo for Stayin' Alive: 104 BPM

digitalWrite(ledPin, HIGH);
digitalWrite(ledPinTwo, HIGH);
delay(30000/104);
digitalWrite(ledPin, LOW);
digitalWrite(ledPinTwo, LOW);
delay(30000/104);

digitalWrite(ledPinTwo, HIGH);
delay(30000/104);
digitalWrite(ledPinTwo, LOW);
delay(30000/104);

digitalWrite(ledPinTwo, HIGH);
digitalWrite(ledPinThree, HIGH);
delay(30000/104);
digitalWrite(ledPinTwo, LOW);
digitalWrite(ledPinThree, LOW);
delay(30000/104);

digitalWrite(ledPinTwo, HIGH);
delay(30000/104);
digitalWrite(ledPinTwo, LOW);
delay(30000/104);

}

Braid by Jonathan Blow

This nifty little arcade game is arguably one of the most creative ones to emerge in recent years. The premise is for the user to solve extremely tricky puzzles when given various opportunities to bend the laws of physics. As I was reading why research in HCI needs to have a theoretical framework that is in the middle of the two extremes (the generalist cognitive science approach and the particularist ethnomethodology approach), this game stuck out from my memory because it ties constructivist learning through the puzzles with reflective thinking through the plot of the game.

While many players of the game are already presumed to be familiar with spatial reasoning (and basic math, an example mentioned by Kaptelinin and Nardi to illustrate how artifacts are used by a subject to aid learning), the game tries to recreate a new learning challenge by presenting an environment where the user can go back in time or where the environment moves only when the user does (see attached GIF). This also reinforces the Activity Theory concept that no property of the subject (player’s avatar Tim) or object (the game environment) exist other than during the activity as each ‘world’ has a different physical curveball, placing the subject in a new unfamiliar learning challenge.

In addition, this game is an homage to the popular game Mario Brothers and feeds into the sociocultural familiarity that users have with the intention of the game, while also providing an outlet to restructure some of those cultural norms – for example, you don’t have to start over because you don’t die in the game; how does that make the subject perceive the object?