Virtual/Augmented Reality – Does Fishkin’s taxonomy still hold up

Going through numerous examples discussed in the readings and how the taxonomy applies to them, I could not help but think about upcoming virtual/augmented reality interfaces and devices. For example looking at Oculus Rift, HTC Vive, and other similar devices and the many interactions they supports through numerous programs it seems a bit difficult to identify a strong place for them within the 2D taxonomy.

Given how one interacts within an augmented/virtual reality setting it seems a bit problematic when thinking about embodiment and metaphor. Let’s look at a tangible interface that mimics clay sculpting – Illuminating Clay. Illuminating clay is an example of Full embodiment and Full metaphor – this is because in this case user needs to make no analogy when looking at the interface – it looks and acts like the same thing. Now what happens if we were to create a similar augmented reality application for clay sculpting. The way the oculus rift would position itself on the 2D taxonomy could be completely different. We can no longer see our output being the same as our input device since there is no tangible element associated to it. But this could taxonomy could work if we were to add a tangible aspect to the augmented reality application, but the output still remains in the reality created by the application.

This is no way means that the taxonomy is wrong. The taxonomy provides a great way for designers to think about their TUI’s and help make design decision that would support the interface. But this taxonomy might be limited when it comes to virtual/augmented reality interfaces. What we see is that it becomes difficult to restrict a single device/interface within one section of the embodiment vs metaphor graph.

Lab/HW 3 – Potentiometer to control LEDs

Description
For the Lab, I used two potentiometers to control the brightness and blinking rate of the three LEDs.

Extra Point
For this portion of the Lab, I used the rotation of one potentiometer to control the brightness of the three LED’s. The other potentiometer controls the gaps and duration between the blinking of the individual LEDs.

Lab Material

 

  • Jumper cables
  • Potentiometers (2)
  • Ping pong ball diffuser
  • Cotton
  • Arduino
  • 3 LED’s
  • Breadboard

 

HW/LAB

int sensorPin_blink = A0;
int sensorPin_bright = A1;// select the input pin for the potentiometer
int ledPin = 13;
int redledPin = 9;
int blueledPin = 10;
int greenledPin = 11;// select the pin for the LED
int sensorValue_blink = 0;
int sensorValue_bright = 0;// variable to store the value coming from the sensor

void setup() {
 // declare the ledPin as an OUTPUT:
 pinMode(ledPin, OUTPUT);
 pinMode(redledPin, OUTPUT); // sets the pins as output
 pinMode(greenledPin, OUTPUT);
 pinMode(blueledPin, OUTPUT);
}

void loop() {
 // read the value from the sensor:
 sensorValue_blink = analogRead(sensorPin_blink);
 sensorValue_bright = analogRead(sensorPin_bright);
 // turn the ledPin on
 analogWrite(redledPin, sensorValue_bright/4);
 analogWrite(greenledPin, sensorValue_bright/4);
 analogWrite(blueledPin, sensorValue_bright/4);
 // stop the program for <sensorValue> milliseconds:
 delay(sensorValue_blink);
 // turn the ledPin off:
 analogWrite(redledPin, 0);
 analogWrite(greenledPin, 0);
 analogWrite(blueledPin, 0);
 // stop the program for for <sensorValue> milliseconds:
 delay(sensorValue_blink);
}

 

Extra Point

int sensorPin_blink = A0;
int sensorPin_bright = A1;// select the input pin for the potentiometer
int ledPin = 13;
int redledPin = 9;
int blueledPin = 10;
int greenledPin = 11;// select the pin for the LED
int sensorValue_blink = 0;
int sensorValue_bright = 0;// variable to store the value coming from the sensor

void setup() {
 // declare the ledPin as an OUTPUT:
 pinMode(ledPin, OUTPUT);
 pinMode(redledPin, OUTPUT); // sets the pins as output
 pinMode(greenledPin, OUTPUT);
 pinMode(blueledPin, OUTPUT);
}

void loop() {
 // read the value from the sensors:
 sensorValue_blink = analogRead(sensorPin_blink);
 sensorValue_bright = analogRead(sensorPin_bright);
 // turn the ledPin on
 analogWrite(redledPin, sensorValue_bright/4);
 delay(sensorValue_blink);
 analogWrite(blueledPin, sensorValue_bright/4);
 // stop the program for <sensorValue> milliseconds:
 delay(sensorValue_blink);
 analogWrite(greenledPin, sensorValue_bright/4);
 delay(sensorValue_blink);
 // turn the ledPin off:
 analogWrite(redledPin, 0);
 analogWrite(greenledPin, 0);
 analogWrite(blueledPin, 0);
 // stop the program for for <sensorValue> milliseconds:
 delay(sensorValue_blink);
}

IMG_7345
Video

 

Who you gonna call? Ghostbusters

For this assignment, I created a diffuser using my LEGO Ghostbusters Car. I wrapped the 3 LED’s within a semi-transparent paper packet and placed the wrapped LED’s within the main section of the car which was surrounded LEGO glass pieces.

I set up the commands as weapon modes of the Ghostbuster Car, in case ghosts were to attack. Each attack had a different effect through the LED.

The three weapon modes were:
ProtonGun – Emitted radiating pulses of Red, Green and Blue light
IonCannon – Alternates between Violet and Yellow
NeutronaGun – The interior of the car becomes Orange

Components:

  • Arduino Uno
  • Breadboard
  • 3 LEDs (rgb)
  • 3 220Ω Resistors
  • jumper wires
  • USB cable
  • laptop
/* 
* Ghostbusters
* ---------------
* Serial commands control the brightness of R,G,B LEDs 
*
* Command structure is "<WeaponMode>", where "WeaponMode" is
* one of "ProtonGun","IonCannon",or "NeutronaWand"
* E.g. "ProtonGun" Emitted radiating pulses of Red, Green and Blue light 
* "IonCannon" Alternates between Violet and Yellow
* "NeutronaWand" The interior of the car becomes Orange
*
* Created 11 September 2016
*/
String weaponmode;
char serInString[200]; // array that will hold the different bytes of the string. 100=100characters;
// -> you must state how long the array will be else it won't work properly
char weaponcolor;
int colorVal;
int wait = 5;
int j = 0;


// Program variables
int redVal = 255; // Variables to store the values to send to the pins
int greenVal = 1; // Initial values are Red full, Green and Blue off
int blueVal = 1;

int redPin = 11; // Red LED, connected to digital pin 9
int greenPin = 10; // Green LED, connected to digital pin 10
int bluePin = 9; // Blue LED, connected to digital pin 11

void setup() {
pinMode(redPin, OUTPUT); // sets the pins as output
pinMode(greenPin, OUTPUT); 
pinMode(bluePin, OUTPUT);
Serial.begin(9600);
analogWrite(redPin, 0); // set them all to mid brightness
analogWrite(greenPin, 127); // set them all to mid brightness
analogWrite(bluePin, 0); // set them all to mid brightness
Serial.println("Ghosts are coming !! Choose your weapon mode:"); 
}

void loop () {
// clear the string
memset(serInString, 0, 200);
readSerialString(serInString);
//read the serial port and create a string out of what you read
 
weaponmode = String(serInString);
weaponcolor = serInString[0];
Serial.println(weaponmode); 
if( weaponcolor == 'p' || weaponcolor == 'i' || weaponcolor == 'n')
{
if(weaponcolor == 'p')
{
 
for(int i=0; i<=5; i++)
{
analogWrite(redPin, 255);
analogWrite(greenPin, 0);
analogWrite(bluePin, 0);
delay(200);
analogWrite(redPin, 0);
analogWrite(greenPin, 255);
analogWrite(bluePin, 0);
delay(200);
analogWrite(redPin, 0);
analogWrite(greenPin, 0);
analogWrite(bluePin, 255);
delay(200);
}
 
}
else if(weaponcolor == 'i')
{
for(int i =0; i<=10; i++)
{
analogWrite(redPin, 153);
analogWrite(greenPin, 51);
analogWrite(bluePin, 255);
delay(500);
analogWrite(redPin, 255);
analogWrite(greenPin, 255);
analogWrite(bluePin, 0);
delay(500);
}
}
else if(weaponcolor == 'n')
{
analogWrite(redPin, 255);
analogWrite(greenPin, 128);
analogWrite(bluePin, 0);
}
}
delay(100); // wait a bit, for serial data
}

//read a string from the serial and store it in an array
//you must supply the array variable
void readSerialString (char *strArray) {
int i = 0;
if(!Serial.available()) {
return;
}
while (Serial.available()) {
strArray[i] = Serial.read();
i++;
}
}

IMG_7295 IMG_7298 IMG_7311 IMG_7312

Creating experiences the mind desires: HTC Vive

I believe that hands are an extension of the mind and not just closely connected to the mind as stated by McCullough (1996). A lot of experiences that our mind creates are based on our senses and touch is major aspect of them. Not only does the mind use our sense of touch to make sense of our environment but when our mind is wandering and creating experiences, it is our hands that extend first which closely ties in with the concept talked about by McCullough.

In a way, our hands let us create whatever the mind desires. A beautiful point that comes across in the readings is that ‘Hands also discover and are themselves led into exploration’. So though the computer is inherently a tool for the mind, hands are the medium that helps us explore the world through the tool. A perfect example such an interface is the HTC Vive (particularly with applications like Google Tilt brush).

Tilt Brush lets you paint in 3D space with virtual reality. The world is set up as our canvas and the only limitation is what our mind can imagine. The way the interface has been created it has been adapted to be as fluent and natural as we can be. Only a few limitations exist to this interface which needs the user to use a gesture or input method that might not be intuitive when using one’s hand naturally. As a user, I am allowed to use my hands to draw and manipulate in a 3D environment. Also since it is virtual reality, the input and output feel more connected. Think of those times, when you want to create something and on unconsciously are waving your hand around trying to imagine how it would get created and how the output would be, the Vive gets you there. It lets the connection between the mind and the hand do what it was meant to do.

Lab – 1 – Alternating Lights

After the initial blink set up, I worked towards setting up two different LEDs to blink alternately.

2 LED
2 Resistors (220Ω)
Jumper wire
1 Arduino
1 Breadboard

##Code##
// the setup function runs once when you press reset or power the board
void setup() {
 // initialize digital pin 13 as an output.
 pinMode(13, OUTPUT);
 pinMode(12, OUTPUT);
}

// the loop function runs over and over again forever
void loop() {
 digitalWrite(13, HIGH); // turn the LED on (HIGH is the voltage level)
 digitalWrite(12, LOW); // turn the LED off by making the voltage LOW
 delay(500); // wait for .5 seconds
 digitalWrite(13, LOW); // turn the LED off by making the voltage LOW
 digitalWrite(12, HIGH); // turn the LED on (HIGH is the voltage level)
 delay(500); // wait for .5 seconds
}

The Click Wheel – iPod

Let us consider for this post, our object to be ‘Listening to music on the go’. As we have seen throughout history , there have been multiple attempts at designing and creating user interfaces to allow users to attain this goal. Technologists knowingly or unknowingly have been disrupting the system by changing, adding or removing subjects from the system designed to support the above goal. As part of this system, we have seen at one time or the other that the daily user, tapes, CDs, etc and the different devices used to play music have all been the subjects. Looking at all the audio players that I owned, a major device that in an inexplicable way cemented my love for music was the first ever classic iPod. The key differentiating factor was the design of the operating method for the iPod – ‘The Click Wheel’.

The iPod revolutionized for me the way I was able to use my iPod and access my entire music collection using just 4 buttons laid out on a click wheel. This design removed multiple unintended and mostly confusing and non-useful actions from the system. The click wheel made me able to swiftly move through my entire library in a manner that felt much more intuitive to my mind. I was essentially scrolling through a list of items (in this case my music). Not only did it beat audio players by removing the hassle of inserting tapes, CDs, but it also made the action of playing your music much easier as compared to other digital audio players. It didn’t take much time for these actions to become an operation when it came to listening to music. No longer did I need to stop, look down and figure out how to get to my next favourite song.