Embodied Geometry – Final Project Proposal

Proposed Interaction: Think Twister meets Tetris.
We are creating a collaborative, embodied geometry game. This game could provide children with a collaborative learning environment in which to explore geometric properties of shapes as well as properties of symmetry and spatial reasoning. We are also exploring possibility of creating a shareable memento/artifact of their game play.

Figure 1 - Embodied Geometry
Figure 1 – Initial Setup

For the scope of the current project, a pair of users will use their limbs to activate circles on an interactive mat (Figure 1). Users must always keep one limb on the center spot. From a geometric stand-point, this center point acts as a frame of reference for body movements (rotations, extensions, etc.). This is an intentional design constraint within the environment that allows for comparison of asynchronously created shapes. We intend for this point to guide their movements such that we believe they will more likely notice patterns and relationships.

Figure 2 - Embodied Geometry
Figure 2 – As the shape approaches towards the users, they are supposed to collaborate and create the shape before it hits the mat.

Users will coordinate their body positions to make composite shapes supplied by the system (Figure 2) paying special attention to the orientation of each component shape. A shape will be projected on the floor near the mat, moving slowly toward the mat. Users must create the shape before it reaches the mat (time runs out). For a shape to be recognized as successfully created, users must be touching all necessary points at the same time. The display will also include a small schematic of all the shapes completed so far. In Figure XXX, the triangle is the challenge shape (component), the image in the upper left represents shapes completed so far (composite). Users will also receive feedback on the mat regarding their touch positions.

The game will have different levels of difficulty that require users to use more limbs (easy: legs only, medium: legs and an arm, hard: both legs and both arms). All successfully created shapes will be overlaid to create a final image that users could take with them (Figure 3 below).

Figure 3 - Embodied Geometry
Fig 3: How the UI could possibly look like in terms of shapes created and to be created. This is an early sample

Trapdoor Cuckoo Clock


Our group wanted to attempt an unconventional cuckoo clock, counter to the example we saw in class with the 4-bar joint. Instead of a cuckoo that emerged horizontally from the door, we designed a cuckoo that plunged through a trapdoor instead. The primary mechanism used to move the trapdoor and lower the cuckoo through the opening is a pulley system. As the pulley is pulled in one direction, the trapdoor moves out of the way to reveal an exit. As the pulley is pulled and the trapdoor is moved back, the cuckoo also lowers. The cuckoo is mitigated from falling outside the bounds of the trapdoor exit until the opening is wide enough for it to plunge through. The cuckoo can then continue to be lowered, or reigned back into the clock by pulling the pulley in the opposite direction. The trapdoor is prevented from collapsing on the cuckoo prematurely by means of (1) the distances used in both the strings affixed to the door and the cuckoo, and (2) the rotational distance between the door and the peg that pushes it back into place.

To hold the system in place and direct accurate movement, we have in place four poles around the trapdoor’s perimeter (but not affixed to the trapdoor). These poles are taped securely to the surface of the foam board/clock base. While there are three gears in our structure, only the center one is unrestrained by a foundational support component. For our design, it is imperative that thread lines are measured to have sufficient tension and slack when appropriate. Otherwise, the system will fail. If the support structure also shifts (whether at foundation or at suspension), the system is likely to fail due to the shifting of the trapdoor in and out of place. Also necessary is a wall to prevent the cuckoo from traveling with the trapdoor as it slides back, as opposed to falling through the exit.

We think this cuckoo clock mechanism design is appropriate, considering the majority of cuckoo clocks hang on the wall, and so have room beneath them that can be utilized.

Components Used:

  • 7 gears (4 for foundational support, 2 for suspended support, 1 for pulley system)
  • 1 long stick
  • 2 mid-length sticks
  • 5 short sticks (sizes must be relative; 3 for pulley system)
  • 1 cuckoo (made out of a spool, suspended by string)
  • two cuts of string (1 for the trapdoor, 1 for the cuckoo)
  • 1 relatively large piece of balsa wood for the trapdoor
  • 3 relatively narrow pieces of balsa wood for support/error-prevention
  • 1 T-pipe
  • 2 straight joints with medial socket
  • 1 claw
  • 1 foam board sheet
  • duct tape
  • X-acto knife and cutting board
  • sufficient string for the pulley system


img_6360 img_6359

Enhancing VR experiences

As someone who spent more time observing others immersed in the VR world during the recent sessions, what I absolutely loved about the whole experience was the incredible capacity of the brain to complete unconnected stimuli and form a coherent impression of the immediate world. For example, standing on the edge of Vesper Peak was enough to trigger the fear of heights in one particular user, despite them being at no risk of falling down a cliff. When Kimiko slightly nudged another user off balance, their brain combined the vision of being on the edge of a mountain and being unstable to very dangerous and hence, they started flailing. This also reminded me of some of the VR rides in Universal Studios where the roller-coaster car is not actually moving anywhere but by tilting it and creating an impression of motion combined with synchronized virtual imagery, it conveys to the brain that you are actually in an animated roller-coaster. This begs the question as to why some rollercoasters (Six Flags) have VR headsets on riders as they seem to underestimate the capacity of the brain to connect these virtual stimuli.

What I liked least about the VR experience was there are still audio limitations where it becomes difficult to create a completely immersive experience. The trouble with headphones was that users would not be able to listen to the conductors of the experiment. How users may benefit from an immersive audio experience is that they might start to spatially relate the source of sound and this creates a greater validation of the virtual space as being something realistic. The most intuitive solution would be to include a small microphone for the moderator to lead the user through the experiment; however, this solution would require validation on whether users prefer to be more disconnected with the real world when navigating the virtual world.

Fun with Drawing Tools – Elena, Ganesh, Leah

Elena Lopez, Ganesh Iyer, Leah Rosenbaum

To create a collaborative puzzle for children which is fun physically and can be digitally enhanced for new ideas.

Fun with Drawing Tools

We envision this project as closely embodied, full-metaphor tangible UI where our main targeted audience – children aged 5 and above – can use drawing tools like a compass, a ruler and maybe a few other daily objects to contribute to a narrative in a game environment on a table top interface. The intention of the team is to encourage children to think of geometric tools in a more intuitive and intimate way. This project can also be thought of something adults can engage in to communicate with each other as conversation starters or as a game to play with over a get-together.

The puzzle is tied into a narrative where the players will use their tools to help a protagonist navigate her way to a reward. This environment would be offset by the tangible constraints that the drawing tools themselves pose and digital constraints through virtual traps in the puzzle which help make the game more difficult and engaging with progress and learned skill. The narrative that we’re looking to tie in this also involves the use of materials – straw, wood and stone – and this led us to exploring possible narratives like Three Little Pigs and the Big Bad Wolf or getting a colony of hardworking ants to secure food amid obstacles.

As a starting point, the most basic game environment could be a fluid path of dots indicating a reward-filled path to the goal and the task of the users is to connect these dots using the drawing tools that they have. To make the learning process more coherent and make gravity more intuitive, we chose to go with a birds’ eye view of the environment, so we are looking at the protagonist from above. The dots can involve materials that stack up across levels and the users may use up materials to build their path to the goal, so they might be nudged to draw optimal paths.

Fun with Drawing Tools 2

The game dynamics physically would be the users drawing paths using soft tools (since the age bar is low, we also want to make sure that the compass and rulers are not sharp/pointed objects). Straight-line paths can be drawn using a ruler and circular-directed paths can be made using a compass for the protagonist to travel along the circumference. An additional advantage is that users can indicate direction of the movement from the way they draw these constituent paths.

Fun with Drawing Tools 3

Next steps:
The next step for us as designers in this particular project would be to scope the project so that our vision is achieved. This would involve detailing out the game narrative – creating a story, reward system – and the virtual game dynamics in addition to understanding intuitive ways in which children use geometric tools so that the physical interactions that we have proposed can be refined.

Looking forward to hearing your feedback! ūüôā

What’s your fortune?

In this one, I tried to create a pulsating effect (as in a fortune-teller’s crystal orb) with a ping-pong ball as a diffuser. I wanted to vary the heart-rate, the baseline intensity and the amplitude of the heartbeat using the potentiometers. I also had to be mindful of how bright my blue LED. One important lesson was to write the code from start to finish even if some of the code can be reused from another source on the internet.

1 ping pong ball
3 potentiometers
3 LEDs (Red, Green and Blue)
3 Resistors (220 ohms each)
Arduino UNO
Breadboard and cables


int heartRatePin = A0; // this controls tempo/delay
int heartRate = 0;

int beatWidthPin = A1; // this controls amplitude of the beat
int beatWidth = 0;

int intensityPin = A2; // this controls baseline brightness around which the amplitude is then set
int intensity = 0;

int crest = 0;
int trough = 0;

// initializing the LEDs
int ledPinGreen = 9;
int ledPinBlue = 10;
int ledPinRed = 11;

void setup(){
// declaring pins as OUTPUTS
pinMode(ledPinGreen, OUTPUT);
pinMode(ledPinBlue, OUTPUT);
pinMode(ledPinRed, OUTPUT);


void driveColor(int pinColor, int c, int t, int hr){
// this will be the cookie-cutter function for starting LEDs and then fading LEDs
int startLED = t;
// int difference = (c-t)/20;
for (startLED = t; startLED <= c; startLED +=30){ analogWrite(pinColor, startLED); delay(90); } delay(hr); for (startLED = c; startLED >= t; startLED -=30){
analogWrite(pinColor, startLED);

// analogWrite(pinColor, c); // don't launch this yet.
// Serial.println("LED brightness value (crest): ");
// Serial.println(c);
// Serial.println("Heart Rate: ");
// Serial.println(hr);
// delay(hr);
// analogWrite(pinColor, t);
// Serial.println("LED brightness value (trough): ");
// Serial.println(t);
// Serial.println("\n");
// delay(hr);

void loop(){
intensity = analogRead(intensityPin);
intensity = intensity/4; // intensity can now take values from 0 to 255
beatWidth = analogRead(beatWidthPin);
beatWidth = beatWidth/8; // reducing the domain for amplitudes
// beatWidth can now only take values from 0 to ~32

heartRate = analogRead(heartRatePin);
heartRate = 2*heartRate;
// this will be used across the LEDs to inform delay

Serial.println("Intensity: ");
Serial.println("Beat width: ");
// Serial.println("Heart Rate: ");
// Serial.println(heartRate);
// Serial.println("--------------");

crest = intensity + beatWidth;
trough = intensity - beatWidth;

// setting the wave limits
// edge case truth table:
// when both intensity and beat are MAXIMUM,
// crest goes beyond 255, trough is okay.
// when both intensity and beat are MINIMUM,
// crest is okay (extreme value = 0), but trough goes below 0.
// when intensity is HIGH but beat is LOW,
// crest can go beyond 255, trough is still okay
// when intensity is LOW, but beat is HIGH,
// crest is okay, but trough again goes below 0

// Conclusion: handle cases for trough < 0 and crest going above 255 // INDEPENDENTLY. if ((crest >=0) && (trough >=0)){
if (crest < 255 ){ if (trough > 0){
driveColor(ledPinGreen, crest, trough, heartRate);
driveColor(ledPinBlue, crest/2, trough, heartRate);
driveColor(ledPinRed, crest, trough, heartRate);
driveColor(ledPinGreen, crest, 0, heartRate);
driveColor(ledPinBlue, crest/2, 0, heartRate);
driveColor(ledPinRed, crest, 0, heartRate);

// set your crests to 255 explicitly
if (trough > 0){
driveColor(ledPinGreen, 255, trough, heartRate);
driveColor(ledPinBlue, 255/2, trough, heartRate);
driveColor(ledPinRed, 255, trough, heartRate);
driveColor(ledPinGreen, 255, 0, heartRate);
driveColor(ledPinBlue, 255/2, 0, heartRate);
driveColor(ledPinRed, 255, 0, heartRate);



Serial.println("Crests and Troughs are now: ");


The Sixth Sense by Pranav Mistry

Pranav Mistry’s famous Sixth Sense interface that made a huge splash in the tech media after Pattie Maes’ TED Talk in 2009 (video here) was what I used as a frame of reference for TUIs in order to evaluate the taxonomy proposed by Fishkin. Why I regard the Sixth Sense as a strong example of TUI because it breaks conventions of desk-constrained computing and made it so that the interactions are intuitive and self-evident.

In terms of embodiment, the input in Mistry’s prototype is through color coded finger tips that are used to encode gestures which are then captured by a camera. The output is then projected onto surfaces in front of the user which are also interactive completing the augmented reality cycle. This kind of I/O touches various types of embodiments – full when the user is touching a projected button hinting at direct manipulation, nearby when the gesture indicates to scroll information on a projected page and environmental when the gesture takes an image (but doesn’t display the same image in front of you after it’s being captured hinting at the ambient nature of the process).

Metaphorically, Mistry’s prototype is heavily gesture-driven and the inputs to the device are based on how other devices act – hence, Sixth Sense falls into the ‘metaphor as verb’ category. Here’s where I think the taxonomy’s intention to pull all TUIs towards the full embodied and metaphor angle is not very convincing. While the prototype does feature a camera and a projector, a finished unified product does not necessarily have to appear like a camera and a projector – there are plenty of creative angles the designer could take on how it could look like and making it look like a camera and a projector does not seem very innovative. So Fishkin’s taxonomy is not very useful to guide design changes in existing innovations. However, it is a very robust concept as it helps place existing examples of TUI in various parts of the spectrum and in my opinion, by seeing innovations like the Sixth Sense appear in the non-extreme portions of the spectrum, it can also help guide what-if questions.

Colored Smoke Cylinder

I used dryer sheets, ping pong balls and the cotton-like material that we had used in class to create an impression of a smoke chimney. The diffuser gives a very close approximation of white and the output is in RGB format. Of the things I had trouble with was using only a single input (no carriage return) and changing the color. The blue was also the strongest LED in the set and hence the resultant white had a tinge of blue.

I used dryer sheets, ping pong balls and the cotton-like material that we had used in class to create an

  • 1 Arduino Uno
  • 1 Breadboard
  • Resistors, Jumper cables
  • Ping pong balls
  • Dryer sheets
  • Cotton-like material for impression of smoke


Smoke cylinder by Ganesh Iyer

char serInString[100]; // array that will hold the different bytes of the string. 100=100characters;
// -> you must state how long the array will be else it won't work properly
char colorCode;
int colorValRed = 0;
int colorValBlue = 0;
int colorValGreen = 0;

int redPin = 11; // Red LED, connected to digital pin 9
int greenPin = 9; // Green LED, connected to digital pin 10
int bluePin = 10; // Blue LED, connected to digital pin 11

void setup() {
pinMode(redPin, OUTPUT); // sets the pins as output
pinMode(greenPin, OUTPUT);
pinMode(bluePin, OUTPUT);
analogWrite(redPin, 0); // set them all to mid brightness
analogWrite(greenPin, 0); // set them all to mid brightness
analogWrite(bluePin, 0); // set them all to mid brightness
Serial.println("Use the r, g and b keys to manipulate and mix colors!");

void loop () {
// clear the string
memset(serInString, 0, 100);
//read the serial port and create a string out of what you read

colorCode = serInString[0];
if (colorCode == 'r'){
serInString[0] = 0;
if (colorValRed == 255){
colorValRed = 0;
analogWrite(redPin, colorValRed);
colorValRed = colorValRed + 20;
if (colorValRed < 255){ analogWrite(redPin, colorValRed); } else{ colorValRed = 255; analogWrite(redPin, colorValRed); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } if (colorCode == 'g'){ serInString[0] = 0; if (colorValGreen == 255){ colorValGreen = 0; analogWrite(greenPin, colorValGreen); } else{ colorValGreen = colorValGreen + 20; } if (colorValGreen < 255){ analogWrite(greenPin, colorValGreen); } else{ colorValGreen = 255; analogWrite(greenPin, colorValGreen); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } if (colorCode == 'b'){ serInString[0] = 0; if (colorValBlue == 255){ colorValBlue = 0; analogWrite(bluePin, colorValBlue); } else{ colorValBlue = colorValBlue + 20; } if (colorValBlue < 255){ analogWrite(bluePin, colorValBlue); } else{ colorValBlue = 255; analogWrite(bluePin, colorValBlue); } Serial.print("RGB( "); Serial.print(colorValRed); Serial.print(", "); Serial.print(colorValGreen); Serial.print(", "); Serial.print(colorValBlue); Serial.print(" )"); Serial.println(); } // if( colorCode == 'r' || colorCode == 'g' || colorCode == 'b' ) { // colorVal = atoi(serInString+1); // Serial.print("setting color "); // Serial.print(colorCode); // Serial.print(" to "); // Serial.print(colorVal); // Serial.println(); // serInString[0] = 0; // indicates we've used this string // if(colorCode == 'r') // analogWrite(redPin, colorVal); // else if(colorCode == 'g') // analogWrite(greenPin, colorVal); // else if(colorCode == 'b') // analogWrite(bluePin, colorVal); // } delay(100); // wait a bit, for serial data } //read a string from the serial and store it in an array //you must supply the array variable void readSerialString (char *strArray) { int i = 0; if(!Serial.available()) { return; } if (Serial.available() > 0) {
strArray[i] = Serial.read();

Smoke cylinder

Multimodal Interactions and Passwords

To me, passwords are no longer just an ordered list of characters. My increasing proficiency with a standard US-keyboard has tied in the motion of my hands with the position and combination of the keys on the keyboard to generate the password. The muscle memory in reference here held me in good stead when remembering passwords across various websites and typing them out effortlessly. However, when I enter a need to type in a password to log into a service using my smartphone, the position and the combination of the keys on the touch keyboard are now altered, breaking my flow. What would follow is my visualizing the password as a string of keys and how that often works is reimagining my the hand movements on the standard physical keyboard.

Alluding to McCullough’s point about computers being a tool for the mind and not for the hands, this resonates strongly with me in almost an ironic way as while the keyboard-based computers are accused of restricting what our hands could do and touch-based devices seem to have freed the range of motion for our hands, touch devices in this example force a cognitive responsibility back on the mind and the hands play a subservient role again. This however, as a trend may not last long as technologies like replacing your password with your brainwaves can bypass the hands altogether and workarounds like Slack sending a ‘magic link’ to your inbox to log in are emerging and extant respectively. This is also ideal as typing a password isn’t exactly pleasurable, but it illustrates the importance that hands have in letting the mind not be burdened by mundane tasks.

In so far as an experience goes in terms of reutilizing the notion of touch and movements of hands, Virtual Reality apps like BowSlinger in The Lab (played on an HTC Vive) and Google’s TiltBrush are on the right track to revive the usage of hands in a more generative way. The former uses a multimodal interaction coupling the pull of the arrow on the bow with the sound of the string stretching to give you an illusion of tension in the string; and it certainly engages the hands more actively than writing out passwords. While the technology may be far too advanced to do away with the Vive hand controllers which hold the hand in a semi-closed position and include non-intuitive motions like having a trackpad near the thumb, we might be able to witness devices which provide feedback to various positions on our palms without it having to be in one type of position and also create resistance if needed. For example, would you be able to feel the weight of a heavy object if you tried to lift it in virtual reality? Or would the sensors on your palms recreate the feel of petting a dog?

Lab 01 – Disco Lights

I tried to play around with the lights and more than a loop, I thought I could make it groovy and hence, chose to sync the lights with the legendary disco number Stayin’ Alive by the BeeGees (click for video; may be too loud though).

To get the different accents and beats, I set three connections in parallel and I placed the green and the yellow LEDs in serial to indicate a stronger third beat. I added the tempo variable in the code itself so if we change the song to, say a slower tempo in 4/4 time, this can be adjusted too! A slightly more varied drumbeat can also be incorporated!


int ledPin = 13; // goes to yellow and green
int ledPinTwo = 12; // goes to red
int ledPinThree = 11; // goes to blue

void setup()
pinMode(ledPin, OUTPUT);
pinMode(ledPinTwo, OUTPUT);
pinMode(ledPinThree, OUTPUT);
void loop()

// Drumbeat pattern (each column is one beat)
// YG: 1100 0000 0000 0000
// R: 1100 1100 1100 1100
// B: 1100 0000 1100 0000
// Tempo for Stayin' Alive: 104 BPM

digitalWrite(ledPin, HIGH);
digitalWrite(ledPinTwo, HIGH);
digitalWrite(ledPin, LOW);
digitalWrite(ledPinTwo, LOW);

digitalWrite(ledPinTwo, HIGH);
digitalWrite(ledPinTwo, LOW);

digitalWrite(ledPinTwo, HIGH);
digitalWrite(ledPinThree, HIGH);
digitalWrite(ledPinTwo, LOW);
digitalWrite(ledPinThree, LOW);

digitalWrite(ledPinTwo, HIGH);
digitalWrite(ledPinTwo, LOW);


Braid by Jonathan Blow

This nifty little arcade game is arguably one of the most creative ones to emerge in recent years. The premise is for the user to solve extremely tricky puzzles when given various opportunities to bend the laws of physics. As I was reading why research in HCI needs to have a theoretical framework that is in the middle of the two extremes (the generalist cognitive science approach and the particularist ethnomethodology approach), this game stuck out from my memory because it ties constructivist learning through the puzzles with reflective thinking through the plot of the game.

While many players of the game are already presumed to be familiar with spatial reasoning (and basic math, an example mentioned by Kaptelinin and Nardi to illustrate how artifacts are used by a subject to aid learning), the game tries to recreate¬†a new learning challenge by presenting an¬†environment where¬†the user can go back in time or where the environment moves only when the user does (see attached GIF). This also reinforces the Activity Theory concept that¬†no property of the subject (player’s avatar Tim) or object (the game environment) exist other than during the activity as each ‘world’ has a different physical curveball, placing the subject¬†in a new unfamiliar learning challenge.

In addition, this game is an homage to the popular game Mario Brothers and feeds into the sociocultural familiarity that users have with the intention of the game, while also providing an outlet to restructure some of those¬†cultural norms – for example, you don’t have to start over because you don’t die in the game; how does that make the subject perceive the object?