Pranav Mistry’s famous Sixth Sense interface that made a huge splash in the tech media after Pattie Maes’ TED Talk in 2009 (video here) was what I used as a frame of reference for TUIs in order to evaluate the taxonomy proposed by Fishkin. Why I regard the Sixth Sense as a strong example of TUI because it breaks conventions of desk-constrained computing and made it so that the interactions are intuitive and self-evident.

In terms of embodiment, the input in Mistry’s prototype is through color coded finger tips that are used to encode gestures which are then captured by a camera. The output is then projected onto surfaces in front of the user which are also interactive completing the augmented reality cycle. This kind of I/O touches various types of embodiments – full when the user is touching a projected button hinting at direct manipulation, nearby when the gesture indicates to scroll information on a projected page and environmental when the gesture takes an image (but doesn’t display the same image in front of you after it’s being captured hinting at the ambient nature of the process).

Metaphorically, Mistry’s prototype is heavily gesture-driven and the inputs to the device are based on how other devices act – hence, Sixth Sense falls into the ‘metaphor as verb’ category. Here’s where I think the taxonomy’s intention to pull all TUIs towards the full embodied and metaphor angle is not very convincing. While the prototype does feature a camera and a projector, a finished unified product does not necessarily have to appear like a camera and a projector – there are plenty of creative angles the designer could take on how it could look like and making it look like a camera and a projector does not seem very innovative. So Fishkin’s taxonomy is not very useful to guide design changes in existing innovations. However, it is a very robust concept as it helps place existing examples of TUI in various parts of the spectrum and in my opinion, by seeing innovations like the Sixth Sense appear in the non-extreme portions of the spectrum, it can also help guide what-if questions.