The second challenge to the community was to consider other senses. The visual channel is dominant in human perception but other senses are also very important and can be used to parallelize the presentation. We need to leverage the full range of human capabilities. Communication skills are very multi-modal, using language, gesture, situation awareness, emotional states, etc. There is a perceived personality to our programs whether we consider it or not. Generally we don’t and the perception is of a cold or neutral system. Adding even small changes in the system messages can drastically change the user’s perception of their system. (The *^#@* paper clip in Office ’97 was brought up as one example – avoid the pollyanna personality!!) George mentioned some studies on bi-manual interfaces – i.e. using two hands. Gestures from the dominant hand are used to control detail, while the other hand controls gross effects. Users adapted quite quickly to these systems. (Papers on this were presented at UIST 98) Perception includes spatialization, pattern recognition, object constancy (use animation between displays instead of jumps), motion parallax (camera based head tracking driving display is possible on desktop systems today) Other senses include audio (spatialized or not), haptics, taste and smell (noted that Ferris Productions is working in this area, as is Myron Kruger).
Human cognition is something of which we need to take advantage in our systems. Cognitive chunking is a known area of psychology. Attention is captured by motion. Curiosity is an important aspect that can be supported in an interface that allows for discoverability of function. This is often suppressed in systems because of the fear of irrevocably screwing up. Systems should implement universal undo.