Making Sense of Sensing Systems: Five Questions for Designers and Researchers

Victoria Bellotti, Maribeth Back, W. Keith Edwards, Rebecca E. Grinter, Austin Henderson, and Cristina Lopes | CHI 2002

In a Nutshell 🥜

Bellotti et al.¹ first identify the challenge of designing interactions for sensing systems as the lack of accepted genres. Genre can be defined as a set of design conventions anticipated with particular usage contexts. For example, users know what to expect when interacting with hoverable and clickable buttons on a traditional GUI interface. By sticking to the accepted GUI genre, designers can utilize standardized toolkits to design GUIs without needing to worry about basic interaction issues. However, the paper argues that there does not exist such standardized answers for the design of interactions for sensing systems.

Norman proposed a seven-stage action model² to describe system interaction:

1. Forming the goal

2. Forming the intention

3. Specifying an action

4. Executing the action

5. Perceiving the state of the world

6. Interpreting the state of the world

7. Evaluating the outcome

This model focuses more on user cognition, positioning computers as rarely taking initiative. This model works well for the GUI genre with pre-packaged solutions. However, the paper argues that an alternative model is needed for sensing systems, one which highlights communicative rather than cognitive aspects of interaction, emphasizing that the joint accomplishments of the user and the system are necessary to complete an interaction, rather than just focusing on the user’s mental model.

The paper presents a framework for the design of interactions with sensing systems, drawing from analysis of human-human communication (HHI). The framework consists of five questions that a system must answer to accomplish some actions:

1. Address. How can the user address the system(s)? For example, the system should be able to separate signals from noise.

2. Attention. How can the user know that the system is attending to the user’s actions? For example, the system needs to give appropriate feedback cues.

3. Action. How can the user create an action, control its extent, and specify its targets? For example, the system should bind actions to targets and also avoid unwanted selections.

4. Alignment. How can the user know that the system is performing correctly? For example, the system should make the current system state interpretable.

5. Accident. How can the user avoid and recover from mistakes? For example, the system should have redo functionality for the user to recover to a previous state.

The paper also introduces several existing sensing systems and discusses how they address or do not address the five questions.

Some Thoughts 💭

• I enjoyed the paper’s insight of incorporating research in HHI into the design of interactions, especially for sensing systems. For example, the Soli radar in the Pixel 4 may perhaps benefit from increasing alignment by visualizing the signals taken in from the radar such that users can make sense of what went well and what went wrong without repeated trial and error.

Figure 1: Using pinching to control a soundtrack with Soli in the Pixel 4 | Source

1. Bellotti, V., Back, M., Edwards, W. K., Grinter, R. E., Henderson, A., & Lopes, C. (2002, April). Making sense of sensing systems: five questions for designers and researchers. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 415-422).

2. Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books.