User Studies Data Analysis

The data collected from the first user study has been harvested and analysed. First thing that comes to eye is, that there do not appear to be many clear differences in the data and it will be difficult to extract valuable information or conclusions, due to the fairly small sample size.

Initially, the performance over time while keeping track of moving metrics has been examined. When comparing the average performance of users working with the visual only approach to the sound approaches, no clear performance increase can be identified. There appear to be performance differences between users working with the "sound alarm only" and the "sound alarm and audification" approach. However, the small sample size does not allow to draw a solid conclusion from this.

In all cases, the performance decreased in a fairly similar amount when an event occurred. Though there where performance differences between each user, the average performance decrease while fixing events was fairly consistent. A notable issue in the data is however, that performances with the visual approach showed a  more consistent performance whereas the audio approaches showed more fluctuations.

A non-intrusive evaluation with a similar prototype installed in a live office environment is planned. This way, the factor time and surprise would take a larger role, as users would not constantly expect event to occur and fully concentrate on their primary task.


In the context identifying tests of auditory alarms, artificial sound design approaches appeared to be clearly more identifiable then recordings of natural sounds. 16 different sound alarms where triggered, eight of them being natural and the other eight being artificially created sounds.

Amount of users guessing context correctly for each context type separated by natural and artificial sounds

Artificial sounds appeared to be more familiar to users whereas natural sounds often caused surprise, confusion or where sometimes completely ignored. 76% of the artificial sounds where referred correctly to what the sound designer intended the sound to represent, whereas only 24% of the naturally recorded sound's context was identified.

This outcome is very insightful and will be further investigated. The focus on further investigations will lie on artificial sounds and different techniques how to transport context.



The analysis of transcript of the interviews is still work in progress.