An Empirical Study of Thinking Aloud Usability Testing from a Cultural Perspective - Qingxin Shi (Copenhagen Business School (CBS), 2010)
Usability evaluation methods are widely used to assess and improve the user interface design. This dissertation investigates the thinking aloud usability testing from a cultural perspective. In a test situation, representative users are required to verbalize their thoughts as they perform their tasks while using the system, and an evaluator observes the user's task performance and comes up with usability problems. The primary goal of a usability test is to find a list of usability problems. In this research, the impacts of evaluators' and users' cultural backgrounds on both the result and the process of the thinking aloud usability testing were investigated. Regarding the results of the usability testing, the identified usability problem was the main focus, whereas for the process of testing, the communication between users and evaluators was the main focus. In this dissertation, culture was regarded as cognitive styles and communication orientations. For the theories of thinking aloud, both Ericsson and Simon's classic model, and Boren and Ramey's revised model for usability testing were taken into account. Based on the culture theories and thinking aloud models, hypotheses were developed to investigate the evaluators' identified usability problems in different cultural settings, and themes were put forward to investigate the evaluators' and users' communications. In order to investigate the hypotheses and themes, an experimental study was conducted. The experimental design consisted of four independent groups with evaluators and users from similar or different cultures (Danish and Chinese). Empirical data were collected by using background questionnaires, usability problem forms, usability problem lists, video recordings of the testing and interviews. The usability testing software “Morae” was used to record the whole testing, including the faces of the evaluators and users, the screen and keyboard activities. Evaluators' and users' communications were analyzed by the behavioural coding and analysis software “Observer XT 8.0” with a well defined coding system. The results of the systematic study of the thinking aloud usability testing in the context of the intra- and inter-cultural usability engineering show that the evaluators' cultural backgrounds do have some influences on the usability testing; however, the influences are different for the tests with Western and East Asian users. The main findings of this research have implications for both usability research and practice. The methodological approach also gives inspiration for usability evaluation studies
hdl:10398/8167
Keywords: 650