PERTANIKA JOURNAL OF SCIENCE AND TECHNOLOGY

Online Roadshow has many benefits in promoting a concept or a product to the public. It is a new model with the same purpose as a physical roadshow but with higher scalability and flexibility in terms of time, location, target audience, resource utilization, and data collection capability. However, the prototype implementation of the new model has not been evaluated for its usability. As ISO standards defined, usability has three key elements: effectiveness, efficiency, and satisfaction. The effectiveness and efficiency are highly dependent on the different systems, while satisfaction is measurable by System Usability Scale (SUS). SUS is a quick and easy technique where the usability of a system can be assessed in a short time. However, the result of SUS is only a grading scale, with no information on the problematic areas. Hence, this paper introduces a practical framework that combines the SUS with the Affinity Diagram. With the intention of maintaining the simplicity and elegance of the SUS, an additional open-ended question is asked to assess the usability problem of the website. The proposed practical SUS plus one open-ended-question usability-testing framework was applied to the Online Roadshow website. Since the average SUS score for the Online Roadshow website from the test was relatively low, the Affinity Diagram was used to analyze the open-ended comments from the user. As a result, the practical usability framework identified the usability problems on the Online Roadshow website to assist the developers in improving the usability.

• Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software Quality Journal, 11(4), 325-338. https://doi.org/10.1023/A:1025869312943

• Alhija, F. N. A., & Fresko, B. (2009). Student evaluation of instruction: What can be learned from students’ written comments? Studies in Educational Evaluation, 35(1), 37-44. https://doi.org/10.1016/j.stueduc.2009.01.002

• Brooke, J. (1996). SUS: A “quick and dirty” usability. In P. W. Jordan, B. Thomas, B. A. Weerdmeester & I. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 89-94). Taylor & Francis. https://doi.org/10.1201/9781498710411-35

• Finstad, K. (2006). The system usability scale and non-native English speakers. Journal of Usability Studies, 1(4), 185-188.

• González-Cancelas, N., Molina, B., & Soler-Flores, F. (2020). Study to improve the digitalization of the Spanish port system through an affinity diagram. Deniz Taşımacılığı ve Lojistiği Dergisi, 1(2), 51-68.

• Hicks, K., Dickinson, P., Holopainen, J., & Gerling, K. (2018). Good game feel: An empirically grounded framework for juicy design. In Digital Games Research Association Conference 2018 (pp. 1-17). Authors & Digital Games Research Association DiGRA.

• Islam, M. N., Khan, S. R., Islam, N. N., Rezwan-A-Rownok, M., Zaman, S. R., & Zaman, S. R. (2021). A mobile application for mental health care during COVID-19 pandemic: Development and usability evaluation with system usability scale. In International Conference on Computational Intelligence in Information System (pp. 33-42). Springer. https://doi.org/10.1007/978-3-030-68133-3_4

• Kaya, A., Ozturk, R., & Gumussoy, C. A. (2019). Usability measurement of mobile applications with system usability scale (SUS). In Industrial Engineering in the Big Data Era (pp. 389-400). Springer. https://doi.org/10.1007/978-3-030-03317-0_32

• Leow, K. R., Leow, M. C., & Ong, L. Y. (2021). Online roadshow: A new model for the next-generation digital marketing. In Proceedings of the Future Technologies Conference (pp. 994-1005). Springer. https://doi.org/10.1007/978-3-030-89906-6_64

• Lewis, J. R. (2014). Usability: lessons learned… and yet to be learned. International Journal of Human-Computer Interaction, 30(9), 663-684. https://doi.org/10.1080/10447318.2014.930311

• Lewis, J. R. (2018). Measuring perceived usability: The CSUQ, SUS, and UMUX. International Journal of Human–Computer Interaction, 34(12), 1148-1156. https://doi.org/10.1080/10447318.2017.1418805

• Lewis, J. R., & Sauro, J. (2018). Item benchmarks for the system usability scale. Journal of Usability Studies, 13(3), 158-167. https://doi.org/10.5555/3294033.3294037

• Lucero, A. (2015). Using affinity diagrams to evaluate interactive prototypes. In IFIP Conference on Human-Computer Interaction (pp. 231-248). Springer. https://doi.org/10.1007/978-3-319-22668-2_19

• Rusu, C., Rusu, V., Roncagliolo, S., & González, C. (2015). Usability and user experience: What should we care about? International Journal of Information Technologies and Systems Approach (IJITSA), 8(2), 1-12. https://doi.org/10.4018/ijitsa.2015070101

• Widjaja, W., Yoshii, K., Haga, K., & Takahashi, M. (2013). Discusys: Multiple user real-time digital sticky-note affinity-diagram brainstorming system. Procedia Computer Science, 22, 113-122. https://doi.org/10.1016/j.procs.2013.09.087