Validation of a four-item questionnaire for assessing student engagement in innovative teaching methods

Andy Chun Yin Chong, Simon Ching Lam, Simon Wing Lung Yau, Chun Wai Tam and Jonathan Ka Ming Ho
The Open University of Hong Kong
Hong Kong SAR, China

Student engagement has become crucial for evaluating the effectiveness of technological innovations applied in higher education, including, but not limited to, virtual reality, mobile learning and simulation. Most of the existing instruments for comprehensively assessing all the three dimensions of students’ engagement level are too lengthy to be applied in the context of a tight lecture schedule. This study aims to develop a four-item questionnaire for measuring the level of student engagement in innovative teaching methods without losing the comprehensiveness of its theoretical framework.

The questionnaire includes three items selected from behavioural, cognitive, and emotional engagement and one item for the overall engagement level. Through a cross-sectional survey study, 301 convenience sample data from Year 1 Bachelor of Nursing students were collected about their engagement in the instant interactive learning system employed in the lecture. One hundred randomly-selected data were split for conducting exploratory factor analysis (EFA, by maximum likelihood factor extraction) and the other 201 data were used for performing confirmatory factor analysis (CFA) in order to delineate the factorial structure of the four selected items of student engagement.

The statistical results revealed a single factor structure in the four-item scale of student engagement through EFA, with acceptable internal consistency. The findings from CFA also confirmed the good model fit and factor validity of the one-factor model of the questionnaire.

With satisfactory reliability and validity, this slimmed down questionnaire would help researchers to assess the level of engagement of students more efficiently in the tight and limited lecture time. Further studies can be conducted on different cohorts or programmes for getting more data in order to provide a more generalized validation of the instrument and investigate its measurement invariance across groups.