The AI Assessment Literacy Gap: Preparedness of Indonesian Pre-Service EFL Teachers for AI-Driven Language Assessment

Abstract

Integrating AI into language assessment requires EFL teachers to acquire/develop new competencies that go beyond the traditional Language Assessment Literacy (LAL). Nevertheless, studies on pre-service teachers’ readiness in AI-driven assessment are still rare, especially in the Indonesian context. The current research examines the readiness of pre-service EFL teachers at Universitas Hindu Negeri I Gusti Bagus Sugriwa Denpasar to competently and ethically design, carry out, and assess language tests using AI. The research design was a mixed-methods sequential explanatory, with data collected through classroom questionnaire from 26 pre-service teachers and semi-structured interviews from 12 participants and 3 EFL lecturers. Quantitative data were analyzed with descriptive statistics, and qualitative data were subjected to thematic analysis. Results found participants held positive attitudes to AI-based assessment (M = 3.41), moderate ethical awareness (M = 3.34), low knowledge (M = 2.89) and very weak skills (M = 2.67). Qualitative findings revealed a total absence of experiential knowledge, with none of the participants interviewed having designed/implemented any AI-driven assessment. Academics confirmed that nothing systematic in the curriculum ensures students develop AI assessment literacy. Qualitative data revealed that there are three major gaps observed: practical skills, technical knowledge and ethical application. The study finds that all the constructs of self-efficacy in the pre-service teachers provide a limited to moderate level of preparedness, and while positive attitudes and ethical awareness are associated with practice skills, they are insufficiently supported by these competencies. Recommendations of the Task Force include: integrating ethical training into the curriculum; investing in faculty development; and institutional investment.