The Effectiveness of Using AI Tool Grammarly in Improving ESL University Students’ Descriptive Writing Scores in English Proficiency Courses

Abstract

The current study investigates the effectiveness of using artificial intelligence (AI) tool Grammarly in improving English as a Second Language (ESL) university students’ descriptive writing scores in English proficiency courses. An explanatory sequential mixed-methods design was employed, involving 150 ESL university students from five Malaysian universities across the Northern (Perak), Central (Selangor), Eastern (Terengganu), Southern (Johor), and Sarawak research zones, all of whom met the English proficiency criteria of MUET Band 3.0 or CEFR Level B1. Data were collected using a quasi-experimental comparative groups pretest–posttest design, a survey questionnaire, and semi-structured interviews. Quantitative data were analysed through independent samples t-tests, one-way ANOVA, post-hoc tests, Spearman’s correlation, and descriptive analysis, while qualitative data were examined through content analysis. The quantitative findings indicate that Grammarly did not produce statistically significant improvements in overall descriptive writing scores. Nevertheless, its effectiveness varied across research zones, with the strongest impact observed in the Southern and Central zones, moderate or inconclusive effects in the Northern and Eastern zones, and negligible impact in Sarawak. A weak but statistically significant positive relationship was observed between the frequency of Grammarly usage and improvements in descriptive writing scores. Complementing these results, the qualitative data revealed that university students perceived Grammarly as a supportive tool that enhanced language accuracy, vocabulary usage, and writing confidence. Importantly, these positive perception did not consistently translate into measurable improvements in overall descriptive writing scores, particularly for higher-order writing skills, and concerns regarding over-reliance were noted. Collectively, the quantitative and qualitative findings provide a nuanced understanding of the role of Grammarly in supporting university students’ development of descriptive writing scores. By addressing a research gap within the Malaysian context, the current study offers a critical, real-world perspective on the integration of Grammarly, highlighting both its pedagogical benefits and limitations. The findings provide practical insights for university students, educators, and higher Education institutions seeking to make informed decisions regarding the use of Grammarly to support the development of descriptive writing scores.