Students' Perceptions about the Impact of AI Chatbots on their Vocabulary and Grammar in EFL Writing

Authors

  • Winia Waziana Technology of Information and Computer Science, Institut Bakti Nusantara, Indonesia
  • Widi Andewi Technology of Information and Computer Science, Institut Bakti Nusantara, Indonesia
  • Tommy Hastomo English Education Program, STKIP PGRI Bandar Lampung, Indonesia
  • Muhamad Hasbi English Education Department, Universitas Islam Negeri Salatiga, Indonesia

DOI:

https://doi.org/10.18326/register.v17i2.352-382

Keywords:

AI chatbots, ChatGPT, EFL students, Writing

Abstract

The rapid integration of AI technologies in education necessitates understanding their potential effects on language acquisition and proficiency, particularly in writing—a critical skill for EFL learners. This study aimed to investigate the types of Artificial Intelligence (AI) chatbots and scrutinize the students' perceptions about the impact of AI chatbots on their vocabulary and grammar in EFL writing. Using a mixed method, this study involved 100 undergraduate English as a Foreign Language (EFL) students from five universities in Indonesia, highlighting the various AI chatbots they employed in their digital activities. Data collection consisted of questionnaires, interviews, and observation sheets which were analyzed using quantitative descriptive statistics and qualitative thematic analysis. ChatGPT, Gemini, Perplexity, Bing Chat, Ernie, Character AI, Discord Bot, Wren, and Ginger emerged as the most widely used chatbots among the students, with reasons such as helpfulness, work simplification, skill and knowledge transfer, and ease of use serving as the primary motivations for their use. The findings revealed that a significant majority of students experienced substantial improvements in their vocabulary range, syntactic variety, and writing quality when using these AI chatbots. They also unanimously reported AI chatbots had a major positive impact on their language proficiency, particularly in the areas of vocabulary and grammar used in writing. The results of this research suggest that incorporating AI chatbots can be advantageous for improving the writing skills of EFL students. The study also addressed its shortcomings and offered recommendations for future research.

References

Algaraady, J., & Mahyoob, M. (2023). ChatGPT’s Capabilities in Spotting and Analyzing Writing Errors Experienced by EFL Learners. Arab World English Journal, 9, 3–17. https://doi.org/10.24093/awej/call9.1

Amin, M. Y. M. (2023). AI and Chat GPT in Language Teaching: Enhancing EFL Classroom Support and Transforming Assessment Techniques. International Journal of Higher Education Pedagogies, 4(4), 1–15. https://doi.org/10.33422/ijhep.v4i4.554

Athanassopoulos, S., Manoli, P., Gouvi, M., Lavidas, K., & Komis, V. (2023). The use of ChatGPT as a learning tool to improve foreign language writing in a multilingual and multicultural classroom. Advances in Mobile Learning Educational Research, 3(2), 818–824. https://doi.org/10.25082/AMLER.2023.02.009

Baskara, F. R. (2023). Integrating ChatGPT into EFL writing instruction: Benefits and challenges. International Journal of Education and Learning, 5(1), 44–55. https://doi.org/10.31763/ijele.v5i1.858

Belda-Medina, J., & Kokošková, V. (2023). Integrating chatbots in education: insights from the Chatbot-Human Interaction Satisfaction Model (CHISM). International Journal of Educational Technology in Higher Education, 20(1), 1–20. https://doi.org/10.1186/s41239-023-00432-3

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated Cognition and the Culture of Learning. Educational Researcher, 18(1), 32–42. https://doi.org/10.3102/0013189X018001032

Chen, M.-R. A., & Lin, Y.-H. (2023). AI Chatbot-Supported Grammar Tense Learning on EFL Students Learning Effectiveness. 17th International Technology, Education and Development Conference, 1207–1216. https://doi.org/10.21125/inted.2023.0353

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: choosing among five approaches. Sage Publications.

Daeun, H. (2021). An analysis of Korean EFL learners’ experience on English classes using AI chatbot. J-Institute, 6(3), 1–9. https://doi.org/10.22471/ai.2021.6.3.01

Deiniatur, M., & Cahyono, B. Y. (2024). Digital literacy practices of novice English as a foreign language teacher in writing research articles for publication. Journal of Education and Learning (EduLearn), 18(1), 165–172. https://doi.org/10.11591/edulearn.v18i1.20899

Donath, L., Mircea, G., & Rozman, T. (2020). E-Learning Platforms as Leverage for Education for Sustainable Development. European Journal of Sustainable Development, 9(2), 1–19. https://doi.org/10.14207/ejsd.2020.v9n2p1

Flower, L., & Hayes, J. R. (1981). A Cognitive Process Theory of Writing. College Composition and Communication, 32(4), 365. https://doi.org/10.2307/356600

Fyfe, P. (2023). How to cheat on your final paper: Assigning AI for student writing. AI & SOCIETY, 38(4), 1395–1405. https://doi.org/10.1007/s00146-022-01397-z

Gozali, I., Syahid, A., & Suryati, N. (2023). Ten Years after Sutton (2012): Quo Vadis Feedback Literacy? (A Bibliometric Study). Register Journal, 16(1), 139–167. https://doi.org/10.18326/register.v16i1.139-167

Han, J., Yoo, H., Kim, Y., Myung, J., Kim, M., Lim, H., Kim, J., Lee, T. Y., Hong, H., Ahn, S.-Y., & Oh, A. (2023). Recipe: How to integrate ChatGPT into EFL writing education. Proceedings of the Tenth ACM Conference on Learning @ Scale, 416–420. https://doi.org/10.1145/3573051.3596200

Handley, Z. (2024). Has artificial intelligence rendered language teaching obsolete? The Modern Language Journal, 108(2), 548–555. https://doi.org/10.1111/modl.12929

Hasbi, M., Alamsyah, A., Faozan, A., Astawa, N. L. P. N. S. P., Fauzi, A. R. (2024). Useful AI Tools For English Teachers. Rizquna.

Hasbi, M., Islamiah, N., Astawa, N. L. P. N. S. P., Hamidah, F. N., Nor, H., Februati, B. M. N. (2024). How to Teach English to Gen Z Students. Rizquna.

Hastomo, T., Mandasari, B., & Widiati, U. (2024). Scrutinizing Indonesian pre-service teachers’ technological knowledge in utilizing AI-powered tools. Journal of Education and Learning (EduLearn), 18(4), 1572–1581. https://doi.org/10.11591/edulearn.v18i4.21644

Hawanti, S., & Zubaydulloevna, K. M. (2023). AI chatbot-based learning: Alleviating students’ anxiety in English writing classroom. Bulletin of Social Informatics Theory and Application, 7(2), 182–192. https://doi.org/10.31763/businta.v7i2.659

Hidayatullah, E. (2024). The Impact of Talkpal.AI on English Speaking Proficiency: An Academic Inquiry. Journal of Insan Mulia Education, 2(1), 19–25. https://doi.org/10.59923/joinme.v2i1.98

Hol, D., & Aydın, I. (2020). Is Technology in Our Classrooms? EFL Teachers’ Beliefs and Engagement with Technology in the Classroom. Journal of Educational Issues, 6(2), 38. https://doi.org/10.5296/JEI.V6I2.17326

Hsu, T.-C., Huang, H.-L., Hwang, G.-J., & Chen, M.-S. (2023). Effects of incorporating an expert decision-making mechanism into chatbots on students’ achievement, enjoyment, and anxiety. Educational Technology & Society, 26(1), 218–231. https://doi.org/10.30191/ETS.202301_26(1).0016

Istiara, F., & Hastomo, T. (2023). Exploring lecturers and administrative staffs’ strategies to hone EFL students’ digital literacy. JOALL (Journal of Applied Linguistics and Literature), 8(1), 151–172. https://doi.org/10.33369/JOALL.V8I1.25568

Jinowat, N., Unapang, J., Kangsawiwat, T., & Sapai, P. (2024). Fostering critical writing excellence: Leveraging Google Gemini and collaborative online writing with Google Docs. In L. Uden & D. Liberona (Eds.), Learning technology for education challenges (LTEC 2024) (Vol. 2082). Springer, Cham. https://doi.org/10.1007/978-3-031-61678-5_25

Johnson, W. L., & Valente, A. (2011). Adaptive learning. In N. M. Seel (Ed.), Encyclopedia of the Sciences of Learning (pp. 107–111). Springer. https://doi.org/10.1007/978-1-4419-1428-6

Kim, H. S., Cha, Y., & Kim, N. Y. (2021). Effects of AI chatbots on EFL students’ communication skills. Korean Journal of English Language and Linguistics, 21, 712–734. https://doi.org/10.15738/KJELL.21..202108.712

Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20(1), 1–17. https://doi.org/10.1186/s41239-023-00426-1

Lalwani, T., Bhalotia, S., Pal, A., Rathod, V., & Bisen, S. (2018). Implementation of a chatbot system using AI and NLP. SSRN Electronic Journal, 6(3), 26–30. https://doi.org/10.2139/ssrn.3531782

Lancu, L., & Lancu, B. (2023). Interacting with chatbots later in life: A technology acceptance perspective in COVID-19 pandemic situation. Frontiers in Psychology, 13, 1–19. https://doi.org/10.3389/fpsyg.2022.1111003

Lin, C. C., Huang, A. Y. Q., & Lu, O. H. T. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: A systematic review. Smart Learning Environments, 10(1), 1–22. https://doi.org/10.1186/s40561-023-00260-y

Liu, G., & Ma, C. (2024). Measuring EFL learners’ use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 18(2), 125–138. https://doi.org/10.1080/17501229.2023.2240316

Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., & Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Applied Sciences, 12(7), 3239. https://doi.org/10.3390/app12073239

Marginson, S., & Dang, T. K. A. (2017). Vygotsky’s sociocultural theory in the context of globalization. Asia Pacific Journal of Education, 37(1), 116–129. https://doi.org/10.1080/02188791.2016.1216827

Martono, Drajati, N. A., Rochsantiningsih, D. R., & Wijaya, S. A. (2023). Intertextuality in Pre-service Teachers’ Argumentative Essay in Raising AI: Practices and Beliefs. Register Journal, 16(2), 186–206. https://doi.org/10.18326/register.v16i2.186-206

Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2), 1–17. https://doi.org/10.1080/2331186X.2023.2236469

Morales-Chan, M., Amado-Salvatierra, H. R., Medina, J. A., Barchino, R., Hernández-Rizzardini, R., & Teixeira, A. M. (2024). Personalized Feedback in Massive Open Online Courses: Harnessing the Power of LangChain and OpenAI API. Electronics, 13(10), 1960. https://doi.org/10.3390/electronics13101960

Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586

Nugroho, A., Putro, N. H. P. S., & Syamsi, K. (2023). The Potentials of ChatGPT for Language Learning: Unpacking its Benefits and Limitations. Register Journal, 16(2), 224–247. https://doi.org/10.18326/register.v16i2.224-247

Okta, Z., Ashadi, A., Triyono, S., & Djoko Pranowo, D. (2023). Thematic Structure in Students’ Writings: Implications on their Ideas Organization and Development. Register Journal, 16(1), 49–72. https://doi.org/10.18326/register.v16i1.49-72

Ono, K., & Morita, A. (2024). Evaluating Large Language Models: ChatGPT-4, Mistral 8x7B, and Google Gemini Benchmarked Against MMLU. COMPUTING AND PROCESSING, 8(1), 1–7. https://doi.org/10.36227/techrxiv.170956672.21573677/v1

Rane, N., Choudhary, S., & Rane, J. (2023). Education 4.0 and 5.0: Integrating Artificial Intelligence (AI) for Personalized and Adaptive Learning. SSRN Electronic Journal, 1(1), 29–43. https://doi.org/10.2139/ssrn.4638365

Rasul, T., Nair, S., Kalendra, D., Robin, M., Santini, F. de O., Ladeira, W. J., Sun, M., Day, I., Rather, R. A., & Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1), 41–56. https://doi.org/10.37074/JALT.2023.6.1.29

Rintaningrum, R. (2023). Technology integration in English language teaching and learning: Benefits and challenges. Cogent Education, 10(1), 1–21. https://doi.org/10.1080/2331186X.2022.2164690

Risko, E. F., & Gilbert, S. J. (2016). Cognitive Offloading. Trends in Cognitive Sciences, 20(9), 676–688. https://doi.org/10.1016/j.tics.2016.07.002

Rudolph, J., Tan, S., & Tan Samson. (2023). War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning & Teaching, 6(1), 364–389. https://doi.org/10.37074/jalt.2023.6.1.23

Safitri, M., & Fithriani, R. (2024). Exploring higher education EFL students’ perception of AI writing tools in the 5.0 era. Cetta: Jurnal Ilmu Pendidikan, 7(1), 267–276. https://doi.org/10.37329/cetta.v7i1.3158

Sarosa, M., Kusumawardani, M., Suyono, A., & Wijaya, M. H. (2020). Developing a social media-based Chatbot for English learning. IOP Conference Series: Materials Science and Engineering, 732(1), 012074. https://doi.org/10.1088/1757-899X/732/1/012074

Shikun, S., Grigoryan, G., Huichun, N., & Harutyunyan, H. (2024). AI chatbots: Developing English language proficiency in EFL classroom. Arab World English Journal, 1(1), 292–305. https://doi.org/10.24093/awej/ChatGPT.20

Slamet, J. (2024). Potential of ChatGPT as a digital language learning assistant: EFL teachers’ and students’ perceptions. Discover Artificial Intelligence, 4(1), 46. https://doi.org/10.1007/s44163-024-00143-2

Sumakul, D. T. Y. G., & Hamied, F. A. (2023). Amotivation in AI injected EFL classrooms: Implications for teachers. Indonesian Journal of Applied Linguistics, 13(1), 26–34. https://doi.org/10.17509/ijal.v13i1.58254

Taecharungroj, V. (2023). “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter. Big Data and Cognitive Computing, 7(1), 1–10. https://doi.org/10.3390/bdcc7010035

Toboula, C. M. Z. T. (2023). Enhancing Post-Pandemic EFL Education by Leveraging Immersive, NLP-Driven, AI-based Tools That Promote Collaboration and Interactivity within an Educational Approach. International Journal of Education (IJE), 11(1), 63–80. https://doi.org/10.5121/ije.2023.11106

Toscu, S. (2024). An Investigation on the Effectiveness of Chatbots in Evaluating Writing Assignments in EFL Contexts. Mehmet Akif Ersoy Üniversitesi Eğitim Fakültesi Dergisi. https://doi.org/10.21764/maeuefd.1425384

Tu, J. (2020). Learn to Speak Like A Native: AI-powered Chatbot Simulating Natural Conversation for Language Tutoring. Journal of Physics: Conference Series, 1693(1), 012216. https://doi.org/10.1088/1742-6596/1693/1/012216

Utami, P. L., Suprapto, N., Hidaayatullaah, H. N., & Cheng, T.-H. (2024). The comparison of Chat GPT, Perplexity AI, and Scopus database to capture Indonesian higher education quality in achieving SDGs 2030. E3S Web of Conferences, 513, 04001. https://doi.org/10.1051/e3sconf/202451304001

Wahyuni, D. S. (2022). Integrated Classroom-Chatbot Experience : An Alternative Solution for English as Foreign Language Learners. English Language Education and Current Trends (ELECT), 7(2), 63–68. https://doi.org/10.37301/elect.v1i1.36

Widianingtyas, N., Mukti, T. W. P., & Silalahi, R. M. P. (2023). ChatGPT in Language Education: Perceptions of Teachers - A Beneficial Tool or Potential Threat? VELES (Voices of English Language Education Society), 7(2), 279–290. https://doi.org/10.29408/veles.v7i2.20326

Yang, H., Kim, H., Lee, J. H., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327–343. https://doi.org/10.1017/S0958344022000039

Young, J. C., & Shishido, M. (2023). Investigating OpenAI’s ChatGPT Potentials in Generating Chatbot’s Dialogue for English as a Foreign Language Learning. International Journal of Advanced Computer Science and Applications, 14(6), 65–72. https://doi.org/10.14569/IJACSA.2023.0140607

Zhang, J., & Zhang, Z. (2024). AI in teacher education: Unlocking new dimensions in teaching support, inclusive learning, and digital literacy. Journal of Computer Assisted Learning, 40(4), 1871–1885. https://doi.org/10.1111/jcal.12988

Downloads

Published

2024-10-22

How to Cite

Waziana, W., Andewi, W., Hastomo, T., & Hasbi, M. (2024). Students’ Perceptions about the Impact of AI Chatbots on their Vocabulary and Grammar in EFL Writing . Register Journal, 17(2), 352–382. https://doi.org/10.18326/register.v17i2.352-382

Issue

Vol. 17 No. 2 (2024): REGISTER JOURNAL

Section

Articles

License

Copyright (c) 2024 Winia Waziana, Widi Andewi, Tommy Hastomo, Muhamad Hasbi

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

COPYRIGHT NOTICE

REGISTER JOURNAL: https://ejournal.uinsalatiga.ac.id/index.php/register/index is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Copyright Notice
  
An author who publishes in  REGISTER JOURNAL agrees to the following terms:

  • The author retains the copyright and grants the journal the right of first publication of the work simultaneously licensed under the Creative Commons Attribution-ShareAlike 4.0 License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal
  • The author can enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book) with the acknowledgment of its initial publication in this journal.
  • The author is permitted and encouraged to post his/her work online (e.g., in institutional repositories or on their website) before and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of the published work (See The Effect of Open Access).

Read more about the Creative Commons Attribution-ShareAlike 4.0 Licence here: https://creativecommons.org/licenses/by-sa/4.0/.
Privacy Statement
The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal. They will not be made available for any other purpose or to any other party.