Generative Artificial Intelligence (GenAI) Meets Assessment: Experimental Insights into Teacher Candidates’ Attitudes and Acceptance
DOI:
https://doi.org/10.46328/ijemst.5190Keywords:
Generative Artificial, Assessment, Pre-service teacher educationAbstract
This experimental study examined the impact of pre-service teachers' development of performance tasks utilizing differentiated generative artificial intelligence tools—textual, visual, and video-based—on their attitudes towards generative artificial intelligence (AI) and their acceptance levels of AI. A quasi-experimental approach was utilized to address the research issues, incorporating one control group and two randomly allocated experimental groups. Ninety-four pre-service teachers voluntarily participated in the study. The groups were categorized based on three distinct generative artificial intelligence tools utilized by the students in the development of a performance task. As a results of analyses, a significant increase in both positive attitude and acceptance level was reported in the Control group using text-based GenAI; an increase only in positive attitude in the Experiment 1 group; and a decrease in negative attitude in the Experiment 2 group. Engagement with differentiated generative AI tools resulted in notable alterations in in-group attitudes and levels of acceptability. This study underscores the essential requirement for pre-service teachers to implement a nuanced and diversified strategy for AI integration, acknowledging the advantages and limitations of different AI tools and their capacity to affect pre-service teachers' attitudes and acceptance in various manners.
References
Ajzen, I., & Fishbein, M. (2000). Attitudes and the attitude-behavior relation: Reasoned and automatic processes. European Review of Social Psychology, 11(1), 1–33. https://doi.org/10.1080/14792779943000116
Aksu-Dünya, B., & Yıldız-Durak, H. (2024). Hi! Tell me how to do it: Examination of undergraduate students’ chatbot-integrated course experiences. Quality & Quantity, 58(4), 3155–3170. https://doi.org/10.1007/s11135-023-01800-x
Aktay, S. (2022). The usability of images generated by artificial intelligence (AI) in education. International Technology and Education Journal, 6(2), 51–62. https://dergipark.org.tr/en/pub/itej/issue/75198/1233537
Arguson, A., Mabborang, M., & Paculanan, R. (2023). The acceptability of generative ai tools of selected senior high school teachers in schools' divisions office-manila, philippines. Journal of Artificial Intelligence, Machine Learning and Neural Network, 3(6), 1-10. 3. https://doi.org/10.55529/jaimlnn.36.1.10
Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.1007/s10639-019-09968-2
Bannister, P., Santamaría-Urbieta, A., & Alcalde-Peñalver, E. (2023). A Delphi Study on Generative Artificial Intelligence and English Medium Instruction Assessment: Implications for Social Justice. Iranian Journal of Language Teaching Research, 11(3), 53-80.
Berk, R. A. (1986). Performance assessment: Methods & applications. Johns Hopkins University Press.
Bezjak, S. (2024). Perceptions and perspectives: Understanding teachers' attitudes towards AI in education. Development, 23, 25. https://toknowpress.net/ISBN/978-961-6914-31-4/68.pdf
Bower, M., Torrington, J., Lai, J. W., Petocz, P., & Alfano, M. (2024). How should we change teaching and assessment in response to increasingly powerful generative artificial intelligence? Outcomes of the ChatGPT teacher survey. Education and Information Technologies, 1–37. https://doi.org/10.1007/s10639-023-12405-0
Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1).
Brandhofer, G., & Tengler, K. (2024). Acceptance of artificial intelligence in education: Opportunities, concerns, and need for action. Advances in Mobile Learning Educational Research, 4(2), 1105–1113. https://doi.org/10.25082/AMLER.2024.02.005
Bugti, F., Sarhandi, S. A., & Bugti, S. M. (2024). A qualitative study on challenges and opportunities for technology integration in pedagogy. Journal of Development and Social Sciences, 5(3), 100–109. https://doi.org/10.47205/jdss.2024(5-III)10
Cabero-Almenara, J., Palacios-Rodríguez, A., Loaiza-Aguirre, M. I., & Rivas-Manzano, M. D. R. D. (2024). Acceptance of educational artificial intelligence by teachers and its relationship with some variables and pedagogical beliefs. Education Sciences, 14(7), 740.
Chandrasekera, T., Hosseini, Z., & Perera, U. (2024). Can artificial intelligence support creativity in early design processes? International Journal of Architectural Computing, 0(0). https://doi.org/10.1177/14780771241254637
Chiu, T. K. F. (2023). The impact of generative AI (GenAI) on practices, policies, and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments, 1–17. https://doi.org/10.1080/10494820.2023.2253861
Chiu, T. K., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, 100–118. https://doi.org/10.1016/j.caeai.2022.100118
Chung, C. J. (2024). Preservice Teachers’ Perceptions of AI in Education. AI-EDU Arxiv, 1-5. https://doi.org/10.36851/ai-edu.vi0.4155
Cohen, J. (1988). Statistical power analysis for the behavioral sciences, 2nd ed. Erlbaum.
Conrad, E. J., & Rees, L. K. (2024). Generative AI as a catalyst to elevate school health education. Journal of Physical Education, Recreation & Dance, 95(6), 45–51. https://doi.org/10.1080/07303084.2024.2355865
Crawford, J., Cowling, M., & Allen, K. A. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice, 20(3), 02.
Creswell, J. W., & Creswell, J. D. (2020). Research design: Qualitative, quantitative, and mixed methods approach. Sage publications.
Darayseh, A. (2023). Acceptance of artificial intelligence in teaching science: Science teachers' perspective. Computers and Education: Artificial Intelligence, 4, 100132.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly 13(3), 319–339.
Driessens, O., & Pischetola, M. (2024). Danish university policies on generative AI: Problems, assumptions, and sustainability blind spots. MedieKultur: Journal of Media and Communication Research, 40(76), 31–52. https://doi.org/10.7146/mk.v40i76.143595
Echave, J. A., Donato, V. J. P., & Acosta, O. D. (2024). Embracing Generative AI in Education: Exploring Teachers' Perceptions, Practices, and Potential Impact. Jurnal Pendidikan Indonesia Gemilang, 4(2), 211-232.
Evmenova, A. S., Borup, J., & Shin, J. K. (2024). Harnessing the power of generative AI to support ALL learners. TechTrends, 1–12. https://doi.org/10.1007/s11528-024-00966-x
Field, A. (2009). Discovering statistics using SPSS. Sage Publications Ltd.
Gunawan, K. D. H., Kaniawati, I., Setiawan, W., Rochintaniawati, D., & Sinaga, P. (2021, March). Profile of teachers’ integrated science curricula that support by intelligent tutoring systems. Journal of Physics: Conference Series, 1806(1), 012139. https://doi.org/10.1088/1742-6596/1806/1/012139
Halagatti, M., Gadag, S., Mahantshetti, S., Hiremath, C. V., Tharkude, D., & Banakar, V. (2023). Artificial intelligence: the new tool of disruption in educational performance assessment. In Smart Analytics, Artificial Intelligence and Sustainable Performance Management in a Global Digitalised Economy (pp. 261-287). Emerald Publishing Limited.
Hamzah, F., Abdullah, A. H., & Ma, W. (2024). Advancing education through technology integration, innovative pedagogies and emerging trends: A systematic literature review. Journal of Advanced Research in Applied Sciences and Engineering Technology, 41(1), 44–63. https://doi.org/10.37934/araset.41.1.4463
Hisan, U. K., & Amri, M. M. (2023). ChatGPT and medical education: A double-edged sword. Journal of Pedagogy and Education Science,2(01), 71–89. https://doi.org/10.56741/jpes.v2i01.302
Hutson, J., & Cotroneo, P. (2023). Generative AI tools in art education: Exploring prompt engineering and iterative processes for enhanced creativity. Metaverse, 4(1), 14.
Jaiswal, A., & Arun, C. J. (2021). Potential of artificial intelligence for transformation of the education system in India. International Journal of Education and Development Using Information and Communication Technology, 17(1), 142–158.
Kadaruddin, K. (2023). Empowering education through generative AI: Innovative instructional strategies for tomorrow's learners. International Journal of Business, Law, and Education, 4(2), 618–625.
Karacaoglan-Yilmaz, F. G., Yilmaz, R., & Ceylan, M. (2023). Generative artificial intelligence acceptance scale: A validity and reliability study. International Journal of Human–Computer Interaction, 1–13. https://doi.org/10.1080/10447318.2023.228873
Kaur, S., Budhraja, K., Pahuja, A., Nayyar, V., & Saluja, S. (2024). Leveraging Artificial Intelligence in Education: Enhancing Learning Experience. In Ethical AI and Data Management Strategies in Marketing (pp. 125-140). IGI Global. https://doi.org/10.4018/979-8-3693-6660-8.ch010
Kaya, F., Aydin, F., Schepman, A., Rodway, P., Yetişensoy, O., & Demir Kaya, M. (2022). The roles of personality traits, AI anxiety, and demographic factors in attitudes towards artificial intelligence. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2022.2151730
Kim, I., & Kim, K. (2024). A Study on Contents Development for the Use of Generative AI in Elementary and Secondary Classes. Journal of The Korea Society of Computer and Information, 29(8), 223-230. https://doi.org/10.9708/jksci.2024.29.08.223
Kim, S. W. (2023). Change in attitude toward artificial intelligence through experiential learning in artificial intelligence education. International Journal on Advanced Science, Engineering & Information Technology, 13(5). https://doi.org/10.1007/s10639-023-12172-y
Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00426-1
Latif, E., & Zhai, X. (2024). Fine-tuning ChatGPT for automatic scoring. Computers and Education: Artificial Intelligence, 6, 100210. https://doi.org/10.1016/j.caeai.2024.100210
Lee, D., Arnold, M., Srivastava, A., Plastow, K., Strelan, P., Ploeckl, F., ... & Palmer, E. (2024). The impact of generative AI on higher education learning and teaching: A study of educators’ perspectives. Computers and Education: Artificial Intelligence, 6, 100221. https://doi.org/10.1016/j.caeai.2024.100221
Lin, H. (2022). Influences of artificial intelligence in education on teaching effectiveness: The mediating effect of teachers’ perceptions of educational technology. International Journal of Emerging Technologies in Learning, 17(24), 144–156. https://doi.org/10.3991/ijet.v17i24.36037
Liu, Q., Zhuang, Y., Bi, H., Huang, Z., Huang, W., Li, J., Yu, J., Liu, Z., Hu, Z., Hong, Y., Pardos, Z. A., Ma, H., Zhu, M., Wang, S., & Chen, E. (2024). Survey of computerized adaptive testing: A machine learning perspective. arXiv preprint arXiv:2404.00712. https://doi.org/10.48550/arXiv.2404.00712
Lu, H., He, L., Yu, H., Pan, T., & Fu, K. (2024). A study on teachers’ willingness to use generative AI technology and its influencing factors: Based on an integrated model. Sustainability, 16(16), 7216. https://doi.org/10.3390/su16167216
Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. Pearson.
Matias, A., & Zipitria, I. (2023). Promoting Ethical Uses in Artificial Intelligence Applied to Education. In International Conference on Intelligent Tutoring Systems (pp. 604-615). Cham: Springer Nature Switzerland.
Metin, M. (2013). Teachers' difficulties in preparation and implementation of performance task. Educational Sciences: Theory and Practice, 13(3), 1664-1673. Doi: 10.12738/estp.2013.3.1452
Montero, J. B. (2024). Revolutionizing creativity: Unleashing the power of AI in upper elementary art education. International Journal of Learning and Teaching, 10(4), 510-515. https://doi.org/10.18178/ijlt.10.4.510-515
Moorhouse, B. L., Yeo, M. A., & Wan, Y. (2023). Generative AI tools and assessment: Guidelines of the world's top-ranking universities. Computers and Education Open, 5, 100-151. https://doi.org/10.1016/j.caeo.2023.100151
Mujtaba, D. F., & Mahapatra, N. R. (2020, December). Artificial intelligence in computerized adaptive testing. In 2020 International Conference on Computational Science and Computational Intelligence (CSCI) (pp. 649-654). IEEE.
Nair, I., & Das, V. M. (2012). Using Technology Acceptance Model to assess teachers’ attitude towards use of technology as teaching tool: A SEM Approach. International Journal of Computer Applications, 42(2), 1-6.
Nyaaba, M., Shi, L., Nabang, M., Zhai, X., Kyeremeh, P., Ayoberd, S. A., & Akanzire, B. N. (2024). Generative AI as a Learning Buddy and Teaching Assistant: Pre-service Teachers' Uses and Attitudes. arXiv preprint arXiv:2407.11983.
Ofosu-Ampong, K. (2024). Artificial intelligence research: A review on dominant themes, methods, frameworks and future research directions. Telematics and Informatics Reports, 100127.
Ou, A. W., Stöhr, C., & Malmström, H. (2024). Academic communication with AI-powered language tools in higher education: From a post-humanist perspective. System, 121, 103225. https://doi.org/10.1016/j.system.2023.103225
Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8), em2307. https://doi.org/10.29333/ejmste/13428
Palm, T. (2008). Performance assessment and authentic assessment: A conceptual analysis of the literature. Practical Assessment, Research & Evaluation, 13(4). https://scholarworks.umass.edu/pare/vol13/iss1/4
Pettersson, J., Hult, E., Eriksson, T., & Adewumi, T. (2024). Generative AI and Teachers--For Us or Against Us? A Case Study. arXiv preprint arXiv:2404.03486.
Picton, I., & Clark, C. (2024). Children and Young People's Use of Generative AI to Support Literacy in 2024. National Literacy Trust.
Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON) (pp. 1-9). IEEE.
Santos, R. P. D. (2023). Enhancing physics learning with ChatGPT, bing chat, and bard as agents-to-think-with: a comparative case study. arXiv preprint arXiv:2306.00724.
Schepman, A., & Rodway, P. (2023). The General Attitudes towards Artificial Intelligence Scale (GAAIS): Confirmatory validation and associations with personality, corporate distrust, and general trust. International Journal of Human–Computer Interaction, 39(13), 2724-2741. https://doi.org/10.1080/10447318.2022.2085400
Schiavo, G., Businaro, S., & Zancanaro, M. (2024). Comprehension, apprehension, and acceptance: Understanding the influence of literacy and anxiety on acceptance of artificial intelligence. Technology in Society, 77, 102537. https://doi.org/10.1016/j.techsoc.2024.102537
Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education, 6(3), 355-366. https://doi.org/10.1177/20965311231168423
Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52(2), 302-312. https://doi.org/10.1016/j.compedu.2008.08.006
Terwiesch, C. (2023). Would Chat GPT3 get a Wharton MBA? A prediction based on its performance in the operations management course. Mack Institute for Innovation Management at the Wharton School, University of Pennsylvania. https://mackinstitute.wharton.upenn.edu/wp-content/uploads/2023/01/Christian-Terwiesch-Chat-GTP.pdf
Vigna-Taglianti, J. (2024). AI-Generated Images as a Teaching Tool in Foreign Language Acquisition. «Technology and language»(Технологии в инфосфере), 5(3), 85-105. https://doi.org/10.48417/technolang.2024.03.07
Wangdi, P. (2023). Integrating artificial intelligence in education: Trends and opportunities. International Journal of Research in STEM Education, 6(2). https://doi.org/10.33830/ijrse.v6i2.1722
Westera, W., Prada, R., Mascarenhas, S., Santos, P. A., Dias, J., Guimarães, M., ... & Ruseti, S. (2020). Artificial intelligence moving serious gaming: Presenting reusable game AI components. Education and Information Technologies, 25, 351–380. https://doi.org/10.1007/s10639-019-09968-2
Xia, L. A., Sun, O. Y., & Reis, C. (2024). Hearing Students’ Voices: Access, Use Patterns and Attitudes toward Generative AI Technologies among Chinese Students. ChatGPT and Global Higher Education: Using Artificial Intelligence in Teaching and Learning, 122.
Yadav, A. (2020). Assessment of Artificial Intelligence virtual Learning Applications & Conventional Classroom Learning Methods. Journal of emerging technologies and innovative research, 7(3):1405-1413-1405-1413. https://www.jetir.org/papers/JETIR2003207.pdf
Yancey, K. P., Laflair, G., Verardi, A., & Burstein, J. (2023). Rating short L2 essays on the CEFR scale with GPT-4. In E. Kochmar, J. Burstein, A. Horbach, R. Laarmann-Quante, N. Madnani, A. Tack, V. Yaneva, Z. Yuan, & T. Zesch (Eds.), Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (pp. 576–584). Retrieved from https://aclanthology.org/2023.bea-1.49
Yin, M., Ma, B., & Pan, X. (2024). Consumer attitudes towards GENAI advertisements: a multi-experiment comparative study based on engagement. Journal of Cases on Information Technology (JCIT), 26(1), 1-35. https://doi.org/10.4018/JCIT.356502
Yusuf, A., Pervin, N., Román‐González, M., & Noor, N. M. (2024). Generative AI in education and research: A systematic mapping review. Review of Education, 12(2), e3489.
Zebua, N. (2024). Evaluating the Impact of GenAI in High School Education: A Critical Review. Polygon: Jurnal Ilmu Komputer dan Ilmu Pengetahuan Alam, 2(6), 79-86. https://doi.org/10.62383/polygon.v2i6.296
Zhai, X. (2024). Transforming teachers’ roles and agencies in the era of generative AI: Perceptions, acceptance, knowledge, and practices. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-024-10174-0
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Education in Mathematics, Science and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Articles may be used for research, teaching, and private study purposes. Authors alone are responsible for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of the research material.
The author(s) of a manuscript agree that if the manuscript is accepted for publication in the journal, the published article will be copyrighted using a Creative Commons “Attribution 4.0 International” license. This license allows others to freely copy, distribute, and display the copyrighted work, and derivative works based upon it, under certain specified conditions.
Authors are responsible for obtaining written permission to include any images or artwork for which they do not hold copyright in their articles, or to adapt any such images or artwork for inclusion in their articles. The copyright holder must be made explicitly aware that the image(s) or artwork will be made freely available online as part of the article under a Creative Commons “Attribution 4.0 International” license.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
