Peer assessment in CFL writing classrooms: An empirical study examining reliability, effectiveness, and teacher acceptance
Article Sidebar
Main Article Content
Abstract
This study investigated the role of peer assessment in Chinese-as-a-foreign-language (CFL) writing classrooms by examining its reliability, effectiveness, and teacher acceptance. Twelve CFL students and six experienced teachers participated as assessors, scoring 20 CFL essays holistically and providing qualitative feedback on content, organization, and language. The findings revealed that while peer assessors showed lower initial reliability compared to teacher assessors (G-coefficients: .53 vs. .66), groups of two to three peer assessors achieved reliability comparable to that of a single teacher assessor. Peer assessors provided qualitative feedback that was as effective as teacher feedback, particularly in areas of content clarity and organizational structure. Teacher acceptance of peer assessment was positive, with all interviewees supporting its use and recognizing its benefits. These results suggest that peer assessment, when properly structured, can be a valuable tool in CFL writing classrooms, enhancing student engagement and improving writing outcomes.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Alqassab, M., Strijbos, J. W., Panadero, E., Ruiz, J. F., Warrens, M., & To, J. (2023). A systematic review of peer assessment design elements. Educational Psychology Review, 35, article number 18. https://link.springer.com/article/10.1007/s10648-023-09723-7
Boud, D., & Molloy, E. (2013). Feedback in higher and professional education: Understanding it and doing it well. Routledge.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. DOI:10.1191/1478088706qp063oa
Braun, Virginia and Clarke, Victoria (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2). pp. 77-101. ISSN 1478-0887
Brennan, R. L. (2001). Generalizability theory. Springer. https://doi.org/10.1007/978-1-4757-3456-0
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. https://doi.org/10.1080/02602938.2018.1463354
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence, 2, 100027. https://doi.org/10.1016/j.caeai.2021.100027
Chen, G. M., & Starosta, W. J. (1998). Foundations of intercultural communication. Allyn & Bacon.
Chen, Y. (2024). The effectiveness of peer feedback on second language writing. Journal of Contemporary Educational Research, 8(2), 68-79.
https://ojs.bbwpublisher.com/index.php/JCER/article/view/6146
Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98(4), 891-901. DOI:10.1037/0022-0663.98.4.891.
Creswell, J. W., & Creswell, J. D. (2023). Research design: Qualitative, quantitative, and mixed methods approaches (6th Ed.). Sage Publications.
Crick, J. E., & Brennan, R. L. (1983). Manual for GENOVA: A generalized analysis of variance system (Version 2.1). The American College Testing Program.
Cronbach, L. J., Gleser, G. C., N. H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. John Wiley & Sons, Inc.
Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32(2), 481-509. https://doi.org/10.1007/s10648-019-09510-3
Ewe, L. C., & Min, F. (2021). Teaching Chinese language outside of China: The case of Chinese teachers in Thailand. 21(4).
Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287-322.
Fathi, J., Ahmadnejad, M., & Yousofi, N. (2019). Effects of blog-mediated writing instruction on L2 writing motivation, self-efficacy, and self-regulation: A mixed methods study. Journal of Research in Applied Linguistics, 10, 159-181.
https://doi.org/10.22055/rals.2019.14722
Gong, Y., Lai, C., & Gao, X. (2020). The teaching and learning of Chinese as a second or foreign language: The current situation and future directions. Frontiers of Education in China, 15(1), 1-13. https://doi.org/10.1007/s11516-020-0001-0
Hanban, T. O. O. C. C. I. (2010). The New HSK test syllabus. The Commercial Press.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.
Hebebci, M. T., Bertiz, Y., & Alan, S. (2020). Investigation of views of students and teachers on distance education practices during the Coronavirus (COVID-19) pandemic. International Journal of Technology in Education and Science, 4(4), 267-282. https://doi.org/10.46328/ijtes.v4i4.113
Huang, J. (2008). How accurate are ESL students’ holistic writing scores on large-scale assessments?—A generalizability theory approach. Assessing Writing, 13(3), 201-218. https://doi.org/10.1016/j.asw.2008.10.002
Huang, J. (2012). Using generalizability theory to examine the accuracy and validity of large-scale ESL writing assessment. Assessing Writing, 17(3), 123-139. https://doi.org/10.1016/j.asw.2011.12.003
Huang, J., & Wang, Y. (2023). Examining Chinese social sciences graduate students’ understanding of research ethics: Implications for their research ethics education. Humanities and Social Sciences Communications, 1-8. DOI: 10.1057/s41599-023-02000-6
Huang, J., Zhou, Y., & Sheeran, T. (2021). Educational researchers’ ethical responsibilities and human subjects’ ethical awareness: implications for research ethics education in China. Ethics & Behavior, 31(5), 321-334.
Huisman, B., Saab, N., Van Den Broek, P., & Van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: A meta-analysis. Assessment & Evaluation in Higher Education, 44(6), 863-880.
https://doi.org/10.1080/02602938.2018.1545896
Kastrati, Z., Imran, A. S., & Kurti, A. (2020). Weakly supervised framework for aspect-based sentiment analysis on students’ reviews of MOOCs. IEEE Access, 8, 106799-106810. https://doi.org/10.1109/ACCESS.2020.3000739
Lee, M., & Evans, M. (2019). Investigating the operating mechanisms of the sources of L2 writing self-efficacy at the stages of giving and receiving peer feedback. Modern Language Journal, 103, 831-847. https://doi.org/10.1111/modl.1259
Li, J., & Huang, J., & Cheng, S. (2022). The reliability, effectiveness, and benefits of peer assessment in college EFL speaking classrooms: Student and teacher perspectives. Studies in Educational Evaluation, 72, 101120
Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education, 45(2),
193-211. https://doi.org/10.1080/02602938.2019.1620679
Liu, N. F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279-290.
https://doi.org/10.1080/13562510600680582
Ma, J., & Zhao, K. (2018). International student education in China: Characteristics, challenges, and future trends. Higher Education, 76(4), 735-751. https://doi.org/10.1007/s10734-018-0235-4
Matsuno, S. (2009). Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms. Language Testing, 26(1), 75-100.
https://doi.org/10.1177/0265532208097337
Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129-144.
Popham, W. J. (2011). Classroom assessment: What teachers need to know. Pearson Education.
Pushpakumara, P. B. S. L. (2024). Use of peer assessment in ESL writing classes: Learners’ perception. Vidyodaya Journal of Humanities and Social Sciences, 9(Special Issue), 1-18. DOI:10.31357/fhss/vjhss.v09irchss2023.a01
Shen, B., Bai, B., & Xue, W. (2020). The effects of peer assessment on learner autonomy: An empirical study in a Chinese college English writing class. Studies in Educational Evaluation, 64, 100821. https://doi.org/10.1016/j.stueduc.2019.100821
Sperber, A., Devellis, R., & Boehlecke, B. (1994). Cross-cultural translation: Methodology and validation. Journal of Cross-Cultural Psychology, 25, 50-524.
https://doi.org/10.1177/0022022194254006
Sun, Q., Chen, F., & Yin, S. (2023). The role and features of peer assessment feedback in college English writing. Frontiers in Psychology, 13, 1070618.
https://doi.org/10.3389/fpsyg.2022.1070618
Tian, J. (2011). The effects of peer editing versus co-writing on writing in Chinese-as-a-foreign language. (Unpublished doctoral dissertation).
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276. https://doi.org/10.3102/00346543068003249
Topping, K. (2017). Peer assessment: Learning by judging and discussing the work of other learners. Interdisciplinary Education and Psychology, 1, 7-23.
https://doi.org/10.31532/InterdiscipEducPsychol.1.1.007
van Zundert, M., Sluijsmans, D., & van Merriënboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20, 270-279. https://doi.org/10.1016/j.learninstruc.2009.08.004
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14.
Wu, W., & Huang, J., Han, C., & Zhang, J. (2022). Evaluating peer feedback as a reliable and valid complementary aid to teacher feedback in EFL writing classrooms: A feedback giver perspective. Studies in Educational Evaluation, 73, 101140. https://doi.org/10.1016/j.stueduc.2022.101140
Xu, C., & Tao, J. (2019). The effects of task types and raters on Chinese writing scoring. Collection of Chinese Language and Literature, 115, 239-250.
Yousefifard, S., & Fathi, J. (2021). Exploring the impact of blogging in English classrooms: Focus on the ideal writing self of EFL learners. International Journal of Instruction, 14, 913-932. https://doi.org/10.29333/iji.2021.14452a
Yu, S., & Lee, I. (2016). Peer feedback in second language writing (2005-2014). Language Teaching, 49(4), 461-493. https://doi.org/10.1017/S0261444816000161
Zhang, F., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.04.002
Zhao, Q. (2010). The scorer reliability of the writing section of the HSK: A case study of experienced and inexperienced scorers. China Examinations, 10, 13-19.