Document Type : research paper
Authors
1 Department of English Language, Payame Nour University
2 Imam Khomeini international University
Abstract
Over the past few years, peer-assessment, as an alternative assessment procedure, has drawn the attention of many researchers. In the study, it was attempted to find what kinds of language components peer-assessors attend to when rating their peers' essays and to investigate whether proficiency levels of peer-assessors make a difference in terms of severity and leniency they exercise. Fifty-eight student raters at Imam Khomeini International University in Qazvin rated five essays, using an analytic rating scale. Paper-based test of English as a foreign language (TOEFL) and five-paragraph essays were used to collect the data. FACETS (version 3.68.1) was used to analyze the data. The results of Facets analysis indicated that advanced peer-assessors had more variability in their severity compared to intermediate peer-assessors. Moreover, the majority of peer-assessors were, on average, more severe than lenient. The results also revealed no statistically significant difference between the ratings of intermediate and advanced peer-assessors. The final finding was that task achievement was the most attended assessment criterion, but grammatical range and accuracy was the least attended assessment criterion. The findings suggest peer-assessors do not attach an equal weight to all assessment criteria. The findings of the study may carry implications for the summative assessment of students' abilities.
Keywords
Aschbacher, P. R. (1991). Performance assessment: State activity, interest, and concerns. Applied measurement in Education, 4(4), 275-288.
Berg, E. C. (1999). The effects of trained peer response on ESL students’ revision types and writing quality. Journal of Second Language Writing, 8(3), 225–241.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: principles, policy & practice, 5(1), 7-74.
Boud, D. (1989). The role of self‐assessment in student grading. Assessment in Higher Education, 14(1), 20-30.
Boud, D. (1995). Assessment and learning: Contradictory or complementary? In P. Knight (Ed.), Assessment for learning in higher education (pp. 35–48). Kogan Page Limited.
Boud, D. (2003). Enhanced learning through self-assessment. Kogan Page.
Boud, D., Cohen, R., & Sampson, J. (1999). Peer learning and assessment. Assessment & Evaluation in Higher Education, 24(4), 413-426.
Boud, D., Cohen, R., & Sampson, J. (Eds.) (2001). Peer learning in higher education: learning from and with each other. Kogan Page.
Brown, G., Bull, J., & Pendlebury, M. (1997). Assessing student learning in higher education. Routledge.
Brown, H. (2004). Language assessment: Principles and classroom practices. Longman
Brown, J. B., & Hudson, T. (1998). The alternatives in language assessment. TESOL Quarterly, 32(4), 653- 675.
Brown, S., & Glasner, A. (Eds.) (1999). Assessment matters in higher education: choosing and using diverse approaches. SRHE and Open University.
Brown, S., & Knight, P. (1994). Assessing learners in higher education. Kogan Page.
Brown, S., Race, P., & Rust, C. (1995). Using and experiencing assessment. In P. Knight (Ed.) Assessment for learning in higher education (pp.75–85). Kogan Page.
Cheng, W., & Warren, M. (1997). Having second thoughts: student perceptions before and after a peer assessment exercise. Studies in Higher Education, 22(2), 233-239.
Cheng, W., & Warren, M. (2005). Peer assessment of language proficiency. Language Testing, 22(1), 93–121.
Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98(4), 891–901
Connor-Linton, J. (1995). Cross-cultural comparison of writing standards: American ESL and Japanese EFL. World Englishes, 14, 99–115.
Dancer, W. T., & Dancer, J. (1992). Peer rating in higher education. Journal of Education for Business, 67(5), 306–310.
DiGiovanni, E., & Nagaswami, G. (2001). Online peer review: An alternative to face-to-face? ELT journal, 55(3), 263-272.
Dochy, F. J., & McDowell, L. (1997). Assessment as a tool for learning. Studies in educational evaluation, 23(4), 279-298.
Dochy, F., Segers, M. & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331–350.
Elder, C., Knoch, U., Barkhuizen, G., & von Randow, J. (2007). Individual feedback to enhance rater training: Does it work? Language Assessment Quarterly, 2(3), 175–196.
Esfandiari, R. (2015). Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model. Iranian Journal of Applied Linguistics, 18(2), 77-107.
Esfandiari, R., & Myford, C. M. (2013). Severity differences among self-assessors, peer-assessors, and teacher assessors rating EFL essays. Assessing Writing, 18(2), 111-131.
Fahim, M, & Bijani, H. (2011). The effects of rater training on raters’ severity and bias in second language writing assessment. Iranian Journal of Language Testing, 1(1), 2251-7324.
Falchikov, N. (1986). Product comparisons and process benefits of collaborative peer group and self-assessments. Assessment and Evaluation in Higher Education, 11(2), 146-166.
Falchikov, N. (1995). Peer feedback marking: Developing peer assessment. Innovations in Education and Training International, 32(2), 175–187.
Falchikov, N. (2003). Involving students in assessment. Psychology Learning and Teaching, 3(2), 102–108.
Falchikov, N., & Goldfinch, J. (2000). Student peer assessment: A meta-analysis comparing peer and teacher remarks. Review of Educational Research, 70, 287-322.
Gibbs, G. (1999). Using assessment strategically to change the way students learn, In S. Brown & A. Glasner (Eds.), Assessment matters in higher Education: Choosing and using diverse approaches (pp. 41–53). SRHE & Open University Press.
Goldfinch, J. M., & Raeside, R. (1990). Development of a peer assessment technique for obtaining individual marks on a group project. Assessment & Evaluation in Higher Education, 15(3), 210–225.
Hargreaves, L., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform. American Educational Research Journal, 39(1), 69–96.
Herman, J. L., Aschbacher, P. R., & Winters, L. (1992). A practical guide to alternative assessment. Association for Supervision and Curriculum Development.
Huerta-Macías, A. (1995). Alternative assessment: Responses to commonly asked questions. TESOL Journal, 5(1), 8–11.
Kolomuç, A. (2017). Subject-specific science teachers' views of alternative assessment. Asia-Pacific Forum on Science Learning and Teaching, 18(1), 124-135.
Kuiken, F., & Vedder, I. (2014). Rating written performance: What do raters do and why? Language Testing, 31(3), 329–348.
Leach, L., Neutze, G. & Zepke, N. (2001). Assessment and empowerment: some critical question. Assessment and Evaluation in Higher Education, 26(4), 293–305.
Lee, H. K. (2009). Native and nonnative rater behavior in grading Korean students’ English essays. Asia Pacific Education Review, 10(3), 387–397.
Lee, I. (2004). Error correction in L2 secondary writing classroom: The case of Hong Kong. Journal of Second Language Writing, 13(4), 285–312.
Li, H., Bialo, J. A., Xiong, Y., Hunter, C.V., Guo, X. (2021). The effect of peer assessment on non-cognitive outcomes: A meta-analysis. Applied Measurement in Education, 34(3), 179-203.
Linacre, J. M. (1989/1994). Many-facet Rasch measurement. MESA Press.
Linacre, J. M. (2011). FACETS (Version 3.68.1) [Computer Software]. Chicago, IL: MESA Press.
Loddington, S. (2008). Peer assessment of group work: A review of the literature. Retrieved from http://webpaproject.lboro.ac.uk/files/WebPA_Literature%20review%20.pdf
Long, M. (1985). Input and second language acquisition theory. In S. M. Gass & C. G. Maddan (Eds.), Input in second language acquisition (pp. 337-393). Newbury House.
Marefat, F., & Heydari, M. (2016). Native and Iranian teachers’ perceptions and evaluation of Iranian students’ English essays. Assessing Writing, 27(1), 24-36.
Matsuno, S. (2009). Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms. Language Testing, 26(1), 75–100.
McDowell, L. & Sambell, K. (1999). The experience of innovative assessment: students’ perspectives. In S. Brown & A. Glasner (Eds.), Assessment matters in higher education: choosing and using diverse approaches (pp. 71–82). SRHE & Open University Press.
Mendonca, C. O., & Johnson, K. E. (1994). Peer review negotiations: Revision activities in ESL writing instruction. TESOL Quarterly, 28(4), 745-769.
Minzi, L., & Zhang, X. (2021). A meta-analysis of self-Assessment and language performance in language testing and assessment. Language Testing, 38(2), 189-218.
Myford, C. M., &Wolfe, E. W. (2004). Detecting and measuring rater effects using many facet Rasch measurement: Part II. In E. V. Smith & R. M. Smith (Eds.), Introduction to Rasch measurement (pp. 460–517). JAM Press.
Nakamura, Y. (2002). Teacher assessment and peer assessment in practice (Educational Studies 44). International Christian University. (ERIC Document Reproduction Service No. ED464483).
Orsmond, P., Merry, S., & Reiling, K. (1996). The importance of marking criteria in the use of peer assessment. Assessment and Evaluation in Higher Education, 21(3), 239–250.
Orsmond, P., Merry, S., & Reiling, K. (2000). The use of student derived marking criteria in peer and self-assessment. Assessment & Evaluation in Higher Education, 25(1), 23-38.
Orsmond, P., Merry, S., & Reiling, K. (2002). The use of exemplars and formative feedback when using student derived marking criteria in peer and self-assessment. Assessment & Evaluation in Higher Education, 27(4), 309-323.
Phakiti, A. (2003). A closer look at gender and strategy use in L2 reading. Language Learning, 53(4), 649–702.
Ross, S. (2005). The impact of assessment method on foreign language proficiency growth. Applied Linguistics, 26(3), 317–342.
Sadler, D. R. (1987). Specifying and promulgating achievement standards. Oxford Review of Education, 13(2), 191–209.
Saito, H. (2008). EFL classroom peer assessment: Training effects on rating and commenting. Language Testing, 25(4), 553–581.
Saito, H., & Fujita, T. (2004). Characteristics and user acceptance of peer rating in EFL writing classrooms. Language Teaching Research, 8(1), 31–54.
Saito, H., & Fujita, T. (2009). Peer-assessing peers’ contribution to EFL group presentations. RELC Journal, 40(2), 149–171.
Searby, M., & Ewers, T. (1997). An evaluation of the use of peer assessment in higher education: A case study in the school of music, Kingston University. Assessment & Evaluation in Higher Education, 22(4), 371–383.
Sluijsmans, D., Dochy, F., & Moerkerke, G. (1999). Creating a learning environment by using self-, peer- and co-assessment. Learning Environment Research, 1, 293-319.
Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249–276.
Topping, K. J. (2005). Trends in peer learning. Educational psychology, 25(6), 631-645.
Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20–27.
Topping, K. J. (2010). Peers as a source of formative assessment. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of formative assessment (pp. 69–75). Routledge.
Topping, K. J., Smith, E. F., Swanson, I., & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment and Evaluation in Higher Education, 25(2), 146–169
Vygotsky, L. S. (1987). Mind in society: The development of higher psychological processes. Harvard University Press.
Weigle, S. C. (1998). Using FACETS to model rater training effects. Language Testing, 15(2), 263–287.
Wright, B., & Linacre, M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), p. 370.
Zhan, Y. (2021). What matters in design? Cultivating undergraduates' critical thinking through online peer assessment in a Confucian heritage context. Assessment & Evaluation in Higher Education, 46(4), 615-630.
Zhang, F., Schunn, C., Li, W., Long, M. (2020). Changes in the reliability and validity of peer assessment across the college years. Assessment & Evaluation in Higher Education, 45(8), 1073-1087.
)