Document Type : Research Paper
Authors
1 English Department, Yasouj University
2 Yasouj University
Abstract
Recent research on self-assessment (SA) has primarily dealt with its relationship with students’ scores. However, few studies have systematically explored SA in language proficiency tests. This mixed-methods study examined differences between IELTS and TOEFL applicants’ self-assessments and their actual test scores. It also explored the sources of variations between these two assessments. The study sample included 81 IELTS (n= 51) and TOEFL (n=30) participants. Data collection involved the applicants' self-assessments, their test scores, and semi-structured interviews. Quantitative data were analysed using correlation and regression analyses, while qualitative data were examined through thematic content analysis. The statistical analyses revealed moderate to moderately high correlations between self-assessed and actual test scores. Self-assessments in speaking, reading and writing for both IELTS and TOEFL showed strong correlations with test scores. However, self-assessments in listening were only moderately correlated with actual test scores of both IELTS and TOEFL. In addition, regression analyses indicated that self-assessments in reading, speaking and writing for IELTS, as well as reading and speaking for TOEFL, were significant predictors of test scores. On the other hand, self-assessments in listening for both IELTS and TOEFL, as well as TOEFL writing, were poor predictors of actual test scores. Furthermore, qualitative data analyses highlighted the influence of factors such as experience, psychological aspects, linguistic abilities, background knowledge and feedback in explaining the variations between self-assessments and actual test performance. In conclusion, the paper discusses the findings and implications of the study in the context of language proficiency testing
Keywords
Main Subjects
Ajjawi, R., & Boud, D. (2018). Examining the nature and effects of feedback dialogue. Assessment & Evaluation in Higher Education, 43(7), 1106-1119. https://doi.org/10.1080/02602938.2018.1434128
Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: a study of washback. Language Testing, 13(3), 280-297. https://doi.org/10.1177/026553229601300304
Alkhowarizmi, A., & Hamdani, H. (2022). The effect of using self-assessment technique towards EFL students’ writing skill. Edulitics (Education, Literature, and Linguistics) Journal, 7(2), 88-100.
Azmoode Sis Abad, M., Kiany, Gh. R., & Abassian, Gh. R. (2024). On the effect of diagnostic self- and peer-assessment on reading comprehension: Examining EFL learners’ diagnostic rating accuracy across various genres. Journal of Modern Research in English Language Studies, 11(2), 177-202. https://doi.org/10.30479/jmrels.2023.18703.2204
Bachman, L. F., & Palmer, A. S. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6(1), 14-29.
Birjandi, P., & Bolghari, M. S. (2015). The relationship between the accuracy of self- and peer-assessment of Iranian intermediate EFL learners and their learning styles. Theory and Practice in Language Studies, 5(5), 996-1006.
Blanche, P., & Merino, B. J. (1989). Self‐assessment of foreign language skills: Implications for teachers and researchers. Language Learning, 39(3), 313-338.
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400-413.
Brantmeier, C. (2005). Nonlinguistic variables in advanced second language reading: Learners' self‐assessment and enjoyment. Foreign Language Annals, 38(4), 494-504.
Butler, Y. G., & Lee, J. (2006). On-task versus off-task self-assessments among Korean elementary school students studying English. The Modern Language Journal, 90(4), 506-518. https://doi.org/10.1111/j.1540-4781.2006.00463.x
Butler, Y. G. (2024). Self-assessment in second language learning. Language Teaching, 57(1), 42-56.
Çakmak, F., Ismail, S. M., & Karami, S. (2023). Advancing learning-oriented assessment (LOA): Mapping the role of self-assessment, academic resilience, academic motivation in students’ test-taking skills, and test anxiety management in Telegram-assisted language learning. Language Testing in Asia, 13(1), 1-19.
Cassady, J. C., & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27(2), 270-295. https://doi.org/10.1006/ceps.2001.1094
Cassidy, S. (2007). Assessing inexperienced students' ability to self‐assess: Exploring links with learning style and academic personal control. Assessment & Evaluation in Higher Education, 32(3), 313-330.
Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (2011). Building a validity argument for the Test of English as a Foreign Language. Routledge.
Cotton, F., & Conrow, F. (1998). An investigation of the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania. IELTS Research Reports, 1(4), 72–115.
Fan, J., & Yan, X. (2017). From test performance to language use: Using self-assessment to validate a high-stakes English proficiency test. The Asia-Pacific Education Researcher, 26, 61-73.
Fan, J. J. (2016). The construct and predictive validity of a self-assessment scale. Papers in Language Testing and Assessment, 5(2), 69-100.
González-Betancor, S. M., Bolívar-Cruz, A., & Verano-Tacoronte, D. (2019). Self-assessment accuracy in higher education: The influence of gender and performance of university students. Active Learning in Higher Education, 20(2), 101-114.
Green, A. (2013). Washback in language assessment. International Journal of English Studies, 13(2), 39-51.
Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation. Teaching and Teacher Education, 36, 101-111. https://doi.org/10.1016/j.tate.2013.07.008
Hasnalia, R., & Rifli, N. R. (2023). Self-assessment of English writing skills. Pedagogical Research Journal, 1(1), 7-12.
Kunnan, A. J. (2007). Test fairness, test bias, and DIF. Language Assessment Quarterly, 4(2), 109-112. https://doi.org/10.1080/15434300701375865
Kun, A. I. (2016). A comparison of self- versus tutor assessment among Hungarian undergraduate business students. Assessment & Evaluation in Higher Education, 41(3), 350-367. https://doi.org/10.1080/02602938.2015.1011602
León, S. P., Panadero, E., & García-Martínez, I. (2023). How accurate are our students? A meta-analytic systematic review on self-assessment scoring accuracy. Educational Psychology Review, 35(4), 106.
Li, M., Zhang, X. (2021). A meta-analysis of self-assessment and language performance in language testing and assessment. Language Testing, 38(2), 189-218. https://doi.org/10.1177/0265532220932481
Liu, J. (2021). Correlating self-efficacy with self-assessment in an undergraduate interpreting classroom: How accurate can students be? Porta Linguarum: Revista Internacional De Didáctica De Las Lenguas Extranjeras, (36), 9-25.
Liu, H., & Brantmeier, C. (2019). “I know English”: Self-assessment of foreign language reading and writing abilities among young Chinese learners of English. System, 80, 60-72.
Ma, W., & Winke, P. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Language Annals, 52, 66-86. https://doi.org/10.1111/flan.12379
Mahmoodi, M. H., & Karampour, F. (2019). Relationship between Iranian intermediate EFL learners' foreign language causal attributions, meta-cognitive self-regulation and their L2 speaking performance. Journal of Modern Research in English Language Studies, 6(2), 77-53. https://doi.org/10.30479/jmrels.2019.11253.1406
Manna, V. F., & Yoo, H. (2015). Investigating the relationship between test-taker background characteristics and test performance in a heterogeneous English as a second language (ESL) test population: A factor analytic approach. ETS Research Report Series, 2015(2), 1-23. https://doi.org/10.1002/ets2.12072
Mohammadi, R., Ghanbari, N., & Abbasi, A. (2024). Perceptual (mis) matches between learners' and teachers' rating criteria in the Iranian EFL writing self-assessment context. International Journal of Language Testing, 14(1), 150-165.
Ohata, K. (2005). Potential sources of anxiety for Japanese learners of English: Preliminary case interviews with five Japanese college students in the US. TESL-EJ, 9(3), 1-21.
Oscarson, M. (2013). Self-assessment in the classroom. In A. J. Kunnan (Ed.), The companion to language assessment: Approaches and development (pp. 712-729). John Wiley & Sons, Inc.
Panadero, E., & Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2), 133-148.
Plonsky, L., & Oswald, F.L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878-912.
Powers, D. E., & Powers, A. (2015). The incremental contribution of TOEIC listening, reading, speaking, and writing tests to predicting performance on real-life English language tasks. Language Testing, 32(2), 151-167.
Richard, J. P. J. (2020). Investigating CEFR-J Self-assessment and TOEIC listening and reading scores. The Global Management, 3, 21-32.
Rezvani, R., & Miri, P. (2021). The impact of gender, nativeness, and subject matter on the English as a second language university students’ perception of instructor credibility and engagement: A qualitative study. Frontiers in Psychology, 12, 1-14. https://doi.org/10.3389/fpsyg.2021.702250
Riazi, A. M., Rezvani, R., & Ghanbar, H. (2023). Trustworthiness in L2 writing research: A review and analysis of qualitative articles in the Journal of Second Language Writing. Research Methods in Applied Linguistics, 2(3), 1-14.
Riazi, A. M., Ghanbar, H., & Rezvani, R. (2023). Qualitative data coding and analysis: A systematic review of the papers published in the Journal of Second Language Writing. Iranian Journal of Language Teaching Research, 11(1), 25-47.
Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1-20.
Runnels, J. (2016). Self-assessment accuracy: Correlations between Japanese English learners' self-assessment on the CEFR-Japan's can do statements and scores on the TOEIC. Taiwan Journal of TESOL, 13(1), 105-137.
Saito, H., & Fujita, T. (2004). Characteristics and user acceptance of peer rating in EFL writing classrooms. Language Teaching Research, 8(1), 31-54.
Sambell, K., McDowell, L., & Montgomery, C. (2013). Developing students as self-assessors and effective lifelong learning. In K. Sambell, L. McDowell, & C. Montgomery (Eds.), Assessment for learning in higher education (pp. 120-146). Routledge.
Skehan, P. (2014). Individual differences in second language learning. In E. Kerz, & D. Wiechmann (Eds.), The Routledge handbook of second language acquisition and corpora (pp. 394-406). Routledge.
Smith, S. (2015). Accuracy in speaking self-assessment among Japanese-speaking English learners and its implications. Polyglossia: The Asia-Pacific's Voice in Language and Language Teaching, 27, 41-55.
Stricker, L. J., Wilder, G. Z., & Rock, D. A. (2004). Attitudes about the computer-based test of English as a foreign language. Computers in Human Behavior, 20(1), 37-54. https://doi.org/10.1016/S0747-5632(03)00046-3
Summers, M. M., Cox, T. L., McMurry, B. L., & Dewey, D. P. (2019). Investigating the use of the ACTFL can-do statements in a self-assessment for student placement in an intensive English program. System, 80, 269-287.
Suzuki, Y. (2015). Self-assessment of Japanese as a second language: The role of experiences in the naturalistic acquisition. Language Testing, 32(1), 63-81.
Takala, S., & Kaftandjieva, F. (2000). Test fairness: a DIF analysis of an L2 vocabulary test. Language Testing, 17(3), 323-340. https://doi.org/10.1177/026553220001700303
Trofimovich, P., Isaacs, T., Kennedy, S., Saito, K., & Crowther, D. (2016). Flawed self-assessment: Investigating self- and other-perception of second language speech. Bilingualism: Language and Cognition, 19(1), 122-140.
Weigle, S. (2010). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing, 27(3), 335-353.
Wind, A. M. (2021). Nonlinearity and inter- and intra-individual variability in the extent of engagement in self-reflection and its role in second language writing: A multiple-case study. System, 103, 102672.
Wind, A. M., & Zólyomi, A. (2022). The longitudinal development of self-assessment and academic writing: An advanced writing program. Language Learning in Higher Education, 12(1), 185-207.
Yan, Z. (2020). Self-assessment in the process of self-regulated learning and its relationship with academic achievement. Assessment & Evaluation in Higher Education, 45(2), 224-239.
Zheng, H., Huang, J., & Chen, Y. (2012). Effects of self-assessment training on Chinese students’ performance on college English writing tests. Polyglossia, 23, 33-42.
Zheng, C., Wang, L., & Chai, C. S. (2023). Self-assessment first or peer-assessment first: Effects of video-based formative practice on learners’ English public speaking anxiety and performance. Computer Assisted Language Learning, 36(4), 806-839.
)