Document Type : research paper

Authors

1 PhD candidate of TEFL,،Department of English Science and Research Branch, Islamic Azad University, Tehran, Iran

2 ELT Professor ،Department of English at TarbiatModares University, Tehran, Iran

3 TEFL Associate Professor ،Department of English at Imam Ali University, Tehran, Iran

Abstract

The present research responds to the call for implementing diagnostic assessment through learner involvement. This study followed two aims: first, examining the distinctive effects of diagnostic self- and peer - assessments on EFL learners’ reading comprehension; second, exploring students’ diagnostic rating accuracy in various reading genres. To achieve this, a number of diagnostic reading comprehension tests and a checklist were developed, the construction of which were based on the learners’ challenging reading subskills. The participants include 60 English translation university students. During 12 weeks of instruction, the rating accuracy of the subjects’ diagnostic self- and peer- assessments were investigated while the instructor ratings were taken as the yardstick. Data analysis, using T-test and MANOVA confirmed that the two groups had improvement in reading comprehension but there was no significant difference between the two groups’ gain. In addition, no statistically significant difference was detected among the accuracy of diagnostic self-, peer- and instructor- ratings except for the assessment of main idea and supporting details subskill in descriptive genre and cause and effect subskill in narrative genre. This study can offer evidence for conducting diagnostic assessment through learner engagement to provide students with appropriate feedback and remedial instruction.

Keywords

Alderson, J. C. (2000). Assessing reading. Cambridge University Press.
Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. Continuum.
Alderson, J. C. (2010). Cognitive diagnosis and Q-Matrices in language assessment”: A commentary. Language Assessment Quarterly, 7(10), 96-103. https://doi.org/10.1080/15434300903426748
Alderson, J. C., Clapham, C. M., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
Alderson, J. C., & Huhta, A. (2005). The development of a suite of computer-based diagnostic tests based on the Common European Framework. Language Testing, 22(3), 301- 320. https://doi.org/10.1191/0265532205lt310oa
Alderson, J. C., & Huhta, A. (2011). Can research into the diagnostic testing of reading in a second or foreign language contribute to SLA research? In L. Roberts, G. Pallotti, & C. Bettoni (Eds.), EUROSLA Yearbook (pp. 30–52). Cambridge University Press. https://doi.org/10.1075/eurosla.11.04ald
Alderson, J. C., Brunfaut, T., & Harding, L. (2015). Towards a theory of diagnosis in second and Foreign language assessment: Insights from professional practice across diverse fields. Applied Linguistics, 36(2), 236- 260. https://doi.org/10.1093/applin/amt046
Alexander, L. G. (1968). For and against. Longman Group Ltd
Aryadoust, V. (2017). An integrated cognitive theory of comprehension. International Journal of Listening, 33(2), 71-100. https://doi.org10.1080/10904018.2017.1397519
Ashton, K. (2014). Using self-assessment to compare learners’ reading proficiency in a multilingual assessment framework, System 42, 105–119. https://doi.org/10.1016/j.system.2013.11.006
Bachman, L. F. (1990). Fundamental considerations in language testing. OxfordUniversity Press.
Birjandi, P., & Siyyari, M. (2010). Self-assessment and peer-assessment: A comparative study of their effect on writing performance and rating accuracy. IJAL, 13(1), 23- 45.
Butler, Y. G. (2018). The role of context in young learners’ processes for responding to self-assessment items. The Modern Language Journal, 102(1), 1-20.https://doi.org/10.111/modl.12459
Butler, Y. G., & Lee, J. (2010). The effects of self-assessment among young learners of English. Language Testing, 27(1) 5–31. https://doi.org/10.1177/0265532209346370
Carrell, P. L., & Connor, U. (1991). Reading and writing descriptive and persuasive texts. Modern Language Journal, 75(3), 314-324. https://doi.org/10.1111/j.1540- 4781.1991.tbo5361.x
Chen, H., & Chen, J. (2016). Retrofitting non-cognitive-diagnostic reading assessment under the generalized DINA model framework. Language Assessment Quarterly, 13 (3), 218-230. https://doi.org/10.1080/15434303.2016.1210610
Chen, H., Cai, Y., & Torre, J. (2023). Investigating second language (L2) reading subskill associations: A cognitive diagnosis approach. Language assessment quarterly, 20(2), 166-189. https: //doi.org/10.1080/15434303.2022.2140050
Davis, F. B. (1944). Fundamental factors of comprehension in reading. Psychometrika,9(3), 185- 197.
Denis, D. J. (2021). Applied univariate, bivariate, and multivariate statistics: Understanding statistics for social and natural scientists, with applications in SPSS and R. https://onlinelibrary.wiley.com
DuBravac, S., & Dalle, M. (2002). Reader question formation as a tool for measuring comprehension: Narrative and expository textual inferences in a second language. Journal of Research in Reading, 25(2), 217-231. https://doi.org/10.1111/1467-9817.00170
Elahi, M. S. (2016). Assessing and improving general English university students’ main sub-skills of reading comprehension: A case of university of Bojnord. Sino-US English Teaching, 13 (4), 245-260. https://doi.org/10.17265/1539-8072/2016.04.002
Esfandiari, R., & Jafari, H. (2021). Morphological complexity across descriptive expository, and narrative text types in Iranian lower- intermediate language learners. Issue in Language Teaching, 10(1), 237-267. https://doi.org/10.22054/ilt.2021.59736.580
Farhady, H. & Daftarifard, P. (2006). On the scalability of the components of the reading comprehension ability: A progress report. In H. Farhady (Ed.), Twenty-five years of living with applied linguistics: Collection of articles (pp. 189-204). Rahnama publisher.
Field, A. (2018). Discovering statistics using IBM SPSS, statistics for statistics. (5th ed.). SAGE Publications.
Finkbeiner, C., & Schluer, J. (2017). Developing prospective teachers’ diagnostic skillsthrough collaborative video analysis: Focus on L2 reading. Language Awareness, 26(4), 282-303. https://doi.org/10.1080/09658416.2017.1415
Fletcher, J. M. (2006). Measuring reading comprehension. Scientific Studies of Reading, 10(3), 323–330. https://doi.org/10.1207/s1532799xssr1003_7
George, D., & Mallery, P. (2020). IBM SPSS statistics 26 step by step: A simple guide and reference. Routledge.
Ghahraki, S., & Sharifian, F. (2005). The relationship between overall reading comprehension and determination of fact/ opinion in L2. The Reading Matrix, 5(1), 36-46.
Goodman, K. S. (1967). Reading: A psycholinguistic guessing game. Journal of the Reading Specialist, 6(4), 126-135. https://doi.
org/10.1080/19388076709556976
Grabe, W. (2009). Reading in a second language: Moving from theory to practice. Cambridge University Press.
Gray, M. J. (1987). Comprehension: Process or product? Reading Horizons: A Journal of Literacy and Language Arts, 27 (2), 146-152.
Han, C., (2018). A longitudinal quantitative investigation into the concurrent validity of self and peer assessment applied to English- Chinese bidirectional interpretation in an undergraduate interpreting course. Studies in Educational Evaluation, 1-10. https://doi.org/ 10.1016/j.stueduc.2018.01.001
Han, C., & Riazi, M. (2018). The accuracy of student self-assessments of EnglishChinese bidirectional interpretation: a longitudinal quantitative study. Assessment and Evaluation in Higher Education, 43(3), 386-398. https://doi.org/ 10.1080/02602938.2017.1353062
Harding, L., Alderson, J. C., & Brunfaut, T. (2015). Diagnostic assessment of reading and listening in a second or foreign language: Elaborating on diagnostic principles. Language Testing, 32(3), 317- 336. https://doi.org/10.1177/0265532214564505
Jang, E. E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. (Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign). https://www.researchgate.net/publication/33746641
Jang, E. E. (2009). Demystifying a Q-Matrix for making diagnostic inferences about L2 reading skills. Language Assessment Quarterly, 6(3), 210-238.https://doi.org/ 10.1080/ 15434300903071817
Javidanmehr, Z., & Anani Sarab, M. R. (2019). Retrofitting non- diagnostic reading comprehension assessment: Application of the G- DINA model to a high stakes reading comprehension test. Language Assessment Quarterly,16(3), 294-311. https://doi.org/ 10.1080/15434303.2019.1654479
Karakoc, A. I. (2019). Reading and listening comprehension Subskills: The match between theory, course books, and language proficiency tests. Advances in Language and Literary Studies, 10(4), 166-17. https://doi.org/ 10.7575/aiac.alls.v.10n.4p.166
Kim, A. (2015). Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2
reading ability. Language Testing, 32(2) 227-258. https://doi.org/10.1177/0265532214558457
Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95 (2), 163–182. https://doi.org/10.1037/0033-295x.95.2.163
Kintsch, W. (2005). An overview of top-down and bottom-up effects in comprehension: The CI perspective. Discourse Processes, 39(2-3), 125-128. https://doi.org/10.1080/0163853X.2005.9651676
Lee, Y.-W., & Sawaki, Y. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly,6(3), 239-263. https://doi.org/10.10.80/15434300903079562
Liu, H. H.-T. (2014). The conceptualization and operationalization of diagnostic testing in second and foreign language assessment. Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics, 14, (1), 1-12. https://doi.org/10.7916/D84F23B7

Lu, L. (2018). An analysis of peer-assessment in Chinese as a second language classroom presentation. Chinese Language Teaching Methodology and Technology, 1(3), 18-30.
Ma, W., & Winke, P. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Language Annals, 52(1), 66-86. https://doi.org/10.1111/flan.12379
Mackey, D., & Savage, A. (2010).Read this 2. Cambridge University Press.
Myers, S. S. (1991) Performance in reading comprehension -product or process? Educational Review, 43(3), 257-272. https://doi.org/
10.1080/0013191910430302
Nikmard, F., & Tavassoli, K. (2020). The effect of diagnostic assessment on EFL learners’ performance on selective and productive reading tasks. Journal of Modern Research in English Language Studies 7(1), 79-104. https://doi.org/ 10.30479/jmrels. 2019.11562.1437
Nuttall, C. (1996). Teaching reading skills in a foreign language (2nd ed.). Macmillan Heinemann.
Oscarson, M. (1989). Self-assessment of language proficiency: Rationale and applications. Language Testing, 6(1), 1-13. https://doi.org/
10.1177/026553228900600103
Paleczek, L., Seifert, S., Schwab, S., & Gasteiger-Klicpera, B. (2015). Assessing reading and spelling abilities from three different angles–correlations between test scores, teachers’ assessment and children’s self-assessments in L1 and L2 children. Procedia - Social and Behavioral Sciences, 174, 2200 – 2210. https://doi.org/ 10. 1016/j.sbspro.2015.01.876
Paris, S.G., & Paris, A. H. (2001). Classroom applications of research on selfregulated learning. Educational Psychology, 36(2), 89–101. https://doi.org/ 10.1207/S15326985EP3602_4
Poulaki, S., Dowlatabadi, H., Ahmadian, M., & Yazdani, H. (2020). Dynamic assessment: A diagnostic tool to capture academic second language reading difficulties in the IELTS context. Journal of Modern Research in English Language Studies 7(4), 55-87. https://doi. Org/10.30479/jmrels.2020.11998.1494
Ravand, H. (2016). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 34(8), 782–799. https://doi.org/ 10.1177/0734282915623053
Rost, D. H. (1993). Assessing different components of reading comprehension: Fact or fiction? Language Testing, 10(1), 79–92. https://doi.org/10.1177/026553229301000105
Şahin, A. (2013). The effect of text types on reading comprehension. Mevlana International Journal of Education, 3(2), 57-67. https://doi.org/ 10.13054/mije.13.27.3.2
Sawaki, Y., Kim, H. J., & Gentile, C. (2009). Q-matrix construction: Defining the link between constructs and test items in large-scale reading and listening comprehension assessments. Language Assessment Quarterly, 6(3), 190–209. https://doi.org/10.1080/15434300902801917
Shefelbine, J., & Shiel, G. (1990). Pre service teachers’ schemata for a diagnostic framework in Reading. Reading Research and Instruction, 30(1), 30-43. https://doi.org/ 10.1080/19388079009558031
Song, M.Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25(4), 435–464. https://10.1177/0265532208094272
Spearritt, D (1972). Identification of subskills of reading comprehension by maximum likelihood factor analysis. Reading Research Quarterly, 8(1), 92-111. https://doi.org/ 10.1002/j.2333-8504.1972.tb00192.x
Tengberg, M. (2018): Validation of sub-constructs in reading comprehension tests using teachers’ classification of cognitive targets. Language Assessment Quarterly, 15(2), 169-182. https://doi.org/10.1080/15434303.2018.1448820
Toledo, P. F. (2005). Genre analysis and reading of English as a foreign language: Genre schemata beyond text typologies. Journal of Pragmatics, 37, 1059–1079. https://doi.org/ 10.1016/j.pragma.2005.01.002
Toprak, T. E., & Cakir, A. (2020). Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework. Language Testing, 1–26. https://doi.org/ 10.1177/0265532220941470
Urmston, A., Raquel, M., & Tsang, C. (2013). Diagnostic testing of Hong Kong tertiary students’ English language proficiency: The development and validation of DELTA. Hong Kong Journal of Applied Linguistics 14(2), 60–82.

Vacca, R. T. (1980). A study of holistic and subskill instructional approaches to reading comprehension. Journal of Reading, 23(6), 512–518.
Yang, S. (2021). Constructing diagnostic reading assessment instruments for low-level Chinese as second language learners. Reading in a Foreign Language, 33(2), 212–237.
Yin, J. (2018). A review on researches of “genre-based teaching approaches” in recent 20 years in China. International Journal of Secondary Education, 6(1), 16-23. https://doi.org/ 10.11648/j.ijsedu.20180601.14
Yoshida, M. (2012). The interplay of processing task, text type, and proficiency in L2 reading. Reading in a Foreign Language, 24 (1), 1-29. https://doi.org/ 10125/66671
Zhou, L., & Siriyothin, P. (2011). Effects of text types on advanced EFL learners’ reading comprehension. Journal of Language and Culture, 30(2), 45-66.