Document Type : research paper
Authors
University of Qom
Abstract
Till recently, text difficulty has commonly been determined by employing readability formulas, however, major criticisms have been leveled against readability formulas (Graves & Graves, 2003).This research project aimed at determining text difficulty through readability formulas and Coh-Metrix. In other words, the study investigated the role of text difficulty in EFL learners’ pedagogic task performances. Since both males and females took part in the study, firstly the researchers aimed at finding out whether a different pattern of task performance existed for each. Secondly, all participants were provided with two different reading passages whose difficulty levels were determined by Coh-Metrix and readability formulas. Finally, a self-efficacy questionnaire was administered to delve into learners’ self-perceptions about their own performances on the pedagogic tasks. Descriptive statistics, paired samples t-test and repeated measures ANOVA were utilized to analyze the data. The results indicated that gender of the students had no significant impact on the learners’ performances on the pedagogic tasks. The findings revealed that text difficulty and the learners’ self-efficacy significantly affected EFL learners’ performance on the pedagogic tasks. The findings suggest that determining difficulty level of the texts through Coh-Metrix could be considered a step forward and will certainly assist language teachers and syllabus designers who strive to tailor the appropriate tasks and materials to learners at differing level of language proficiency. The results also imply that self-perceptions of learners might be a true predictor of their own performances on different tasks in general, and on pedagogic tasks in particular
Keywords
Agnihorti, R.K., & Khanna, A.L. (1992). Evaluating the readability of school text books: An Indian study. Journal of Reading, 35(4), 282-288.
Alexander, P. A, & Jetton, T. L. (2000). Learning from text: A multidimensional and developmental perspective. In M. Kamil, P. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research, volume III (pp. 285-310). Mahwah, NJ: Erlbaum.
Anderson, T. H., & Armbruster, B. B. (1984). Content area textbooks. In R. C. Anderson, J. Osborn, & R. J. Tierney (Eds.), Learning to Read in American Schools (pp. 193-226). Hillsdale, NJ: Erlbaum.
Badgett, B.A. (2010). Toward the development of a model to estimate the readability of credentialing-examination materials. UNLV, Paper 185. Beck, I. L., & McKeown, M. G. (1989). Expository text for young readers. The issue of coherence. In L. Resnick (Ed.), Essays in honor of Robert Glaser (pp. 47-66), Hillsdale, NJ: Erlbaum.
Beck, I. L., McKeown, M. G., & Gromoll, E. W. (1989). Issues that may affect social studies learning: Examples from four commercial programs. Cognition and Instruction, 6(1), 99-158.
Beck, I. L., McKeown, M. G., & Worthy, J. (1995). Giving a text voice can improve students' understanding. Reading Research Quarterly, 30(2), 220-238.
Britton, B. K., & Black, J. B. (Eds.). (1985). Understanding expository text: A theoretical and practical handbook for analyzing explanatory text. Hillsdale, NJ: Erlbaum
Britton, B. K; Guelgoez, S. (1991). Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83(3), 329-345.
Berardo, S. A. (2006). The use of authentic materials in the teaching of reading. The Reading Matrix,6(2), 60-68.
Brown, J. D. (1998). An EFL readability index. JALT Journal, 29(2), 7–36.
Carrell, P.L. (1987). Readability in ESL. Reading in a Foreign Language, 4(1), 21-40.
Chall, J. S., Bissex, G. L., Conard, S. S., & Harris-Sharples, S. (1996). Qualitative assessment of text difficulty: A practical guide for teachers and writers. Cambridge, MA: Brookline Books.
Chall, J., & Dale, E. (1995). Readability revisited: The new Dale-Chall readability formula. Cambridge, MA: Brookline Books.
Chambliss, M., & Calfee, R. C. (1998). Textbooks for learning: Nurturing children's minds. Cambridge, MA: Basil Blackwell.
Crossley, S.A., Louwerse, M. M., McCarthy, P.M., & McNamara, D. (2007). A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91(1), 15-30.
Crossley, S.A., Greenfield, J., & McNamara, D.S. (2008). Assessing text readability using cognitively based indices. TESOL Quarterly, 42(3), 475-493.
Crossley, S.A., & McNamara, D.S. (2008). Assessing L2 reading text at the intermediate level: An approximate replication of Crossley, Louwerse, McCarthy, & Mcnamara (2007). Language Teaching, 41(3), 409-429.
Crossley, S. A., Allen, D.B., & McNamara, D.S. (2011). Text readability and intuitive simplification: A comparison of readability formulas. Reading in a Foreign Language, 23(1), 84-101.
Davison, A., & Kantor, R. (1982). On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly, 17(2), 187–209.
Flesch, R. (1948). A new readability yardstick. The Journal of Applied Psychology, 32,221–233.
Fulcher, G. (1997). Text difficulty and accessibility: Reading formulae and expert judgment. System, 25(4), 404-519.
Gernsbacher, M. (1997). Coherence cues mapping during comprehension. In J. Costermans & M. Fayol (Eds.), Processing inter clausal relationships. Studies in the production and comprehension of text (pp. 3–22). Mahwah NJ: Erlbaum.
Goldman, S. R., & Rakestraw, J. A., Jr. (2000). In M. Kamil, P. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of Reading Research, volume III (pp. 3-11-335). Mahwah, NJ: Erlbaum.
Graesser, A. C., McNamara, D. D., Louwerse, M. L., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36, 193–202.
Graesser, A. C., McNamara, S. D., & Kulikovich, M. J. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223-234.
Graves, M.F., & Graves, B.B. (2003). Scaffolding reading experiences: Designs for student success. Christopher-Gordon.
Greenfield, G. (1999). Classic readability formulas in an EFL context: Are they valid for Japanese speakers? Unpublished doctoral dissertation, Temple University, Philadelphia,PA, United States.
Greenfield, J. (2004). Readability formulas for EFL.JALT Journal, 26(1), 5–24.
Haberlandt, K. F., & Graesser, A. C. (1985). Component processes in text comprehension and some of their interactions. Journal of Experimental Psychology. General, 114, 357–374.
Hamsik, M. J. (1984). Reading, readability, and the ESL reader. Unpublished doctoral dissertation, University of South Florida.
Hess, K. & Biggam, S. (2004). A discussion of increasing text complexity. Appendices for New England Common Assessment Program (NECAP).
Homan, S., Hewitt, M., Linder, S. (1994). The development and validation of a formula for measuring single-sentence test item readability. Journal of Educational Measurement, 31, 349-358.
Jurafsky, D., & Martin, J. H. (2000). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice-Hall.
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354.
Just, M. A., & Carpenter, P. A. (1987). The psychology of reading and language comprehension. Boston: Allyn & Bacon, Inc.
Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel, Research Branch Report 8–75, Millington,TN: Naval Technical Training, U. S. Naval Air Station, Memphis, TN.
Kitikanan, P. & Sasimonton, P. (2017). The relationship between English self-efficacy and English learning achievement of L2 Thai learners. Language Education and Acquisition Research Network Journal, 10(1),148-163.
Koda, K. (2005). Insights into second language reading. Cambridge: Cambridge University Press.
Linderholm, T., Everson, M. G., van den Broek, P., Mischinski, M., Crittenden, A., & Samuels, J. (2000). Effects of causal text revisions on more- and less-skilled readers' comprehension of easy and difficult texts. Cognition & Instruction, 18, 525-556.
Lu, X. (2008). Automatic measurement of syntactic complexity using the revised developmental level scale.American Association for Artificial Intelligence.
McCarthy, M. P., Lightman, J. E., Dufty, F. D., & McNamara, S. D. (2019). Using Coh-Metrix to assess cohesion and difficulty in high-school textbooks.
McKeown, M. G., Beck, I. L., Sinatra, G. M., & Loxterman, J. A. (1992). The contribution of prior knowledge and coherent text to comprehension. Reading Research Quarterly,27, 78-93.
McNamara, D. S., Kintsch, E., Butler-Songer, N., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43.
McNamara, D. S., Louwerse, M. M., & Graesser, A. C. (2002). Coh-Metrix (Version 2.0) [Software]. Memphis, TN: University of Memphis, Institute for Intelligent Systems. Available from http://cohmetrix.memphis.edu/cohmetrixpr/index.html
Oakland, T. & Lane, H. (2004). Language, reading, and readability formulas: Implications for developing and adapting tests. International Journal of Testing 4, 239-252.
Perfetti, C. A. (1985). Reading ability. Oxford: Oxford University Press.
Rayner, K., &Pollatsek, A. (1994). The psychology of reading. Englewood Cliffs, NJ: Prentice Hall.
Prabhu, S. N. (1987). Second language pedagogy. Oxford: Oxford University Press.
Robinson, P. (2001). Task complexity, task difficulty, and task production: exploring interactions in a componential framework. Applied Linguistics, 22(1), 27-57.
Rubin, A. (1985). How useful are readability formulas? In J. Osborn, P. T. Wilson, & R. C. Anderson (Eds.), Reading education: Foundations for a literate America (pp. 61–77). Lexington, MA: Lexington Books.
Sawyer, M. H. (1991). A review of the research in revising instructional text.Journal of Reading Behavior, 23(3), 307-333.
Sinha, M., Sharma, S., Dasgupta, T., &Basu, A. (2012). New readability measure for Bangla and Hindi texts. Proceedings of COLING, Mumbai.
Skehan, P. (2001). Tasks and language performance assessment. In M. Bygate et al. (Eds). Researching pedagogic tasks. Second language learning and testing (pp.99-118). Harlow: Longman.
Tamor, L. (1981). Subjective text difficulty: an alternative approach to defining the difficulty level of written text. Journal of Reading Behavior, XIII (2), 165-172.
)