Mixed Methods Research in an EFL Context: A Quality Assessment Perspective

Amini Farsani, Mohammad; Mohammadi, Vali

doi:10.30479/jmrels.2021.14436.1771

Document Type : research paper

Authors

¹ دانشگاه علم و صنعت ایران

² University of Mohaghegh Ardabili

https://doi.org/10.30479/jmrels.2021.14436.1771

Abstract

Beyond the mono-method quantitative and qualitative research syntheses (e.g. meta-analysis and meta-ethnography, respectively) and with a pragmatic perspective on conducting mixed methods research (MMR), recently a very few research synthesists have adopted a Mixed Methods Research Synthesis (MMRS) approach to answer complex review questions. Therefore, to better understand the issue of quality, this study takes the initiative in aligning the mixed methods research quality with the Plonskyian views with specific reference to study quality proposed in methodological synthesis literature. The main purpose of the methodological synthesis here was to provide empirically-based evidence for describing and evaluating mixed methods studies in an Iranian EFL (English as a Foreign Language) context. We synthesized mixed methods theses in an Iranian EFL context by describing and evaluating three interrelated components of study quality through focusing on transparency and reporting practices related to: (a) MMR formulation stage (or MMR problem specification stage), (b) MMR design-related features, and (c) MMR interpretation and integration (or MMR implementation stage). The cumulative findings highlighted a set of deficiencies and strengths across MMR studies in the respective EFL setting. Of notable results were unsatisfactory attention to the issue of integration and transparency at the levels of design, method, and interpretation. The study has implications for designing and implementing sound MMR studies. Furthermore, it includes suggestions for doing solid MMR research as well as writing and preparing well-founded mixed methods articles.

Keywords

20.1001.1.26765357.2022.9.2.5.2

References

Amini Farsani, M., & Babaii, E. (2020). Applied linguistics research in three decades: a methodological synthesis of graduate theses in an EFL context. Quality and Quantity, 54, 1257–1283.

Byrnes, H. (2013). Notes from the editor. The Modern Language Journal, 97, 825-827.

Brown, J. D. (2014). Mixed methods research for TESOL. University of Edinburgh Press.

Bryman, A. (2014). June 1989 and beyond: Julia Brannen’s contribution to mixed methods research. International Journal of Social Research Methodology, 17, 121-131.

Christ, T. W. (2007). A recursive approach to mixed methods research in a longitudinal study of postsecondary education disability support services. Journal of Mixed Methods Research, 1, 226-241.

Collins, K. M. T., Onwuegbuzie, A. J., & Jiao, Q. G. (2006). Prevalence of mixed methods sampling designs in social science research. Evaluation and Research in Education, 19, 83–101.

Collins, K. M. T., Onwuegbuzie, A. J., & Johnson, R. B. (2012). Securing a place at the table: Introducing legitimation criteria for the conduct of mixed research. American Behavioral Scientist, 56, 849–865.

Cooper, H. (2016). Research synthesis and meta-analysis: A step-by-step approach. Sage.

Creamer, E. G. (2018). An introduction to fully integrated mixed methods research. Sage.

Creswell, J. W. (2009). Mapping the field of mixed methods research. Journal of Mixed Methods Research, 3(2), 95-108.

Creswell, J.W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4^th ed.). Sage.

Creswell, J. W. (2015). A concise introduction to mixed methods research. Sage.

Creswell, J. W., & Plano Clark, V.L. (2011). Designing and conducting mixed methods research (2^nd ed.). Sage.

Dörnyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford University Press.

Duff, P., & Lazaraton, A. (2000). Current trends in research methodology and statistics in applied linguistics. TESOL Quarterly, 34, 175-181.

Fàbregues, S. & Molina-Azorín, J.F. (2017). Addressing quality in mixed methods research: a review and recommendations for a future agenda. Quality and Quantity, 51, 2847-2863.

Fetters, M. D. (2020). The mixed methods research workbook: Activities for designing, implementing, and publishing projects. Sage.

Fetters, M. D., & Freshwater, D. (2015a). Publishing a methodological mixed methods research article. Journal of Mixed Methods Research, 9, 203-213.

Fetters, M. D., & Freshwater, D. (2015b). The 1 + 1 = 3 integration challenge. Journal of Mixed Methods Research, 9, 115-117.

Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-analysis in social research. Sage.

Greene, J. C. (2007). Mixed methods in social inquiry. Jossey-Bass.

Greene, J. C. (2015). Preserving distinctions within the multimethod and mixed methods research merger. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp. 606-615). Oxford University Press.

Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.

Ivankova, N.V., & Greer, J. L. (2015). Mixed methods research and analysis. In B. Paltridge & A. Phakiti (Eds.), Research methods in applied linguistics: A practical guide (pp.73-91). Bloomsbury.

Henning, G. (1986). Quantitative methods in language acquisition research. TESOL Quarterly, 20, 701-708.

Hashemi, M. R. & Babaii, E. (2013). Mixed methods research: toward new research designs in applied linguistics. The Modern Language Journal, 97(4), 828-852.

Heyvaert, M., Hannes, K., & Onghena, P. (2016). Using mixed methods research synthesis for literature reviews. Sage.

Johnson, R.B., & Christensen, L. (2014). Educational research: Quantitative, qualitative, and mixed methods research (5^th ed.). Sage.

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133.

Ioannidis, J. P. A. (2018). Meta-research: why research on research matters. PLoS Biol, 16(3), 1-6.

Lazaraton, A. (2005). Quantitative research methods. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 209–224). Routledge.

Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66-81.

Maxwell, J.A. (2016). Expanding the history and range of mixed methods research. Journal of Mixed Methods Research, 10(1), 12-27.

Molina-Azorin, J. F., and Fetters, M. D. (2016). Mixed methods research prevalence studies: field- specific studies on the state of the art of mixed methods research. Journal of Mixed Methods Research, 5(1), 7-24.

Morse, J. M., & Niehaus, L. (2009). Mixed method design: Principles and procedures. CA.

Norris, J. M., & Ortega, L. (2006). The value and practice of research synthesis for language learning and teaching. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 3-50). John Benjamins.

Norouzian, R. & Plonsky, L. (2018). Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research, 34(2), 257-271.

Onwuegbuzie, A. J. (2012). Introduction: Putting the mixed back into quantitative and qualitative research in educational research and beyond: Moving towards the radical middle. International Journal of Multiple Research Approaches, 6, 192–219.

Onwuegbuzie, A. J., & Corrigan, J.A. (2014).Improving the quality of mixed research reports in the field of human resource development and beyond: A call for rigor as an ethical practice. Human Resource Development Quarterly, 25(3), 273-299.

Onwuegbuzie, A.J. & Leech, N.L. (2006). Linking research questions to mixed methods data analysis procedures. The Qualitative Report, 11(3), 474–498.

Onwuegbuzie, A. J., Johnson, R. B., & Collins, K. M. T. (2009). A call for mixed analysis: A philosophical framework for combining qualitative and quantitative. International Journal of Multiple Research Approaches, 3(2), 114–139.

Onwuegbuzie, A. J., & Corrigan, J.A. (2014). Improving the quality of mixed research reports in the field of human resource development and beyond: A call for rigor as an ethical practice. Human Resource Development Quarterly, 25(3), 273-299.

Ortega, L. (2015). Research synthesis. In B. Paltridge & A. Phakiti (Eds.), Research methods in applied linguistics: a practical guide (pp. 219-237). Bloomsbury Publications.

Plano Clark, V. L., & Badiee, M. (2010). Research questions in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2^nd ed., pp. 275–304). Sage.

Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655-687.

Plonsky, L., & Gass, S. (2011). Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61, 325-366.

Plonsky, L., & Gonulal, T. (2015). Methodological synthesis in quantitative L2 research: A review of reviews and a case study of exploratory factor analysis. Language Learning, 65, Supp. 1, 9-36.

Plonsky, L., & Oswald, F. (2015). Meta-analyzing second language research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 106-128). New York, NY: Routledge.

Poortman, C. L., & Schildkamp, K. (2012). Alternative quality standards in qualitative research? Quality and Quantity, 46, 1727-1751.

Riazi, A. M. (2017). Mixed-methods research in language teaching and learning. Equinox Publishing.

Riazi, A. M. & Candlin, C. N. (2014). Mixed-methods research in language teaching and learning: Opportunities, issues and challenges. Language Teaching, 47, 135-173.

Tashakkori, A., & Creswell, J. W. (2007). Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research, 1(3), 207-211.

Tashakkori, A., & Teddlie, C. (2008). Quality of inferences in mixed methods research: calling for an integrative framework. In M. M. Bergman, (Ed.), Advances in mixed methods research (pp. 101–119). Sage.

Tashakkori, A., Teddlie, C., & Sines, M. C. (2012). Utilizing mixed methods in psychological research. In Weiner, I., Shinka, J. A. & Velicer, W. F. (Eds.) Handbook of psychology (2^nd ed.), Volume 2. (Research methods in psychology). Wiley.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

Yin, R. K. (2009). Mixed methods research: Are the methods genuinely integrated or merely parallel? Research in the Schools, 13(1), 41-47.

Journal of Modern Research in English Language Studies

Mixed Methods Research in an EFL Context: A Quality Assessment Perspective

References

References

Volume 9, Issue 2
April 2022
Pages 99-122

Mixed Methods Research in an EFL Context: A Quality Assessment Perspective

References

References

Volume 9, Issue 2April 2022Pages 99-122

Volume 9, Issue 2
April 2022
Pages 99-122