Keywords: language assessment, second language acquisition, assessing speech, methodology, validity, reliability
Assessing speech is an essential element of many studies on second language learning or acquisition. For statistical analysis, the participants are often grouped by proficiency. For ample examples, see studies in edited volumes like Martohardjono and Flynn (2021), Ionin and Rispoli (2019), VanPatten and Jegerski (2010). Yet, very little to no elaboration in those studies is given to how the students are tested in their language proficiency. Routinely, various tests are used and then results of the studies are compared.
In this paper, we argue that such comparisons are unwarranted. We focus on speech, and argue that while testing second language speech is a vital element of most papers in the field, considerations of validity (Chapelle and Voss 2021) and reliability for such testing are ubiquitously ignored. The objective of the paper is thus to highlight the drawbacks of using a diversity of tests for assessing second language speech (Michigan test, Goethe, DELF etc.) and argue for a unified framework across the board which would aid interpretation of the results of many studies.
To argue against a wide range of tests, we overview the implications this has for comparing studies. To advocate for the unified framework of assessing speech, we sketch a proposal of a rubric following these requirements:
- is easy to use for inexperienced raters,
- is following the CEFR (CoE 2000, 2020) requirements hence can be applied for any language, even those where validated and reliable tests are not developed yet,
- offers a holistic assessment of performance regardless of the area of the study the participants of which are being tested
- is applicable for any participants aged from ~7 on. We conclude with prospects on further research into the proposed framework of assessing speech, including research on assessing proficiency in signed languages.
References
- Chapelle, Carol A., and Erik Voss, editors. Validity Argument in Language Testing: Case Studies of Validation Research. Cambridge University Press, 2021. Cambridge University Press, https://doi.org/10.1017/9781108669849.
- Council of Europe. Common European Framework of Reference for Languages: Learning, Teaching, Assessment – -Companion Volume. Council of Europe Publishing, 2020, www.coe.int/lang-cefr.
- –. Common European Framework of References. 2000, https://rm.coe.int/1680459f97.
- Ionin, Tania, and Matthew Rispoli, editors. “Three Streams of Generative Language Acquisition Research.” Lald.63, John Benjamins Publishing Company, 2019. benjamins.com, https://benjamins.com/catalog/lald.63.
- Martohardjono, Gita, and Suzanne Flynn, editors. Language in Development: A Crosslinguistic Perspective. MIT Press, 2021.
- VanPatten, Bill, and Jill Jegerski, editors. “Research in Second Language Processing and Parsing.” Lald.53, John Benjamins Publishing Company, 2010. benjamins.com, https://benjamins.com/catalog/lald.53.