ESTABLISHING PERFORMANCE INDICATORS FOR ACADEMIC ENGLISH PROFICIENCY: A CASE FOR EFL UNIVERSITY STUDENTS
DOI:
https://doi.org/10.20319/pijtel.2020.42.264286Keywords:
Ability Bands, Academic English, Concurrent Validity, English ProficiencyAbstract
In Thailand, the B1 level of the CEFR was specified as the goal of English proficiency for high school or vocational certificate graduates. Therefore, for university levels, students’ English ability equal to the B1 level or above is expected. English proficiency tests are needed to fulfill this assessment purpose. Moreover, one of the considering points is that the test results are generally reported quantitatively without meaningful interpretations. This study attempts to investigate the Target Language Use (TLU) domain of academic English proficiency for EFL university students so that performance indicators can be established. The quality of this developed test was evaluated and the concurrent validity was examined to demonstrate how well the developed test correlated with a well-established test. The sample of the study included 39 informants that provided information about the TLU domain and 30 test takers for the main study. The instruments consisted of a questionnaire asking about the TLU domain and the developed academic English proficiency test. The arithmetic mean and Standard Deviation (S.D.) were employed for the analysis of the TLU domain. Construct validity, reliability, and item analysis were analyzed for test quality. Finally, Pearson’s product-moment was implemented in order to examine concurrent validity. Besides, regression analysis was used to demonstrate if the obtained scores could predict the scores from the standardized test. The findings are significant in several ways. First, these findings are able to provide a clearer concept of the performance indicators of academic English proficiency. The developed test can be a useful instrument for several purposes, for example for placement or achievement. Finally, the assessment of concurrent validity can be useful in terms of test score interpretations.
References
Ansastasi, A. & Urbina, S. (1997). Psychological Testing. Upper Saddle Rever, NJ: Prentice Hall.
Athiworakun, C. & Wudthayagorn, J. (2018). Mapping Srinakharinwirot University – Standardized English Test (SWU-SET) onto the Common European Framework of Reference (CEFR). Suranaree J. Soc. Sci. (12) 2: 69 – 84.
Bachman, L. F. (2004). Statistical Analyses for Language Assessment. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511667350
Bachman, L.F. & Palmer, A.S. (1996). Language Testing in Practice: Designing and Developing Useful Language Tests. Oxford: Oxford University Press. https://doi.org/10.1177/026553229601300201
Bachman, L.F. (1990). Fundamental Considerations in Language Testing. Hong Kong: Oxford University Press.
Bamford, J. (2006). International students and their experiences of UK higher education. Paper presented at the Society for Research into Higher Education Annual Conference, Brighton, UK.
Brindley, G. (1998). Describing language development? Rating scales and second language acquisition. In Bachman, L. F. and Cohen, A. D. (eds.), Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press, 112-140. https://doi.org/10.1017/CBO9781139524711.007
Buck, G. (1991). The testing of listening comprehension: An introspective study. Language Testing 8: 67—91. https://doi.org/10.1177/026553229100800105
Cambridge. (2020). International language standards. Retrieved from
https://www.cambridgeenglish.org/exams-and-tests/cefr/
Common European Framework of Reference for Languages-CEFR. (2018). Learning, Teaching, Assessment. Cambridge: Cambridge University Press.
Craighead, W.E. & Nemeroff, C.B. (2004). The Concise Corsini Encyclopedia of Psychology and Behavioral Sciences (3rd Ed.).
Darr, Charles. (2005). A hitchhiker’s guide to validity and reliability. SET Magazine. 55-60. https://doi.org/10.18296/set.0639
Dornyei, Z. (2011). Research Methods in Applied Linguistics. New York: Oxford University Press.
Drummond, R.J. (1996). Appraisal Procedures for Counseling and Helping Professions- (4th ed.). Upper Saddle Rever, NJ.: Prentice Hall.
Ebel R, L. (1979). Essentials of education measurement. New Jersey: Prentice Hall.
Glen, S. (2015). Statistics How To: Concurrent Validity Definition and Examples Retrieved from https://www.statisticshowto.datasciencecentral.com/concurrent-validity/
Han, E. (2007). Academic Discussion Tasks: A Study of EFL Students’ Perspectives EFL Journal 9,1: 8-12.
Hawkey, R. (2004). A modular approach to testing English language skills: the development of the Certificates in English Language Skills (CELS) examinations. New York: Cambridge University Press.
Hinkle, D. E, William, W. and Stephen G. J. (1998). Applied Statistics for the Behavior Sciences. 4th ed. New York: Houghton Mifflin.
Hiranburana, K. et al. (2018). Framework of Reference for English Language Education in Thailand – (FRELE-TH) Based on the CEFR: Revisited in the English Educational Reform. Pasaa Paritat 33, 51 – 91.
Kenyon, D.M. & Romhild, A. (2013). Standard Setting in Language Testing. Retrieved from https://onlinelibrary.wiley.com/doi/epdf/10.1002/9781118411360.wbcla145 https://doi.org/10.1002/9781118411360.wbcla145
Knoch, U. & Frost, K. (2017). Setting Empirical Standards on the Diagnostic English Language Needs Assessment (DELA). The University of Melbourne: Language Testing Research Centre.
Lin, W.L. & Yao, G. (2019). Concurrent Validity. Springer Nature: Switzerland.
McIntire, S. A., & Miller, L. A. (2005). Foundations of psychological testing (2nd ed.). Thousand Oaks: Sage Publishing Co.
McNamara, T., & Roever, C. (2006). Language Testing: The Social Dimension. UK: John Wiley & Sons.
Murphy, K.R. & Davidshofer C.O. (1998). Psychological Testing: Principles and Applications. Prentice Hall: USA.
O'Connell, M. (2016). What role should universities play in today’s society? Retrieved from https://theconversation.com/what-role-should-universities-play-in-todays-society-63515
Office of Educational Assessment. (2020). Understanding Item Analyses. Retrieved from https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/
Read, J. (2015). Assessing English Proficiency for University Study. UK: Palgrave Macmillan. https://doi.org/10.1057/9781137315694
Sattle, J. (1992). Assessment of Children: Revised and Updated Third Edition. San Diego, CA: Jerome Sattler, Inc.
Scarcella, R. (2003) Academic English: A conceptual Framework. Linguistic Minority Research Institute Newsletter. University of California, Santa Barbara. Retrieved from: http://iteslj.org/Articles/Uribe-AcademicEnglish.html
Sornkam, B, Person, K. R., & Yordchim. S. (2018). Reviewing the Common European Framework of Reference for English Language in Thailand higher education. Paper presented at Graduate School Conference, Bangkok, Thailand.
Spolsky, B. (1989). Conditions for second language learning. Oxford: Oxford University Press.
Stevens, Stanley. (n.d.) On the Theory of Scales of Measurement. Retrieved from https://cehs01.unl.edu/aalbano/intromeasurement/mainch2.html
Wudthayagorn, J. (2018). Mapping the CU-TEP to the Common European Framework of Reference (CEFR). LEARN Journal: Language Education and Acquisition Research Network Journal (11) 2: 163 – 180.
Yuyun, I., Meyling, M., Laksana, N., & Abenedgo, D. (2018). A Study of English Proficiency Test among the First Year University Students. Journal of Language and Literature, 18(1), 1 – 8. https://doi.org/10.24071/joll.2018.180101
Downloads
Published
How to Cite
Issue
Section
License
Copyright of Published Articles
Author(s) retain the article copyright and publishing rights without any restrictions.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
