logo logo European Journal of English Language Studies

EJELS is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

RHAPSODE
Eurasian Society of Educational Research
College House, 2nd Floor 17 King Edwards Road, Ruislip, London, HA4 7AE, UK
RHAPSODE
Headquarters
College House, 2nd Floor 17 King Edwards Road, Ruislip, London, HA4 7AE, UK
Research Article

Developing a Word Family-Based Depth of Academic Word Knowledge Test

Paul Mathieson , Francesco Bolstad , Yosuke Sasao

Despite the continued widespread use of the academic word list, there is a paucity of comprehensive and easily replicable tests which measure learners.


  • Pub. date: September 15, 2024
  • Online Pub. date: September 10, 2024
  • Pages: 137-154
  • 18 Downloads
  • 43 Views
  • 0 Citations
D

Despite the continued widespread use of the academic word list, there is a paucity of comprehensive and easily replicable tests which measure learners’ depth of knowledge of academic word list items. This paper outlines the development and initial validation of a test which measures multiple aspects of learners’ knowledge of academic vocabulary. The test comprises four sections – one productive knowledge section (sentence writing), and three receptive knowledge sections (collocations, synonyms, and L1 meaning). Each section includes 30 items selected from the academic word list, 10 of which are derived forms containing one or more affixes. The test was administered to 193 Japanese students studying at a public medical university in Japan. All four sections of the test showed a high level of internal consistency. In addition, the inter-rater reliability between the two different markers for the productive use (sentence writing) section was also moderately high. Furthermore, Rasch item analysis showed that the test items were well fitted to comprehensively measure learners’ knowledge of academic vocabulary. These encouraging initial results suggest that there may be merit in the test being used, evaluated, and possibly adapted in other EAP teaching and learning contexts.

Keywords: Academic vocabulary, depth of vocabulary knowledge, diagnostic test.

cloud_download PDF
Cite
Article Metrics
Views
18
Download
43
Citations
Crossref
0

References

Ackermann, K., & Chen, Y.-H. (2013). Developing the Academic Collocation List (ACL) - A corpus-driven and expert-judged approach. Journal of English for Academic Purposes, 12(4), 235-247. https://doi.org/10.1016/j.jeap.2013.08.002

Bauer, L., & Nation, P. (1993). Word families. International Journal of Lexicography, 6(4), 253-279. https://doi.org/10.1093/ijl/6.4.253

Beglar, D., & Nation, P. (2013). Assessing vocabulary. In A. J. Kunnan (Ed.) The companion to language assessment (pp. 172-184). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla053

Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman grammar of spoken and written English (Vol. 2). Longman.

Boone, W. J., & Staver, J. R. (2020). Advances in Rasch analyses in the human sciences. Springer. https://doi.org/10.1007/978-3-030-43420-5

Brown, D., Stoeckel, T., McLean, S., & Stewart, J. (2022). The most appropriate lexical unit for L2 vocabulary research and pedagogy: A brief review of the evidence. Applied Linguistics, 43(3), 596-602. https://doi.org/10.1093/applin/amaa061

Browne, C., Culligan, B., & Phillips, J. (2013). The New Academic Word List (NAWL). New General Service List Project. https://www.newgeneralservicelist.com/new-general-service-list-1

Chui, A. S. Y. (2006). A study of the English vocabulary knowledge of university students in Hong Kong. Asian Journal of English Language Teaching, 16, 1-23.

Cobb, T., & Laufer, B. (2021). The nuclear word family list: A list of the most frequent family members, including base and affixed words. Language Learning, 71(3), 834-871. https://doi.org/10.1111/lang.12452

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104

Cons, A. M. (2012). The use and misuse of academic words in writing: Analyzing the writing of secondary English learners and redesignated learners. TESOL Journal, 3(4), 610-638. https://doi.org/10.1002/tesj.36

Coxhead, A. (n.d.). Academic word list sublist families. Victoria University of Wellington School of Linguistics and Applied Language Studies/Te Herenga Waka Te Kura Tātari Reo. https://www.wgtn.ac.nz/lals/resources/academicwordlist/sublist

Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238. https://doi.org/10.2307/3587951

Coxhead, A. (2016). Reflecting on Coxhead (2000),"A New Academic Word List". TESOL Quarterly, 50(1), 181-185. https://doi.org/10.1002/tesq.287

Daller, M. H., & Xue, H. (2009). Vocabulary knowledge and academic success: A study of Chinese students in UK higher education. In B. Richards, M. H. Daller, D. D. Malvern, P. Meara, J. Milton, & J. Treffers-Daller (Eds.). Vocabulary studies in first and second language acquisition (pp. 179-193). Palgrave Macmillan. https://doi.org/10.1057/9780230242258_11

Dang, T. N. Y., Coxhead, A., & Webb, S. (2017). The academic spoken word list. Language Learning, 67(4), 959-997. https://doi.org/10.1111/lang.12253

Durrant, P. (2014). Discipline and level-specificity in university students’ written vocabulary. Applied Linguistics, 35(3), 328-356. https://doi.org/10.1093/applin/amt016

Fernández, B. G., & Schmitt, N. (2015). How much collocation knowledge do L2 learners have? The effects of frequency and amount of exposure. ITL - International Journal of Applied Linguistics, 166(1), 94-126. https://doi.org/10.1075/itl.166.1.03fer

Gardner, D., & Davies, M. (2014). A new academic vocabulary list. Applied Linguistics, 35(3), 305-327. https://doi.org/10.1093/applin/amt015

Hunston, S. (2022). Corpora in applied linguistics. Cambridge University Press. https://doi.org/10.1017/9781108616218

Hyland, K. (2016). General and specific EAP. In K. Hyland & P. Shaw (Eds.), The Routledge handbook of English for academic purposes (pp. 17-29). Routledge. https://doi.org/10.4324/9781315657455

Hyland, K., & Tse, P. (2007). Is there an "academic vocabulary"? TESOL Quarterly, 41(2), 235-253. https://doi.org/10.1002/j.1545-7249.2007.tb00058.x

Ishii, T., & Schmitt, N. (2009). Developing an integrated diagnostic test of vocabulary size and depth. RELC Journal, 40(1), 5-22. https://doi.org/10.1177/0033688208101452

Kaneko, M. (2020). Lexical frequency profiling of high-stakes English tests: Text coverage of Cambridge first, EIKEN, GTEC, IELTS, TEAP, TOEFL, and TOEIC. JACET Journal, 64, 79-93.

Knoch, U., Rouhshad, A., Oon, S. P., & Storch, N. (2015). What happens to ESL students’ writing after three years of study at an English-medium university? Journal of Second Language Writing, 28, 39-52. https://doi.org/10.1016/j.jslw.2015.02.005

Kremmel, B., & Schmitt, N. (2016). Interpreting vocabulary test scores: What do various item formats tell us about learners’ ability to employ words? Language Assessment Quarterly, 13(4), 377-392. https://doi.org/10.1080/15434303.2016.1237516

Laufer, B. (2021). Lexical thresholds and alleged threats to validity: A storm in a teacup? Reading in a Foreign Language, 33, 238-246.

Laufer, B. (2023). Understanding L2-derived words in context: Is complete receptive morphological knowledge necessary? Studies in Second Language Acquisition, 46(1), 200-213. https://doi.org/10.1017/S0272263123000219

Laufer, B., & Cobb, T. (2020). How much knowledge of derived words is needed for reading? Applied Linguistics, 41(6), 971-998. https://doi.org/10.1093/applin/amz051

Laufer, B., Webb, S., Kim, S. K., & Yohanan, B. (2021). How well do learners know derived words in a second language? The effect of proficiency, word frequency and type of affix. ITL-International Journal of Applied Linguistics, 172(2), 229-258. https://doi.org/10.1075/itl.20020.lau

Linacre, J. M. (2022). WINSTEPS® Rasch measurement computer program (Version 5.3.2.0). [Computer software]. https://www.winsteps.com/index.htm

Linacre, J. M., & Tennant, A. (2009). More about critical eigenvalue sizes in standardized-residual principal components analysis (PCA). Rasch Measurement Transactions, 23(3), 1228. https://www.rasch.org/rmt/rmt233f.htm

Malmström, H., Pecorari, D., & Warnby, M. (2023). Teachers’ receptive and productive vocabulary sizes in English-medium instruction. Journal of Multilingual and Multicultural Development. Advance online publication. https://doi.org/10.1080/01434632.2023.2260781

Masrai, A., & Milton, J. (2018). Measuring the contribution of academic and general vocabulary knowledge to learners' academic achievement. Journal of English for Academic Purposes, 31, 44-57. https://doi.org/10.1016/j.jeap.2017.12.006

Masrai, A., Milton, J., El-Dakhs, D. A. S., & Elmenshawy, H. (2021). Measuring the contribution of specialist vocabulary knowledge to academic achievement: Disentangling effects of multiple types of word knowledge. Asian-Pacific Journal of Second and Foreign Language Education, 6, Article 8. https://doi.org/10.1186/s40862-021-00114-5

Mathieson, P., & Bolstad, F. (2020). A spoonful of humour helps the medicine go down: Enlivening academic English vocabulary learning in a medical English course. The Asian ESP Journal, 16(3), 64–87.

Mathieson, P., & Murray, C. (in press). Nurturing academic vocabulary knowledge growth with learner-inspired materials. [Manuscript submitted for publication]. Department of Clinical English, Nara Medical University.

McLean, S. (2018). Evidence for the adoption of the flemma as an appropriate word counting unit. Applied Linguistics, 39(6), 823-845. https://doi.org/10.1093/applin/amw050

McLean, S. (2021). The coverage comprehension model, its importance to pedagogy and research, and threats to the validity with which it is operationalized. Reading in a Foreign Language, 33(1), 126-140.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Macmillan.

Mochizuki, M., & Aizawa, K. (2000). An affix acquisition order for EFL learners: An exploratory study. System, 28(2), 291-304. https://doi.org/10.1016/S0346-251X(00)00013-0

Nadarajan, S. (2011). The challenges of getting L2 learners to use academic words in their writings. Electronic Journal of Foreign Language Teaching, 8(2), 184-200. https://e-flt.nus.edu.sg/v8n22011/nadarajan.pdf

Nagy, W., & Scott, J. A. (2000). Vocabulary processes. In M. L. Kamil, P. G. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 269-284). Lawrence Erlbaum Associates.

Nagy, W., & Townsend, D. (2012). Words as tools: Learning academic vocabulary as language acquisition. Reading Research Quarterly, 47(1), 91-108. https://doi.org/10.1002/RRQ.011

Nagy, W. E., Herman, P. A., & Anderson, R. C. (1985). Learning words from context. Reading Research Quarterly, 20(2), 233-253. https://doi.org/10.2307/747758

Nation, I. S. P. (2011). Vocabulary research into practice. Language Teaching, 44(4), 529-539. https://doi.org/10.1017/S0261444811000267

Nation, I. S. P. (2022). Learning vocabulary in another language (3rd ed.). Cambridge University Press. https://doi.org/10.1017/9781009093873

Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Heinle Cengage Learning.

Nation, P. (2021). Thoughts on word families. Studies in Second Language Acquisition, 43(5), 969-972. https://doi.org/10.1017/S027226312100067X

Nizonkiza, D. (2016). First-year university students’ receptive and productive use of academic vocabulary. Stellenbosch Papers in Linguistics, 45, 169-187. https://doi.org/10.5774/45-0-215

Phung, D. H., & Ha, H. T. (2022). Vocabulary demands of the IELTS Listening Test: An in-depth analysis. SAGE Open, 12(1), 1-13. https://doi.org/10.1177/21582440221079934

Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. Routledge. https://doi.org/10.4324/9781315621395

Qian, D. D., & Lin, L. H. F. (2020). The relationship between vocabulary knowledge and language proficiency. In S. Webb (Ed.), The Routledge handbook of vocabulary studies (pp. 81-96). Routledge. https://doi.org/10.4324/9780429291586-5

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danmarks Paedagogiske Institut.

Read, J. (2004). Plumbing the depths: How should the construct of vocabulary knowledge be defined? In P. Bogaards, & B. Laufer (Eds.), Vocabulary in a second language (pp. 209-227). John Benjamins. https://doi.org/10.1075/lllt.10.15rea

Read, J. (2023). Towards a new sophistication in vocabulary assessment. Language Testing, 40(1), 40-46. https://doi.org/10.1177/02655322221125698

Read, J., & Dang, T. N. Y. (2022). Measuring depth of academic vocabulary knowledge. Language Teaching Research, Advance online publication. https://doi.org/10.1177/13621688221105913

Schmitt, N., & Meara, P. (1997). Researching vocabulary through a word knowledge framework: Word associations and verbal suffixes. Studies in second language acquisition, 19(1), 17-36. https://doi.org/10.1017/S0272263197001022

Schmitt, N., Nation, P., & Kremmel, B. (2020). Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 53(1), 109-120. https://doi.org/10.1017/S0261444819000326

Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55-88. https://doi.org/10.1177/026553220101800103

Snoder, P., & Laufer, B. (2022). EFL learners' receptive knowledge of derived words: The case of Swedish adolescents. TESOL Quarterly, 56(4), 1242-1265. https://doi.org/10.1002/tesq.3101

Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed.). Psychology Press. https://doi.org/10.4324/9781410604491

Storch, N., & Tapper, J. (2009). The impact of an EAP course on postgraduate writing. Journal of English for Academic Purposes, 8(3), 207-223. https://doi.org/10.1016/j.jeap.2009.03.001

Therova, D. (2020). Review of academic word lists. TESL-EJ, 24(1). https://tesl-ej.org/wordpress/issues/volume24/ej93/ej93a5/

Warnby, M. (2024). Relating academic reading with academic vocabulary and general English proficiency to assess standards of students' university-preparedness - The case of IELTS and CEFR B2. Scandinavian Journal of Educational Research, Advance online publication.
https://doi.org/10.1080/00313831.2024.2318434

Webb, S. (2005). Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27(1), 33-52. https://doi.org/10.1017/S0272263105050023

Webb, S. (2007a). Learning word pairs and glossed sentences: The effects of a single context on vocabulary knowledge. Language Teaching Research, 11(1), 63-81. https://doi.org/10.1177/1362168806072463

Webb, S. (2007b). The effects of repetition on vocabulary knowledge. Applied Linguistics, 28(1), 46-65. https://doi.org/10.1093/applin/aml048

Webb, S. (2008). Receptive and productive vocabulary size. Studies in Second Language Acquisition, 30(1), 79-95. https://doi.org/10.1017/S0272263108080042

Webb, S. (2009). The effects of receptive and productive learning of word pairs on vocabulary knowledge. RELC Journal, 40(3), 360-376. https://doi.org/10.1177/0033688209343854

Webb, S. (2019). Incidental vocabulary learning. In S. Webb (Ed.), The Routledge handbook of vocabulary studies (pp. 225-239). Routledge. https://doi.org/10.4324/9780429291586-15

Webb, S., & Kagimoto, E. (2009). The effects of vocabulary learning on collocation and meaning. TESOL Quarterly, 43(1), 55-77. https://doi.org/10.1002/j.1545-7249.2009.tb00227.x

Webb, S., & Nation, P. (2017). How vocabulary is learned. Oxford University Press.

Webb, S., Sasao, Y., & Ballance, O. (2017). The updated Vocabulary Levels Test: Developing and validating two new forms of the VLT. ITL - International Journal of Applied Linguistics, 168(1), 33-69. https://doi.org/10.1075/itl.168.1.02web

West, M. (1953). A general service list of English words. Longman Green.

Wolfe, E. W., & Smith E. V., Jr. (2007). Instrument development tools and activities for measure validation using Rasch models: Part II - Validation activities. Journal of Applied Measurement, 8(2), 204-234.

Yanagisawa, A., & Webb, S. (2020). Measuring depth of vocabulary knowledge. In S. Webb (Ed.), The Routledge handbook of vocabulary studies (pp. 371-386). Routledge. https://doi.org/10.4324/9780429291586-24

Zhou, S. (2010). Comparing receptive and productive academic vocabulary knowledge of Chinese EFL learners. Asian Social Science, 6(10), 14-19. https://doi.org/10.5539/ass.v6n10p14

...