MAAL6014 Language testing and assessment
Lecturer
Dr. Michelle Raquel
Introduction
The aims of this elective course are to:
a) introduce participants to the fundamental principles of language testing and language test evaluation
b) develop skills in the design, trialling, moderation and validation of testing instruments for a range of purposes including: formative and summative assessment, self-assessment, language profiling
Intended Learning Outcomes (ILOs)At the end of the course you should be able to: i. describe and discuss a range of contexts, purposes and methods for language testing ii. discuss and critique the appropriacy and usefulness of various methods for testing writing, speaking, listening, reading and integrated skills iii. analyze, describe and report data derived from test scores using a range of techniques including item analysis, estimation of reliability, investigation of validity and potential test bias iv. describe and discuss appropriate techniques for the evaluation of language tests |
Core text
Carr (2011). Designing and analyzing language tests. Oxford: OUP Oxford.
Green, A. (2021). Exploring language assessment and testing. Abingdon: Routledge
- Ebook available via: https://www.routledge.com/Exploring-Language-Assessment-and-Testing-Language-in-Action/Green/p/book/9781138388789
Course Outline
| Session | Session Topic and Description | Related CLOs |
| Session 1 | Course introduction
Types of test & purposes of assessment Qualities of effective assessment systems |
1, 4 |
| Session 2 | Test specifications, test tasks, item writing
Assessing reading and listening |
1, 4 |
| Session 3 | Assessing speaking and writing
Assessing grammar and vocabulary |
2, 3 |
| Session 4 | Gen AI & language assessments | 2, 3 |
| Session 5 | Data analysis | 2, 3 |
| Session 6 | Formative assessment
Assessment literacy Feedback |
1, 4 |
* Each session is 3 hours
Assessments
Part 1: Language Test Portfolio (50%)
Group Work
This project aims to determine your ability to apply the skills on test design and feedback that you learnt in class.
You should do the following in groups:
- Assessment problem – Think of a situation where you could have to design, administer, and give feedback on an English language diagnostic or placement test. This could be for any level of students. Assume you will have unlimited budget and resources to do this. Write out this assessment problem in 300 words max.
- Test specifications – You will now write up the test specifications for this test. Think about the issues of practicality, validity, reliability, and beneficial consequences. There is no word or page limit for this but remember that it must be comprehensive and concise.
- Use generative AI (e.g., ChatGPT) to initially write up the test specifications for your test. Evaluate the gen AI output and continue to modify it yourself or with more support from gen AI until you come up with the final version.
- The test specification will include 3 appendices:
i. Expected outcome: In 1 paragraph, briefly describe the expected results of a cohort or batch of candidates taking your test.
ii. Timeline: include a timeline that outlines how this test could be operationalized (from creation, administration, and feedback) This timeline will be in table format with the columns: date, activity, expected outcome).
- Sample test items
- Use gen AI to help you generate sample items for any subtest in your test specification.
- You can choose any item type.
i. Selected response (e.g., multiple-choice, fill in the blanks, T/F): 5 items + answer key; you must include the full text (e.g., reading text or listening script and recording).
ii. Constructed response: 3 items + assessment rubric
| Assessment Part 1 Criteria | Match of criteria to Programme-level criteria |
| Quality of the specifications in terms of comprehensiveness | 1,2,3,4,5 |
| Quality of the content of appendices in terms of practicality | 1,2,4,5 |
| Quality of sample items | 1,2,4,5 |
| Mechanics (format, layout, etc.) | 4,5 |
* Each member of the group will receive the same grade, so you are expected to share the work equally.
* Please indicate member contribution to the group project on the front page.
* Submit a softcopy via Turnitin
Part 2: Reflective Essay (50%)
Individual
Part 1: Impact on student learning: Describe how you will use the results of the test for student learning (i.e., formative purposes). The written section can be supplemented with graphics (e.g., infographic, diagram, lesson plan). You do not need to include samples of teaching material. (max 1000 words)
Part 2: Reflection on test construction.: Reflect on the experience creating any 1 aspect of this test portfolio and the use of generative AI. This is not just a report of what you did but rather a reflection of the test development experience. (max 1000 words)
Your reflection could include the following:
- Give 1 example of how you used Gen AI in this portfolio (either in developing test specification, item creation, etc.)
- Discuss how you evaluated the gen AI output, and subsequently what you did.
- Discuss what you learnt from this experience and what would do differently if given the chance to do this one aspect of this test portfolio again.
| Assessment Part 2 Criteria |
| Understanding of the task and key concepts/issues involved |
| Depth of analysis and/or critique in response to the task |
| Use of appropriate professional and/or research literature to support response |
| Structure and organization of response; |
| Presentation of response according to appropriate academic and linguistic conventions |
Sources of additional readings and bibliographic references
Alderson, J.C., Clapham, C. & Wall, D. (1995). Language test construction and evaluation. Cambridge. Cambridge University Press
Bachman, L.F. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press
Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice: designing and developing useful language tests. Oxford. Oxford University Press
Berry, V. and Lewkowicz, J. (2000). Assessment in Chinese Contexts: Special Issue of the Hong Kong Journal of Applied Linguistics. The English Centre, The University of Hong Kong
Carr (2011). Designing and analyzing language tests. Oxford: OUP Oxford.
Fulcher, G. (2010). Practical language testing. London: Hodder Education.
Fulcher, G. and Davidson, F. (2007). Language Testing and Assessment. Abingdon: Routledge
Hughes, A. (2003). Testing for language teachers 2nd Edition. Cambridge: Cambridge University Press
McNamara, T. (1996). Measuring Second Language Performance. Harlow: Pearson Education Limited
McNamara, T. (2000). Language Testing. Oxford: Oxford University Press
Weir, C.J. (2005). Language Testing and Validation. Basingstoke: Palgrave Macmillan
Relevant Journals
Assessing Writing: http://www.sciencedirect.com/science/journal/10752935
International Journal of Language Testing: http://ijlt.ir/journal/
Language Assessment Quarterly: www.tandf.co.uk/journals/titles/15434303.asp
Language Testing: www.sage.com
Language Testing in Asia: http://link.springer.com/journal/40468
International/Regional Language Testing and Assessment Associations
Asian Association for Language Assessment www.aalawebsite.com
Association for Language Testing and Assessment of Australia and New Zealand www.altaanz.org
European Association for Language testing and Assessment www.ealta.eu.org
International Language Testing Association www.iltaonline.com
Web-based resources:
http://languagetesting.info/ is a one-stop site for references on testing specific language skills and using statistics. There are also short video clips of topics in language testing. It is maintained by Dr. Glenn Fulcher.
http://www.antonykunnan.com/ Personal website of Professor Antony John Kunnan. He has free chapters on assessment topics related his book.
http://taleproject.eu/index.php? Teachers’ Assessment Literacy Enhancement website. This is a free online module to develop teachers’ classroom assessment literacy.