FacebookInstagram
Faculty of Arts

MAAL6014 Language testing and assessment

  1. Home
  2. MAAL6014 Language testing and assessment
Start main Content

MAAL6014 Language testing and assessment

Lecturer

Dr. Michelle Raquel

michelle.raquel@hku.hk

 

Introduction

The aims of this elective course are to:

  1. introduce participants to the fundamental principles of language testing and language test evaluation
  2. develop skills in the design, trialling, moderation and validation of testing instruments for a range of purposes including: formative and summative assessment, self-assessment, language profiling

 

Intended Learning Outcomes (ILOs)

At the end of the course you should be able to:

  1. describe and discuss a range of contexts, purposes and methods for language testing
  2. discuss and critique the appropriacy and usefulness of various methods for testing writing, speaking, listening, reading and integrated skills
  3. analyze, describe and report data derived from test scores using a range of techniques including item analysis, estimation of reliability, investigation of validity and potential test bias
  4. describe and discuss appropriate techniques for the evaluation of language tests

 

Core texts

Dimova, S, Yan, X., & Ginther, A. (2020). Local Language Testing Design, Implementation, and Development. Abingdon: Routledge

Green, A. (2021). Exploring language assessment and testing. Abingdon: Routledge

 

Course Outline

Session Session Topic and Description Related CLOs
Session 1 Course introduction

Types of test & purposes of assessment

Qualities of effective assessment systems

 1, 4
Session 2 Test specifications, test tasks, item writing

Assessing reading and listening

 1, 4
Session 3 Assessing speaking and writing

Assessing grammar and vocabulary

 2, 3
Session 4 Gen AI & language assessments 2, 3
Session 5 Data analysis 2, 3
Session 6 Formative assessment

Assessment literacy

Feedback

 1, 4

* Each session is 3 hours

 

Assessments

Part 1: Language Test Portfolio (50%)

Group Work

This project aims to determine your ability to apply the skills on test design and feedback that you learnt in class.

You should do the following in groups:

1. Assessment context – Think of a real situation where you have to design, administer, and give feedback on an English language diagnostic test. This could be for any level of students. Write out this assessment context in 300 words max.

2. Test specifications – You will now write up the test specifications for this test. Think about the issues of practicality, validity, reliability, and beneficial consequences. There is no word or page limit for this but remember that it must be comprehensive and concise.

    • Use generative AI (e.g., ChatGPT) to initially write up the test specifications for your test. Evaluate the gen AI output and continue to modify it yourself or with more support from gen AI until you come up with the final version.
    • The test specification will include 3 appendices:

i. Expected outcome:
1. Briefly describe the expected results of a cohort or batch of candidates taking your test.
2. Briefly describe the predicted performance of your test items, testlets, or whole test.

ii. Timeline:
Include a timeline that outlines how the diagnostic test could be operationalized (from creation, administration, and feedback) This timeline will be in table format with the columns: date, activity, expected outcome).

iii. Sample student diagnostic report

3. Sample test items

    • Use gen AI to help you generate sample items for any subtest in your test specification.
    • You will generate two types of items (total 8 items).
      i. Five (5) selected response items (e.g., multiple-choice, fill in the blanks, T/F and answer key; you must include the full text (e.g., reading text or listening script and recording). If you attach a link to the recording, please make sure the link is accessible when the file is converted to PDF format.
      ii. Three (3) constructed response items with assessment rubric/s or answer key.

 

Assessment Part 1 Criteria Match of criteria to Programme-level criteria
Quality of the specifications in terms of comprehensiveness 1,2,3,4,5
Quality of the content of appendices in terms of practicality 1,2,4,5
Quality of sample items 1,2,4,5
Mechanics (format, layout, etc.) 4,5

 

*Each member of the group will receive the same grade, so you are expected to share the work equally.

*Please indicate member contribution to the group project on the front page.

*Submit a softcopy via Turnitin

 

Part 2: Reflective Essay (50%)

Individual

Part 1: Impact on student learning: Describe how you will use the results of the test for student learning (i.e., formative purposes and development of assessment literacy). The suggested plan must be appropriate to your assessment context. The suggested action plan can be supplemented with graphics (e.g., infographic, diagram, lesson plan). Do not include samples of teaching material. (max 1000 words)

Part 2: Reflection on test construction: Reflect on your experience creating this test portfolio through a SWOT analysis (strengths, weaknesses, opportunities, threats). The goal of this assessment is for you to demonstrate your ability to consider all aspects of the assessment context, not just the learners but all stakeholders. This is not a report of what you did but rather a reflection of the test development experience. (max 1000 words)

Your reflection could include the following:

  • Discuss what you, as an individual, learnt from this experience and what would do differently if given the chance to do this one aspect of this test portfolio again.
  • Working on a test as a group
  • How you used Gen AI in this portfolio (either in developing test specification, item creation, etc.)
  • How you evaluated the gen AI output, and subsequently what you did.

 

Assessment Part 2 Criteria
Understanding of the task and key concepts/issues involved
Depth of analysis and/or critique in response to the task
Use of appropriate professional and/or research literature to support response
Structure and organization of response;
Presentation of response according to appropriate academic and linguistic conventions

 

Sources of additional readings and bibliographic references

Alderson, J.C., Clapham, C. & Wall, D. (1995). Language test construction and evaluation. Cambridge. Cambridge University Press

Bachman, L.F. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press

Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice: designing and developing useful language tests. Oxford. Oxford University Press

Berry, V. and Lewkowicz, J. (2000). Assessment in Chinese Contexts: Special Issue of the Hong Kong Journal of Applied Linguistics. The English Centre, The University of Hong Kong

Carr (2011). Designing and analyzing language tests. Oxford: OUP Oxford.

Fulcher, G. (2010). Practical language testing. London: Hodder Education.

Fulcher, G. and Davidson, F. (2007). Language Testing and Assessment. Abingdon: Routledge

Hughes, A. (2003). Testing for language teachers 2nd Edition. Cambridge: Cambridge University Press

McNamara, T. (1996). Measuring Second Language Performance. Harlow: Pearson Education Limited

McNamara, T. (2000). Language Testing. Oxford: Oxford University Press

Weir, C.J. (2005). Language Testing and Validation. Basingstoke: Palgrave Macmillan

Relevant Journals

Assessing Writing: http://www.sciencedirect.com/science/journal/10752935

International Journal of Language Testing: http://ijlt.ir/journal/

Language Assessment Quarterly: www.tandf.co.uk/journals/titles/15434303.asp

Language Testing: www.sage.com

Language Testing in Asia: http://link.springer.com/journal/40468

International/Regional Language Testing and Assessment Associations

Asian Association for Language Assessment www.aalawebsite.com

Association for Language Testing and Assessment of Australia and New Zealand www.altaanz.org

European Association for Language testing and Assessment www.ealta.eu.org

International Language Testing Association www.iltaonline.com

Web-based resources:

http://languagetesting.info/ is a one-stop site for references on testing specific language skills and using statistics. There are also short video clips of topics in language testing.  It is maintained by Dr. Glenn Fulcher.

http://www.antonykunnan.com/ Personal website of Professor Antony John Kunnan. He has free chapters on assessment topics related his book.

http://taleproject.eu/index.php? Teachers’ Assessment Literacy Enhancement website. This is a free online module to develop teachers’ classroom assessment literacy.