Assessment Design and Specifications

What makes assessment necessary?
Teachers want to know what parts of the course cause difficulties and require special attention
Teachers want to know what progress students have made
Students want to know their strengths and weaknesses
Parents want to know how their children are doing
PROSET - TEMPUS

What makes assessment necessary?
Universities want to choose the best applicants for enrollment
Teachers want to know whether the students are coping with the programme
Employers want to choose the best applicant for the job
PROSET - TEMPUS

What forms of assessment do you use with your students?
Test
Portfolio
Project work
Observation
Case study
Interview, oral questions
Experimental work, etc
PROSET - TEMPUS

Testing & assessment
PROSET - TEMPUS
EVALUATON
TESTING

What is a test?
Any procedure for measuring
ability
knowledge, or performance.
(Longman Dictionary of Language Teaching and Applied Linguistics)
PROSET - TEMPUS

What is a test?
A test is a measurement instrument designed to elicit a specific sample of individual’s behaviour. … a test necessarily quantifies characteri-stics of individuals according to explicit procedures. (L.F. Bachman. Fundamental Considerations in Language Testing, 1990)
PROSET - TEMPUS

Test as an instrument of assessment (measurement)
WHY
WHAT
HOW
PROSET - TEMPUS

What types of tests do we use?
Test Purposes
Proficiency tests
Achievement tests
Placement tests
Diagnostic tests
PROSET - TEMPUS

What types of tests do we use?
Direct vs Indirect
Discrete Point vs Integrated
Objective VS Subjective
Low stakes vs High stakes
Norm-referenced vs Criterion-referenced
PROSET - TEMPUS

Norm-referenced testing: the bell curve
Number
of
students
0 25 50 75 100
Score (points)
PROSET - TEMPUS

The Assessment Cycle: where does a test begin?
PROSET - TEMPUS

What kinds of information should be available about a test?
WHY
WHAT
HOW
PROSET - TEMPUS

Test specification
– the official statement about what the test tests and how it tests it … to be followed by test and item writers
Multilingual Glossary of Language Testing Terms. CUP, 1998
Test design statement
Blueprint
Task and item specifications
PROSET - TEMPUS

Test design statement
the purpose, the knowledge, skills or abilities it is intended to assess
the resources available
the uses of the results, the intended impact of its use
PROSET - TEMPUS

Blueprint
content
methods
scoring
PROSET - TEMPUS

Test tasks
Prompt (a reading/listening text, an essay question, a picture to describe)
Response (ticking a box, giving a short answer, describing a picture, etc)
PROSET - TEMPUS

The building blocks of a test
Instructions
Item
Stem
Options
Distractors
Key
PROSET - TEMPUS

Responses
Brown & Hudson, 1998
Selected response
Constructed response
Personal response
PROSET - TEMPUS

Responses: EGE
Selected response
Выбор ответа
(A1-A7, A8-A14, A15-A23)
Short response
Краткий ответ
(B1, B2, B3, B4-B10, B11-B16)
Constructed response
Свободно конструируемый ответ (C1, C2)
PROSET - TEMPUS

Task & item specifications
How many items are included in each task?
What area of knowledge, skill or ability is assessed in each task?
What should the test-taker do (instructions)?
How are scores on each item and task determined?
PROSET - TEMPUS

One and the same specification for all?
Test-takers
Item writers
Test validators
Users of test results
PROSET - TEMPUS

The same type of specification for a classroom test and EGE?
EGE specification:
11 pages
12 parts (sections)
4 tables
an appendix
PROSET - TEMPUS