ASSESSING LISTENING: purposes and technique

There was an old owl who lived in an oak,
The more he heard, the less he spoke
The less he spoke, the more he heard -
Oh, if men were all like that wise bird!
(Ogden Nash)
The Wise Owl

Listening is often seen as a two-stage process involving, in the first instance, the extraction of basic information followed by the use of that language for a communicative purpose.
(Buck, 2001 p.51)
What is listening

Features of listening
Active: The listener has to remake the speaker’s intended meaning.
Time-constrained: The input is not under the control of the listener
Transitory: No text to refer back to. The listener has to carry forward a recall of what has been said in his/her mind.
Oral: The input contains features which reflect the way speech is produced

Top Down
Bottom Up

Processing in listening
Predict what is to be said
Infer implied meanings and intentions
Recognize cohesive devices in discourse
Hold the information in short-term memory
Construct the literal meaning of the sentence
Sentence processing
Word recognition
Speech perception
Top Down
Bottom Up

Comprehension as a matter of listeners…
Decoding smallest elements of what they hear – the sounds.
Sounds combined and individual words decoded.
Words combined into sentences: listener works out meaning.
Add recognition of features such as intonation and so on…
Finally reach non-linguistic content.
(Rumelhart and Ortony, 1977)
Bottom-up approach to listening

Top-down approach
You hear:
McKenzy brought me another present today. It was too late to save it so I buried it in the garden. I think I’m going to have to put a bell round his neck..
(from eltnotebook.blogspot.co.uk)
What is the relationship between the speaker and McKenzy?

Listening comprehension
Is not either top-down or bottom-up processing, but an interactive, interpretive process where listeners use both prior knowledge and linguistic knowledge in understanding messages.

What makes listening difficult?
Phonological modification
Clustering
Reduced forms
Colloquial speech and accents
Prosodic features
Speech rate
Performance variables

Spoken sounds vary. A lot.
In writing we have standard spelling – no spoken equivalent.
Sounds of words change according to context:
I met her /t/ I met you /t∫/
Sounds of words depend on gender, age, shape and size of vocal tract of speaker.
Phonological modification

Clustering
Spoken language is “chunked” into phrases and clauses.
Can be difficult to separate out individual words from these chunks or clusters.

Reduced forms
More reduced forms in speech than in writing.
I am going to win = I’m gonna win
In speech a lot of information comes from context and is not put into words:
Not that one.
On top!

Colloquial speech and accents
Local colloquial expressions and culturally specific patterns
Accents
Standard British/ American /Australian/ New Zealand etc.
Non-standard
L2 accent

Rhythm, stress, and intonation of speech.
Prosody varies according to…
Emotional state of speaker
Form of utterance (statement, question, or command)
Irony, sarcasm, emphasis, contrast
How many different meanings can you express with just the word ‘No’? Can you make a conversation with three turns?
Prosodic features

Fast speech is harder for learners to follow.
Those familiar with text book speech are often unable to keep up with a natural pace of speech.
Speech rate

Performance variables
Hesitations
False starts
Pauses
Corrections

Assessing listening is similar to reading
Both are receptive skills;
Can use many of the same types of items often with the same issues;
Need a reason for listening / reading;
Items can be marked objectively or subjectively;
Discrete point or integrative;
Both include bottom-up, top-down and interactive processing;

Assessing listening is different from reading
Real-time processing -> requires automaticity;
Specifications need to include information about speed of delivery wpm or sps (Hughes, 2003:163)
‘Mental gymnastics’ (backtracking) is more difficult;
Environment is more important (acoustics, quality of recording, quietness of location) (Hughes, 2003:163) ->construct –irrelevant variance (Messick, 1993)
Live delivery: must have equal input/speed/intonation/empgasis in delivery (between speakers and for the same speaker)

Types of classroom listening skills
(Brown 2001, p.273ff.)
Reactive (listen and repeat)
Intensive (listen for specific sounds, discourse markers, intonation patterns, etc.)
Responsive (listen and respond – briefly)
Selective (listen for particular items in a longer stretch of discourse)
Extensive (listen for global comprehension)
Interactive (authentic communication listening as part of discussion, conversation, debate, etc.)

Listening is found in most general proficiency tests, academic tests, tests of language for business purposes
In achievement tests it is used to assess achievement and to encourage students to practise listening
Listening is widely used in placement testing to identify the level of class a student should enter.
Purposes of assessing listening comprehension

Which kind of language should learners be able to understand at different levels?
OVERALL LISTENING COMPREHENSION
(Common European Framework of Reference for Languages)
C2 Has no difficulty in understanding any kind of spoken language, whether live or broadcast, delivered at fast native speed
C1 Can understand enough to follow extended speech on abstract and complex topics beyond his/her own field, though he/she may need to confirm occasional details, especially if the accent is unfamiliar.
Can recognise a wide range of idiomatic expressions and colloquialisms, appreciating register shifts. Can follow extended speech even when it is not clearly structured and when relationships are only implied and not signalled explicitly.
Can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure etc., including short narratives.

B2 Can understand standard spoken language, live or broadcast, on both familiar and unfamiliar topics normally encountered in personal, social, academic or vocational life. Only extreme background noise, inadequate discourse structure and/or idiomatic usage influences the ability to understand.
Can follow extended speech and complex lines of argument provided the topic is reasonably familiar, and the direction of the talk is sign-posted by explicit markers. ...
B1 Can understand straightforward factual information about common everyday or job related topics, identifying both general messages and specific details, provided speech is clearly articulated in a generally familiar accent.
Which kind of language should learners be able to understand at different levels?

A2 Can understand enough to be able to meet needs of a concrete type provided speech is clearly and slowly articulated.
Can understand phrases and expressions related to areas of most immediate priority (e.g. very basic personal and family information, shopping, local geography, employment) provided speech is clearly and slowly articulated.
A1 Can follow speech which is very slow and carefully articulated, with long pauses for him/her to assimilate meaning.
Which kind of language should learners be able to understand at different levels?

Measurement principles of assessing listening
Reliability
Validity
Fairness

Cognitive validity
Do the cognitive processes in which test-takers engage during a test of language skills correspond to those which expert users of the L2 would employ in the real-world events the test is intended to predict?

‘Authenticity’
Recordings should be as close as possible to natural everyday speech.
‘Authentic’ material not necessarily informal. Some formal listening conditions provide useful test material: job interviews, discussions, lectures.
Key consideration: What kind of speech will test takers be exposed to: informal everyday – broadcast – professional/academic?
Especially important in some tests is naturalness of delivery. Are there: natural planning pauses – hesitations – relatively short utterances - lexical chunks etc.?
*
University of Bedfordshire

Number of participants (Hughes A. 2003)
Monologue
Dialogue
Polylogue/ Multiparticipant Conversation.

Text purposes (Hughes 2003)
Description
Narration
Explanation
Exposition
Argumentation
News item
Review
Instruction

The Common European Framework of Reference for Languages: Learning, Teaching, Assessment.
(Council of Europe 2001)
CEFR, Chapter 4.6.3 Text types include:
Public announcements and instructions
Public speeches, lectures, presentations, sermons
Rituals (ceremonies, formal religious services)
Entertainment (drama, shows, readings, songs)
Sports commentaries (football, cricket, boxing, etc.)
News broadcasts
Public debates and discussion
Inter-personal dialogues and conversations
Telephone conversations
Job interviews
etc.

Task formats (Buck 2001)
Multiple matching
Multiple choice
Dictation
Open ended comprehension questions
Gap filling
Note taking
Written response
True/false
Map labelling

Advantages and disadvantages of different task types (Buck, 2001)

Advantages and disadvantages of different task types (Buck, 2001)

Formats
Most of the traditional test formats are open to question because of the extent to which they are dependent upon the reading skill. This is especially true of multiple choice questions.
Traditional formats require test takers to map from a statement on the page to one that they hear. This matching and checking very rarely occurs in real-world listening.
Pre-set questions tell test takers a great deal of what they are about to hear. This encourages test-wise strategies based upon anticipating what is to come.
Test takers exploit the knowledge that items follow the order of the recording. They listen out for words that appear in the items. They tune in and out of the recording, knowing that items will be evenly spaced.