To test, or not to test

Placement, diagnostic, achievement, progress.What do these four words collocate with?

Here’s a hint: It’s something that all students say they hate, but some start to ask for it when it’s missing from a language course.

No, it’s not grammar.
It’s tests.

Those of us who work at schools are most definitely familiar with placement tests – used when a student first comes to a school and needs to be placed at the right level. This is often done by a discrete item test that might include multiple choice questions, gap-fills, sentence transformations and/or error analysis (more on discrete item testing later).  In some schools, a spoken interview or a freer writing task is given to help ascertain the student’s level.

Diagnostic tests are used to find out what students of a class already know so as to help define what should go into a lesson or course plan. Diagnostic tests can be formal tests with a formal marking system, or an informal test whereby a teacher uses his/her observation of the learner’s task performance and homework to determine what they are able or not able to do in the foreign language, and what they might need.

While diagnostic tests are forward-looking and involve future course planning, progress tests are retrospective, and are set to assess how much the student has learnt and how much progress they have made. In some schools, teachers are required to carry out such tests at the end of each course, and sometimes every month or even every week, so as to demonstrate the effectiveness of teaching/learning and to help determine if the student is ready to move to the next level.

Many students apply and pay for achievement tests in order to have a more formal paper qualification that would give credibility to their claimed level of English. Achievement tests are often administered by an external examining body like Cambridge or Oxford, and the growing popularity of tests like the Cambridge First Certificate and IELTS suggests that a high percentage of students expect and want to be tested in their language ability.

While some students take these tests because of the necessary qualifications needed for job applications, securing a place at university, or immigration purposes, other students take tests because they find tests motivating. Tests can give them a sense of progress, a sense of achievement, a sense of having reached a certain level in their language study.

Tests can show teachers and learners what has been learnt and ‘absorbed’. Tests can create a comforting sense that language learning is indeed linear – that there is a learning continuum whereby every step would lead them to towards their goal of being an advanced, proficient, ‘native-like’ user of the language.

But is language learning really such a neat process?

More importantly, do tests really show us what has been learnt? Do they really measure what they are supposed to be measuring?

Here are some reasons why you might want to think twice before assuming that formal language tests are an essential and indispensible part of language learning.

• Tests are unnerving and the results often depend on how the candidate performs under stress.
  
  
• Tests are often unreliable. If there was a test that could accurately measure a person’s language ability in terms of their entire range of skills and knowledge, that test would probably need to be extremely flexible and tailored to the individual, would take several days to administer, and would unavoidably incur very high costs. This would not only be impractical, but also unprofitable for the examining body.  The balancing act between practicality and reliable is a difficult one to achieve without sacrificing one or the other.

• Many tests still involve discrete item testing.
  This refers to tests that try to reveal the candidate’s knowledge of an area of the language e.g. the 2nd conditional, the present perfect, the use of articles with uncountable nouns, the collocations of the words ‘make’ and ‘do’, the correct phrases used when making requests, the stress patterns of three syllable nouns ending with ‘-ity’, the distinction between the phonology of /r/ vs /l/, etc.
  Discrete item tests often take the form of gap-fills, multiple choice questions, sentence transformation exercises, error analysis, matching exercises, jumbled words/sentences, translation exercises, etc. and are considered to be objective tests as the answer is usually either correct or wrong, and does not require the subjective judgement of a teacher or examiner.

Here are some examples of discrete item tests:

(1) Choose the correct answer.
If I ___________  a millionaire, I would buy my husband a helicopter.
(a) am  (b) is  (c) were  (d) are

(2) Fill in the gaps with ‘make’ or ‘do.
______  a mistake      ______ one’s duty      ______ amends

(3) Correct the following sentence.
There are five waters on a table. The life is beautiful. The priest played the music on a organ.

Based on the belief that an accumulation of different isolated elements of a language would result in the knowledge and proficiency in that language, discrete item tests fail to focus on how the learner combines the different bits of knowledge to actually produce language in an ordinary discourse situation and in context.

As a result, discrete item tests are not able to truly test one’s communicative competence.

• Integrative tests, on the other hand, focuses on a more global proficiency by attempts to measure skills such as writing and speaking through freer writing and speaking tasks that allows the candidate to utilize their knowledge of the different areas of language all at once.

Here are some examples of integrative tests (I apologise for being purposefully banal here):

1. Write a letter to the record company asking for a recording contract.
2. Write an essay about the advantages and disadvantages of water beds.
3. Look at these pictures of mammoths. Compare and contrast what you see.
4. Talk about a building you dislike for 2 minutes. You have 1 minute to prepare.
5. Design an ideal hospital with your partner.

Such integrative tests are better able to assess the candidates’ real use of authentic communicative language, and therefore are the preferred method of testing productive skills by most achievement tests; e.g. the Cambridge main suite exams, the Trinity exams, IELTS, BULATS, BEC, etc.

However, integrative tests are inevitably subjective by nature and require the examiner to make judgements on the candidate’s language ability based on a criteria provided by the examining body. This criteria is still very much focused on lexico-grammatical accuracy, with examiners listening out for whether relative clauses are present in the spoken production and if collocations used are ones that are commonly known to the native speaker.

Less attention is given to the discursive/pragmatic competence, socio-cultural competence, and the candidate’s adaptation and accommodation strategies.

And perhaps as demonstrated by my exaggerated use of banal examples above, many of the spoken and written tasks used in integrative tests still struggle to reflect an authentic discourse situation that the candidate might actually find themselves in.

• Many exam preparation classes and books are now resorting to purely equipping students with exam-taking strategies, memorizable fixed phrases, and hoop-jumping techniques, defeating the true purpose of the test and bringing the validity and reliability of the formal testing procedure into question.

Despite these issues with tests, government bodies, schools, companies and institutions still regard the face validity of an exam as all important and take the exams results as reflecting the gospel truth about the candidate’s language ability. 

And these tests and exams end up dictating entire curricula and courses – curricula and courses that are designed not to produce real progress in the learners’ language ability and usage, but to enable the students to pass the required exams.

Oh, what a dilemma this is.

About English Teaching professional’s regular blogger:

Chia Suan Chong is a General English and Business English teacher and teacher trainer, with a degree in Communication Studies (Broadcast and Electronic Media) and an MA in Applied Linguistics and English Language Teaching from King’s College London.

A self-confessed conference addict, she spends a lot of her time tweeting (@chiasuan/@ETprofessional), Skyping, and writing. You can find out more about her on her blogsite: http://chiasuanchong.com