Playing Russian roulette with the test takers*

*B. Spolsky, Measured Words

Language tests and the UK Border Agency

In response to a parliamentary question, Damian Green, UK Minister of State (Immigration), is reported to have stated:

As part of its current exercise to revise its list of secure English language test providers, the UK Border Agency conducted a verification exercise of the mapping of prospective providers of secure English language tests to the CEFR. Dr North (one of the CEFR co-authors) was invited to play the role of an independent expert on 10 December 2010 and the consultation took place during the second week of March 2011.

Quite deservedly, since this news was announced, there’s been some very heated discussion  on the Language Testing Research and Practice (LTEST-L) mailing list. In brief, the facts are as follows:

  • The UK Border Agency requires that immigrants (including students) produce evidence of competence in the English language at a stated CEFR level.
  • Acceptable evidence is in the form of a test score or certificate from an approved test provider.
  • Whether a particular test score or certificate proves that the holder is at the required CEFR level can be determined via a “verification exercise of the mapping” of the test to the CEFR; this verification exercise can be in the form of a consultation with an expert, as long as the expert is “one of the CEFR co-authors”

The CEFR and the Use of Language Tests

These are rather disturbing developments for the ELT community as well as society at large. Here is why:

  • The UKBA’s decisions on what minimum CEFR levels are acceptable for different types of migrants (Tier 1 workers, Tier 2 workers, students, etc) are quite arbitrary – this was proven in quite a spectacular manner when the UKBA had to change their minds last summer about whether B1 or B2 was a more appropriate threshold for foreign students.
  • The UKBA’s decisions on which examinations and which scores constitute proof that one has reached the arbitrarily determined level are equally arbitrary: some of the approved test providers have produced evidence of having completed a CEFR mapping project as suggested in the Council of Europe’s Manual for Relating language exams to the CEFR, while others have not; in addition, there are examination providers who have produced such evidence, but their examinations are not on the UKBA approved list.
  • Even when such mapping has been undertaken, it is questionable that a link has been established between the CEFR levels and the levels that the test tasks reflect or the levels of the cut scores of the various “CEFR-mapped” examinations. This is mainly because the CEFR-mapping procedure descibed in the Council of Europe’s manual is actually very much a fool-proof exercise, which guarantees that once the stages of the “linking” process have been completed, some sort of link will have been established; in other words, it is not possible to do the mapping exercise and conclude that there is no link between your exam and the CEFR!
  • The CEFR levels themselves do not represent the nature of communication or the stages of acquisition of a foreign language; in the words of Dr North (the same Dr North that was involved in the UKBA consultation that started the heated discussion on LTEST-L), one of the co-authors of the CEFR, the CEFR levels are merely “teachers’ perceptions of language proficiency (appropriate for a common framework of reference) not validated descriptions of SLA processes.”
  • Even if the CEFR levels did represent actual stages of language acquisition, rather than the collective impressions of teachers, linking an examination to the CEFR does not make the examination valid. To prove that a test is valid, you have to provide evidence that the decisions based upon the test are rational (cf. McNamara, T and C Roever. 2006. Testing: the social dimension, Chapter 2). Unfortunately, what often happens is that examination bodies use CEFR-mapping procedures in lieu of validation rather than in addition to validation of their tests.
  • In actual fact, the uses to which the UK Border Agency puts some of the tests on its “approved” list constitute, in themselves, an invalidating argument: a test designed to measure competence in Academic English cannot be a valid measure of an unskilled worker’s ability to use English for occupational purposes, even if that test has hastily been claimed to be linked to the CEFR!

The CEFR as Shibboleth

In the critical language testing literature, mention is often made of the biblical pronunciation test of the Shibboleth (Judges, 12:6): in the biblical account, those who could pronounce “shibboleth” correctly passed the test, proved their origin and were spared their lives; those who failed were instantly killed, all forty two thousand of them. The present-day Shibboleth is, it appears, not the language test itself, but the CEFR and its test alignment procedures: should a language exam prove to be “aligned” to the CEFR, it can live and prosper, as CEFR mapping has become all that is required for the exam to gain recognition and therefore be commercially viable; if an exam is not linked to the CEFR, or cannot, or does not, claim to be, it does not seem to matter whether it is valid, whether it is fair, whether it serves the purpose for which it was designed: it will not be recognised and the examinees that have successfully taken the exam will not be given, for instance, the opportunity to study, work or live in the country of their choice.

Rehashed content in language teacher education: why not!

I’ve done this session so many times I am sick and tired of it. I need to find a new approach, design some new materials.

My colleague was talking about an input session that she was doing the following day on a DELTA course. The session was on approaches and methods in English language teaching and the truth is that over the last twenty years both of us have covered this topic a number of times in a number of different ways. I can remember sessions we have designed and taught in which:

  • We have taught trainees mini lessons in different languages using different methods and then discussed what the principles behind the lessons were
  • We have shown trainees videos of English language lessons that demonstrate the techniques used in various methods and then discussed the underling assumptions
  • We have asked each trainee to read up on a method and prepare a poster presentation on it for the rest of the class
  • We have looked at lesson plans that the trainees themselves had prepared and identified influences of different approaches and methods in the procedures described therein
  • We have played games like TEFL pursuits (obviously, an adaptation of Trivial Pursuit) in which all the questions had to do with the beliefs and assumptions behind various methods
  • We have organised balloon debates with characters like Caleb Gattegno, Stephen Krashen, Scott Thornbury and Georgi Lozanov in which the trainees have to decide who should not be thrown out of the balloon
  • We have delivered mini-lectures in which we give trainees the main facts about each approach and method, followed by workshops in which they discuss what techniques and activities are compatible with each.
  • We have delivered loop input sessions in which trainees read and translate texts about the grammar translation method, do drills based on statements about audiolingualism, or take part in jigsaw reading activities on the communicative approach.

On the basis of this, you might claim that between us we now have a wealth of material that we can use on the next DELTA course that we teach. And it is true that we do. And it is also true that a lot of that material is very good material indeed, material which has worked in exactly the way we were hoping it would, material that our trainees both appreciated and found useful.

Why, then, do we keep re-designing sessions and trying out new things? After all, there are many apparently successful teacher trainers out there who have happily been using the same material for decades and no-one seems to be complaining! And even ELT celebrities keep presenting the same things in the same ways in plenary addresses in conferences and again no-one seems to be complaining! Perhaps my colleague and I have too much time on our hands, or do not have a life outside TEFL? I think not. The reasons why we feel we still need to spend a lot of time preparing sessions that we have repeatedly taught over the years are of a different order.

First of all, TEFL is a dynamic, developing field. Having attended a Diploma or an MA in TEFL course in the early eighties and not having followed the developments in the field since then does not just mean that you now know very little about the current state of TEFL, it means that you know very little about TEFL in general, as TEFL in the 2010s is a completely different activity from what it was in the 1980s and the body of knowledge on which TEFL professional activity is based has expanded so much that most of what you may have learnt in the eighties is now not just dated but simply irrelevant. For example, when I started teaching on what was then the RSA DTEFLA and the RSA DOTE in the nineties, many of the approaches and methods that are currently all the rage (e.g. whole language, CLIL, competency-based instruction, data-driven language learning), had not even been invented; the research that led to the Common European Framework of Reference was still in its infancy; Internet social media did not exist; the British National Corpus was still being designed and corpora and concordances were still alien concepts to the ELT community. While it still is unlikely that what we know about language, language acquisition and language teaching methodology will change radically over the course of two or three years, the emphases do change quite rapidly and a tutor on a course like the DELTA can simply not afford to assume that the current state of TEFL is what it was two years ago.

The most important reason, however, why it is essential to keep rethinking your approach, redesigning your sessions and producing new materials has to do with the nature of teaching and learning per se. Planning a DELTA input session is very similar to planning any kind of lesson. You make certain assumptions about the students and their current state of knowledge and experience, you study the syllabus, you determine the specific objectives of the session, you plan a sequence of activities that will help your students achieve those objectives and you select or produce the materials that will enable them to do these activities. However, you cannot be sure that the materials and activities will work until you actually try them out! And it is only after teaching the lesson that you can reflect on what happened during the lesson, what worked and what didn’t work, and, most importantly, why the things that worked worked and why the things that didn’t work didn’t work. So the next time you teach a similar group of students the same topic you have the advantage that the experience of teaching it before and reflecting upon it gives you; not to use that advantage for the benefit of your students would be unprofessional and even unethical, as it would mean that even though you now know what you need to do to improve your students’ chances of learning, you refuse or neglect to do it because you simply can’t be bothered! Now if your students happen to be experienced teachers themselves, who are investing a lot of time and money in their professional development, and part of their assessment focuses on whether they can reflect upon and improve their professional practice (as is the case on the DELTA course), it is completely unacceptable for their trainer to demonstrate that she/he cannot or will not do exactly that.

Conference sponsored by…

A very interesting blog post published a couple of days ago by David Dodgson got me thinking about teachers’ conferences and what they are about. I am not talking about the very large scale conferences organised by IATEFL or TESOL, but rather events organised by local associations in various parts of the world, with more or less success. I have myself been involved in organising such conferences for my local association of teachers, TESOL Greece, of which I was a board member in the distant past. It seems, though, things have changed quite drastically since then.

Teachers’ conferences some time ago

This was an era before the world wide web was invented: there were no blogs, no internet forums, no tweets, no chats, no online events, so teachers really depended on such conferences to get information on what was going on in the profession, to share ideas with colleagues and to further their own professional development. A teachers’ conference, even in a small country like Greece, was an important event, which attracted quite a few hundred teachers as well as some of the biggest names in English Language Teaching globally. This may also have been due to the fact that there were not that many ELT conferences worldwide, ELT publishers did not run as many road shows as they do these days, free seminars for teachers sponsored by publishers, examination bodies and language school chains were very sparse, and perhaps the big names were not in such high demand and had time and energy to devote to preparing and delivering original talks and workshops even if the conference was not very widely publicised.

Organising a teachers’ conference in the 21st century

It seems that organising a conference of this scope now is a completely different story. At a time when face to face conferences are becoming less and less relevant, it has become more essential than ever to ensure that they are attended by a large number of people, so that the conference fees collected can at least meat the ever rising cost of the conference itself.  However, to attract those large numbers, the conference must be worth its salt. As a bare minimum, I would expect the following:

  • The keynote speakers must be people who have something to say, who can say what they have to say attractively, and who have not said the exact same thing in the exact same way before.
  • The focus of the talks and workshops cannot be to promote a commercial product or service (such as a book, an exam or a course) but must be on teacher development: the exchange of ideas, the evaluation of current practices, the discussion of new developments in our field, and so on.
  • The premises must be decent, the equipment the presenters need must be in good working order and the room available should be adequate for the turnout expected.

But face to face conferences have become very expensive: venue costs are rising (a conference centre in Athens, Greece, can charge € 20,000 + for a two-day conference attended by 400 people or so); most keynote speakers ask for hefty honoraria in addition to their travel and accommodation expenses; conference publicity, invitations, badges, etc., can also cost quite a bit; insurance may well be another unavoidable cost; and there may well be a need for paid admin staff to assist in the conference (security, ushers, etc.) in addition to the volunteers who organise it. So even if the conference managed to attract 400 people, and they each paid a € 70 conference fee for the privilege of attending, the costs would still not be met.

Enter the sponsors

What is a poor teachers’ organisation to do? Seek sponsors, of course! This is how most small-scale conferences manage to survive! This would not be a problem if the sponsors were hotel chains, airlines, mobile phone companies or e-shops; this kind of sponsor, however, is highly unlikely to be interested in a teachers’ conference with a few hundred attendees. The people who are interested in reaching a few hundred English teachers are those who have things to sell to the teachers and their students: ELT publishers, exam providers, language school chains, teacher training institutions, and so on.

Let’s have a look at two examples of ELT conference sponsorship:

  • The conference organisers will naturally ask ELT publishers if they can sponsor a speaker; the publisher will equally naturally sponsor the author of one of their latest course books to give a talk and indirectly (or occasionally not so indirectly) promote their book.
  • The conference organisers may ask a large teacher training institution to offer their premises; the institution will gladly host the conference as it gives them the unique chance of promoting their services to four hundred of their most likely customers, i.e. teachers evidently interested in professional development and willing to pay for it!

Exit the point of the conference

Clearly, there is no such thing as a free lunch and there is no such thing as a disinterested sponsor. The ELT-related businesses that sponsor teachers’ conferences do so for the exact same reasons that businesses that have nothing to do with ELT do not: because they are focused on what there is to gain. Relying exclusively on sponsors to pay for the venue and determine the content of the conference inevitably results in a conference that will, more or less shamelessly, promote the products and services of the sponsors rather than the professional development of the attendees.

Sooner or later, conference goers do realise precisely how irrelevant all this is to their own development. If insult has been added to injury and they have also been asked to pay for the privilege of having products and services advertised and sold to them, then they may well feel cheated out of their money and start to resent the conference, the conference organisers and the association that is behind the conference. And if they have made the mistake (as I know I have) of actively participating in the conference by presenting a paper or leading a workshop which they have carefully prepared so that no product or service is promoted which they themselves may have an interest in promoting, then I would argue that they rightly feel doubly cheated: out of both their money and their self-respect.

The more teachers realise how irrelevant many of these conferences are, the more commercialised the conferences have to become, as the only alternative to commercialisation is to organise a conference that people will want to go to, will need to go to, and will be willing to pay for. Otherwise, there eventually comes a point when even the ELT-related sponsors lose interest in the few faithful (or perhaps naïve) conference goers, withdraw their support and the conference dies the death that it should have died some time before.

I have attended many seminars…

I was looking at yet another CV of an ELT teacher applying to work with us as a part-time educational consultant. This teacher had a degree in English Literature, quite a few years of experience and an impressive two pages of “Seminars attended”. The “seminars attended” included, almost exclusively, items like the following:

  • Preparing students for [exam A]
  • Using [coursebook B] with young learners
  • Making the most of [brand C] interactive whiteboards
  • The assessment of writing skills in [exam D]
  • Teaching adults with [publisher E’s] brand new course

I scanned through the long list looking for courses or workshops that this teacher had attended that actually focused on teacher development rather than promoting educational products and services; there were none. And yet, when I interviewed the teacher in question, she said “I am very interested in professional development and, as you can see from my CV, I have attended a very large number of seminars; I understand the need to keep abreast of current developments in our field!”

Fair enough! It was evidently true that this teacher had, in fact, given up quite a few free mornings, especially at weekends, to “attend” these “seminars,” and it was also evidently true that she devoted time and energy to collecting certificates that the “seminar” organisers produced and it was also evidently true that she actually believed these activities to constitute teacher development.

While I agree that such informative, or even commercial, events can indeed contribute to a teacher’s professional development, as they can potentially give teachers useful information on products and services that they might find useful in their profession, it does bother me that a lot of teachers seem to think that  such free seminars and workshops sponsored by publishers, examination bodies and other organisations primarily interested in promoting their products are in fact equivalent to teacher training / teacher development courses that other teachers invest in, both in terms of time (often quite considerable) and in terms of hard-earned money! Some of these teachers are even more unaware than the teacher I have described here; you ask them “have you completed any teacher training courses?” and they respond “oh, absolutely, lots! I go to seminars all the time” meaning, of course, that they routinely attend those free promotional events.

It is true that some of these commercially driven “seminar” providers do occasionally make an effort to offer teachers workshops and talks that do, indeed, focus on teacher development or methodology issues which are quite unrelated to the products and/or services they promote. I have organised a number of such events myself and in the “seminars” the organisation I work with (PeopleCert, which represents the City & Guilds English language exams in Greece) offers, I always make sure that we include one talk that directly relates to the exams we promote and at least one workshop that focuses on a topic in methodology that has nothing to do with our exams. Over the last three years, we have covered in such workshops topics like developing speaking skills, using dictionaries, using a genre-based approach to teaching writing, teaching listening, classroom management, and using Web 2.0 tools in the classroom. And yet, I would not dream of pretending that these workshops and seminars can do anything more than give participants a taster of how much more there is to learn about their profession.

It is clear that such events are by definition limited and that attending a talk or workshop on a Saturday morning cannot really, by itself, support a teacher’s effort for professional development. It is also clear that any certificates of attendance teachers collect from such events have a rather limited value: at best, they certify that the teacher was physically present in a room where something to do with ELT was being talked about.

This is one more example of the term “teacher development” being misused and applied to what is in fact lack of development. As I have argued elsewhere, teacher development presupposes teacher education, and teacher education is a long, often painful, and usually expensive process. The kind of awareness that can enable teachers to question their current practices, to improve their teaching and ultimately to improve their students’ chances of learning, learning faster and learning more enjoyably is an awareness that can only be developed through a teacher educational process that is far more demanding, and therefore far more far-reaching, than the free seminars we can attend at publishers’ events, conferences and ELT book exhibitions.

Once you have completed a course or two that involve critically examining your current practices, planning lessons, teaching, and getting feedback on your teaching so that you can confront and re-evaluate your assumptions about teaching and learning, the free workshops and seminars can indeed help you develop further; and so can blogging, interacting with other teachers on the web, attending conferences, and so on. The difference is, of course, that you know and understand more about what you are talking about and you are, actually, in a position to process what you are being told by the various “presenters,” to evaluate it, to relate it to your own practice, and to reject it if it deserves to be rejected!

encouraging submission

A teachers’ newsletter that supports teacher development

The TESOL Greece Newsletter is, as the name suggests, the newsletter of TESOL Greece, one of the first professional associations of teachers of English in Greece and an affiliate of both TESOL and IATEFL. It is not a newsletter in the strict sense of the word, as it has always published, in addition to the news of the association (letter from the chair, calendar of events, meet the board, etc), practical articles on methodology, teaching ideas and suggestions, reports on conferences, etc; in short, the TESOL Greece Newsletter (from now on, TGNL) publishes the sort of material that one might expect to find in a practical magazine for teachers of English as well as newsletter material. In this respect it is not unlike other association newsletters like IATEFL Voices or the now discontinued Essential Teacher, which was published by TESOL; indeed, at least some of the articles in TGNL have, as a matter of fact, in recent years, been reprinted from IATEFL Voices and from the newsletters of other TESOL affiliates.

The main function of this type of publication seems to be to allow teachers to share ideas and exchange views on issues of professional concern, as well as to disseminate information on the latest developments in the field of English Language Teaching. Many of the articles are written by practising teachers, who do not necessarily have (or need!) an academic background. The idea is that the periodical will contribute to what must be one of the main aims of a professional teachers’ association: to foster and support teachers’ professional development. Indeed, the TESOL Greece website clearly states with reference to writing for the TGNL:

Many of the best teachers develop their skills by interacting with other teachers and hearing about their teaching practices. We also develop ourselves professionally by writing about and reflecting on our experiences as teachers.
The Newsletter caters for the needs of diverse target groups ranging from students and young teachers to experienced teachers and people working in the publishing field. In all likelihood, there is an audience out there waiting for your contribution.

It is clear that those seeking a scholarly journal to which to submit their research reports or research articles should look elsewhere. The envisaged readership of the TGNL is not academics, the TGNL is not a refereed journal, submissions are not peer-reviewed and, consequently, having an article published in the TGNL does not have the kind of value that forces academics all over the world to continually publish (sometimes articles based on mundane or insignificant, yet “original” reasearch) so that they do not perish. Those members of TESOL Greece that are, indeed, academics will submit their research papers to journals such as  Applied Linguistics, TESOL Quartely, Language Learning and the like, i.e. journals which do, in fact, have an established gate-keeping function within the disourse communities of the academic disciplines to which they relate. Thus, publishing a piece in, for example, TESOL Quarterly is not about sharing your ideas with colleagues, it is about being accepted into an academic discourse community, whose gate-keepers are the academics who blindly review submissions and the journal editors; these gate-keepers are necessarily respected, senior members of the academic discourse community, who have had a number of articles of their own published in such journals.

TESOL Greece has always, as far as I know (and I have been a member since 1987, so I really ought to know), been aware of the main function of its newsletter as one more tool for supporting teacher development. It has always encouraged teachers to contribute articles and teaching ideas, it has always published articles written by teachers for teachers and the editors of the TGNL (who are unpaid volunteer members of the association, as are the members of the elected board) have always encouraged teachers to write, even if they are unsure of their writing style or the relevance of their ideas. The TESOL Greece website still pleads: Please DO write the article! Your contribution may be invaluable to many colleagues. And to those worried about the accuracy or appropriacy of their English it even goes so far as to patronisingly suggest: You don’t have to worry about making mistakes. All articles are proofread and edited -if necessary -by experienced teachers who are native speakers or bilingual. (Believing that the intentions behind this statement were good, i.e. to encourage more teachers to share their ideas and views, I will forgive the faux-pas of implying that native speakers are necessarily good writers and editors who do not make mistakes and whose help the nonnative speaker teacher needs and necessarily appreciates). Further, the TESOL Greece site reassures prospective writers that we ALL have something to contribute to our profession, even if we are not “experts” or ”authorities.”

When I received the latest issue of the TGNL (#106, April-June 2010), it did not look as though anything had changed. In addition to the association news and updates, it contains the kind of articles that one would expect: some on methodology and class management (e.g. on reducing teacher talking time), some on professional development (e.g. reflective blogging for teachers), some containing views on broader issues related to ELT (e.g. the relationship between theory and practice). The Editor’s Introduction (page 3) contains, among other things, the usual words of encouragement to prospective writers:

all members should take the opportunity , or challenge, to share their ideas, methods, successes (and failures, because we can all learn from our mistakes) and even just your thoughts. By sending us your work, your articles and your comments you will be making this Newsletter more truly representative of all those different areas, people and places that make TESOL Greece the organization that it is. Please contribute! (See Article Guidelines in this issue).

Hmm, I thought, they have even included article guidelines in an effort to get more people to contribute to the newsletter! I knew, from my experience of serving on the TESOL Greece board and working closely with the newsletter editors, that it can be difficult to get enough contributions to fill the 36 pages of a quarterly newsletter with material that will be relevant to the readership and worth reading. And things, I thought, must be even more difficult now that teachers are forced to work more and get paid less, therefore have less, if any, time to devote to any kind of personal or professional development; not to mention the fact that those who do feel like writing about their professional lives and sharing their ideas with the world have, in the era of Web 2.0., far more and far more convenient publishing platforms at their disposal and can easily reach audiences far larger than the few hundred members of TESOL Greece who might read the TGNL a few times a year. When I looked at the contents of the issue, my suspicion was, unfortunately, confirmed: 40% of the content was actually reprinted from other newsletters (mainly IATEFL Voices).

This apparent reluctance of teachers to write for the TGNL explained, I thought, the recurring “please send us your contributions” leitmotiv in the “Editor’s Letter” in the last few issues as well as the friendly suasive (to the point of being patronising) tone of the Newsletter FAQs on the web: the TGNL, I thought, may be in danger of not getting any contributions at all in the near future if the current trend continues.

A teachers’ association that encourages submission

I quickly turned to page 12, where the article guidelines referred to in the Editor’s Introduction were to be found, mainly out of curiosity to see what ideas the Editor and the Board of TESOL Greece had come up with this time to persuade, coerce or inveigle the members of the association into writing something for the newsletter. What I found on page 12, however, was very different from what I was expecting: far from encouraging people to contribute articles to the newsletter, the supposed guidelines positively discourage people from contributing anything and might even discourage certain people (me, for one) from ever reading this publication again.

The first thing that struck me was the tone of the guidelines. This was no longer the friendly pat on the back that said “don’t worry, even if you make mistakes in your writing, we’ll fix them,” nor was it the plea “please contribute” that I had seen on the website and in the Editor’s letter. The title on page 12 is actually “TESOL Greece Newsletter Submission Guidelines,” while the style of the whole text is clearly formal and distant, characterised by the following:

  • high lexical density, containing a large number of latinate words (such as, concision, construed, elucidate, endorsement, indicate, significance)
  • high incidence of nominalisations (e.g. encourages submission of articles, under consideration for publication, having the endorsement of)
  • total lack of personal subjects (I, we, you)
  • high incidence of passive voice constructions
  • circumlocutions aiming to increase the distance between reader and writer (e.g. individuals concerned with English language teaching and learning)

The second, even more surprising, feature of these guidelines is that they have nothing to do with either the actual content of the TGNL or what is explicitly stated about this content in the editor’s letter and on the TESOL Greece site. In fact, according to the “Submission Guidelines,” it appears that articles about ideas that worked in class or just the members’ thoughts will most definitely not be accepted for publication, as the factors that are considered (significantly, we are not told by whom) when evaluating a submission for publication include the following:

  • the submission elucidates the relationship between theory and practice: practical articles must be anchored in theory […]
  • the submission offers a new, original insight or interpretation and not just a restatement of others’ ideas and views.
  • the submission makes a significant (practical, useful, plausible) contribution to the field and reflects sound scholarship and research design with appropriate, correctly interpreted references to other authors and works

Moreover, according to these guidelines, the areas in which the TGNL accepts submissions include topics like the psychology and sociology of language learning and teaching, psycholinguistics, sociolinguistics, research and research methodology, but not topics like class management or reflection on practice (which is, oddly, claimed to be the main theme of this issue).

On the basis of these guidelines, assuming no prior knowledge of what kind of publication they refer to, I do not think there is any doubt that one would have thought this is, indeed, a refereed journal. Which, clearly, the TGNL is not. And has never been. These formidable guidelines contradict all previous TGNL guidelines, misrepresent the content of the publication and positively discourage the average teacher from considering writing for the TGNL. What is worse, if they were to be followed, and if the TGNL did in fact publish only articles that were in accordance with these guidelines, the majority of the readers of the TGNL would simply find it irrelevant; it might, however, if it managed to establish itself as a scholarly journal, gain a new following in academic circles and a new life on the shelves of tertiary institution libraries – but even if that were possible, which it is not, for reasons that I will try to explain below, the TGNL as TGNL would need to cease to exist: it could no longer be the newsletter of a professional association that is, as the cover advertises, distributed free to all members, it would have to turn into a subscription journal available to all academics who are interested in it, whether they chose to be members of a teachers’ association or not.

Seriously, though, I do not believe that these submission guidelines mean that the TGNL has changed into a scholarly journal overnight. I would very much like to think that the people who came up with these pseudo-guidelines were misguided or simply mistaken, or that the pseudo-guidelines were put together as a joke that accidentally found its way to page 12 of the TGNL. After all, the TGNL does not have an editorial board and does not use a peer review system, and there is simply no way on earth any volunteer TESOL Greece member that takes on the responsibility of the TGNL Editor can have the necessary authoritative expertise and academic prestige required in all of the areas that the TGNL accepts submissions in so as to act as the sole reviewer; but even if such a person existed, it is doubtful that they would agree to do this kind of work for free; and even if that mythical creature existed and consented, the academics would never come forward that would even consider submitting their work to a journal that had no reputation in academic circles, no editorial board, no expert advisors, and merely one, albeit omniscient, volunteer editor to review their work.

An identity crisis?

The people who came up with these pseudo-guidelines are, in fact, named in a note at the bottom of the page. The note reads: These Guidelines have been adapted from those in use by TESOL Inc. and have been approved by the TESOL Greece Executive Board. This then, one might innocently think, explains some of the confusion: the TESOL Greece elected board members simply adapted some of the guidelines that another teachers’ association uses; the fact that this other association is one of the longest-standing and most respected associations in the world, and that TESOL Greece has almost always been an affiliate of this association would seem to further justify the decision made by the TESOL Greece board.

The problem is that the guidelines were in fact adapted from the guidelines of TESOL Quarterly, a professional, refereed journal that happens to be published by TESOL Inc. Indeed, the most objectionable of the guidelines seem to have been lifted verbatim from the TESOL Quarterly General Submission Guidelines. TESOL Inc does not, of course, use these formidable guidelines for any of its other publications, and did not use them for its own newsletter/practical teachers’ magazine, Essential Teacher, which ceased publication last year. Neither TESOL Inc nor any other teachers’ association would ever dream (or has ever dreamed) of using the submission guidelines of a refereed journal for its members’ newsletter.

I really find it difficult to believe that the TESOL Greece members who comprise the elected board of the association could have made such an embarrassing mistake. Most of them evidently know what a refereed journal is and does and in what ways it differs from the TGNL and I am sure they know that TESOL Quarterly is a refereed journal, not just because it says so on the web page from which they lifted most of the guidelines they approved, but also because many of the TESOL Greece board members, according to their bios published in the very same issue of the TGNL, have completed post-graduate studies in ELT, so must be very familiar with the genres publishable in TESOL Quarterly.

Nor do I believe that the board of the teachers’ association I have been a member of for 23 years suffered a collective identity crisis, spiced with a massive dose of folie de grandeur, which led them to the delusion that, by virtue of having been elected to a volunteer association’s board, they now form the academic editorial committee of a non-existent refereed journal that used to be TGNL.

There is no doubt, I hope, that these guidelines are a mistake, regardless of the facts that I think led to their publication (more on which below). They are a mistake for which I think the elected board of TESOL Greece owe the members an apology, because these guidelines misrepresent the content and function of the TGNL and discourage members from contributing to it. They are also a mistake because they misrepresent the TESOL Greece Board members, who thus falsely appear to have assumed the roles of gatekeepers rather than facilitators.

A few explanatory facts

It is still not clear, however, how and why the board suddenly decided, out of the blue, that they needed to publish these new submission guidelines. And in fact, the publication of the new TGNL submission guidelines cannot be fully accounted for unless certain facts are taken into account that are not, yet, public knowledge.

Some time ago, a friend and colleague, Marisa Constantinides, told me she was writing an article for the TESOL Greece Newsletter. Marisa is a very well-known teacher educator, Director of CELT Athens, a teacher education centre that, among other activities, runs Cambridge CELTA and DELTA courses. She is also one of the founding members of TESOL Greece and has, like myself, repeatedly served on the board of the association. The article, she explained, was about what it takes to be a teacher educator and she was very excited to be writing about this topic. I, too, was very excited and told her I was looking forward to reading it.

A few weeks later, Marisa told me her article had been rejected by TGNL on the grounds that it did not fit with the theme of the issue that was being put together. We both felt this was rather strange, as this was an article that she had been asked to write, not an unsolicited contribution. In fact, I thought it was rather shocking that Marisa’s article was rejected, not just because it had been solicited, but because I believe she is a very good writer and was writing about a topic that she knows better than most, having had over 25 of experience as a teacher educator.

The article in question was then published as a guest post in the extremely popular TEFL blog Kalinago English and you can read it here. Later, Marisa’s post was nominated for the category of “most influential post” in the Edublogs Awards 2009. In retrospect, I think Marisa should be thankful that her article ended up being published in Karenne Sylvester’s blog, read by thousands of people, and nominated for this award, rather than having been published in a print newsletter with very limited readership.

What is interesting, though, is that Marisa found out later, in informal conversation with one of the members of the (elected) board of TESOL Greece, that the reason she had been given earlier for the rejection of her article was not true, the real reason being that the board had felt the article was self-promotional (!) and could therefore not allow it to be published! If this version of the story is true, the fact that Marisa did not need to promote herself to any of the five hundred members of TESOL Greece, as they already know her and her school, did not, apparently, cross their minds. That it is impossible to talk about your professional experience without talking about… well, your professional experience did not cross their minds either. But the most frightening thing is that they felt they had the right to act as gatekeepers of the TESOL Greece member community and that they did, in fact, prevent an article from being published for reasons that have more to do with their personal beliefs, ideology and preferences than with the job they were elected to do.

We have no way of knowing whether this version of the events is what truly happened. However, Marisa tells me she wrote to the TESOL Greece board a few months ago requesting an explanation, but has so far not even received an acknowledgement.

An original guideline

I suspect that it was instead of replying to Marisa that the TESOL Greece board decided to publish the Submission Guidelines which I have been discussing in this post. What I saved for last is the only guideline that was not, in fact, lifted from the TESOL Quarterly guidelines but must have been composed by the elected members of the board themselves. It reads as follows: Submissions promoting commercial or personal interests will not be published.

How it can be determined that a particular “submission” promotes commercial or personal interests is not clear. For example, could an article on the construct validity of the TOEFL be claimed to promote the TOEFL? Could an article about the revisions to the Cambridge DELTA scheme be claimed to promote the Cambridge DELTA? Could an article about the use of Fidel charts in the Silent Way be claimed to promote Educational Solutions, Inc? My answer to all three questions is no. Others might think that the answer to any or all of these questions should be yes. In any case, these three questions could all be claimed to make reference to actual commercial products, regardless of the fact that these products are actually part of the everyday reality of many English language teachers!

But when it comes to personal interests and their promotion in a teachers’ association’s newsletter, which explicitly invites teachers to share their experience with the readers, it is even more difficult to define what constitutes “promotion”. I have been a teacher educator for almost twenty years, a materials writer for ten and an academic manager for seven. If you ask me to share my experience with you, I might tell you about the DELTA and CELTA courses that I have taught on, the rewards and the challenges, the things I have learnt and the things that I am still finding it difficult to learn; I might also tell you about the City & Guilds ESOL exams which I am now working with and the Cambridge ESOL exams for which I have worked in the past; I might also tell you about the challenges of designing the syllabuses and training and assessing the teachers and the Directors of Studies at the Eurognosi language school chain, of which I was Academic Director; and I might also tell you about Action, my first coursebook series, Lift off, my second, and Code, the most recent coursebook I worked on and what I learnt from my experience as a coursebook author. Does this mean that I am promoting DELTA, CELTA, City & Guilds, Cambridge ESOL, Eurognosi, Action, Lift off or Code? Does it mean that I am promoting myself? And if the latter, what exactly am I promoting myself as? And for what? And to whom?

There are, I think, some very good reasons why publications like TESOL Quarterly, but also IATEFL Voices and the various teachers’ associations newsletters do not include, in their notes to contributors, any clauses like the one that TGNL came up with regarding the promotion of personal or commercial interests:

  • It is not always possible to define what constitutes promotion, especially of personal interests. As a result, this criterion can easily be used to conceal the hidden agendas and personal interests (or likes and dislikes) of those who make the editorial decisions.
  • When talking about teaching it is often necessary to refer to specific examples from your experience, which would immediately rule your contribution out as it might be taken to be promoting yourself.
  • When talking about research you have conducted (in case TGNL seriously wants to become a scholarly journal!) it is essential to refer to your specific context; not to mention that the whole point of submitting an article to a refereed journal is indeed to promote yourself in the academic world (publish or perish)!

Self-appointed (and self-accountable?) gatekeepers

If, then, the motivation behind the rushed and misguided decision made by the elected members of the board of TESOL Greece was, as I suspect, not to prevent teachers from contributing to the TGNL, but to secure an alibi, a retrospective, fool-proof justification for rejecting articles contributed to the TGNL, then the situation is much worse than it would have been if the publication of these submission guidelines were merely a matter of ignorance and confusion. In any case, the questions that remain open are the following:

  • What should be the criteria used by the editor of a teachers’ association’s newsletter in accepting or rejecting an article that has been submitted or, worse still, one that has been solicited?
  • What is the role and function of the elected board of the association with regard to the association’s newsletter? Do they act as a collective editorial board? If so, does this apply to all submitted articles? Are the members of the association who elected them aware of this function? Are the members of the board competent to make such decisions? And, most importantly, are the criteria and the process used transparent?

Are ELT materials purged of ideology?

I started thinking about the topic of this post last Thursday, when I read Lindsay Clandfield’s article on course books and the curse of celebrity in the Guardian Weekly online. The article reminded me (because my brain works in chaotic ways) of Scott Thornbury’s Window-dressing vs cross-dressing in the EFL sub-culture, published more than ten years ago, but still relevamt and easily accessible via Scott’s website. And that in turn reminded me of Althusser’s seminal essay Ideology and Ideological State Apparatuses, which I keep returning to ever since I first read it in my early twenties.

The topic is, of course, the subject-matter of English Language Teaching materials, and more specifically, the ideological content of such materials. Lindsay writes about how international celebrities have permeated the world of ELT course books in the last 20 years or so and makes the valid point that this kind of material does not conform to many educators’ ideas of what education is about; clearly, he is thinking about educators with a more critical attitude to what education should be about, and in fact implies, in his last paragrpah, that he believes education is about thinking critically about the world we live in and making more sense of it. Lindsay is, of course, right: the inclusion of international celebrity culture in teaching materials may be motivating in some contexts, appreciated even, on a superficial level, by many learners, expedient insofar it makes it easy to present and practise language based on a common core of shared (albeit stereotypical) cultural knowledge, but the inclusion of and constant allusion to celebrity culture is also an ideological statement that the course book writers, editors and publishers are not just making but also imposing upon the users of the course book (teachers and students), who may not necessarily consciously subscribe to the underlying ideology.

However, as Scott Thornbury put it 11 years ago, ideology is not just present in the content that is included in course books, but also in the content that is, conspicuously or surreptitiously, excluded from them. Scott’s point is that the total absence of any allusion to GLBT (gay, lesbian, bisexual and trans) culture in coursebooks is also ideological, especially given the fact that a lot of ELT practitioners, including publishers, authors, teacher educators and teachers, are in fact members of the GLBT community. The forced invisibility of gay, lesbian, bisexual and trans people in ELT materials is still evident 10+ years later, even though GLBT professionals in the ELT world are now a lot more visible, as are GLBT people in general in most of the countries where these materials are used. I am not saying (and nor was Scott) that it is easy to include GLBT role models in ELT materials (although it is clearly easy to include characters that might be GLBT); but I am saying that the reason why most course book content is bland, vapid, uncontroversial is because publishers cannot afford to have absolutely anyone object to their content (there are, for instance, at least two publishers’ guidelines that state the word bacon cannot be included in any ELT material, lest teachers and students in Moslem countries be offended) and, at the end of the day, the educational value of course books is indeed subordinate to their market value, as far as publishers are concerned.

Which brings me back to Althusser’s views on ideology and the ideological apparatuses of the state. In brief, Althusser holds that the reproduction of the relations of production, i.e. the capitalist relations of exploitation, is achieved via the Repressive State Apparatus (the legislature, the police, the army, etc) as well as the Ideological State Apparatuses (communications, culture, religion, school, etc), both of which function both by repression and by ideology. The dominant Ideological State Apparatus is, of course, the educational state apparatus. The ideological manner in which education works to reproduce the given relations of production is, in Althusser’s words, via

an ideology which represents the School as a neutral environment purged of ideology (because it is …lay), where teachers respectful of the ‘conscience’ and ‘freedom’ of the children who are entrusted to them (in complete confidence) by their ‘parents’ (who are free, too, i.e. the owners of their children) open up for them the path to the freedom, morality and responsibility of adults by their own example, by knowledge, literature and their ‘liberating’ virtues.

In this light, the ideological nature of the ELT materials I have been talking about becomes clearer:

  • Nothing can be included that could be considered ideologically or culturally sensitive, because ELT materials must appear to be non-ideological; however, the appearance of non-ideologicalness is, in actual fact, merely the perpetuation of the ruling ideology which typically masquerades as a non-ideological norm.
  • Content that is included must be harmless; but by “harmless” is meant harmless to the ruling ideology. Thus, the life of David Beckham is perfectly accaptable content; so are Madonna’s charitable acts; so is universally recognised “high culture,” such as Victorian literature or the paintings of the Great Masters. But nothing that is remotely critical of the supposedly non-ideological ruling ideology, whether in terms of discourse (e.g. atheist views) or of action (e.g. gay “families”), can even be alluded to, on the grounds that ELT materials must be neutral, must not take sides!

The practical reasons why non-harmless material simply does not make it to the course books that are published are an excellent illustration of how this particular ideological apparatus, ELT publishing, functions both by ideology and by repression. Publishers will simply censor any material that is culturally or ideologically powerful: this is a repressive mechanism that has been put into place precisely so that the ideological content of ELT materials can be controlled. The market (the people who will consume the books, i.e. directors of studies and teachers with decision-making powers) will reward, and therefore reproduce, these ideological choices precisely because they appear to be non-ideological. Thus, nothing is problematised, the process of reproducing the relations of production continues stronger, and educators are reduced to mere agents of an ideological struggle they may not even be consciously aware of.

Horror, horror: there were no native speakers present at all!

One of my responsibilities where I work is the recruitment of oral examiners. In fact, in our case they are not exactly “oral examiners,” as they simply act as Interlocutors, i.e., they are not expected to assess the students they are examining, but simply to use the “interlocutor framework” that the examination body provides in order to manage the interaction and elicit as good a sample of spoken English from the candidates as they can. The spoken interviews are recorded and the recordings are then sent to a different group of people, the markers, whose job is to decide whether the language sample elicited by the interlocutors fulfills certain pre-defined criteria.

What was not clearly pre-defined when we started offering these exams in Greece was the Interlocutor selection criteria. The examination body suggested that the Interlocutors should be teachers with some experience teaching the level that the students were being examined at. That made sense, of course, but I felt that it would not be enough in a country where (1) English language teachers do not always have a good command of the language; (2) English language teachers do not necessarily have much methodology training; (3) most of the stakeholders expect, or even demand, that the people who “examine” candidates in a spoken test should be BANA* natives. So my colleagues and I came up with some minimum professional requirements (which include a very high level of spoken English, a minimum of five years of classroom experience, and some basic methodology training) as well as a structured selection, training and assessment procedure, which we have so far used quite successfully to recruit and train more than 200 Interlocutors in the space of two years.

There have, of course, been problems: unavoidably, perhaps, some of the Interlocutors prove not to be as good in practice as we thought they would be when we trained and assessed them, there have been occasional complaints about Interlocutors not being helpful enough or courteous enough to the candidates and we have also had Interlocutors who realise they do not want to continue doing this either because of the working conditions or because of the nature of the work itself. Having worked as an Oral Examiner for other examination bodies myself for a number of years, and having worked with other Oral Examiners for a number of years, I can’t say I wasn’t expecting these problems: I have been in examination rooms with oral examiners who may have been perfectly normal people otherwise but would openly smirk at the candidates, or be outright rude to them, or be offensively unhelpful; and I know from first-hand experience how taxing it can be to have to travel to far-away provinces to examine candidates for 6 to 8 hours and then travel back home only to face another long examining day the following morning!

What I wasn’t expecting is the kind of fierce reactions we have been getting from some of the teachers who apply to become Test Interlocutors and whose applications are turned down. I would like to look at one such reaction today, because I find it symptomatic of a number of attitudes which, surprisingly, are still with us, which is why I feel they should be exposed and discussed. One prospective Test Interlocutor, who clearly did not fulfill our minimum requirements as she had not trained to be a teacher and had only taught English for two years, but happened to be a BANA native, decided that she would not bother to call or write to us at all, but sent an email to our “parent” organization in the UK, obviously believing that they would put some sense into our depraved nonnative brains. I quote, changing only some of the details that might give away personal information about the complainant:

I recently had a student of mine sit the B2 level exam in English. While attending the oral exam, in the city of [large provincial city], Greece, I realised that there were no native speakers present at all! When requiring [SIC] about this, an examiner told me, Greek speakers of English were better as the students feel more ‘comfortable’ with a Greek examiner when participating in the oral exam!
 Subsequently, I have in fact sent my C.V. to the Greek office in Athens, and have never received recognition in any form, or indeed an answer in any form. I do know the importance of native speakers in the oral examinations, if the said students plan to study in Britain. However, I have to say that I will seriously consider not sending my future students for [SIC] your examinations in the near future. 
As I will be teaching this summer in England, at the [name of a university], during the months of July and August, please feel free to contact me. I am open to solutions and a better experience of your examinations in Greece.

Let’s have a look at exactly what this person is saying and doing before we look at why she might think she is justified in saying and doing the things she is saying and doing:

  1. She is suggesting, at first indirectly by means of the exclamation mark at the end of the second sentence, and then very directly and emphatically (“I do know the importance of native speakers in the oral examinations”), that it is necessary for an oral examiner to be a BANA native.
  2. She is suggesting, by virtue of applying despite the minimum professional requirements being clearly stated in the ad she was replying to and on our website, that BANA natives need no other qualifications nor experience: they are, by definition, the right people for the job. She even explains that the reason why she decided to apply was precisely because she noticed there were no other BANA native “examiners”!
  3. To make her case strong (which makes me suspect that at some level she must know it is difficult to justify her stance), she falsely claims that she was told non-BANA speakers are preferred for some ridiculous reason; she also claims her applications (both of which were sent to us two days before the complaint was sent to our UK partners) were not acknowledged or processed.
  4. To make her case even stronger, she threatens to stop “sending students for” our exams; in other words, she is saying that, if it were up to her, the commercial success of the exam is dependent upon the examination body’s compliance with her demands and acceptance of her views.
  5. To make herself sound more credible, without risking mentioning her qualifications and experience, she mentions a UK university where she will be teaching summer courses, confident that the mere fact that a BANA institution has offered her summer work proves her professional worth as a language teacher.
  6. In a final patronising move, she offers her services to the examination body so that a better examination service can be offered in Greece, since, evidently, these non-BANA speakers cannot do the job properly! This must also be the reason why she did not bother calling or emailing us, the incompetent Greeks, but went straight to her compatriots to seek a solution to what she believes is a grave problem.

To say that I am angry at this person’s reaction, her ideas and her attitude would be an understatement. However, let me clarify, it is not that I have personally or professionally suffered because of it in the narrow sense. It is because I find it sad and dangerous that, even in this day and age, more than ten years after the ideas about nativespeakerism, the ownership of English and linguistic imperialism were first articulated and widely discussed in academic circles, in practice, the ELT community in Greece and many other non-BANA countries is still structured in such a way that voices like the one I have been writing about are still heard and listened to.

What angers me is not what this person says in her email, but rather, the fact that she feels she can say these things, and, worse, that the majority of teachers in Greece would even agree with her! In my experience, most teachers would expect Oral Examiners to be BANA natives, if possible recently imported for the purpose of conducting oral exams; most language schools will employ BANA natives without even enquiring about their teaching qualifications and experience; most parents will demand that their children are taught, at least once a week, by a BANA native; and both the parents and the children themselves have learnt that in a Spoken English examination the examiner must have a perfect BANA accent, so that, even if they are not a BANA native, they can pass for one; and then, of course, teachers like myself are made to feel that they have to make an effort to pass for BANA natives in the same manner that many male to female transsexual people are made to feel, for very similar ideological reasons, that they have to make an effort to pass for biological women! Sad, sad, sad!

Continue reading

Make my students pass or else!

For the last few years I have been working with an awarding body which, among other things, offers examinations in English for speakers of other languages. In Greece, English language exams are big business: more than a hundred thousand students sit for such exams every year and fourteen different examination bodies have so far had their qualifications recognised by the Greek state.

EFL teachers and language schools in Greece therefore have, one would think, a lot of choice. One would imagine that the teachers and language school owners look at what each exam is about, how the various language skills and/or systems are tested, which exams are more compatible with the syllabus they are following, whether they are appropriate for the age of their students, and so on.

In actual fact, though, the format, structure, content and syllabus of each exam are immaterial! Most schools make their choice of exam to prepare their students for based on a solitary criterion: how easy it is for the students to pass, regardless of their actual level, regardless of the course they have, or have not, attended, regardless of whether they actually deserve to get the qualification that the exam leads to.

And because there are so many exams to choose from, and because the exam market has become a very competitive one, teachers and language school owners actually feel that they can unashamedly voice their demands. Many of them will not hesitate to call or email the academic director of the examination body (in our case, that person is unfortunately me) and say, in so many words: ‘Five of my students took your exam last month and only one passed! I demand that you reconsider your decision, revise the results and pass at least four out of the five. Otherwise, I will not only allow none of my students to sit for your exam again, I will also bring the matter to my language school owners’ association and you will never get any candidates from my area again.’

What I find really sad (not even shocking any longer, just sad) is that these people, who are supposedly educators, are not necessarily aware, on a conscious level, of what they are doing. When I tell them that we are not in the business of selling valueless certificates, they say ‘of course not; I didn’t say anything like that!’ When I tell them that we refuse to be blackmailed into cooking the results to make them more favourable (!) because that would turn our exams into a sick joke and because that would simply be unethical, they are shocked and surprised and start shouting at me ‘how dare you suggest that I am attempting to blackmail you?’

And yet, when I finally suggest to them that perhaps their students are not, after all, at the level they thought they were and perhaps they need more language development work before they attempt an exam at this level again, the standard answer is invariably that they themselves think their students deserve to pass and must pass; otherwise (i.e, if we insist on offering an exam that is valid and reliable and a certificate that actually means something) we should forget about getting any candidates ever again!

Yes, I am aware of the pressure on teachers and language schools to produce certificate holders; I am aware of the fact that teachers and schools are evaluated almost solely on the basis of exam success rates; I am aware of the fact that many language schools are striving to survive in an impossibly competitive post-capitalist environment. But it still saddens me that people who call themselves educators are so cynically demanding that awarding bodies certify that their students know what they evidently do not know simply because these students, or their parents, happen to be paying customers.