Tony Gurr

Posts Tagged ‘assessment’

LEARNing to Cope with Exams (Guest Post from Laurence Raw)

In Adult Learners, Assessment, Guest BLOGGERS, Our Universities on 24/07/2013 at 3:04 pm

Assessment (David Boud quote) Ver 02

8

Many learners from all over Europe will have taken exams this summer; the results might yet not be known.  My fourteen-year-old niece had this experience, and unfortunately she did not do so well.  I realized that the results bore little or no relationship to her intellectual capabilities; she obtained a poor grade on account of what might be termed TESTaphobia.  As I listened to her, I recalled my days at school and university, when I was so scared of exams that I used to imagine myself suffering from chest pains, so that I could go to hospital and obtain some kind of tranquillizers.

8

I read recently that British Education Secretary Michael Gove insisted that “exams matter because motivation matters … Human beings are hard-wired to seek out challenges … the experience of clearing a hurdle we once considered too high spurs us on to further endeavours and deeper learning”

But what if the need to jump that hurdle prevents learners from achieving success?  What happens to those whose wires are configured in different ways, and might need to discover alternative means of achieving “further endeavours and deeper LEARNing?”  Many websites offer advice as to how to deal with this condition (by learning from experiences, devising a realistic revision schedule, taking time off or relaxing), but they’re actually missing the point.

8

Assess Lit 03

The only way to change attitudes towards exams is to change the LEARNing cultures in which they take place.  Learners have to understand that passing exams is not simply about “clearing a hurdle,” but rather providing an opportunity for them to express what they have learned.  Educators should help them to approach an exam in a positive frame of mind; rather like an actor giving a performance in front of the camera, they need to perform to the best of their ability.  And even if they do not do as well as they should, exams are not the be-all and end-all of their educational lives; what matters more is that they should feel they have achieved their own personal goals through the courses that they have taken.

8

Assidere (original meaning) Ver 02

8

Perhaps it’s time to go back to first principles; to understand that any program of study is not primarily concerned with the exam but with the experience of LEARNing.  This can only be achieved through negotiation; the working out of a series of mutually shared goals that educators and learners alike feel happy to pursue.  As the course unfolds, so everyone should be encouraged to reflect on its usefulness; this might be achieved through discussion, or by encouraging everyone to keep a journal to record feelings and experiences.  Learners can use this as a means to develop their self-esteem, to discover for themselves what they have LEARNed.

In this type of model, the exam functions as an extension of the journal, enabling learners to expound at greater length what they might have already recorded in their journals, and (in an ideal world) thereby manage to deal successfully with their fears.

8

However this can only be achieved through educator support.  This is one thing that Gove and his fellow-mandarins in politics will never understand: learners can only develop themselves when they feel that they are part of a community.  A piece in The Guardian written by a practising  educator asks whether there is a line to be drawn between ‘helping’ and ‘hindering’ learners; whether too much support for learners taking exams is not counter-productive: “What do they learn about self-motivation and independence?  If we want them to become lifelong learners, don’t they at some point need to learn how to teach themselves?

I think this is a comment of mind-blowing fatuity, implying that there is some kind of distinction to be drawn between “TEACHing,” and “LEARNing.”

In a LEARNing community in which everyone participates and helps one another, the problem of developing motivation simply doesn’t arise.  Learners might have to take exams, but they can approach them in a positive frame of mind if they are supported by their peers as well as their educators.

8

Assessment (fattening pigs)

The question here is one of shifting focus, of understanding the psychological reasons why learners fear exams, and restructuring the course of study to help deal with them.  However I fear that no one will be too interested in this solution, especially those politicians who believe that standards can be improved through quick fixes.  At the classroom level, however, I think that improvements can be made, or at least I’d like to think so.

8

Laurence Raw
(aka @laurenceraw on Twitter)
Baskent University – Ankara, Turkey
Editor: Journal of American Studies of Turkey
http://baskent.academia.edu/LaurenceRaw
http://www.radiodramareviews.com

Advertisements

TESTing Assessment in HigherEd…

In Assessment, Our Universities on 03/03/2012 at 9:08 am

 

In an earlier post, I discussed the idea of how students are often treated as “outsiders” in allthingsassessment……especially in higher education.

If we look at what students “want” from assessment (and “need”), they do ask much of us. Seriously…they do not really care that much about reliabilityvalidity and fitness-for-purpose (they are things we have to worry about…and worry about them we should)!

 

if we take a quick look at the research, and contrary to all the “folk wisdom” that students just want “easy exams”, all they really want is:

  • Unambiguous expectations…because they value, and expect, transparency in the way they will be assessed
  • Authentic tasks…because they value assessment that they perceive as “real”
  • Choice and flexibility…because they value the opportunity to showcase their particular talents in the best light


Don’t YOU feel the same way? I know I do…

 

So, here’s the first “test” for your institution:

  • Do you give every student a detailed “coversheet” at the beginning of each year / semester explaining how they will be assessed, the different types of assessment activities you will use, the “weights” of those different assessment events, any specific performance criteria that you will use to “grade” or “mark” student work – and also details of how (and when) you will provide them with feedback on their growth and development? 
  • Do you take the time to go over the “coversheet” outlining its key elements on Day ONE and provide students with the opportunity to ask questions and clarify any issues? 
  • Do you and your institution see this documentation and wider process as a hard and fast “contract” between you and your learners – a promise of the service you will provide to them?


It ain’t rocket-science…and it’s how we would expect to be treated ourselves.

As teachers or lecturers we often “joke” about how many of us have been asked the “mother” of all assessment questions from students:

“Um, do we have to know this? Will it be on the test?”

Wouldn’t it be great if we simply removed the “need” for this question to ever be asked – again…ever? The bottom line is that most students ask this question because our behaviours and actions have ensured they are “assessment outsiders”. Even worse, their experiences with us over the years have learned them that there will always be a “test”…

 

When will we learn…and help learn our students…that:

Assessment is something that teachers

DO WITH students BEFORE, DURING and AFTER learning

NOT, just:

Assessment is something that teachers

DO TO students AFTER learning

OK, while we are on this point – let’s try a second “test”:

  • Does your institution have a written statement of what it values in assessment, a set of principles that informs the development of assessment matrices, events and tasks and also guidelines that help teachers be as effective as they can be when they assess students?
  • Do all your teachers own these values and principles – know them (or at least are able to locate them), “live” them as part of the way they interact with students and contribute to upgrading or improving them over time?
  • Do students know them and can they make connections between the way their teachers do the business of assessment – and the way students are expected to do the business of learning?

As far as students are concerned, there is nothing more central to the learning experience than assessment…and they know that assessment can affect their whole future lives and careers.

Surely, we owe it to them to have all of these things in place…or, at least, have a thunk about them!

 

If, as we noted earlier, assessment is the “engine”:

The real problem here is, of course, that many institutions and educators still view assessment as “weighing and measuring”. Yes, it is a fact of life that some form of measurement needs to take place…how else would we know who to “graduate”?

But, assessment is not just about the “assessment-of-learning” (if we assume that learning can, in fact, be measured at all – indeed, it might be said that teachers can never truly understand what has been learned – only the students know this)! Assessment is also an integral component of LEARNing – and learning-orientated assessment processes can not only help us engage students in rich, authentic tasks, but also contribute (in an on-going manner) to the growth and development of all our learners.

I am, of course, talking about “assessment-for-learning” (the term coined by Caroline Gipps in 1994) and “assessment-as-learning” – the innovative approach adopted and developed by the faculty at Alverno College from 1973 onwards.

As Boud (1995) noted, “all assessments lead to some kind of student learning” – however, approaches that emphasise the “for” and “as” varieties of assessment connect the assessment of student learning to programme and institutional outcomes.

These approaches to assessing student learning do not only rely on testing a student’s “possession of knowledge” – they focus on the “use of knowledge” (in action) and what students can “do” with what they know and learn.

As Englemann (2007)a member of the Alverno philosophy faculty, notes:

As an Alverno faculty member, it is no longer possible to imagine teaching without assessing, because for us to teach is to assess, continuously, what our students are learning, and what they can do with what they know. We assess in order to improve the learning process, to give each student, and groups of students, guidance for their learning. At this point in the life of our curriculum and our academic culture, if our accrediting body were to say: “You no longer have to go to the trouble of assessing student learning,” we would still do it anyway.

 

This not “new” – as the original definition of assessment illustrates:

However, these models of “assessment-as-learning” and “assessment-for-learning” rely  heavily on student self-assessment and teaching students how to observe, analyse, and judge their own performance (on the basis of explicit and published criteria) – not just delivering “content” in pre-packed and off-the-shelf “course units” or using a textbook  (but actively developing meta-cognition in all students).

These approaches, it could be argued, require a fundamental “belief” that all students are capable of becoming adaptableflexible, and independent in their learning and decision-making…they are BTW!

Gulp – values, beliefs and principles again!

Double gulp – new roles, new skills and new abilities for educators!

One of the most important of these is the provision of quality diagnostic feedback from teachers (remember the “lubricant”) – feedback that allows students to further reflect and then determine how they can act to improve their own performance levels.

Not just a “grade”…

We all know:

Anyways, for many institutions and educators, this involves a serious re-thunk of what assessment means to those involved in the process, in addition to the teacher-student relationship in that process.

This was noted by teachers and researchers working on the REAP Project (in Scotland) – and involves the re-conceptualisation of assessment as a collaborative process where students are viewed as “partners in assessment” – they realised that we cannot continue to keep students on the “outside” anymore!

Like Alverno, the guys on the REAP Project have realised that taking this kind of LEARNing and ASSESSMENT PERSPECTIVE results in:

 

REAL LEARNing, REAL TEACHingMEANINGFUL Assessment!

 

OK – ready for the penultimate “test”? Go on – be a devil!

  • Does your assessment of students begin with educational values, reflect an understanding of learning as multi-dimensional and also respect the diverse talents and ways of learning of your students?
  • Do the assessment tasks and events you currently use communicate high expectations to your students, encourage contact between teachers and learners and develop reciprocity and collaboration between groups of students?
  • Do your assessment events emphasise “time-on-task” and going that “extra mile”, give credit for using both active learning techniques and creativity – and also support this with timely, on-going expert feedback?

A couple more – come on, you can do it;

  • Do your assessment tasks include an element of self- and peer-assessment and offer possibilities for students to explore a wider range of “knowledge”: procedural (“knowing how”), schematic (“knowing why”), and strategic (“knowing when certain knowledge applies, where it applies, and how it applies”)?
  • Do you support this type of self-assessment by “teaching” students to understand and interpret the criteria by which they are assessed – and assess themselves?

If you got through those last ones without feeling the need to grab the closest bottle of aspirin (or Prozac)you are probably ready for the next “big” question (inspired by Race, 2002):

  • What is “broken” with assessment at your institution? How do you know? What are YOU going to do about it?

But, remember – first have a thunk about:

  • What do you want to keep that you already have or do in allthingsassessment?
  • What do you want that you don’t already have or do in allthingsassessment?
  • What do you have or do now that you don’t want to keep in allthingsassessment?
  • What don’t you have or do that you don’t want in allthingsassessment?


YOUR students deserve nothing less…as “assessment insiders”!

Bringing students in from the “cold”…

In Assessment, Classroom Teaching, Quality & Institutional Effectiveness, The Paradigm Debate on 07/10/2011 at 8:27 pm

An old friend of mine called me earlier this week. He works for a major Turkish bank up in İstanbul…can’t say which one!

 

After we exchanged the usual pleasantries…(and how the hell he knew I had put on some weight, over the bloody phone, I will never know – maybe my voice gets “fatter”, too – ne se)…he launched into a huge “bitching” session about his bank (now you know why I can’t say which one)!

 

It seems his bank’s senior “my-way-or-the-highway” management team have kicked off a new project…and set up a new project team to ensure all the other divisions are played off against each other (do you really need to know why I “left” banking in my youth). The problem is that this “new” project team is now looking into all those areas that my friend and his colleagues “manage”…and care about deeply (especially, their jobs)!

Being the sensitive and constructive pal I am – I suggested he “walk” and find a new jobNo, I didn’t…I asked him to think about what would need to change to make him and his team feel better and what he could do to try and make that happen.

His response:

“Nothing, we’re out in the cold…we have no voice whatsoever…and will get walked all over…as usual…”

What my friend was really talking about was the fact that he had been “excluded”excluded from a very important process that would impact him, his whole life, his future…

So, I told him to “walk”Come on, he’d find a job before he even started lookingHave you seen the Turkish banking industry these days…many in Europe are already talking of turfing out Greece…and (just) asking İstanbul (the “city” – not the “country”) to join the EU…

Seriously, though…we have all probably felt like this from time to time. Being made to feel like an “outsider” is never a positive experience…think about the last time it happened to you!

All this made me think about studentsand, the ways in which we, as educators, may (inadvertently) make learners feel like this.

 

In an earlier post, I discussed the idea of how students are often treated as “outsiders” in allthingsassessment……and, reflecting on how my dear friend was feeling, I though to meself… “time for this to STOP”!

Come on…if we look at what students want from assessment, they do not have many needs really. Seriously…they do not really care that much about reliability, validity and fitness-for-purpose (they are things we have to worry about…and worry about them we should)!

if we take a quick look at the research, and contrary to all the “folk wisdom” that students just want “easy exams”, all they really want is:

  • Unambiguous expectations…because they value, and expect, transparency in the way they will be assessed
  • Authentic tasks…because they value assessment that they perceive as “real”
  • Choice and flexibility…because they value the opportunity to showcase their particular talents in the best light

 

Don’t YOU feel the same way? I know I do…

So, here’s the first “test” for your institution:

  • Do you give every student a detailed “coversheet” at the beginning of each year / semester explaining how they will be assessed, the different types of assessment activities you will use, the “weights” of those different assessment events, any specific performance criteria that you will use to “grade” or “mark” student work – and also details of how (and when) you will provide them with feedback on their growth and development? 
  • Do you take the time to go over the “coversheet” outlining its key elements on Day ONE and provide students with the opportunity to ask questions and clarify any issues? 
  • Do you and your institution see this documentation and wider process as a hard and fast “contract” between you and your learners – a promise of the service you will provide to them?

 

It ain’t rocket-science…and it’s how we would expect to be treated ourselves.

 

As teachers or lecturers we often “joke” about how many of us have been asked the “mother” of all assessment questions from students:

“Um, do we have to know this? Will it be on the test?”

Wouldn’t it be great if we simply removed the “need” for this question to ever be asked – again…ever? The bottom line is that most students ask this question because our behaviours and actions have ensured they are “assessment outsiders”. Even worse, their experiences with us over the years have learned them that there will always be a “test”…

 

When will we learn…and help learn our students…that:

Assessment is something that teachers

DO WITH students BEFORE, DURING and AFTER learning

NOT, just:

Assessment is something that teachers

DO TO students AFTER learning

 

OK, while we are on this point – let’s try a second “test”:

  • Does your institution have a written statement of what it values in assessment, a set of principles that informs the development of assessment matrices, events and tasks and also guidelines that help teachers be as effective as they can be when they assess students?
  • Do all your teachers own these values and principles – know them (or at least are able to locate them), “live” them as part of the way they interact with students and contribute to upgrading or improving them over time?
  • Do students know them and can they make connections between the way their teachers do the business of assessment – and the way students are expected to do the business of learning?

As far as students are concerned, there is nothing more central to the learning experience than assessment…and they know that assessment can affect their whole future lives and careers.

Surely, we owe it to them to have all of these things in place…or, at least, have a thunk about them!

 

If, as we noted earlier, assessment is the “engine”:

The real problem here is, of course, that many institutions and educators still view assessment as “weighing and measuring”. Yes, it is a fact of life that some form of measurement needs to take place…how else would we know who to “graduate”?

But, assessment is not just about the “assessment-of-learning” (if we assume that learning can, in fact, be measured at all – indeed, it might be said that teachers can never truly understand what has been learned – only the students know this)! Assessment is also an integral component of LEARNing – and learning-orientated assessment processes can not only help us engage students in rich, authentic tasks, but also contribute (in an on-going manner) to the growth and development of all our learners.

I am, of course, talking about “assessment-for-learning” (the term coined by Caroline Gipps in 1994) and “assessment-as-learning” – the innovative approach adopted and developed by the faculty at Alverno College from 1973 onwards.

As Boud (1995) noted, “all assessments lead to some kind of student learning” – however, approaches that emphasise the “for” and “as” varieties of assessment connect the assessment of student learning to programme and institutional outcomes.

These approaches to assessing student learning do not only rely on testing a student’s “possession of knowledge” – they focus on the “use of knowledge” (in action) and what students can “do” with what they know and learn.

As Englemann (2007), a member of the Alverno philosophy faculty, notes:

As an Alverno faculty member, it is no longer possible to imagine teaching without assessing, because for us to teach is to assess, continuously, what our students are learning, and what they can do with what they know. We assess in order to improve the learning process, to give each student, and groups of students, guidance for their learning. At this point in the life of our curriculum and our academic culture, if our accrediting body were to say: “You no longer have to go to the trouble of assessing student learning,” we would still do it anyway.

 

This not “new” – as the original definition of assessment illustrates (OK – we can let the Greeks stay in the EU – just because of this – the Italians, too):

However, these models of “assessment-as-learning” and “assessment-for-learning” rely  heavily on student self-assessment and teaching students how to observe, analyse, and judge their own performance (on the basis of explicit and published criteria) – not just delivering “content” in pre-packed and off-the-shelf “course units” or using a textbook  (but actively developing meta-cognition in all students).

These approaches, it could be argued, require a fundamental “belief” that all students are capable of becoming adaptable, flexible, and independent in their learning and decision-making…they are BTW!

 

Gulpvalues, beliefs and principles again!

Double gulpnew roles, new skills and new abilities for educators!

One of the most important of these is the provision of quality diagnostic feedback from teachers (remember the “lubricant”) – feedback that allows students to further reflect and then determine how they can act to improve their own performance levels.

Not just a “grade”…

We all know:

Anyways, for many institutions and educators, this involves a serious re-thunk of what assessment means to those involved in the process, in addition to the teacher-student relationship in that process.

 

This was noted by teachers and researchers working on the REAP Project (in Scotland) – and involves the re-conceptualisation of assessment as a collaborative process where students are viewed as “partners in assessment” – they realised that we cannot continue to keep students on the “outside” anymore!

Like Alverno, the guys on the REAP Project have realised that taking this kind of LEARNing and ASSESSMENT PERSPECTIVE results in:

 

REAL LEARNing, REAL TEACHingMEANINGFUL Assessment!

 

OK – ready for the penultimate “test”? Go on – be a devil!

  • Does your assessment of students begin with educational values, reflect an understanding of learning as multi-dimensional and also respect the diverse talents and ways of learning of your students?
  • Do the assessment tasks and events you currently use communicate high expectations to your students, encourage contact between teachers and learners and develop reciprocity and collaboration between groups of students?
  • Do your assessment events emphasise “time-on-task” and going that “extra mile”, give credit for using both active learning techniques and creativity – and also support this with timely, on-going expert feedback?

 

A couple more come on, you can do it;

  • Do your assessment tasks include an element of self- and peer-assessment and offer possibilities for students to explore a wider range of “knowledge”: procedural (“knowing how”), schematic (“knowing why”), and strategic (“knowing when certain knowledge applies, where it applies, and how it applies”)?
  • Do you support this type of self-assessment by “teaching” students to understand and interpret the criteria by which they are assessed – and assess themselves?

 

If you got through those last ones without feeling the need to grab the closest bottle of aspirin (or Prozac), you are probably ready for the next “big” question (inspired by Race, 2002):

  • What is “broken” with assessment at your institution? How do you know? What are YOU going to do about it?

 

But, remember first have a thunk about:

  • What do you want to keep that you already have or do in allthingsassessment?
  • What do you want that you don’t already have or do in allthingsassessment?
  • What do you have or do now that you don’t want to keep in allthingsassessment?
  • What don’t you have or do that you don’t want in allthingsassessment?

 

YOUR students deserve nothing less…as “assessment insiders”!

 

Besides, you never know what may appear when the snow thaws….

Back to School – ASSESSMENT for DUMMIES…

In Assessment on 08/09/2011 at 7:17 am

I thought I’d continue with the “back-to-school” series with something on assessment – the guys at Wiley Publishing (or rather their “army of dummies’ lawyers”) have not caught up with me yet and the “other series” is also proving popular!

I’m also running this series for people new to allthingslearning – helping them get a better feel of the blog and all the links that can be used to explore the various posts). Guys – remember, all the “links” are in red (not blue) so just “click” to follow up on a post…

 

Assessmentthe area we love to hate!

As teachers, our lives in the world of education, whether we like it or not, are dominated by assessment. Indeed, it is this fact that has led commentators to suggest that “assessment is the sharp end of teaching and learning” (Race).

However, did you know:

 

There are three posts you might want to start with:

 

All good educators know that assessment is a critical component of any successful learning and teaching experience. They also recognise that assessment is one of the main parts of the “job” of teaching or supporting learning.

Students know MORE!

Students know that it is assessment that mediates all teaching and learning relationships. They know there is nothing more central to the learning experience than assessment.

They know assessment can affect their whole future careers and…lives!

For students, the type of assessment they know is “around the corner” determines:

  • what they define as important;
  • when they tune in and tune out during formal learning opportunities;
  • what they study in their own time and how much they engage with their subject matter;

and,

  • how they come to see themselves as students, graduates and people in later life

 

In a nutshell, and to borrow the words of John Cowen:

– and it determines not only what students learn but how they go about learning it.

 

If this is the case (and it is – trust me on this), maybe we should be doing “better”if we let the “examocracy” win, we only have ourselves to blame!

Is SPEAKING the “lost eagle” of ELT? [Part THREE]

In Classroom Teaching, ELT and ELL, Our Universities on 30/05/2011 at 11:39 am

I thought it was about time we got “serious” about this mini-series (and for me to tell you what actually happened to Marcus and Esca, too – see the bottom of the page if the movie is more important to you)…

A few weeks ago I met a young university student getting ready to graduate – she was so happy to have someone to “practice” her English with. I got chatting with her and asked how she had learned how to “speak” English.

She thought for a minute and said “by myself”.

Now, I knew she had been to a prestigious private high school and completed a year of university prep classes (at one of our best universities) – so I asked the “obvious” next question.

“They taught us nothing but grammar at school – and I did a year of the same in hazırlık without being asked to speak once – she had more than a slight “edge” in her voice!

She clearly “blamed” her teachers for doing so little to help her with what is the most “visible” of the language skills – and the one most widely regarded as the “step-child” of ELT here in Turkey.

 

As we noted in Part One of this mini-series, learning how to speak a foreign or second language is an amazingly complex task. English is tougher than most.

 

 

But, engineering is tough – as is rocket science and brain surgery – but that doesn’t mean we “avoid” them. Although the teaching and learning of “speaking” might be a serious challenge for institutions and teachers, it is certainly not brain-surgery – it is very doable!

 

Many institutions would also have us believe that it is the students themselves that are to blame for their own poor speaking skills; they do not want to speak, they are overly-concerned with not making mistakes (or “losing face” in front of their peers), or they just want to do grammar (to pass “the test”).

There may be more than a word of truth in these observations!

The problem is that these students have “learned” many of these behaviours and attitudes at school, from teachers…often unconsciously after years and years of exposure to the way we “do business” in our schools and universities. They have, if you will, been “created” in institutions that more often than not undervalue oral communication skills at the level of curriculum, classroom practice and assessment.

Remember also that teachers are “created” by their educational experiences, their training and the way their institutions “do business”! There’s many a teacher out there who knows what is “right” – but also blames the “system” for not being able to “do the right thing” by their learners.

 

Most teachers and institutions sing the praises of a skills-based, integrated language programme these days – focussed on listening, reading, writing and speaking. At a very basic level that suggests to me that 25% of our curriculum, classroom time and assessment matrices should be given over to the “step-child”.

OK, OK – we have to do some grammar and some vocabulary work, too – but again (very roughly, of course) that means speaking should get at least 16-17% of our “attention”.

Let’s be honest! How many teachers come close to these sorts of numbers in terms of how they use their GLHs?

Digging deeper! How many pages of the typical language learning curriculum are given over to learning outcomes that “balance” real-life interactions, transactions and performances?

One more! How many institutions do not even “assess” speaking in their proficiency exams (let alone use systematic processes to promote classroom-based assessment and learner self-assessment)?

 

It might appear, from the simple logic I used above, that the “solution” is easy – self-evident, even.

Do more “speaking”!

The problem is that improving the speaking skills of our learners is not just a “numbers game” – before we tackle the “class-scape” or “exam-scape”, we have to look at the “mindscape” of learners, teachers and institutions.

In the past (and still today), we have prioritised “grammar” and the assumption here is clearly that if we give them enough “grammar input”, they’ll be able to “speak” at some time in the “future”.

The problem is – this does NOT work and we have known this for years!

What did Einstein say about “doing the same thing over and over again and expecting different results” – insanity, pure insanity!

 

The student I chatted to said she had gone a whole yearwithout any serious instruction on how to speak, without any meaningful classroom speaking activities and (I would hazard a guess) without any relevant assessment or feedback on her spoken communication skills.

But, she “passed” hazırlık…and, I bet she was not alone that year!

 

If we are to have any success in “teaching” speaking (or facilitating its learning):

  • We have to have a decent “roadmap” for speaking
  • We have start “walking-our-talk”
  • We have to know how well we doing (and let learners know about this, too)

That having been said – what institutions need to do to redress this critical problem is actually quite simple:

  • STEP ONE – fix the curriculum!
  • STEP TWO – support your teachers in bringing more speaking into the classroom!
  • STEP THREE – assess speaking (and give learners decent feedback)!

I think we have the basis for the follow-up trilogy – watch this space!

 

BTW, and as promised…

Marcus (after waking up to what it’s like being a “slave” for a few days) discovered that his father had fought bravely to protect the eagle – but was murdered by the Seal People. Esca found a way to recover the golden standard – and both he and Marcus did a “runner”.

The problem was that the Seal People were all bloody marathon runners – and lasted longer than the horses Marcus and Esca used to escape. Exhausted, trapped in a dead-end and destined to re-live the fate of his daddy, Marcus decided to make his last stand but granted young Esca his “freedom”. Esca did the wise thing and left his ex-master to his fate…only to return with the sad, rag-tag remains of the Ninth Legion (the buggars who’d all done runners and left Marcus’ daddy to be sliced and diced by the Seal People). Determined to earn back their own honour and help Marcus prove his daddy was a true Roman-among-Romans…all hell broke loose in the shallow waters of a Scottish river…

You know the rest…

Çay ve Simit – and a bit of education…

In Educational Leadership, Guest BLOGGERS, Our Universities, Uncategorized on 07/05/2011 at 11:02 am

A few months ago I did a post on the inspirational videos that can be found on the TED (Ideas Worth Sharing) site. I was inspired to do this post by Aybike Oğuz, Head of Curriculum at SELI (Özyeğin University, İstanbul) – after she told me how she had “jumped on a plane to Bali” to check out John Hardy’s “Green School” (as soon as she saw the TED video from John).

Since then Aybike and I have been looking at doing some form of “blog project” together.

Çay ve Simit – and a bit of education… is the project we came up with.

The first of a series of interviews looking at educational issues across Turkey – with educators from Turkey.

Aybike, you have been involved in setting up one of the newer university-level language centres in Turkey. Tell us about a few of the major challenges you and the guys at Özyeğin faced – and what you did to overcome these. What did you learn about “what matters” when setting up a new ”language centre”?


It’s quite new, actually, but this is our third year now. I prefer to use the term ‘preparatory school’ instead of ‘language center’ because teaching English is only one of the things we do. I see preparatory schools as the foundation year of the university education in Turkey. Our aim is to prepare our students for what they will experience in their departments. This includes teaching English, academic study, critical thinking, research and independent learning skills.

There were two things that I found challenging about my job, especially when we first started. The first was to contribute to the establishment of a shared vision and a shared organizational culture for the prep school among the Board of Trustees, Senate, Rector, prep faculty, students, and parents. And the second was creating growth opportunities for the faculty and students. We had to do this under serious time constraints. The activities for the foundation of Özyeğin University were initiated by the Hüsnü M. Özyeğin Foundation in the autumn of 2005. However, the prep school faculty started working with three people in February 2008 and the rest of the faculty, including myself, joined the team in August 2008 and we welcomed our first group of students in September. So when all of us came together for the first time in August, all the deadlines were basically ‘yesterday’.

Our biggest advantage was the quality of the prep school faculty. Everyone in the team was very knowledgeable, experienced and passionate about what they do and they left the universities they had worked at because they believed in ‘Özyeğin vision’; I was the youngest in the group actually, so we had the ‘know-how’ but the real challenge was to define and implement what we believed mattered the most.

This required a lot of thinking on ‘creating communication opportunities’. We discussed our vision with the Board of Trustees and the Senate. We had bi-weekly update meetings with the Rector, Erhan Hoca. We had whole group and small group faculty meetings on a regular basis. We also held ‘class representative meetings’ with the students every four weeks to listen to their side of the story and to involve them in the decision-making process. Erhan Hoca met the faculty and the students time to time.

At some point, we realized that even these channels were not enough because there was too much going on at the school so we started publishing a weekly news letter called ‘SELI in a Nutshell’ (SELI: School of English Language Instruction), which was emailed to the prep faculty and Erhan Hoca every week. Student and parent orientation programs also have been useful to communicate our vision to the students and the parents. It was a good start for expectation management. We spent a lot of time and energy on involving as many stakeholders as possible in the design and the decision making process of the school and to keep everyone updated on the issues discussed in different forums.

There’s a lot of talk about the quality of language teaching across schools, colleges and universities across Turkey – and not much of it is positive. What do you think are some of the key directions that we need to take or the initiatives we need to kick start, to really make a difference to how our students learn English?

I don’t think the problem is specifically about the quality of language teaching. It’s about the quality of teaching in general.

My first suggestion would be to ask the students what’s going wrong. Nobody seems to care about what they think. We need student leaders to gain an insight into the root causes of the problems and what can be done to solve them. Student representation is very limited in decision making in our education system. Each school and university needs to have student representatives in management and student representatives should work as actively as any other management member. Now, I hear some people say “Yeah, right! Get real!” Well, actually the reality is some schools in Turkey have been doing this for centuries. The best example that comes to mind is Galatsaray Lisesi.

My second suggestion is to have confidence in the teaching talent already available here in Turkey.  Once you start listening to that talent, you will see that we have the teaching talent and also the expertise to create that talent. Last week, I was in IATEFL. Just out of curiosity, I checked the conference program to see how many people from Turkey and Cyprus presented in the conference. I don’t know the official numbers but I counted 34 workshops from Turkish and Cypriot universities and three K12 schools. And we were not represented only by Istanbul, Ankara, and İzmir but also by Zonguldak, Bursa, and Denizli. And there was also one workshop by the Ministry of Education. Except for TESOL, this is the biggest TEFL conference in the world. What does that tell you?

What’s even more promising is that we are not taking part only in one-shot events. We are also taking a leading role in Special Interest Groups, which requires long-term commitment at an international level. Just a few examples, Zeynep Ürkün from Sabancı University in Testing SIG, Ayşen Güven from Bilkent University in EAP SIG, Birsen Tütüniş from Istanbul Aydın University in Teacher Training SIG. And is it just the universities? No! I met Özge Karaoğlu and Esra Girgin Akışkalı from Şişli Terakki (a private K12 school in Istanbul). Meeting them and talking about the kind of work they do at K12 level was inspirational. It gave me a lot to think about primary and secondary education. And it raised my awareness of the importance of understanding K12 education to understand university education.

Can you imagine the impact of creating a forum which brings all of these people together? Ask them what’s wrong about English teaching. They are the experts. Get them to work together. We don’t need to wait for international conferences. We can easily create such forums ourselves. A good example is the forum Sabancı University School of Languages initiated two years ago. The FOCI (Forum on Curricular Issues) events bring together curriculum coordinators in prep schools in Turkey and Cyprus twice a year to discuss the common challenges we face at prep schools in terms of curriculum design and delivery. I have learnt so much in these meetings about the English teaching at tertiary level in Turkey. Maybe a similar initiative needs to start between K12 schools and universities. Can you imagine the impact of all of that expertise working together?

You’re Head of Curriculum at Özyeğin. Do you think our schools and universities have the “right take” on what curriculum and assessment is all about – should be about? How have you tried to “renew” the Özyeğin curriculum and assessment model?

Ours is quite new actually as this is our third year. Based on the feedback we receive from our colleagues and students throughout the year, we make revisions every summer but currently we are still working around the curriculum model that we designed in 2008.

I think before asking ourselves if we have the “right take” on curriculum and assessment, we need to ask if we have the “right take” on university education and the role of prep schools in universities because curriculum and assessment, to me, are the manifestations of our educational philosophy and organizational culture. So we are all tested in walking the talk through the curriculum and assessment models we design in our programs. And naturally prep schools are manifestations of the universities they are a part of. I will take the liberty of adapting a Turkish saying. ‘Bana hazırlık okulunu söyle, sana kim olduğunu söyleyeyim.’

Unfortunately, the power and the role of prep schools in universities in Turkey have been undermined for a very long time. Very recently, we see some examples of universities which understand that ‘it’ all begins at prep schools. So it is, at best, naive to expect to have “super” graduates if universities do not invest in prep schools, especially considering the fact that the majority of high school graduates in Turkey start their university education life with little English. This means that most of the student population in Turkish Higher Education spend at least one year in prep schools when they start studying at university. One of the things we have started doing at Özyeğin is delivering a course called ‘Üniversiteye Giriş Dersi’ (Introduction to University Life) for prep school students. What you might find interesting is that Erhan Hoca, the Rector, has designed and has been delivering this course. You can think of it as a-year long orientation program where the students are exposed to different topics like plagiarism, discrimination, team work, CV building, entrepreneurship, creative thinking, student exchange and internship programs, research and presentation skills, etc. We also invite guest speakers to share their success stories with us, one of the guest speakers being Hüsnü Özyeğin.

As for the English component of our prep school curriculum, we decided on a couple of things that we all agreed on as of day one. These were putting ‘learning’ and ‘investing on student competence’ at the core of all the activities, using educational technology resources intensively to extend the limits of learning from ‘classroom’ to ‘student life’, assessing student competence both through standardized tests and projects which give students multiple opportunities to prove themselves.

To achieve this, one of the first things we did was to write a five-level, integrated-skills syllabus, which describes our overall program and level objectives. The empowering power of this ‘50-page document’ is that it gives the stake holders (the faculty and the students) a clear understanding of where we want to go in the program overall and at the end of each level. It also gives the faculty the space they need to use their individual expertise to design courses based on “their’ students’” learning preferences and needs. From the students’ perspective, having such a document gives the students a sense of direction and a framework to guide them in managing their own learning. Finding the middle ground between ‘standardization’ and ‘customization’ is always an issue in prep programs (unlike department courses) as you know so what we aim to do is to define where want to go and respect individual differences in choosing the path.

One of the challenges we all have is that our students study English in a non-English speaking environment. Therefore, in order to maximize their exposure to the target language, we use Moodle to continue interacting with our students when they leave the classroom. This helps us provide extra resources for practice, carry out online class discussions, give online feedback on student writing and speaking (via video journals uploaded into Moodle) skills. In order to cater for all of these needs, we work very closely with the Information Technology and Library Services teams in our university. For example, at the beginning of each course, our library team gives training sessions to our students on the online resources available in our library, which students need to do the tasks we set. To achieve this, we had a series of meetings with the library staff to present our curriculum to them so that they know what to focus on in their training sessions.

As for testing, in addition to mid-terms and finals, our students also do projects which involve a series of integrated tasks around one theme. These tasks require them to do very basic research on a problem, analyze the findings, and to synthesize them to propose solutions. In the project component, the students are given multiple opportunities, through feedback processes, to work on and improve the outcome.

Another focus we have is creating opportunities for peer-learning and support. We have a student buddy system called SBSS (SELI Buddy Support System), which we are very proud of because it is a student initiative. One day, when I was working in my office, one of our students, Gökhan, (the followers of this blog might remember his name), came to my office and shared this idea with me. And now that idea is a fully functioning student buddy support system designed, introduced, and managed by the Student Union with Gökhan’s leadership. The idea is that the students who finish our prep program and move on to study in their departments help prep students in their studies and adaptation to life at Özyeğin. I think this is what happens when you involve students in the decision making of the school. When students are invited to do things together, they do feel the responsibility of contributing to the learning environment.

OK, let’s come back to teachers. You’ve heard about the MEB initiative to bring in 40,000 native speakers to teach in our schools. With all the talk about global varieties of English (or “Englishes”), do you think this is the best strategy? What else could, should we be doing?

Well, I think I kind of answered this question already. I can’t help but ask “why”. What is this decision based on? On what needs? One of the first things that can be done before implementing such projects is to do research to identify the needs. If research has been done, then the findings need to be shared with the public. The first example that comes to my mind on this the kind of work is Bill and Melinda Gates Foundation. Do research, share the findings with the public, based on the finding do projects, evaluate and improve them.

One of the MEB projects that I appreciate is the intensive training programs they organize where they bring together teacher trainers at universities with MEB teachers. Some of our colleagues at Özyeğin University have given sessions in these training programs and they were so energized by the motivation and commitment of MEB teachers. They advised us all to experience this great opportunity. We will continue working with MEB in similar projects. Maybe one of the things we can work on together is offering a follow up program for these training sessions.

I recently did a session with a group of young, Turkish “trainers-in-training” and we were discussing “teacher motivation” (because in all our discussions on motivation we tend to focus on the “student variety”). Like many other countries, Turkey is facing challenges with teacher motivation. What do you think our priorities should be? What do you do differently at Özyeğin to deal with these challenges?

We need to invest in managing human resource. The whole Human Resource Management cycle basically. What are recruitment, orientation, retention, mentoring and coaching, and performance appraisal and review policies?  We need to look at each of these stages carefully. How do we recruit teachers? Do we have orientation programs to support the faculty in their adaptation to the organization? Do we support them in their professional development and create career opportunities? Do we acknowledge and reward their performance? And when they decide to leave the institution, do we respect their decision, acknowledge and praise their contribution and accept the fact that people move on?

What we do at Özyeğin is to follow an individual approach in supporting our colleagues in personal and professional development. Each of us is given different opportunities based on our interests and needs. We are very lucky in this sense because the university has the budget to support us financially to attend and present in national and international conferences and to take courses. We have presented papers and workshops in Greece, England, Holland, Malaysia, UAE, Cambodia and some other places I can’t think of right now.  We have also started benefiting from Erasmus teaching exchange opportunities. One of our colleagues gave a two-week teacher training course at a college in Holland last summer. Another colleague is going to the same college this summer. Some of us have taken courses on educational technology, teaching Academic English, teacher training, etc.

We also share our experiences with the other universities in Istanbul. For example, one of our colleagues was invited to Doğuş University prep school to give a presentation on how we use Moodle in our courses. Next week, I’ll meet a group of people from Kadir Has University prep school to discuss the curriculum changes they are planning to make in their program. Two of our colleagues were invited to Fatih University to present how we adapt a course book both universities use in their programs. And what you might find surprising is that one of our colleagues who presented this workshop is in her second year in her career. As I said before, we also work with MEB in the teacher training programs they offer. A group of us will visit Sabancı University soon to exchange ideas on curriculum and assessment issues in our program.

We also take courses with our students in the departments. For example, currently I’m taking a BA level course on Modern Middle East from Cengiz Çandar with two of my colleagues at the prep school. Some other colleagues are taking Italian courses with the freshman students. It’s a great experience. I mean experiencing what we are preparing our students for first hand.

Our sports center also works for us very actively. Another course I took two years ago was Latin Dances. You can also see some of us at the Sports Center doing ‘Zumba’! As a university (academic and admin staff and the students), we also attend Eurasia Marathon every year. You can see us anywhere, doing anything. We are a crazy bunch…

Another “hot topic” across the country is that of “distance language learning” and even “distance teacher education”. What are your views on this – are we heading in the right direction or perhaps another “dead end”?

I’m not sure if we are heading in the right direction but I’m sure that it’s definitely not a dead end.  There is a huge market out there promoting this idea and offering different programs for different purposes. What we need to be careful about is the content of these programs and what they offer to the participants. One of our colleagues is doing an online EAP course at Nottingham University, for example, and she is very happy with the quality of the education. I believe it all depends on what the design is.

This area touches on “teacher learning” – a complex area. If you could wave “a magic wand”, what type of teacher learning programmes would you recommend? What about trainee teachers on undergraduate programmes – what needs to change to improve the quality of teachers our education faculties creating?

I can’t suggest a learning program right now. It would take another interview to do that but I can share my latest self-reflection on my own learning for whatever it’s worth.

I learn from people (professors, teachers, students, parents, friends, managers, taxi drivers, curriculum designers, doctors, test writers, etc.)  who inspire me. Obviously, their concern is not to inspire me. That’s who they are and their ‘vibe’, in some cases, is independent of their educational background. I want to have people around me who I respect and look up to. And that is what I hope to be for my students.

Therefore, I think our first duty, as teachers, is to be keen learners. And my first suggestion would be reading outside the field. Deniz (Kurtoğlu Eken) Hoca has been highlighting this issue in her training sessions and talks. Working at Özyeğin University has raised my awareness a lot in this sense. When everybody talks about economy, finance, and entrepreneurship, you feel the need to read about it. We need to read about economy, history, psychology, philosophy, whatever we find interesting. I totally agree on the importance of the environment in growth but I believe in the power of the individual more than anything else. We are responsible from creating ourselves learning opportunities.

Reading outside the field is one way of doing it. Another is going to the museums, concerts, exhibitions, spending time with people from different walks of life, dancing, doing yoga, jogging, whatever we find interesting and is convenient for us.   The best way to inspire others is to inspire ourselves and to me learning is all about inspiration.

Aybike, thank you…more Çay…Simit?

Seriously…what is CURRICULUM…Seriously? [Part THREE of ???]

In Curriculum, Our Schools, Our Universities on 21/03/2011 at 4:12 am


Nature is one. It is not divided into physics, chemistry, and quantum mechanics.

Albert Szent-Gyorgi

In the very first part of this mini-series (and, I’m rapidly discovering it will not quite “fit” into a neat, little “trilogy”) I mentioned that the starting point of a “curriculum rEvolution” was the question:

What are your beliefs about what type of “curriculum model” you (and your students) need?

However, and I as I tried to point out, many institutions are just not used to this type of approach – an approach that makes “explicit” those underlying assumptions that “drive” the way we “do business”.

What we have to remember, however, is that institutions are made up of people. I’m guessing that you are “people” (institutions do not read blogs) so I’d like to offer you a version of those mini-quizzes we find in magazinesto see if you are ready for a “curriculum rEvolution”.

 

Let’s start with a couple of “YES” or “NO” questions remember, you have to say “YES” to make it worth reading the rest of the post:

Q1. Do you (really) believe that every single student can learn or at least grow as the result of further educational and learning opportunities?

Q2. Do you (in your heart of hearts) believe that schools, colleges and universities should be organised around the notion of student learning?

Q3. Do you (and this is the “killer” question) believe that a curriculum should be more than a “teaching plan” – that we should conceptualise of curriculum as the expression of “educational beliefs in practice” and that we need to think of curriculum in terms of the “whole educative process” (rather than simply “content”)?

If you answered “YES” to two out of three of these questions, you should probably read on – if not, read on anyway (or take a look at Part Two)!

Answering “YES” to questions like these suggests (to me, at least) that you see your role (as a teacher or educator) as involving the creation of meaningful change in all your students and that you would perhaps want to see “curriculum” as more of a coherent roadmap for supporting the holistic development of all your learners.

OK – it might just mean that you did what I asked you to do and answered “YES” – but, I am (forever) an optimistic kind of guy.

 

Let’s try a couple of others:

Q4. Do you believe learning is more than “knowing” – it is about doing something with what we know and our ability to continue to learn and grow after “formal education” is over?

Q5. Do you believe learning is a “complex process” that involves the whole person in a c0-constructed, situated (context-specific) and collaborative exercise of sense-making?

Q6. Do you believe learners develop attitudes, skills and knowledge best when they are connected to and transformed by their learning – in addition to taking responsibility for that learning?

I’m hoping we are on a “roll” here – and again you said “YES” to at least two of those.

 

Let’s push the envelope a little more:

Q7. Would you agree with the statement “real education involves making a sustained, substantial and positive impact on how learners think, act and feel”?

Q8. Would you agree with the statement “real learning involves making a sustained, substantial and positive impact on what learners can do with what they know and learn”?

Q9. Would you agree with the statement “real curriculum renewal involves moving to the creation of a living curriculum that reflects the development and growth of the whole student, in addition to developing assessment processes that can evidence the quality of learning that has taken place and what student can do with what they know and learn”?

I never said these questions would not give you a headache…and, I know it might have been better if we had done some form of likert scale for this last set of questions. You know:

5 – Strongly Agree

4 – Agree

3 – Neutral (I never really “got” why we use this one – I have not met many educators are “neutral” about their students. If I have met teachers like this, I have usually advised them to simply “get out of Dodge”).

2 – Disagree

1 – Strongly Disagree

Then, asked you to add up all your points and compare your totals to a number of descriptions that tell you where you are on the “learning teacher” to “teaching teacher” continuum…but that would have been too much hard work for a lowly blogger.

Besides, Oprah and Cosmopolitan have done this to death…

You get the idea.

 

Question nine is a nice one. It introduces the idea of a “living curriculum”.

A living curriculum is not chopped up into discrete courses, decontextualised skills and “chunks” of knowledge; rather, it facilitates the holistic performance of meaningful, complex tasks in increasingly challenging environments.

A living curriculum promotes collaborative co-creation of knowledge, understandings and abilities by learners and teachers (and teams of teachers from different disciplines) and in doing so creates a sense of efficacy and confidence in learners.

Learning opportunities, challenge or problem-based projects, materials and content are structured so that students gradually regulate their own learning over their whole “learning career” and so that learning is always meaningful and makes sense.

These goals are promoted in a variety of ways through living curricula. For example, a living curriculum encourages students to clarify their purposes in performing a task, to assess what they already know, and to predict what is to be learned on various learning opportunities.

A living curriculum helps learners highlight what is most important and thereby fosters feelings of control over subject matter. It explores students’ attitudes about themselves as learners and about learning in topic and discipline areas. It provides opportunities for students to assess difficulties they have in learning and consider strategies they could use to overcome learning difficulties. It stresses continuing to work in the face of ambiguity, solving problems despite unexpected difficulties, and looking at problems as challenges to learn more and better.

By being engaged in curriculum in this manner, students come to see themselves as successful, capable learners.

And, then there are curricular that resemble the Dead Sea Scrolls…hidden away for years, collecting dust, never questioned…well, not by many.

 

OK, back to some questions:

Q10. Choose the correct answer:

a)    A school, college or university is an institution that should exist to provide instruction.

b)    A school, college or university college is an institution that should exist to produce learning.

 

Q11. Choose the correct answer

a)    Most of our schools, colleges and universities exist to provide instruction.

b)    Most of our schools, colleges and universities exist to produce learning.

 

Do I need to say more?…Perhaps, a couple of words.

It should be clear that the view of curricular I have in mind represents a significant change for many educators (especially academics in higher education). The type of  paradigm shift required of educators and academics is quite significant and we cannot expect this to occur overnight.

Many may, in fact, wish to challenge whether we need such a shift and could suggest that the assumptions a teacher or faculty member holds about curriculum constitute a grounded source of principled ideas about how to promote learning in a given discipline.

In truth, many of these objections are often “stalling tactics” – designed to minimise the disruption of lesson plans, materials and textbooks – prepared years ago (IMHO)…but let’s give people the benefit of the doubt.

While this is often true, the evidence base for “newer thinking” on teaching and learning is grounded on a belief in a “scholarship of connection and engagement” with all those in our wider learning communities – these communities both “start” and “finish” with learners.

As with any respectable scholarship, most professional educators today believe that the assumptions that underpin action and decision-making should be continually challenged and that it is this very process that contributes to the learning of educators and academics – and, more importantly, students.

However, it has to start with those educators and academics “challenging” themselves – then, perhaps challenging the way their institutions “do business”.

But, more on that…tomorrow.


ASSESSING How We ASSESS Learners (Part Üç)

In Assessment, Our Schools, Our Universities on 11/03/2011 at 8:00 am


The best years of your life are the ones in which you decide your problems are your own. You do not blame them on your mother, the ecology, or the president. You realize that you control your own destiny. 

Albert Ellis

Ellis had it right – has it right.

Pointing the finger, looking for scapegoats or burying our heads in the sandare all just, plain dumb!

And, just “so… 2oth Century, darling“!

In Part Two of this little “dizi”, I tried to highlight some of the challenges facing learners, educators and administrators with the way we currently “do assessment”. I noted that all these challenges have a kind of “symbiotic relationship” and that it’s difficult to tease them apart.

The bottom line is that all of them “belong” to us all.

Those of you that want to get that “cushy, civil servant’s job” in the Ministry of Passing-The-Buck should stop reading now – and go to another blog!

As I said, in a much earlier post:

  • Change involves going to some of our “darker places” (as Luke Skywalker had to), ending the blame game – and it also takes more than an ounce of courage

  • Courage is helped along best through truth, acceptance, forgiveness and connecting with others on things that “matter”

 

So, what is the “püf noktası” in all of these discussions about assessment in education?

There are three:

  • All of the problems we examined come down to the fact that we still haven’t quite “got” the fact that assessment is something that teachers DO WITH students BEFORE, DURING and AFTER learning not simply something we DO TO students AFTER learning
  • There has been a “surge of interest” in the assessment of student learning and its links to learning across education over the past ten years. Despite this interest, assessment across our schools, colleges and universities remains under-conceptualised and very little has changed in how we “do” assessment
  • We haven’t quite got to the point of “walking-our-talk” and much of this comes down to the fact that we have avoided meaningful discussion of the assumptions we hold about assessment – we haven’t gone to that dark place enough and we haven’t connected enough with “what really matters” in assessment

 

Let’s take that last one first.

There are more student complaints and litigation about unfair assessment than in any other area in university life – and if primary kids knew how to save enough of their pocket money to hire lawyers, we’d be in trouble there, too.

Legal action in “all things assessment” is telling us more than we need to “watch our backs”, it lets us know that we are stillin danger of using assessment to stop learning instead of to start learning” (Race).

Let’s take the example of an academic trying to get something published (and, yes I know I have been critical of this in the past – but many research papers help us learn, grow and get off the planet faster).

When researchers and academics submit articles for consideration by refereed journals, they do not expect them to be rejected out of hand without any comments or feedback.  What they expect is “constructive feedback” about how the article could be improved, suggestions related to what may be missing and possibly some encouragement to re-submit a revised version.

In short, they expect to be treated as “insiders”.

The development of “assessment thinking” across education has been part of a wider paradigm shift in how we approach learning and teaching. The research of the past 25 years into education and learning has seen the learner become of central importance.

The motives, activities and feelings of the learner (regardless of “age”) have become much more important than those of the teacher – and also content. Teaching itself has become redefined as the “facilitation of learning” and content has been redefined by reference to learning outcomes (rather than teaching inputs).

Both students and educators are evolving as “insiders” in the process of teaching and learning – and needs to be also true for assessment.

If we see students as assessment “outsiders”, we miss many valuable opportunities to encourage reflection and support them in evaluating their own progress and to obtain feedback on how to improve our own teaching.

It’s that simple!

Educators also need to recognise that assessment is not only central to learning and teaching, but that it is something that comes naturally to students. All learners, to various degrees, are all involved in regulating their own learning through processes of self-assessment and self-monitoring (Nicol, 2007).

 

What about learning?

We all know that surface or superficial learning is a key challenge for all those involved in education – and other educational stakeholders who “hire” the talent educational institutions “create”. We also know that this type of superficial learning is encouraged by:

  • Lack of choice of subjects and methods of study
  • An excessive amount of material in the curriculum
  • Lack of opportunity to pursue subjects in depth
  • Relatively high class-contact hours
  • Threatening and anxiety-provoking assessment systems
  • Assessment methods that emphasise recall or application of trivial knowledge
  • Poor or absent feedback on progress

We all know this, right?

 

OK – who shall we blame? Who should we tar n’ feather this time?

Race (2002) gives us a list of “targets” – and reminds us “we can only ever really solve problems which we own. But the assessment problem is so widely owned. It’s dangerously easy to feel there’s just nothing that we can do about it. It’s easy enough to identify scapegoats, including:”

  • Professional bodies, in whose name we feel we need to stick to the status quo;
  • Pre-university education systems, which cast the die and train pupils into particular expectations of learning and assessment;
  • Institutional, faculty and departmental assessment regulations, which limit our room for manoeuvre;
  • Teaching and learning strategies, which are so all-encompassing that we can’t suspend belief and start afresh again;
  • Heads of department or school, who are often seen (sometimes seen wrongly) to be content with the status quo;
  • External examiners who would have to be convinced when radical changes may need to be made;
  • Students themselves who could or would complain about rapid changes to the level of the playing field or the position of the goalposts (even if the whole is enveloped in thick fog at present);
  • The world outside academe, where there’s a view about what a graduate should be, and so on;
  • Journalists, broadcasters and editors who would give us a hard time if anything were to be found wrong in the way we did the job we are paid to do;
  • Politicians and policy-makers who got to where they are by succeeding in the system of assessment we already have, and dare not admit that it might have been flawed;
  • Parents, employers, taxpayers and others who foot the bill for higher education.

Thank God – for scapegoats!

 

It is often said that, “‘if you want to change student learning, then change the method of assessment” (Brown, Bull and Pendlebury, 1997).

Easier said than done, Tony!

It’s true – because the core purposes of assessment do involve tensions and multiple demands, reform is difficult to achieve (Carless, 2007) – difficult, but not impossible.

The starting point is “us” – as students, as educators, as institutions – not our mothers!

 

As Ghandi said:

“Be the change you want to see in the world.”

 

I’d like to thank everybody who made winning this Emmy possible – I couldn’t have done it without you all! Clem & Alison – for showing me the value-added a teacher can have on a working-class “kid” growing up on a council estate in Manchester. My family – for helping to shape the man I am today, even if they sometimes never totally got it. My assessment “gurus” and “mentors” – Phil Race, Sally Brown, David Carless, Peter Knight, Caroline Gipps – plus, the visionaries and risk-takers at Alverno (for “opening my eyes”), my plain-speaking friends at the Centre for the Study of Higher Education in Melbourne and the guys on the REAP Project for bringing “the good fight” into the 21st Century.


ASSESSING How We ASSESS Learners (Part Two)

In Assessment, Our Schools, Our Universities on 11/03/2011 at 5:17 am
Our lives in the world of education, whether we like it or not, are dominated by assessment. Indeed, it is this fact that has led commentators to suggest that “assessment is the sharp end of teaching and learning” (Race).

In Part One of this little “series”, I tried to emphasise the importance of principles or “guiding lights” for how we “run the business” of assessment in schools, colleges and universities. I genuinely believe that this type of approach is the best way to really make a difference – firstly, by helping us review our assumptions about assessment and then by allowing us to take a “perspective” on assessment. Taking a perspective is very different to simply “having a perspective” (bit like having an opinion but not doing much “with” the opinions we have) – I learned this from Alverno.

 

However, there is another way – a way that many people find useful. Less “philosophical” – as if that should be a problem for all us “thinking doers” in education.

This second approach is grounded on “problem-solving”.

The problem-solving approach, however, relies on the understandings we have about the nature of the problem itself – as my dad used to tell me, “Lad, you can’t fix it, till you know what’s up”!

The factors affecting the quality of assessment practices are woven so tightly together that they must first be teased apart before an effective strategy can be developed for learners, educators and staff.

So, what is the problem? Depends who you speak to…



What do STUDENTS say?

  • Assessment “overload”
  • Assessment is not related to what we do in class
  • Insufficient time to do assignments, projects and study
  • Too many assignments with the same deadline
  • Not enough information on criteria or marking schemes – we do not know what teachers want
  • Inadequate or superficial feedback
  • Teacher and institutional “obsession” with what’s on the test
  • Different teachers have different “expectations”
  • Too little choice and flexibility
  • Assessments are not authentic or based on real-life
  • Grades tell us very little – but we like it when we get high ones (!)
  • Some teachers just enjoy making things “hard”
  • Testing does not help us “learn”
  • Teachers do not always follow up on the learning from assessments

What do TEACHERS say?

  • Assessment “overload”
  • Marking “overload”
  • Institutional assessment is not related to what “we do in class”
  • Difficulty of assessing independent critical thinking, creativity, academic or life-skills as opposed to “subject content”
  • Insistence on reliability has resulted in curriculum areas that are inadequately represented in examinations and tests
  • Some students do very well in tests, others do better in other forms of assessment
  • Student “obsession” with what’s on the test

What do ADMINISTRATORS and the EXPERTS say?

  • Assessment matrixes are rarely, if ever, based on the principles that follow a specific vision for education (AAHE, 2006)
  • “Overuse” of certain modes of assessment (e.g. written tests, essays) – 90 percent of typical university degrees depend on unseen, time-constrained written examinations, and [instructor]-marked essays and/or reports (Race, 2002)
  • The vast majority of assessment tools in education still focus on declarative knowledge (“knowing that”), frequently overlook procedural (“knowing how”), schematic (“knowing why”), and strategic knowledge (“knowing when certain knowledge applies, where it applies, and how it applies”). Even less attention is paid to personal, social, and civic abilities (Shavelson and Huang, 2003)
  • There remain many practical issues related to validity, reliability, transparency and “fitness-for-purpose”
  • Many quality issues in assessment come down to a “poverty of practice” among teaching communities (Black and Wiliam, 1998)
  • Many institutions have been charged with “abiding amateurishness” (Elton and Johnson, 2002) in the way they put together assessment tools
  • Many institutions still use “folkloric systems of equivalence” (e.g. a three-hour paper is “equivalent” to a 3000-word assignment)
  • Assessment tasks often distribute effort and “assessment burden” unevenly across a course (Gibbs & Simpson, 2004)
  • Examinations and terminal assignments are frequently critiqued for encouraging memorisation or surface approaches to learning (Ramsden, 2003)
  • Many educators (esp. in higher education) have never received formal training in curriculum and assessment design
  • The majority of traditional assessment tools rarely resemble the “tests” students will face after they graduate and even fewer help prepare them for a “career” of lifelong learning (Lombardi, 2008)
  • Student achievement is still frequently only measured “within courses” – with limited attention to cumulative learning outcomes (AAHE, 2006)
  • Change in the area of assessment practice has been surprisingly slow – many of the innovative approaches, or at least critiques challenging traditionalism posed in the late 1960s, have failed to bear fruit in any meaningful way across education.

 

That’s a pretty impressive set of “challenges” (don’t you just love that word – makes it all seem so harmless)!

For a change, I’m not going to say much – I think perhaps David Boud said it best:

Students can escape bad teaching; they can’t escape bad assessment.

Over to you…problem-solvers!

 

P.S: I actually missed “Fatmagülün Suçu Ne” to write this one – but then I guess one “dizi” is pretty much the same as another!

ASSESSING How We ASSESS Learners (Part One)

In Assessment, Our Schools, Our Universities on 10/03/2011 at 8:25 am


“…we have to see that learning – deep learning, learning that matters, learning that lasts – is not something that teachers do to students or even that students do for themselves. Rather it is the product of action in a context shaped by goals, performance, feedback, time horizon, and community – of all of the principles that define the cognitive economy, acting to create an environment that empowers and engages students.

Tagg, 2003

One of the things I am often asked on my travels around Turkey is:

  • What can we do to improve the way we assess student learning?

I love this question – it shows an “improvement-orientated ethos” and a recognition of just how important assessment is to “learning”. After all, and as Lauren Resnik so powerfully put it, “What you assess is what you get; if you don’t test it you won’t get it.

BUT, I usually answer the question with a question (yes, my “big little girl” always hated this habit of her daddy):

  • What principles are your assessment systems and tools built on?

 

I often get a lot of “blank” looks.

Many respond by talking about reliability, validity or “professional levels of quality”.

To be sure, these are important notions – but my question is digging a little deeper.

As teachers, educators and assessment experts we all discuss “quality issues” in assessment but we forget one thing:

The REAL assessment “experts” are STUDENTS!

All good educators know that assessment is a critical component of any successful learning and teaching experience. They also recognise that assessment is one of the main parts of the “job” of teaching or supporting learning.

Students know MORE!

Students know that it is assessment that mediates all teaching and learning relationships. They know there is nothing more central to the learning experience than assessment.

They know assessment can affect their whole future careers and…lives!

For students, the type of assessment they know is “around the corner” determines:

  • what they define as important;
  • when they tune in and tune out during formal learning opportunities;
  • what they study in their own time and how much they engage with their subject matter;

and,

  • how they come to see themselves as students, graduates and people in later life

In a nutshell, and to borrow the words of John Cowen, “assessment is the engine which drives student learning” – and it determines not only what students learn but how they go about learning it.

If assessment is so important, why are we more interested in issues of reliability and validity than say the guiding principles that help us imagineer the type of assessment systems, protocols and tools that can really make a difference to the way our students learn?

 

The fundamental challenge for our schools, colleges and universities is to stimulate the “right kind of learning” – this challenge brings together our beliefs and assumptions, our purposes and missions, the way we plan – and also highlights the importance of “walking one’s talk”.

Educators have little difficulty in appreciating the power principles in education – guiding and informing practice across the wide spectrum of learning and teaching activities we carry out on a day-to-day basis.

However, as we scan the websites and documentation of our institutions we discover that many schools, colleges and universities do not explicitly state their values and principles on learning – let alone assessment.

Of those that do, sadly, many fail to live in alignment with their espoused principles.

But, there is hope!

Nichols tells us that for principles to be effective, they should capture a “guiding light” (preferably linking explicit assumptions and beliefs to research evidence) and they should be “doable” (so as to guide practitioners towards effective implementation).

So, what are some of the principles that could perhaps act as an effective guiding light?

 

As far back as 1991 (that’s 20 years ago), Chickering & Gamson summarised the “secrets” of good practice in education:

  1. Encourages contacts between students and faculty,
  2. Develops reciprocity and cooperation among students,
  3. Uses active learning techniques
  4. Gives prompt feedback,
  5. Emphasizes time on task,
  6. Communicates high expectations
  7. Respects diverse talents and ways of learning.

 

In 1992, the American Association for Higher Education (AAHE) also developed a set of Principles of Good Practice for Assessing Student Learning.

This was heavily influenced by the ideas developed by Alverno College in the 1970s.

The principles developed by AAHE made deeper and more explicit connections between values, the type of learning envisioned and guidance to practitioners as to how to make this happen. The set of nine principles started at the level of values and sought to link the choices higher education institutions make about assessment to the values that these institutions should subscribe to – they are relevant to primary and secondary education as much as they are to tertiary level learning and…

They are classic bedtime (or snowtime) reading:

1)    The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what’s easy, rather than a process of improving what we really care about.

2)    Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer bases for improving our students’ educational experience.

3)    Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations – those derived from the institution’s mission, from faculty intentions in program and course design, and from knowledge of students’ own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful.

4)    Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students “end up” matters greatly. But to improve outcomes, we need to know about student experience along the way — about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning.

5)    Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, “one-shot” assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time. This may mean tracking the process of individual students, or of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights.

6)    Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment’s questions can’t be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better informed attention to student learning by all parties with a stake in its improvement.

7)    Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return “results”; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement.

8)    Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution’s planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought.

9)    Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation – to ourselves, our students, and society – is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement.

 

Gibbs and Simpson highlighted the practical implications of these principles when they emphasised the conditions that best support student learning:

  • Assessed tasks capture sufficient student time and effort
  • These tasks distribute student effort evenly across topics and weeks
  • These tasks engage students in productive learning activity
  • Assessment communicates clear and high expectations to students

Central to all these principles and recommendations for implementation is the notion of “feedback” – both its quantity and quality. And, the remaining conditions are:

  • Sufficient feedback is provided, frequently enough & in enough detail
  • The feedback is provided quickly enough to be useful to students

And that this feedback is:

  • Focused on learning rather than on marks or the students
  • Linked to the purpose of the assignment and to stated criteria
  • Understandable to students, given their sophistication
  • Received by students and attended to
  • Acted upon by students to improve their work or their learning

 

More recently, the discussions on this line of thinking have been taken to new levels by the REAP Project in Scotland.

The project run by the University of Strathclyde and other REAP Project partners is firmly grounded on more recent research evidence into assessment, best practice in the quality management of assessment systems and the practices that are associated with high levels of student success.

The principles developed by the REAP team renew and expand many of the principles noted above – and do this by asking questions to both teachers and institutions):

1. Help clarify what good performance is (goals, criteria, standards).

To what extent do students in your course have opportunities to engage actively with goals, criteria and standards, before, during and after an assessment task?

2. Encourage ‘time and effort’ on challenging learning tasks.

To what extent do your assessment tasks encourage regular study in and out of class and deep rather than surface learning?

3. Deliver high quality feedback information that helps learners self-correct.

What kind of teacher feedback do you provide – in what ways does it help students self-assess and self-correct?

4. Encourage positive motivational beliefs and self-esteem.

To what extent do your assessments and feedback processes activate your students’ motivation to learn and be successful?

5. Encourage interaction and dialogue around learning (peer and teacher-student).

What opportunities are there for feedback dialogue (peer and/or tutor-student) around assessment tasks in your course?

6. Facilitate the development of self-assessment and reflection in learning.

To what extent are there formal opportunities for reflection, self-assessment or peer assessment in your course?

7. Give learners choice in assessment – content and processes.

To what extent do students have choice in the topics, methods, criteria, weighting and/or timing of learning and assessment tasks in your course?

8. Involve students in decision-making about assessment policy and practice.

To what extent are your students in your course kept informed or engaged in consultations regarding assessment decisions?

9. Support the development of learning communities.

To what extent do your assessments and feedback processes help support the development of learning communities?

10. Help teachers adapt teaching to student needs.

To what extent do your assessment and feedback processes help inform and shape your teaching?

BUT, the “real elegance” of the REAP model is in the way specific principles have been engineered to “use” assessment as a tool to foster learner independence or learner self-regulation (“empowerment”) or to promote time on task and productive learning (“engagement”) – and draw on technology to make this a reality.

 

The root of the word assessment is from the Latin “assidere”, which means “to sit beside.” As teachers and learners sit and work together, communication about the ongoing learning and teaching naturally occurs.

If we, as educators, want to enhance the learning of our students and capture the full educational benefits of well-designed assessment, we have to reconsider the conventional assumptions about assessment in education – we have to look to the principles that guide us.

Perhaps, it’s time we took some time to look at the principles our institutions “sit beside” – those principles, if chosen well and “lived” will help us “see” what the real assessment experts have known for years.

 

Then, a bit later, we can move onto another question:

What does a [school or] college career made up of high exam scores really tell us about a student’s readiness to put knowledge into practice in creative ways?” (Race and Brown)