Tony Gurr

ASSESSING How We ASSESS Learners (Part One)

In Assessment, Our Schools, Our Universities on 10/03/2011 at 8:25 am


“…we have to see that learning – deep learning, learning that matters, learning that lasts – is not something that teachers do to students or even that students do for themselves. Rather it is the product of action in a context shaped by goals, performance, feedback, time horizon, and community – of all of the principles that define the cognitive economy, acting to create an environment that empowers and engages students.

Tagg, 2003

One of the things I am often asked on my travels around Turkey is:

  • What can we do to improve the way we assess student learning?

I love this question – it shows an “improvement-orientated ethos” and a recognition of just how important assessment is to “learning”. After all, and as Lauren Resnik so powerfully put it, “What you assess is what you get; if you don’t test it you won’t get it.

BUT, I usually answer the question with a question (yes, my “big little girl” always hated this habit of her daddy):

  • What principles are your assessment systems and tools built on?

 

I often get a lot of “blank” looks.

Many respond by talking about reliability, validity or “professional levels of quality”.

To be sure, these are important notions – but my question is digging a little deeper.

As teachers, educators and assessment experts we all discuss “quality issues” in assessment but we forget one thing:

The REAL assessment “experts” are STUDENTS!

All good educators know that assessment is a critical component of any successful learning and teaching experience. They also recognise that assessment is one of the main parts of the “job” of teaching or supporting learning.

Students know MORE!

Students know that it is assessment that mediates all teaching and learning relationships. They know there is nothing more central to the learning experience than assessment.

They know assessment can affect their whole future careers and…lives!

For students, the type of assessment they know is “around the corner” determines:

  • what they define as important;
  • when they tune in and tune out during formal learning opportunities;
  • what they study in their own time and how much they engage with their subject matter;

and,

  • how they come to see themselves as students, graduates and people in later life

In a nutshell, and to borrow the words of John Cowen, “assessment is the engine which drives student learning” – and it determines not only what students learn but how they go about learning it.

If assessment is so important, why are we more interested in issues of reliability and validity than say the guiding principles that help us imagineer the type of assessment systems, protocols and tools that can really make a difference to the way our students learn?

 

The fundamental challenge for our schools, colleges and universities is to stimulate the “right kind of learning” – this challenge brings together our beliefs and assumptions, our purposes and missions, the way we plan – and also highlights the importance of “walking one’s talk”.

Educators have little difficulty in appreciating the power principles in education – guiding and informing practice across the wide spectrum of learning and teaching activities we carry out on a day-to-day basis.

However, as we scan the websites and documentation of our institutions we discover that many schools, colleges and universities do not explicitly state their values and principles on learning – let alone assessment.

Of those that do, sadly, many fail to live in alignment with their espoused principles.

But, there is hope!

Nichols tells us that for principles to be effective, they should capture a “guiding light” (preferably linking explicit assumptions and beliefs to research evidence) and they should be “doable” (so as to guide practitioners towards effective implementation).

So, what are some of the principles that could perhaps act as an effective guiding light?

 

As far back as 1991 (that’s 20 years ago), Chickering & Gamson summarised the “secrets” of good practice in education:

  1. Encourages contacts between students and faculty,
  2. Develops reciprocity and cooperation among students,
  3. Uses active learning techniques
  4. Gives prompt feedback,
  5. Emphasizes time on task,
  6. Communicates high expectations
  7. Respects diverse talents and ways of learning.

 

In 1992, the American Association for Higher Education (AAHE) also developed a set of Principles of Good Practice for Assessing Student Learning.

This was heavily influenced by the ideas developed by Alverno College in the 1970s.

The principles developed by AAHE made deeper and more explicit connections between values, the type of learning envisioned and guidance to practitioners as to how to make this happen. The set of nine principles started at the level of values and sought to link the choices higher education institutions make about assessment to the values that these institutions should subscribe to – they are relevant to primary and secondary education as much as they are to tertiary level learning and…

They are classic bedtime (or snowtime) reading:

1)    The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what’s easy, rather than a process of improving what we really care about.

2)    Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer bases for improving our students’ educational experience.

3)    Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations – those derived from the institution’s mission, from faculty intentions in program and course design, and from knowledge of students’ own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful.

4)    Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students “end up” matters greatly. But to improve outcomes, we need to know about student experience along the way — about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning.

5)    Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, “one-shot” assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time. This may mean tracking the process of individual students, or of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights.

6)    Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment’s questions can’t be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better informed attention to student learning by all parties with a stake in its improvement.

7)    Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return “results”; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement.

8)    Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution’s planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought.

9)    Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation – to ourselves, our students, and society – is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement.

 

Gibbs and Simpson highlighted the practical implications of these principles when they emphasised the conditions that best support student learning:

  • Assessed tasks capture sufficient student time and effort
  • These tasks distribute student effort evenly across topics and weeks
  • These tasks engage students in productive learning activity
  • Assessment communicates clear and high expectations to students

Central to all these principles and recommendations for implementation is the notion of “feedback” – both its quantity and quality. And, the remaining conditions are:

  • Sufficient feedback is provided, frequently enough & in enough detail
  • The feedback is provided quickly enough to be useful to students

And that this feedback is:

  • Focused on learning rather than on marks or the students
  • Linked to the purpose of the assignment and to stated criteria
  • Understandable to students, given their sophistication
  • Received by students and attended to
  • Acted upon by students to improve their work or their learning

 

More recently, the discussions on this line of thinking have been taken to new levels by the REAP Project in Scotland.

The project run by the University of Strathclyde and other REAP Project partners is firmly grounded on more recent research evidence into assessment, best practice in the quality management of assessment systems and the practices that are associated with high levels of student success.

The principles developed by the REAP team renew and expand many of the principles noted above – and do this by asking questions to both teachers and institutions):

1. Help clarify what good performance is (goals, criteria, standards).

To what extent do students in your course have opportunities to engage actively with goals, criteria and standards, before, during and after an assessment task?

2. Encourage ‘time and effort’ on challenging learning tasks.

To what extent do your assessment tasks encourage regular study in and out of class and deep rather than surface learning?

3. Deliver high quality feedback information that helps learners self-correct.

What kind of teacher feedback do you provide – in what ways does it help students self-assess and self-correct?

4. Encourage positive motivational beliefs and self-esteem.

To what extent do your assessments and feedback processes activate your students’ motivation to learn and be successful?

5. Encourage interaction and dialogue around learning (peer and teacher-student).

What opportunities are there for feedback dialogue (peer and/or tutor-student) around assessment tasks in your course?

6. Facilitate the development of self-assessment and reflection in learning.

To what extent are there formal opportunities for reflection, self-assessment or peer assessment in your course?

7. Give learners choice in assessment – content and processes.

To what extent do students have choice in the topics, methods, criteria, weighting and/or timing of learning and assessment tasks in your course?

8. Involve students in decision-making about assessment policy and practice.

To what extent are your students in your course kept informed or engaged in consultations regarding assessment decisions?

9. Support the development of learning communities.

To what extent do your assessments and feedback processes help support the development of learning communities?

10. Help teachers adapt teaching to student needs.

To what extent do your assessment and feedback processes help inform and shape your teaching?

BUT, the “real elegance” of the REAP model is in the way specific principles have been engineered to “use” assessment as a tool to foster learner independence or learner self-regulation (“empowerment”) or to promote time on task and productive learning (“engagement”) – and draw on technology to make this a reality.

 

The root of the word assessment is from the Latin “assidere”, which means “to sit beside.” As teachers and learners sit and work together, communication about the ongoing learning and teaching naturally occurs.

If we, as educators, want to enhance the learning of our students and capture the full educational benefits of well-designed assessment, we have to reconsider the conventional assumptions about assessment in education – we have to look to the principles that guide us.

Perhaps, it’s time we took some time to look at the principles our institutions “sit beside” – those principles, if chosen well and “lived” will help us “see” what the real assessment experts have known for years.

 

Then, a bit later, we can move onto another question:

What does a [school or] college career made up of high exam scores really tell us about a student’s readiness to put knowledge into practice in creative ways?” (Race and Brown)

Advertisements
  1. […] 7 – Remember that for real change in learning and teaching, we need to build in an “assessment perspective” into our planning (after all, we all know that if it ain’t “tested”, it […]

  2. I’ve bookmarked your web site and I’m together with your RSS feeds to my Google account.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: