Update

Hi there tout la gang,

We don't have much to say about research in practice at the Café right now

but we are talking policy and practice over here now: Literacy Enquirers.

Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Monday, June 16, 2008

to ireland today!

Good Monday morning.

I have just spent a rather lazy weekend in London. It takes a bit of effort to be lazy in this city but we managed. Toronto feels like a pretty big city when I am in Canada, but this is a really big city.

I am heading to Ireland this afternoon. To Dublin now and then to Galway for the RaPAL conference, Inclusion and Engagement in Adult Literacy, Numeracy and ESOL . Tannis is coming too. And Bonnie Soroke is doing a workshop.

And to get in the mood for the conference here is an an Irish perspective on Middle School from irishdancr.com. Plus ça change...

More soon, Tracey

P.S. The photo is one I took from the plane to Heathrow. It is the south coast of Ireland.

Thursday, September 27, 2007

apples

Here is a whole bushel of apples for this teacher.

On Tuesday, September 25, 2007, teachers from across the province will rally at the Sooke Board of Education [British Columbia] office in support of a colleague who may be disciplined for refusing to administer a standardized test to her Grade 3 students.

Motivated by concern for her students, Kathryn Sihota, a Grade 3 teacher in the Sooke school district and 27-year veteran in primary teaching, took a stand last spring and refused to give the DART (District Assessment of Reading Team) to her young charges. "I've administered the test for years and I'm not going to do it anymore," Sihota says. "The last time I gave the test, a child dissolved in tears from anxiety. I'd put her in a situation I didn't want her to be in.

Teachers oppose testing when its purpose is simply to satisfy the government and school district agenda of data collection for political purposes, rather than to assist them in finding ways to enrich students' learning experiences. An increasing number of tests are now being imposed on students and are detracting from a quality learning experience.

"Teachers work with students in classrooms every day and understand that students need support and encouragement to be successful," says Irene Lanzinger, president of the British Columbia Teachers' Federation. "We use a variety of assessment methods, including tests, to help students learn. The over-emphasis on standardized exams and data collection is putting students and their success at risk while doing nothing to enhance learning. Teachers throughout the province are speaking out on behalf of their students and the support they need in order to be successful."
September 24, 2007
September
The breezes taste
Of apple peel.
The air is full
Of smells to feel-
Ripe fruit, old footballs,
Burning brush,
New books, erasers,
Chalk, and such.
The bee, his hive,
Well-honeyed hum,
And Mother cuts
Chrysanthemums.
Like plates washed clean
With suds, the days
Are polished with
A morning haze.

by John Updike

Tuesday, September 11, 2007

what can we count?

Here is a poem by a tutor from St. Christopher House Adult Literacy Program.

Anniversary Literacy Poem

How do we love thee, St. Christopher House Literacy Program,
can we count the ways?
Take thirty years of group and individual classes (in the evenings, along with the days)
times the number of completed grammar exercises
then subtract the mistakes that we made:
Do you arrive at the number that quantifies a literacy protégé?

Literacy and . . .
numeracy, of course, that’s what this is all about
Learn the basic functions, count and conquer!
. . . the job maket, that is. And then shuffle us out?

Or should we take a different approach to our equation
starting with the people in the organization
Fran, Judi and Joanna, divining the perfect combination of learners, tutors, kids, activities and inspiration on any given night.
We can even take it right down to the number, if you like,
of pens, pencils, and computers used to enlight-

-en us to a new and better understanding
an accounting of our programming
without the messiness of sentiment, aesthetics or creativity
showing up in the assessments of our ability.
Dollars and scores keep us going—those are the measures that impress.
Even though such external parameters don’t, for us, convey success.

Tenses, agreements, plurals, to the exponential amount of words we’ve learned
Do we add or divide by the people we’ve met, the pages we’ve turned?
Squared up with the compositions: poems, stories, essays and reports
the letters we’ve written to favourite authors, and interest groups of all sorts

And must we keep with the hard work only? Or can we count the parties too?
One every month or so – New Year’s, birthdays, the summer bbq –
And how about special projects – the Tree Tour, workshops, the Link, to name a few,
does this make us eligible for a little more dough to get us through

next class, next week, next tutor orientation,
next group meeting, next time we gather to critique the state of the nation
and other important topics: the Beckhams, Angelina and Brad,
the TTC cuts, crime, health, family, landlords, and whatever else we’ve had
to discuss – most eloquently, by the way – and write about as well.
We do good work here,
so how about stop treating literacy like something we’ve got to sell?

Now—
perhaps it’s time to test your literacy skills by asking: how
much do you recognize here?
This isn’t just any poem, after all,
It’s an adaptation of one by Shakespeare.
You know, the famous bard who,
some four centuries before our program was mounted
taught us that literacy isn’t something that can be counted.

Wednesday, July 11, 2007

time to learn

The majority of learners (35) reported that estimating how long it will take to reach a goal was difficult. Many learners seemed to link time limits with being pressured or rushed, something they needed to avoid to be successful. One learner reported, “For me, I don’t count times. If you put a time, it’s like a rush and you do everything wrong.” George felt that “sometimes timed goals lead to failure and depression.” These typical responses did not appear to be linked to learners’ length of time in their programs. ...

Fifteen learners said that to commit to a timeline would be setting themselves up for disappointment and perhaps failure. For many of these learners, estimating a timeline for their learning brought up negative memories from past school experiences where they had not completed their education, a particular course or covered an academic concept in the time allotted. Some expressed a fear that if you did not reach your stated goal within an estimated time frame, you would be asked to leave the program, even when clearly this was not a program policy or practice.

Five learners described learning as a life-long pursuit, so timelines were not necessary. Three learners in this category participated in their respective programs, worked towards and achieved specific short-term goals and then exited. They returned to the program when they required specific literacy skills and knowledge for another goal. For example, Venus participated in her literacy program a few years ago with the aim to read and write for everyday purposes, a goal she accomplished. Recently she returned to her literacy program to work on understanding and managing household bills.

Five learners said it was difficult to determine how long the learning would take due to diverse changing life circumstances. Estimating how long it will take to reach goals could be affected by issues such as daycare, employment, health, or part time versus full time programs. For many of the women who were single parents, the availability of daycare and the health of their school age children often interrupted participation in their programs.

Eight learners were not sure how long it would take to reach their goals; however, they did not provide explanations for their uncertainty.

from I Open Up: Exploring Learners' Perspectives on Progress
by Susan Lefebvre, Patricia Belding, Mary Brehaut, Sarah Dermer, Anne-Marie Kaskens, Emily Lord, Wayne McKay and Nadine Sookermany

You can listen to Nadine and Susan talk about this project here.

Friday, July 6, 2007

learn anything you want

Coming here as soon as Nancy wins the lottery...

The learners’ stories combine to create an eloquent narrative about the value of adult and community learning, celebrating the energising, life-enhancing, health-giving and career-building consequences of non-accredited programmes.

Others spoke of being unexpectedly and irrevocably ‘hooked’ by their experience into becoming committed learners. This was underpinned by an appreciation of the intrinsic value of learning as well as the contingent economic, social and intellectual benefits. In some cases this extended to a sense of entitlement: ‘You should be allowed to learn just for the hell of it.’ In many ways, their enthusiasm echoes the eclectic vision offered by David Blunkett in the Foreword to The Learning Age – of learning as a nutrient for civic, social, economic, artistic and spiritual activity.

The opportunity to study without qualifications was widely appreciated. The benefits were seen as wider than the achievement of agreed learning outcomes, including the recognition of unanticipated (or unspecified) but valued gains such as greater confidence and self-awareness. Reference was made both to the lack of pressure and competition associated with qualifications, and to the dignity of having greater choice about what was learned. This was described as an ‘adult’ approach and something that distinguished their experience from school. ‘At school you are told what you will learn. It’s an adult way of learning, to make your own choices.’
from the executive summary of the NIACE project
(National Institute of Adult Continuing Education - England and Wales)

Proof Positive: a report on research into learners’ views on approaches to identifying achievement in non-accredited learning

by Kate Watters and Cheryl Turner

Happy weekend everybody. See you Monday.

Thursday, July 5, 2007

critical assessment

Assessment and evaluation also bear the effects of our failure to fully contextualize the lives of learners. Union educators have pointed out that the assumption underlying much of workplace literacy is that workers and management have the same interests in education. Certainly, some interests are shared, but others aren't.

More importantly, educational programs that direct learners toward participating in and measuring up to existing standards, understanding existing systems, and complying with organizational goals usually leave out avenues for conceptualizing, supporting, and making change.

What the work of anthropologists shows is that the population in our programs needs not only literacy but also an expansion of existing opportunities for both work and education. We and our students need to understand not only how education affects work, but also how racism and sexism, and the social capital that comes with class status, determine which jobs are available to whom.
(p. 13)
Politics, Policy, Practice And Personal Responsibility:
Adult Education In An Era Of Welfare Reform

by Deobrah D’Amico,
Ph.D. Consultant to the Adult Literacy Media Alliance,
the Consortium for Worker Education and the Literacy Assistance Center

NCSALL REPORTS #10 April 1999


click on cartoon for more legible version
originally published october 2004 in the literacy enquirer

Thursday, June 7, 2007

program evaluation in ontario

Ontario literacy workers are just being told about CIPMS (Continuous Improvement of Performance Management System) and how this is to be implemented in Literacy and Basic Skills programs.

CIPMS includes a performance measurement process made up of three elements: 50% effectiveness (learner skills attainment or essential skills gain), 30% customer satisfaction (?), and 20% efficiency (contract compliance).

I won’t drag you through all the details here – you can read about it at Community Literacy Ontario – Literacy Basics: Outcomes-based Program Evaluation, the May 14, 2007 Update and in Our Voice, Community Literacy Ontario Newsletter, June 2006, p.3.

The short story is that the ministry hired Vubiz Ltd to research how best to measure learner achievement. What they came up with was an eight-point framework for use with all learners and a 500-point test of Reading Text, Document Use and Numeracy (the International Adult Literacy Survey assessment) for use with 5-10% of learners for ‘corporate reporting purposes’.

In 2005, Jo Lobianco wrote in Becoming Policy Literate, (PDF at the Centre for Literacy of Quebec Online Articles):

An ideological shift occurred between 1990 and 1996. During 1990, International Literacy Year, adult literacy thinking was still characterized by reformist social ambition… By the time the International Adult Literacy Survey was conducted [1996 – see Literacies fall 2003 web forum for more about IALS] … the various participating countries … typically replaced social transformation ideologies with prevailing ideas about labour market reforms, efficiency, and enhanced competitiveness in globalizing markets.
Is CIPMS just the next logical step in the shift Jo describes?

Here is the antidote: Making Assessment Personally Relevant
I want my students to realize that learning is not about making your work conform to some standard imposed by the teacher. Learning is about creating your own standards and adjusting them based on your goals. Learning is about setting your own goals and monitoring your own progress. It is about having conversations with yourself and others.
As Stephen Downes says,
There are many things I like about this post, but I'll highlight two: first, assessments are conducted by the students themselves, not some arbiter of learning achievement. Second, the scales use no numbers. They're not needed, not unless you want to portray (inaccurately) learning as some sort of contest. Which it's not.