program evaluation in ontario

Ontario literacy workers are just being told about CIPMS (Continuous Improvement of Performance Management System) and how this is to be implemented in Literacy and Basic Skills programs.

CIPMS includes a performance measurement process made up of three elements: 50% effectiveness (learner skills attainment or essential skills gain), 30% customer satisfaction (?), and 20% efficiency (contract compliance).

I won’t drag you through all the details here – you can read about it at Community Literacy Ontario – Literacy Basics: Outcomes-based Program Evaluation, the May 14, 2007 Update and in Our Voice, Community Literacy Ontario Newsletter, June 2006, p.3.

The short story is that the ministry hired Vubiz Ltd to research how best to measure learner achievement. What they came up with was an eight-point framework for use with all learners and a 500-point test of Reading Text, Document Use and Numeracy (the International Adult Literacy Survey assessment) for use with 5-10% of learners for ‘corporate reporting purposes’.

In 2005, Jo Lobianco wrote in Becoming Policy Literate, (PDF at the Centre for Literacy of Quebec Online Articles):

An ideological shift occurred between 1990 and 1996. During 1990, International Literacy Year, adult literacy thinking was still characterized by reformist social ambition… By the time the International Adult Literacy Survey was conducted [1996 – see Literacies fall 2003 web forum for more about IALS] … the various participating countries … typically replaced social transformation ideologies with prevailing ideas about labour market reforms, efficiency, and enhanced competitiveness in globalizing markets.

Is CIPMS just the next logical step in the shift Jo describes?

Here is the antidote: Making Assessment Personally Relevant

I want my students to realize that learning is not about making your work conform to some standard imposed by the teacher. Learning is about creating your own standards and adjusting them based on your goals. Learning is about setting your own goals and monitoring your own progress. It is about having conversations with yourself and others.

As Stephen Downes says,

There are many things I like about this post, but I'll highlight two: first, assessments are conducted by the students themselves, not some arbiter of learning achievement. Second, the scales use no numbers. They're not needed, not unless you want to portray (inaccurately) learning as some sort of contest. Which it's not.