report cards

The news hit the pages of the Globe and Mail under this headline last Friday: Conservatives stop funding for learning organization.

The news is that the federal government will not provide further funding for the Canadian Council on Learning, an organization established in 2004 by a Liberal government with a five-year grant of $85-million to promote lifelong education.

The CCL was established because "In 2004, Canada saw that it had some catching up to do. Canadians were falling behind the rest of the world in some crucial areas. Innovation. Creativity. Skills development. Learning. There was no debate about what we had to do to stop the decline, and begin to improve. We had to figure out what works in education and learning, from early childhood to post-secondary schooling, from job training through adult literacy improvement, and we had to monitor our progress so that we were certain we were always on the right path."

I am not sure that I agree with the basic premise. I see little real world evidence that Canadians are less innovative, creative, skilled and/or learned than their counterparts around the world. In fact, the solutions to our recent economic woes seem to point to a past underuse of the innovation, creativity, skills and learning of many Canadians rather than any deficits in those areas. But the creation of the CCL intrigued me. It seemed to hold promise - perhaps we could create an education think tank that would shine a light on the innovation, creativity, skills and learning of educators in Canada and that this knowledge would inform policy.

The CCL divided its work into 5 knowledge centres: Aboriginal Learning (based in the Prairies, NWT and Nunavut); Adult Learning (based in Atlantic Canada); Early Childhood Learning (based in Quebec); Health and Learning (based in British Columbia and Yukon); and Work and Learning (based in Ontario).

I have had the most experience with the Adult Learning Knowledge Centre. In 2007 I worked on a research-in-practice project that was funded through the ALKC and, in June 2007, I went to an ALKC conference. At the conference I was impressed by the grassroots nature of some of the projects but became aware of a focus on measuring. As time went on, I realized that the CCL was less a "learning organization" and more a measuring organization.

The Globe and Mail article quotes Mr. Cappon about the loss of funding: “What Canada would lose without CCL would be like being a student without a report card of any kind. And we'd be prevented from knowing how far behind the competition we're slipping."

CCL brought us several report cards over the years.

The Composite Learning Index (CLI) measures Canada’s progress in lifelong learning based on statistical indicators that reflect the many ways Canadians learn, whether in school, in the home, at work or within the community. The CLI tells us things like this: "For the first time, Canada's overall score on the Composite Learning Index has declined, dropping two points to 75 in 2009, from 77 in 2008. The decrease is being driven by the Learning to Be pillar, even though there has been an increase on the Learning to Do pillar."

The Projections of Adult Literacy: Measuring Movement (PALMM) provides a statistical “snapshot” of Canada’s adult literacy future through 2031; calculates future adult literacy rates according to province/territory and specific population groups; and generates graphs for incorporating into planning for literacy policy and program interventions. The PALMM tells us things like this: "Although the proportion of immigrants with low literacy skills will decrease by 2031, the actual number of low-skilled immigrants will increase by 61%. However, there will also be an increase in the number of immigrants with higher literacy levels."

The July 2008 literacy educators got a CCL report card. Reading the Future: Planning to meet Canada’s future literacy uses the PALMM to assess the state of adult literacy in Canada and makes recommendations on how literacy programming should be structured and delivered. This report was well discussed on this blog (and here) and in our special bulletin. One of our concerns was that this report makes recommendations about professionalization, teaching methods and time limits that overlook "the rich and varied techniques Canadian practitioners use to teach adults successfully and share in practitioner-based research reports." Reading the Future was a signal to the literacy field that the CCL was not developing knowledge based on the innovation, creativity, skills and learning of educators but was taking a top down approach to develop policy recommendations based on survey data.

In March of last year, CCL presented the adult literacy community with their online literacy assessment tool, also well discussed on this blog. This tool allowed us to develop our own report card on our own IALSS level. CCL told us we could also use the tool to assess learner needs overlooking the fact that the IALSS is a survey tool and is not an appropriate assessment for learning

"In anticipation of renewed funding, CCL had proposed an exciting slate of projects for the coming years."

Many in the list of 10 proposed projects, not surprisingly, seem to be about collecting more data to create more measuring tools. The Literacy Self-Assessments looks to be an expansion of the above mentioned online assessment: "CCL plans to launch a series of free, online tests that will allow individual Canadians to measure their competencies in three areas: prose literacy, document literacy and numeracy. Based on the International Adult Literacy and Skills survey, these tests will also provide teachers and instructors with a convenient means of assessing strengths and weaknesses to ensure programs meet learner needs." Once again CCL is overlooking the folly of using a survey tool to assess for learning and using a standardized test for self-assessment.

That report card argument may resonate in some sectors but I am not sure what literacy workers will think about it. You can probably tell what I think :), but what about you? Do we need these report cards? How do they help us in our work? How do they help learners in their learning? How does the report card argument resonate with you?

monday morning blues

Here is something I read over at the LearningWork blog:

Many Canadians experience a significant loss of literacy skills during adulthood, and this loss appears to be concentrated in adults from lower socio-economic backgrounds, according to a new study. ...

The loss of literacy skills in Canada appears to be a gradual process that begins at the age of about 25, peaks at around 40, and tapers off during late middle age (55).

For example, adults aged 40 in 1994 had average scores on the IALS literacy test of about 288. When this test was implemented nine years later, those who were aged 49 had average scores of about 275.

Really? I am not sure what this reported 13 point drop in scores could mean, but anecdotally I notice that what we read and how we read changes with age and with technology. I think that the possibilities and strategies for engaging with print material changed quite significantly between 1994 and 2003.

How many of us, when working with newcomers preparing for citizenship tests, or helping young friends do homework, or watching that show where people pit their wits against those of elementary school students, have reflected on how much of what we learned about math, civics, history, natural science in our school days has fled from our knowledge bank? Speaking personally, I am sure that any tests on geometry or the life cycle of frogs would show a drop of much more than 13 points over the years.

Come on everybody. Stop cluttering up the conversations about literacy and education with this weird, incomprehensible data. Every time I read one of these reports I think of this:

This is not the information age.
It is an age of data with delusions of grandeur.

Sorry, I cannot remember where I heard that or who said it. But I used to.

program evaluation in ontario

Ontario literacy workers are just being told about CIPMS (Continuous Improvement of Performance Management System) and how this is to be implemented in Literacy and Basic Skills programs.

CIPMS includes a performance measurement process made up of three elements: 50% effectiveness (learner skills attainment or essential skills gain), 30% customer satisfaction (?), and 20% efficiency (contract compliance).

I won’t drag you through all the details here – you can read about it at Community Literacy Ontario – Literacy Basics: Outcomes-based Program Evaluation, the May 14, 2007 Update and in Our Voice, Community Literacy Ontario Newsletter, June 2006, p.3.

The short story is that the ministry hired Vubiz Ltd to research how best to measure learner achievement. What they came up with was an eight-point framework for use with all learners and a 500-point test of Reading Text, Document Use and Numeracy (the International Adult Literacy Survey assessment) for use with 5-10% of learners for ‘corporate reporting purposes’.

In 2005, Jo Lobianco wrote in Becoming Policy Literate, (PDF at the Centre for Literacy of Quebec Online Articles):

An ideological shift occurred between 1990 and 1996. During 1990, International Literacy Year, adult literacy thinking was still characterized by reformist social ambition… By the time the International Adult Literacy Survey was conducted [1996 – see Literacies fall 2003 web forum for more about IALS] … the various participating countries … typically replaced social transformation ideologies with prevailing ideas about labour market reforms, efficiency, and enhanced competitiveness in globalizing markets.

Is CIPMS just the next logical step in the shift Jo describes?

Here is the antidote: Making Assessment Personally Relevant

I want my students to realize that learning is not about making your work conform to some standard imposed by the teacher. Learning is about creating your own standards and adjusting them based on your goals. Learning is about setting your own goals and monitoring your own progress. It is about having conversations with yourself and others.

As Stephen Downes says,

There are many things I like about this post, but I'll highlight two: first, assessments are conducted by the students themselves, not some arbiter of learning achievement. Second, the scales use no numbers. They're not needed, not unless you want to portray (inaccurately) learning as some sort of contest. Which it's not.

who benefits?

In a few weeks I will be in Belfast at the annual conference of Research and Practice in Adult Literacy (RaPAL). I've been pondering how to convey the realities of literacy work on this side of the Atlantic.

I've also been reading Valuing Literacy: Rhetoric or Reality (Detselig, 2006). This book reports on research by Veeman, Ward and Walker comparing adult literacy in Canada and Sweden. I like the succinct summary included in their conclusion:
In Canada, adult literacy is an individual rather than a community problem and it is dealt with as a charitable cause. There is no universal publicly-funded system of adult basic education to provide compensatory education for adults in any jurisdiction. Instead, undereducated adults must avail themselves of a patchwork of volunteer programs or projects offered by community-based organizations. There is no reliable schedule of adult learning opportunities, nor do all Canadians have access to funding for study at the basic level. The charity nature of literacy and the name literacy itself are disincentives – and often a disservice – to adults who might lack self-confidence or need special help in addressing learning needs. (p.102)

The aim of the research was to explore why Canada and Sweden fared so differently in the OECD's 1994 International Adult Literacy Survey, which found that "adult literacy levels in Scandinavia were significantly higher than those in English-speaking countries." What did the researchers conclude?

The social democratic approach of Northern Europe sees adult education as a public investment that helps raise the educational level of the whole society for the benefit of all. This is in contrast to the economic and social policies of Anglophone countries, where the emphasis has been on sustaining meritocracy rather than on producing social equity. Literacy in countries such as Canada is seen as the individual’s problem to be solved, and the rampant individualism that has damaged trust, fairness, and social bonds has not served to raise literacy levels. (p. 105)

I've often thought that adult literacy programs in the UK had enviable levels of infrastructure and sustained support. At RaPAL I'm sure I'll learn more about how conditions in Ireland, England and Scotland might be similar to Canada. I'll use this space to share what I learn!