report cards

The news hit the pages of the Globe and Mail under this headline last Friday: Conservatives stop funding for learning organization.

The news is that the federal government will not provide further funding for the Canadian Council on Learning, an organization established in 2004 by a Liberal government with a five-year grant of $85-million to promote lifelong education.

The CCL was established because "In 2004, Canada saw that it had some catching up to do. Canadians were falling behind the rest of the world in some crucial areas. Innovation. Creativity. Skills development. Learning. There was no debate about what we had to do to stop the decline, and begin to improve. We had to figure out what works in education and learning, from early childhood to post-secondary schooling, from job training through adult literacy improvement, and we had to monitor our progress so that we were certain we were always on the right path."

I am not sure that I agree with the basic premise. I see little real world evidence that Canadians are less innovative, creative, skilled and/or learned than their counterparts around the world. In fact, the solutions to our recent economic woes seem to point to a past underuse of the innovation, creativity, skills and learning of many Canadians rather than any deficits in those areas. But the creation of the CCL intrigued me. It seemed to hold promise - perhaps we could create an education think tank that would shine a light on the innovation, creativity, skills and learning of educators in Canada and that this knowledge would inform policy.

The CCL divided its work into 5 knowledge centres: Aboriginal Learning (based in the Prairies, NWT and Nunavut); Adult Learning (based in Atlantic Canada); Early Childhood Learning (based in Quebec); Health and Learning (based in British Columbia and Yukon); and Work and Learning (based in Ontario).

I have had the most experience with the Adult Learning Knowledge Centre. In 2007 I worked on a research-in-practice project that was funded through the ALKC and, in June 2007, I went to an ALKC conference. At the conference I was impressed by the grassroots nature of some of the projects but became aware of a focus on measuring. As time went on, I realized that the CCL was less a "learning organization" and more a measuring organization.

The Globe and Mail article quotes Mr. Cappon about the loss of funding: “What Canada would lose without CCL would be like being a student without a report card of any kind. And we'd be prevented from knowing how far behind the competition we're slipping."

CCL brought us several report cards over the years.

The Composite Learning Index (CLI) measures Canada’s progress in lifelong learning based on statistical indicators that reflect the many ways Canadians learn, whether in school, in the home, at work or within the community. The CLI tells us things like this: "For the first time, Canada's overall score on the Composite Learning Index has declined, dropping two points to 75 in 2009, from 77 in 2008. The decrease is being driven by the Learning to Be pillar, even though there has been an increase on the Learning to Do pillar."

The Projections of Adult Literacy: Measuring Movement (PALMM) provides a statistical “snapshot” of Canada’s adult literacy future through 2031; calculates future adult literacy rates according to province/territory and specific population groups; and generates graphs for incorporating into planning for literacy policy and program interventions. The PALMM tells us things like this: "Although the proportion of immigrants with low literacy skills will decrease by 2031, the actual number of low-skilled immigrants will increase by 61%. However, there will also be an increase in the number of immigrants with higher literacy levels."

The July 2008 literacy educators got a CCL report card. Reading the Future: Planning to meet Canada’s future literacy uses the PALMM to assess the state of adult literacy in Canada and makes recommendations on how literacy programming should be structured and delivered. This report was well discussed on this blog (and here) and in our special bulletin. One of our concerns was that this report makes recommendations about professionalization, teaching methods and time limits that overlook "the rich and varied techniques Canadian practitioners use to teach adults successfully and share in practitioner-based research reports." Reading the Future was a signal to the literacy field that the CCL was not developing knowledge based on the innovation, creativity, skills and learning of educators but was taking a top down approach to develop policy recommendations based on survey data.

In March of last year, CCL presented the adult literacy community with their online literacy assessment tool, also well discussed on this blog. This tool allowed us to develop our own report card on our own IALSS level. CCL told us we could also use the tool to assess learner needs overlooking the fact that the IALSS is a survey tool and is not an appropriate assessment for learning

"In anticipation of renewed funding, CCL had proposed an exciting slate of projects for the coming years."

Many in the list of 10 proposed projects, not surprisingly, seem to be about collecting more data to create more measuring tools. The Literacy Self-Assessments looks to be an expansion of the above mentioned online assessment: "CCL plans to launch a series of free, online tests that will allow individual Canadians to measure their competencies in three areas: prose literacy, document literacy and numeracy. Based on the International Adult Literacy and Skills survey, these tests will also provide teachers and instructors with a convenient means of assessing strengths and weaknesses to ensure programs meet learner needs." Once again CCL is overlooking the folly of using a survey tool to assess for learning and using a standardized test for self-assessment.

That report card argument may resonate in some sectors but I am not sure what literacy workers will think about it. You can probably tell what I think :), but what about you? Do we need these report cards? How do they help us in our work? How do they help learners in their learning? How does the report card argument resonate with you?

it is all fun and games until...

The Canadian Council on Learning has posted their online literacy assessment tool.

The tool can be used to assess your IALSS level. IALSS is the International Adult Literacy and Skills Survey:

"The International Adult Literacy and Skills Survey was a seven-country initiative conducted in 2003. In every country nationally representative samples of adults aged 16-65 were interviewed and tested at home, using the same psychometric test to measure prose and document literacy as well as numeracy and problem-solving skills.

• ...The main purpose of the survey was to find out how well adults used printed information to function in society.
• Another aim was to collect data on the incidence and volume of participation in adult education and training, and to investigate the relationships between initial and adult education, on the one hand, and literacy, numeracy and problem-solving proficiency and wider economic and social outcomes, on the other.
• In addition, a subsidiary goal was to provide information regarding change in the distribution of skills over the years since the previous survey (the 1994, International Adult Literacy Survey).
• ...The data are used to inform policy decisions, help effectively allocate resources where needed and inform decisions on the composition and content of remedial skill development course and adult education."

Ever since the first IALS survey in 1994, there has been debate* about what exactly these data tell us, but, until now, I do not think that anyone has suggested that an IALSS-based test, designed to survey a population, ever be used to assess and assign levels to individuals. If they have, they should be thoroughly ashamed of themselves.

A couple of us tried the test. It took ages, the fonts were teeny weeny, the interface did not work in any browser, and it was incredibly dull. I think that the test could take quite a long time to complete. We did not want to spend too long on it so we tried not to get bogged down. We just clicked on our first good idea for the right answer and let it go.

We both scored Level 4 (Strong skills -- individuals at these levels can process information of a complex and demanding nature) in Document Literacy (The ability to find and use information in forms, charts, graphs and other tables).

My colleague scored Level 4 in Numeracy (The ability to use basic math skills in everyday life) and I scored Level 3 (Adequate to cope with the demands of everyday life and work in an advanced society -- roughly denotes the skill level required for successful high school completion and college entry). I must admit I did not feel like doing some of the calculations and guessed at some of the answers but Level 3 is what I would guess is correct for me in any case and, as it turns out, I am a pretty good guesser.

Now the bad news. We both scored Level 2 (A capacity to deal only with simple, clear material involving uncomplicated tasks -- people at this level may develop everyday coping skills, but their poor literacy skills make it hard to conquer challenges such as learning new job skills) in Prose Literacy (The knowledge and skills required to understand and appropriately use information from print materials). CCL recommends that all Canadians need to be at Level 3 if they are to achieve prosperity, health and happiness.

My colleague is very clever and has exemplary prose literacy skills. I am using this fact to cheer myself up a little. I am surprised to find that despite my rather cavalier attitude to this test, the result made me feel a little crappy. I usually feel fine about my ability to understand and appropriately use information from print materials. There are things that challenge me but I have confidence that if I persevere I will get it.

Our Level 2 scores prompted another colleague to wonder, "If the CCL online test is based on or reflects the IALSS test and if [these two] only scored 2 on the Prose test, perhaps the IALSS test is underestimating people’s literacy levels."

One of the questions that has been raised about IALS and IALSS is why the results show such high numbers of people at Level 1 and 2 when people self-assess at a much greater ability to understand and appropriately use information from print materials.

But the biggest question for me remains, “Why do I want to know what IALSS level I am at?” or “Why does anyone want to know what IALSS level any individual is at?”

What we want to know is how do we work with text now and how do we want or need to be able to work with text to do what we want or need to do. To do this we can use a method such as the Canadian Adult Reading Assessment that assesses reading skills and strategies in a manageable, non-invasive, non-judgmental way and demonstrates how learner-centred assessment leads to the development of a learner-centred curriculum that is relevant and effective.

C'mon CCL, stop now before somebody gets hurt.

*For some of that debate see Literacies Issue #2 and the Literacies IALS web forum.

what did you learn?

Here are the program response recommendations from the Canadian Council on Learning as stated in their report Reading the Future (page 52). Click the graphic to see it full size :P






Here are the "Best Practice" program responses to each market segment from DataAngel as stated in their report Addressing Canada's Literacy Challenge: A Cost/ Benefit Analysis (page 37). As above.

And here is the trailer from Entre les murs (The Class):

Teacher and novelist François Bégaudeau plays a version of himself as he negotiates a year with his racially mixed students from a tough Parisian neighborhood. Directed by Laurent Cantet.

This film shows learning the way everybody who has ever tried to help anybody learn anything understands it. It shows how learning does not happen by tick boxes, charts, timelines and goalsetting. It shows how learning is chaotic, free-ranging, exasperating and energizing;

• that 17 'teachable moments' fly past every minute and no-one can catch them all;
• that the moments when a teacher and student find a way to communicate authentically are transcendent but often fleeting; 
• that when students can control the context, process and content, learning is meaningful and deep; 
• that everything matters forever; and 
• that the most difficult thing to do is to say what you learned at the end of it all.

reading the reports

Recently a group of literacy workers and researchers came together to apply their expertise and analysis to two reports that could inform education policy decisions in Canada.

Learning Literacy in Canada: Evidence from the International Survey of Reading Skills, was released by Statistics Canada in January 2008. It used data from the International Survey of Reading Skills (ISRS) to offer information that can help “policy makers, researchers and practitioners” decide “how to plan and deliver appropriate and efficient reading instruction for different adult learners” (p. 19).

Reading the Future: Planning to meet Canada’s future literacy needs is a synthesis of work by a number of researchers who were involved in developing IALSS. This report, released by the Canadian Council on Learning in July 2008, offers program recommendations, practices, and strategies for improving the skills of the low-skilled readers identified by the ISRS.

We have created Reading the Reports (PDF download) to let you know what they are saying. We hope you find this brief summary of their discussion helpful and informative.

The group is: Tannis Atkinson (editor of Literacies), Dr. Pat Campbell (Grass Roots Press / Education for Change), Dr. Richard Darville (Carleton University), Brigid Hayes (Brigid Hayes Consulting), Dr. Nancy Jackson (Ontario Institute for Studies in Education of the University of Toronto), and Tracey Mollins (publisher of Literacies).

What conversations have you been having about these reports? Let us know your thoughts, ideas and questions.

conferencing literacy

I am just back from the Adult Learning Knowledge Centre 2007 National Symposium in Halifax. Still trying to sort through all that I learned.

The highlights:

1. Invigorating and refreshing conversation with some of the good literacy folks who were able to attend ~ Nancy Jackson, Nadine Sookermany, Suzanne Smythe, Janet Skinner, Sue Shore, Arthur Bull, Charles Ramsey and Wendy DesBrisay (in order of appearance).
2. Presentations that brought learner voices to the conference.
   
   • Corrina Craig from Our Place Society, Victoria, BC told us about Street Life’s Creative Turn
   • Arthur Bull of the Marine Resource Centre, Digby, NS told us about Visual Media as a Tool for Community Development; and
   • Jason Brown of the University of Western Ontario, London, ON told us about Adult Learning in Aboriginal Community-Based Inner-City Organizations.
3. A presentation that brought literacy practitioner voices to the conference.
   • Wendy DesBrisay, Movement for Canadian Literacy, Ottawa, ON told us about A Fully Literate Canada: What does it mean? What is it going to take?
4. The brave, clear literacy voices. The above group conferenced with passion and compassion and, for those who could not be there, I think that you would be proud of how well our field was represented by this tiny, fierce contingent.
5. Support. Some of the ALKC people and conference participants responded with passion and compassion.
   

The less-than-high lights:

1. The lack of diversity among participants. And along the same lines ~ the reports on projects about work in aboriginal communities not being presented by aboriginal people.
2. The pervasive view that sets of numbers and/or tiny boxes can describe learning as measurable, quantifiable sets of skills rather than what it actually is: a process of developing self, community and awareness (see the scottish approach).
   
3. The pervasive sadness of literacy workers. This ordinarily innovative and brave group is finding it difficult to envision a future for the field. I think that we all know why.