Skip to content

EDTalkNZ Posts

Research-based teaching

Published on February 13, 2009

Stuart Middleton
New Zealand Education Review
Vol. 14 No.5, 13 February 2009, p.16
APN Educational Media (NZ) Ltd.
Wellington

I once went to the theatre to see the complete works of Shakespeare delivered in the space of a little under two hours. It was dazzling and in a bizarre way succeeded in getting across the essence of the Bard both in terms of language and the larger themes and concerns. There is something of a feeling like this in the release of John Hattie’s Visible Learning, and in trying to write about it in 1,000 words!

This book is a synthesis of more than 800 meta-analyses from the education research on effective learning and achievement. In short, it sets out to grapple with the answer to the question “What really makes a difference in helping learners to reach high levels of achievement?” It brings together a huge body of research – 15 years work as an educational researcher, 50,000 quantitative studies reworked in 800 meta-analyses and, the easy bit (I think not!), the writing of the book.

All of this seemed to matter very little to many of those who responded to early media interest in the book over the holiday period. The media, in their inexhaustible quest for the story and the controversy, highlighted the low ranking of class size as a factor and the stories focused largely on this. Hattie concludes that class size ranks 106th in a list of 138 factors that influence achievement. Cries of “rubbish”, “I know better!”, “we need more teachers” pretty well sums up the responses proving once again the desire of education to not be a research-driven profession in the way that characterises law, medicine, engineering and technology. Oh no! We know best! 

I have refrained from comment until I have read the book unlike all those who grabbed a radio opportunity.

It is easy to be impressed by the statistics of the study itself but that misses the point. This book deals honestly with the issues of meta-analysis and disarmingly sets out to be just that – a synthesis of meta-analyses. But this is exactly the process by which educational research can hope to have an impact. The world of teaching and learning is not short of research, it is simply ignorant of much of it and books such as this set up a group of professionals to engage with a body of research in chunks that can be understood and more importantly acted on by classroom teachers.

There seems both a degree of irony and appropriateness that I write this while the TV shows pictures of bush fires in Australia. A useful device in the book is a kind of Forest Fire Danger barometer, an arrow positioned over an arc that portrays the level of impact of a factor on student achievement.

So what does make a difference? The top five factors in terms of measured impact on achievement are:

Self-report grades

Hattie concludes that students have a “reasonably accurate understanding” of their achievement and that this is based on their experiences except for minority students who were a little less realistic in self reported grades. Do teachers, I wonder, inflate their assessment of minority students? But the message is clear, expectations are important so make sure they are right! Students need to know where they are, where they are headed and what they must do to get there.

Piagetian programmes

The conclusion is that Piaget got it right for mathematics and almost right for reading. The point is that there are stages and as teachers we had better understand them.

Providing formative evaluation

When I was doing my PhD at an advanced age I constantly asked my supervisor when he enthused about a chapter or something “But is this going to be a PhD? How am I going? Will I get there? Am I at the right place? Learners need to know where they are and who can tell them if the teachers can’t? A PhD candidate is in no different a position from a Year 3, Year 10, Year 13 student in this regard.

Micro Teaching

Goodness me, this seems like a theme from the past. But it is a wake-up call for teacher education. Teachers will become more effective when they develop their skills in a micro-teaching environment. This happens in some teacher education programmes but this begs the question of what happens with beginning and new teachers and too experienced teachers. Medicine wouldn’t risk it but we do in education. In at the deep-end depends entirely on whether you can swim or, perhaps even, whether you want to stay alive. Worse, while a lack of micro-teaching is not ideal for teachers, it is less satisfactory for students who clearly benefit in terms of achievement.

The challenge for schools is to organise so as to allow for micro-teaching.

Acceleration

When I got this bit I really could hear celestial choirs! At last, I thought, at last! The evidence is clear – that students should advance as quickly as they are able. But what do we do? Year by year, happy birthday by happy birthday, four seasons in a row, curriculum Year X to curriculum Year Y – the modern western education system is the myth of Sisyphus and we do untold damage to our best students. Perhaps we want to own them as slaves for our own gratification in the gladiatorial battle of assessment. 

Put more crudely and without the benefit of x studies and y meta-analyses – why do we restrict students to the lock-step system that is demonstrably in favour of teachers and schools. It is certainly not a reflection of what we know about learning. And NCEA gives us the ideal vehicle to achieve increased levels of acceleration.

So they are the top five factors (don’t forget that the book covers a further 133 factors that have varying degrees of impact on achievement.) Education research relies on such syntheses and analyses – there must be brokers who translate the outputs of research for the practitioner. They interpret, they generalise, they simplify, they turn a myriad of detail into a clear single command.

John Hattie is inarguably one of our best education researchers. This book should be read by each and every school. It should inform teacher education programmes. It should be an almanac of research evidence kept within reach and consulted.

Or we could ignore it and just carry on.

Leave a Comment

Tomorrow’s schools today

Published on February 6, 2009

Stuart Middleton
New Zealand Education Review
Vol. 14 No.4, 6 February 2009, p.16
APN Educational Media (NZ) Ltd.
Wellington

There are currently 28 schools in New Zealand where a commissioner has replaced the Board of Trustees for one reason or another. This is roughly 1% of schools so that puts the issue into perspective.

In 1998 it was the 10th anniversary of the Picot Report into the administration of education in New Zealand that led to Tomorrow’s Schools and the administrative system that we currently have. A conference was held that marked this end of the first decade and my contribution was a paper that I called Planting Cabbages and Expecting Cauliflowers : The Reforms and Multicultural Schools.

We mark the 20th birthday of the Tomorrow’s Schools approach with two high profile instances of school Boards being dismissed to be replaced by a Commissioner.

In the paper I wrote ten years ago, I focussed on the impact on the multicultural schools in my patch in Manukau and argued that the reforms had not in themselves produced administrative weaknesses but had made explicit weaknesses that had always been there masked by various elements of the previous system which I will come to shortly.

The Picot Report (Administering for Excellence) wrote about the problems faced by the administration of education in New Zealand. It was a report that spoke in generalities and at a macro level. It did not speak about our schools in any detail.

Where it did speak of “South Auckland” its comments were at best polite and at worst mealy-mouthed. Take for example, the comments on failure. Cutely called a form of “consumer disaffection”, the report (p36) referred to the 26% of pupils who leave school with no qualification of any kind. “We are told,” the report states “[that] these students leave school thoroughly disenchanted with a school environment they have merely endured, rather than enjoyed.”

20 years later and the rhetoric booms out a similar message. Where’s the action?

Curiously in other specific references to South Auckland the Picot Report created a new area called Southern Auckland. Perhaps this was a tactic to soften the negative impact of the media’s use of “South Auckland”! Whatever the reason it was noted that “half of the 26 secondary schools in Southern Auckland had more than 25 percent of the students leaving with no qualifications” and that “seven schools had more than 50% leaving with no qualifications.”

In stentorian tone the report thundered that “This clustering of failure is certain to lead to personal, social, and economic catastrophe. It cannot be allowed to continue!”

20 years later……. where’s the action?

Back in 1988 the report failed to mention such factors as the generally benevolent attitude of the Department of Education at a regional level which as a “near-centre” appeared to be more capable of understanding the situation than was the “remote-centre” (i.e. the national office). But there were other issues too: the way new schools were planned, difficulties with staffing secondary schools, the growth of communities that put challenges into a small number of schools that most of the rest of the country had little or no understanding of, the lack of capacity in some communities to support the schools with cash, the growth of formulaic approaches to resourcing, and so on. There were winners and losers in the imnplementation. For instance, the creation of the level playing field saw the very schools that needed benevolent staffing levels lose significant numbers of teachers. (It had been a favourite Department of Education regional office response to offer a bit of additional staffing to solve any issue but now the formula was written on a tablet of stone.)

The worst element of the reforms might have been the placing of the burden of governance onto community members with little experience of it and with little support to do it. Would the business community risk such an approach?

Now this issue is confused by the reaction to such a suggestion which righteously argues that local communities should have a voice, that the local community should be involved in the governance of their own schools and that governance should emerge from the streets and houses served by the school. Well of course all that should happen. But the distance between a group brought together to do this in some communities and the central administration is not only measured in terms of the kilometres to Wellington but also in terms of huge distances in experience, understanding, values and aspirations between these communities and a central administration.

The reform in New Zealand turned each school into a school district in USA terms and therein is a possible solution.

It might well be time to consider establishing some form of Local School District governance provision to work with a cluster of schools at a governance level and with authority to back up and support the school-based groups. Such groups would be appointed rather than elected and would ensure the balance of skill and experience that good governance demands. Each school would continue to elect its “governance group” that would contribute to the process and would be a special reflection of the precise community served by each school. But they would not serve alone – rather they would be backed up by skill sets and experience that local election processes might not necessarily produce.

Such Local School District Boards would be governance groups not administrative groups (albeit that one role of governance is to ensure sound administration of resources, processes and outcomes). A Local School District Board serving state early childhood, primary and secondary institutions in a defined area that has a community of interest would enhance the work of the local Boards of Trustees. It w also stop fractious behaviour and outside interference of the kind that has disrupted Boards in some instances.

How many decades should pass before the template put in place by Picot, Lange et. al. is modified? Will we simply continue to smugly believe that one size really does fit all?

T S Eliot in Choruses from The Rock tells us that the “end of all our exploring will be to arrive at the beginning and to know the place for the first time.” Covey said much the same – “the end is where you start from.” Do we know and understand where to start from in all this?

The implementation of the Tomorrow’s Schools reforms was flawed – schools are not generally free to purchase services and some are less free than others, Community Education Forums were not established, education service centres were not put in place, some schools will never be able to buy in additional resources, the contract between “the school and the state” is a very lop-sided arrangement.

It would be a brave assertion to claim that those reforms of the state sector in the 1980’s got it all right in one hit. And nowhere is this more open to question than in the governance of education.

Leave a Comment

The definition of success

Published on January 30, 2009

Stuart Middleton
New Zealand Education Review
Vol. 14 No.3, 30 January 2009, p.16
APN Educational Media (NZ) Ltd.
Wellington

A headline caught my eye – “As Talk about Retention Rises, Rates Drop.” There has been a clear increase in recent years of interest in addressing issues related to student retention in tertiary institutions and at long last around the world a shift was talking place it seemed. Talk of “success was replacing talk about “access”.

When tertiary systems were more elite than they are now, it seemed enough to say “Well you got your chance at tertiary and you didn’t take advantage of it.” The onus was clearly on the student to shape up.

But as tertiary education widened both the scope of the programmes offered and the range of students admitted into an increased range of education and training settings, interest shifted to the outcomes. Recognising that higher levels of qualification and increased levels of training were critically important to economic performance let alone transformation, retention in programmes was a goal of administrators, support staff in institutions and teaching staff.

But an issue with the discussion on retention is how to measure it and the United States does this by using the fact of a student returning to their institution, especially from the first to the second year, as evidence of retention. That is a crude and unsatisfactory measure. It fails to take account of students who are transferring to other institutions.

In New Zealand such a measure would also fail to take account of those doing courses of less than a year’s duration – and that is a huge number. 38% of tertiary students in New Zealand are enrolled at a certificate level which would be typically a programme of a duration equivalent to one year or less. 

What would be a better measure of “success”? Successful completion can be the only measure of success. I know that there is a wide array of elegant arguments that claim that students seek partial qualifications, that they are after other outcomes such as increased self esteem, that they are completing qualifications within unconventional timeframes and so on. These just don’t stand up.

If students want only partial qualifications then what is wrong with the qualifications being offered? Have they become obsolete? Do they not meet the needs of students and the settings into which they take these “qualifications? And why do I not see employment advertisements that state “successful applicants have probably done part of a diploma in whatever?

The self esteem argument can only be pursued by people with complete qualifications surely. When did you last hear a student say “I failed ABC and did not finish my Certificate in XYZ but I feel I am a much better person”? And if we have unique student identifiers why can’t we track students who change education and training providers, or work towards qualifications over a period of time?

Being able to state firmly that the successful completion of qualification is the only measure of student retention and success would be a mark of maturity in the tertiary education system.

The reports on retention must also take into account the greatly increased range of students entering a greatly widened variety of programmes of education and training now called tertiary. Reports of this percentage or that have to be careful to measure a similar sample. Remember that tertiary now even has programmes about tertiary. If you are going to admit a greater range of students then the very real issues of academic preparation have to be considered. All the evidence runs in support of the view that levels of academic preparation for postsecondary education and training has declined and continues to decline.

But there is good news in the latest retention reports we are told. Community Colleges in the United States have retained 53.7% of their students from Year 1 to Year 2. This is an increase from the low point of 51% in 2004.

You just have to ask yourself whether or not a system is working when having attracted 73% of a cohort of students a sector as important as the community colleges can only retain a little over half of them. (Remember that the community colleges are based on a two year programme offering two years qualifications.) This means that having entered an open access institution in the US, 40% of the cohort of students has left by the start of the second year!

The argument against using qualification completion as the measure of success in postsecondary education and training has usually been based on a fear that this could or even would lead quickly to designing funding arrangements based on it. This might not be a bad thing. Certainly providers that are doing a good job would be rewarded and those who are not would not. But it would also be simplistic because the hard yards done by tertiary providers are equally hard.

There is no argument that students entering a selective or elite institution require anything like the same degree of support required by those entering an open access situation for which they are insecurely equipped in academic terms. Different sections of the community require different levels of support. For instance, students who are first in family to enter tertiary education require a different level of commitment from an institution than do those who are not. And the statistics related to Maori and Pasifika performance at tertiary do not allow for any reduction in commitment and effort in the retention and success direction! Those who cater for greater numbers from those groups clearly have a greater load when it comes to support for retention and success.

So funding based on successful completion would have to take account of the different levels of support that are require and this is probably too hard for officials to contemplate and for systems to devise. There would also be vigorous arguments against differential funding based on student group just as there has been over the years for the retention of the total EFTS base for calculating equity funding. 

I am excited by the international trends in all these issues. It suggests that they are reflective of certain kinds of education systems and certain policy settings rather than a bad reflection of this country or that. It also might mean that approaches and solution found in one system might have relevance for progress here.

Surely it is not beyond the US, UK, Australia, Canada and New Zealand to collectively develop pathways which lead to improvement. But are we talking with each other?

Leave a Comment

The answers are obvious

Published on January 23, 2009

Stuart Middleton
New Zealand Education Review
Vol. 14 No.2, 23 January 2009, p.16
APN Educational Media (NZ) Ltd.
Wellington

The 1960 Master Plan for postsecondary education in California confirmed the place of the community college in creating pathways that young Californians could trek in the fulfilment of the American Dream – a college education. These postsecondary institutions are somewhat akin to the New Zealand polytechnic but offer a wider range of courses that includes a rich range of personal enrichment courses. Qualifications that theses courses lead to are usually the Associate Degree which allows students to transfer to a university or a General Diploma in Education.

Putting aside the issues of secondary school disengagement – an issue that bedevils the USA as much as it does us here in New Zealand – those who proceed from high school to an undergraduate tertiary programme in a postsecondary institution in California head in three directions. The University of California system (Berkeley, Davis etc) accepts 9% of the undergraduates, 18% are in the State University system (Sacremento, Monterey, etc) while 73% are in the community colleges. So the split is 27% into the universities and 73% into the community college system.

This compares to the picture of New Zealand undergraduates of 49% in the universities and 51% in the polytechnics and wananga. So we have a higher proportion of undergraduates in the university system which could be a reflection of the absence of any historical transfer track between the parts of our system. And we have a much higher proportion of students who enter Private Training Providers than do overseas systems.

All western education systems are struggling with a clear demand for increased numbers of graduates and the persistent difficulty of achieving this through the simple increase in participation remains a puzzle.

California – like New Zealand, Australia, Canada, the USA generally and the UK – has come to realise that the key to this issue is to get increased numbers of graduates from its community college system. This is based on a recognition that if gains are to be made, they will be achieved in the community college system because it is in the non-university tertiary sector that the required scale of increased achievement among traditionally underserved groups will be won.

But a recent report[1] from  the Institute for Higher Education Leadership and Policy at CSU Sacramento, hit the nail on the head with a major chapter heading We Know What Works But We Don’t Do It. There is a major industry on student retention and success in the USA supported internationally that has clarified clear actions that can easily be taken that will increase student success in undergraduate programmmes. Some of the agreed actions and responses we should be pushing in undergraduate programmes are promoted by this report and come as no surprise.

  • Make sure that students’ are ready for tertiary study. .

 A clear outcome of schooling should be preparation for tertiary education and training. But does school curriculum reflect this? Is there a shared understanding between educators at secondary and tertiary levels as to what readiness would mean?

  • Provide conditions that allow students to achieve success right from the beginning of the programme.

 Enrolling students in the right classes at the beginning of the year is a good place to start. And providing remediation in basic skill areas where it is needed. But do we really take the care we should with enrolment? Do we know clearly whether a student has basic skill weaknesses when they start or do we wait for it to become apparent?

  • Provide clear educational goals and pathways.

 Fortunately we are seeing some progress on individual education plans and personal pathway plans and such approaches that will help.

  • Make clear to students that the best study options are to study full-time and continuously.

The gap year – both the intentional of the moneyed and the accidental of the struggling student – is disruptive and will not increase success. And once they have started they should carry on without breaks. This is easier for the young than for the older after responsibilities have come along that compete for time.

  • Provide intensive support for students.

 This is, as they say, a no-brainer!

The report goes on to highlight other aspects in areas such as finance and institutional culture. For instance in finance, it notes, institutions that have multiple missions created by their relatively heavier load and more demanding tasks of catering for higher numbers of under-prepared students are simply not funded adequately to do the work.

And that old chestnut – funding systems reward enrolment but not success. Are the changed approaches in tertiary funding being worked through in New Zealand going to achieve what the Californian system has not been able to?

But perhaps the biggest challenge in the report is the authors’ assertion that “entrenched assumptions prevent considerations of new approaches” and that “there is a disinclination to consider policy change as the system seeks to improve student success.” 

The world has changed and the communities that we now serve have changed. The challenges of providing success in tertiary education have also changed. Simply increasing participation will not be enough. That must be matched with increased student success and the base on which this increase will be achieved is in the work of the secondary school and in the parts of the tertiary system that serve the under-served communities which represent the future.

If New Zealand does not respond to issues of access to and success in postsecondary qualifications that are industry recognised for these under-served groups then talk of economic transformation is plain nonsense.

In the Californian community college system 14.5% of students succeed in completing a certificate or associate degree or transfer (to a university). Comparisons are difficult, New Zealand Polytechnics offer a wider range of qualifications than does a Californian community college but certainly the comparable figure is under 50% and may even be as low as 40% for fulltime students,

Do we wait until we start getting the levels of achievement in undergraduate postsecondary education that our colleagues in California are faced with before we respond? As they say: “We know what works but we don’t do it.”

[1] Shulock, Moore, Offenstein, Kirlin (2008) It Could Happen: Unleashing the Potential of California’s Community Colleges to help Students Succeed and California Thrive, Sacramento, IHRELP , USC Sacramento

Leave a Comment

Never Mind the bullocks

Published on January 16, 2009

Stuart Middleton
New Zealand Education Review
Vol. 14 No.1, 16 January 2009, p.16
APN Educational Media (NZ) Ltd.
Wellington

At last, the year of the horned, cloven-footed ruminant, the Ox no less. We are told that those who belong to the Year of the Ox are patient, speak little, are excellent organisers and inspire confidence in others. Who would have thought that one animal could have been responsible for so many leaders in education!

Any new year is a time of wondering what will prevail this year – delight or despair? Well early indications are a little confusing.

First there is the death of Pakaarriki Harrison late in December.

There is delight in the thought that one person can bring together knowledge and skill as a tohunga whakairo to such a high level in ways that leave communities all that much richer. The story of the master carver who completed a series of great carved wharenui is well documented in the recent biography by Dr Ranginui Walker (Tohunga Whakairo: Paki Harrison) which was a delightful read just before Christmas. Harrison leaves us houses such as Nga Kete Wananga (Manukau Institute of Technology), Whaiora (Otara), Te Otawhao (Te Awamutu College), Tanenuiarangi (University of Auckland) and others such as those at Manurewa, St Stephens School, Kennedy Bay, Edgewater College. The houses are a place in which communities find expression of their histories, their beliefs and their people. They are also places where talent flourishes – whaikorero, kapahaka and art.

Paki Harrison was a teacher and like so many teachers was also greatly talented in other areas. These houses are the artefacts of this great New Zealander. The Walker biography tells the tale of a man who was humble, who learned his craft over a lifetime and whose legacy is in New Zealand terms very significant.

So where was the despair? The pakeha media, over-stuffed on the holiday diet of the trivial and the trite that typifies summer newspapers and TV, ignored all this. One of New Zealand’s real taonga is now gone but lives on in such a forceful way in many educational institutions and communities. But who knows? The communities know, te Ao Maori knows, but mainstream media simply shared their silent ignorance of such matters. While our major newspapers continue their obsession with Jesse Ryder, Sione Lauaki and Paris Hilton, communities mourn the loss of this great New Zealander.

But it was a different story with the anniversary of last January’s death of another great New Zealander – Sir Edmund Hillary. This occasion received appropriate attention and there was some gentle coverage that allowed us to recall the special days a year ago when the country came together in a way so seldom seen. It was a delight to be able to recall that time in calmness. A single artefact, the Hillary ice-axe, seemed to symbolise everything that this great New Zealander achieved.

So where was the despair? A weekend paper thought it clever – they surely never thought it appropriate – to ask the full page question Who is the next Hillary? And they had a parade of minor “celebrities” as part of a tasteless and rather stupid “competition” asking members of the public to nominate the “next Hillary.”

Does it not occur to the newspaper that one doesn’t elect a Hillary? That finding a Hillary isn’t something like those summer competitions of the past such as the Mt Maunganui Beauty Pageant or Miss Whangamata? Thankfully stronger and more sustainable values have driven those oglefests to one side. But now we have the competition to nominate the next Hillary!

People like Hillary emerge as a result of their deeds over a period of time. They frequently build their fame on a solid block of humility and even some obscurity. They achieve greatness rather than have it conferred on them by newspapers and weekly magazines. It is simply an insult to parade a collection of people and ask the question. We will know when someone with the mana and achievement of a Hillary comes along.

I wonder if in both the above instances there is an issue with our attitude towards death. I received a card a few days before Christmas that read: “Our thoughts are with you at this time of loss and even though X is no longer with you we hope that you will have the memories and thoughts of his long life to help you through this time.”  This card, handwritten and no doubt sincerely meant, was referring to the death of my cat, a 17 year old fellow who simply ran out of steam. Dr A.Vet, BVSci. seemingly follows up on the death of an animal with these responses. This anthropomorphic response was not only unnecessary, it is also more than a little demeaning to the memory of our fellow humans who have passed on. It is the thin edge of a wedge that allows the careless and cavalier responses mentioned above.

George Orwell wrote about the degradation of thinking that resulted from the frivolous use of language and he argued strongly that sloppy use of language encouraged sloppy thinking. Clear and precise thinking was, in his view, at the heart of communities that were strong intellectually.

So the current overuse of “loved ones” to refer to the living rather than the dead, encourages confused thinking. This also robs us of a useful cliché for referring to the dead in those times of stress when those who are loved “pass away”. Such clichés are the lifejackets we need to survive the rough seas of sadness. To not treat dead people with great respect (ignoring the death of great New Zealanders is at best disrespectful while running trivial competitions is at worst offensive) is a serious comment on the values we hold as a community.

So it is the Year of the Ox.  They tend, however, to be eccentric, and bigoted, and they anger easily.

Leave a Comment