ThinkEd: Counting the education beans

Stuart Middleton

EdTalkNZ

27 August 2010

Knowing what is happening in education requires us to “tell the story” with some accuracy. But that is not easy. We end up having to apply statistics and measures and known performance measures at different point. This shows us the trend and the pattern but it is also something of a fiction because of our inability to actually talk about what happens to real people as they progress through.

The development of the unique student identifier, a number given to a student on entry into ther education system that they carry with them right through their schooling and even into the post-school education and training will be an enormous help. For it will enable cohort stories to develop. What happens to 100 real children who enter the education system?

Many of the current measures probably overstate success. It happens like this. One hundred students start school. Let’s say that 70% get NCEA Level 2 – this is probably too high but will serve this example. We then go on to talk about completion rates in tertiary and perhaps they are reported as 75%. This is probably too high and it is important to know what completion means – course completion or qualification completion.

The 75% tertiary figure is actually 75% of the 70% who succeeded at school and went on to study further. Of the young people who started school this means that 53% were successful. True reporting od educational success should be on a system basis not on parts of it. It gets back to the dis-integration of the education system into sectors. The true story would be the whole story.

The issue of course completion against qualification completion is also an interesting issue. New Zealand has a large number of people who at tertiary are successful in passing courses but who never complete qualifications. We seek comfort by claiming that “they” don’t want the qualification or that “they” only come for specific purposes. These are simply untested assumptions. I cannot for the life of me see why someone would set out to get an incomplete qualification.

There could be other explanations of the incomplete qualification phenomenon. Perhaps the length of programmes is simply too long and students, particularly part-time students, have to get on with other things in their lives. Perhaps it is students in employment who are disproportionately represented in this group. What is the gender split of this group? The ethnic split? We simply don’t know. The untested assumptions that we use tend to be ones which seek explanations not on what we do but what “they” do.

For my money, qualification completion trumps everything else.

It also seems to me that our method of collecting information about the life-long impact of education is also self-serving. For instance, take the Bachelor of Business as a qualification. Again using fictitious numbers – 100 students enrol in the Bachelor of Business degree programme and about 60 complete it. Do we know the proportion of successful graduates who are able to make appropriate use of this qualification by entering qualification related employment? No. Do we measure the worth of a B.Bus. across the entire graduating class of 60? No. We look at the returns to people working in the sorts of jobs that some of the 60 will do.

This is because we have no mechanism for matching the graduating student to the earnings performance of workers in employment. We could do this if we would need to have a unique student identifier that also became a unique worker identifier. Well, we already have such a number, the IRD Number. Think of what we would actually know about education and its value of students were given their IRD number at birth and this was used to track performance both of the individual and the education system.

We would know a lot more about access and equity for a start. We would know about performance of cohorts of various kinds. We would know about the return on investment of different qualifications much more precisely. We would know more about the career and education trajectory of different qualifications. In other words we wouldn’t have to rely on untested assumptions and guesses.

Already I hear the sounds of protests developing – human rights, the invasive state, the “they already know too much” brigade gathers. I have never quite understood such protests. Knowledge of these things would be able to encourage performance of systems that are resource hungry and which seem to never quite satisfy. There is simply too much nystery about what is happening.

Years ago I knew a fellow who had an intriguing approach to identity. If I phoned him at home I used what I believed to be his real name – “Is Tom there please?” – a name that also reached him at one of his jobs. But he had two jobs and at the other one I had to say – “Is Harry there please?”  To this day I am not sure whether either of the names was his birth name.

 Tom / Harry would not have wanted a robust unique student or worker indentifier.

Read or add to the 3 comments

  1. Phil Ker says:

    Hi Stuart,
    I agree with you that we should be trying to understand whole of system performance, and to get sensible data that allows us to do so. I am not one who objects to using unique identifiers for such purposes.
    But if we cannot get a whole systems approach, let’s at least get it right within sub systems. The tertiary sector is a case in point. We have the means through a unique identifier mechanism already in place to track cohort performance – particularly of qualification completions. Yet TEC in its wisdom has dropped cohort measurement for a highly flawed EFTS calculation. All in the interests of simplicity! And herein lies the current problem – we have officials, probably doing the bidding of politicians, taking simplistic views of complex phenomena. Worse, they will dish their findings up to a public that does not know any better.
    BTW, there definitely are people who enrol in courses and who do not want to complete the qualification in which that course is located. We see it all the time in health where already qualified people take courses as professional development. We also see people “tasting” a vocational pathway, and coming to the conclusion that it is not a pathway for them. Both of these are good things for people to do. Of course, we could be vigilant to ensure that such dabblers used the vehicle called “Certificate of Proficiency” – which we do, if we see what is going on before it is too late!
    Cheers
    Phil

  2. Sandy Barnett says:

    I totally agree. In tertiary education at least we have been concentrating for years on course completion statistics as relates to individual papers (I will call them this old fashioned term for sake of clarity) but we have no idea about programme /qualification completion rates particularly completion of longer more complex qualifications such as degrees. This gives us a completely erroneous picture of what is going on.

    For instance a student ‘A’ could do a couple of semesters on a degree, pass all papers with C s, and leave without a qualification because they either
    don’t like it
    or are bored with the course or teaching,
    or don’t think it is relevant or useful,
    or get distracted by other things in life
    or don’t think they can manage to pass the higher level papers
    or maybe think the standards are too low
    etc.

    and even may be completely put off education as a result.

    This outcome is seen as a 100% success by our present measurements. despite the waste of time money to student and the taxpayer.

    Student ‘B’ could complete the whole degree, having had to re-sit a few papers during the process particularly at the beginning, (say 5 out of the total 24) , become totally engaged with the teaching and learning , get straight A’s in their final year, get the degree qualification in about 3 1/2 years and then get a good job.

    By our present statistical measurement this student would be contributing to the total failure rate on that qualification i.e. less than 75% success rate.

  3. Sandy Barnett says:

    I totally agree. In tertiary education at least we have been concentrating for years on course completion statistics as relates to individual papers (I will call them this old fashioned term for sake of clarity) but we have no idea about programme /qualification completion rates particularly completion of longer more complex qualifications such as degrees. This gives us a completely erroneous picture of what is going on.

    For instance a student ‘A’ could do a couple of semesters on a degree, pass all papers with C s, and leave without a qualification because they either
    don’t like it
    or are bored with the course or teaching,
    or don’t think it is relevant or useful,
    or get distracted by other things in life
    or don’t think they can manage to pass the higher level papers
    or maybe think the standards are too low
    etc.

    and even may be completely put off education as a result.

    This outcome is seen as a 100% success by our present measurements, despite the waste of time money to student and the taxpayer.

    Student ‘B’ could complete the whole degree, having had to re-sit a few papers during the process particularly at the beginning, (say 5 out of the total 24) , become totally engaged with the teaching and learning , get straight A’s in their final year, get the degree qualification in about 3 1/2 years and then get a good job.

    By our present statistical measurement this student would be contributing to the total failure rate on that qualification i.e. less than 75% success rate.

Leave a Reply to Sandy Barnett Cancel reply

Your email address will not be published. Required fields are marked *