8 Responses

  1. Kimani
    Kimani June 27, 2016 at 7:34 pm

    The concept of sustainability is very critical and its more than environmental though many still believe environmental sustainability is still key to development and growth.

  2. Geoff Sanderson
    Geoff Sanderson June 13, 2016 at 3:14 pm

    As Simon reminds us above, the sustainability of behavioural and organisational change is difficult to achieve. That is true whether we are working in Australia or in Indonesia. However, I think that there are some approaches that can increase the chance of success.
    The first is for sustainability to be systematically addressed in designs and annual plans, and be a focus of monitoring and evaluation. Leaving the serious thinking on sustainability to the exit report stage (as happens) is too late.
    Some people take the view that an activity will be sustainable just because it aligns with the recipient government’s priorities, and has a senior-level steering committee. If that was the case, every planned educational change in Australia would have been sustainable and that has not been the case. Local ownership is necessary but not sufficient; and ownership at Secretary General level is not the same as ownership at district level.
    AusGuideline 6.4 Promoting Practical Sustainability may be dated (2005), but it remains a useful and relevant guide and checklist for planning, management and monitoring.
    The second factor is for a gradual handover of financing. It is common for donor financing to continue at 100% until the last day, then stop. It would not be surprising if the activity gradually disappeared into the machinations of the budget cycle. A better approach would be for a staged withdrawal that has the recipient government funding some activities, but still with ongoing support. (Bob has noted the need for this above.) This stage would occur once benefits start to become clear and the credibility of the activity is established – not less than 3 years. Dan notes the need for time for the development to occur. Another reason for time, and flexibility, is for cultural issues to emerge and be addressed. It is unlikely that they can all be identified at the design stage.
    Finally, we are concerned about sustainaining organisational change. There is a considerable body of management literature on the management of change, which people working in development could be utilising -and change models are consistent with the sustainability principles. For example, a plan to develop ‘champions of change’ is a plan to develop local ownership. Perhaps a design could be specifically structured as change model; and perhaps the most important technical assistance may be on the management of change, rather than, for example, on school management.

  3. Mark Heyward
    Mark Heyward May 27, 2016 at 2:00 pm

    Bob, many thanks for this insightful posting – and for raising some key issues.

    The key point you make, for me, is the following: “With more than 40 years of Indonesian development experience, it is reasonable to enquire why we do not have stronger evidence of the actual sustainability of benefits from aid to education. Why have the ten donors in the analysis, who have provided over USD 4.5 billion in assistance since 1973, been so incurious and for so long about sustainability arising from this massive investment to support Indonesian education?”

    What has this money achieved? An optimistic response will point to the huge gains in access. According to Indonesian Government data, 81 per cent of children aged 6 to 18 are now enrolled in school (net enrolment rate). When children (or adults) outside the specified age ranges are included (gross enrolment rate), the percentage increases to 91 per cent. Gross enrolment in primary school is 109%, in junior-secondary, 101%. Participation in early childhood has reached about 70% (a massive increase over the last decade) and in senior secondary 76%. These are significant achievements for a young nation which at the time of independence provided schooling to less than six per cent of its citizens.

    Dropout rates have reduced dramatically. In 2011-12, just one per cent of children enrolled did not complete primary school, with over 95 per cent continuing on to junior secondary schooling. The dropout rate for junior-secondary is 1.7%. While still an issue in remote areas, attendance rates (for both teachers and children) are also improving. Gender parity has largely been achieved. Adult literacy is around 95%.

    But when we look for gains in quality and in learning outcomes, the story is rather different. While we might argue about the validity and relevance of international tests, there do tell us something. Indonesia ranked 64 out of 65 participating countries in the last PISA test (2012). The country’s ranking dropped from 57 in 2009. TIMSS and PIRLS rankings are similar – and, similarly, have declined over the years of testing.

    After nearly 25 years working in education in Indonesia, a question I often ask myself is, if we already know what works (as we sometimes claim), then why are we still here, and why are doing pretty much the same thing, after 30-40 years? Why hasn’t active learning become mainstream? Why isn’t school-based management working as effectively as it could be? Why are the management and governance of basic education still mired by corruption and inefficiency?

    The answer you offer is a good one: We need to “approach educational development with a better balance of cultural, political and technical realities in both design and implementation.”

    This is well demonstrated in the studies referred to in the post. It is the key to Guthrie’s argument. And I think it could be the key to the future in Indonesia. We know that Indonesian teachers can ‘get it’. They are just as capable as any of implementing active learning approaches in their classrooms, for example. But how many actually sustain the changes beyond the life of a project? I am not convinced that many of the changes are sustained in any real way. Sustained change requires changes in cultural perspectives (beliefs, norms, relationships, roles, expectations …). And this is unlikely to be achieved with a few cascade training events – even when followed up with coaching.

    Donors also need to work in the political context in order to get sustained change. People behave as they do (in the classroom, the office or the community) because they are motivated to do so by complex sets of incentives and disincentives. If we continue to ignore this and assume that teachers will change their practice (despite the technical and cultural hurdles) – with no real incentive to do so, we are surely kidding ourselves! I don’t just mean financial incentives, of course. But why would a teacher adopt an active learning approach if it (1) takes a lot more effort, (2) earns disfavor from colleagues, parents and supervisors who don’t understand or appreciate the approach, (3) confuses kids, sometimes making them unruly and disruptive in the short term, (4) does not help them get better exam scores (the only vaguely objective measure of their performance), and (5) does not advance their career?

    And here, perhaps is the key thing. In order to work in the political and cultural dimensions, foreign assistance programs must be far more integrated and ‘owned’ by local partners. On a slightly optimistic note, I do see that this is increasingly the case. Foreign consultants, like me, are a very small part of the picture these days. We bring something that I think is still needed – a certain broader perspective, a kind of objectivity, a proficiency in English and familiarity with international standards, rigorous thinking and so forth that is still needed – but it is only a small part of the equation.

    But the real problem, highlighted in this post, is that we really don’t know for sure. We don’t know what impact all those programs, all that funding, has had. We really don’t know what works, from a technical, cultural and political perspective.

  4. Christopher Knapper
    Christopher Knapper May 25, 2016 at 11:36 pm

    Bob, I’m struck by the bridge analogy, and wonder if the original project goal was to replace the old bridge or supplement it (to provide the two-lane highway). What I’m getting at here is that the outcomes of a project might not always (or often) be the ones the funder planned for, and in some cases there may be unanticipated changes, both for better or for worse. This suggests to me that when we revisit projects to evaluate sustainability we cannot simply rely on quantitative indicators devised to measure predicted outcomes from the project proposal — we need to be much more flexible and insightful. On the whole, my impression is that revisiting projects after completion, even very expensive projects, is the exception rather than the rule, and that such ex-post-facto evaluations are not done well or over sufficient time to judge sustainability with any sophistication. I used to teach planners and architects, and there is a well-know concept in the field known as post-occupancy evaluation, which involves visiting built structures some time after completion to compare the designers’ conceptions of how the space would be used with the way that the occupants in fact use it. Such evaluations are in fact extremely rare, and most architects regard them with indifference. Hmmm.

  5. Simon Payne
    Simon Payne May 24, 2016 at 1:40 pm

    Bob, thanks for a thought provoking post. It makes me wonder if we have really grasped how to even think about this issue properly.

    Just as environmental sustainability has a specific meaning other types of projects and programs may need to adopt different concepts of sustainability. The author contrasts the evident sustainability of a bridge project with the harder to demonstrate and lower reported levels of sustainability of education projects. But couldn’t this be because we are again talking about different types of sustainability?

    In one sense sustainability is intrinsic to a bridge. It fulfils its purpose by not falling down. Once built a bridge should continue to be a bridge for at least a minimum projected lifetime and we should regard it as a failure if it were not to do so long past the aid project that built it.

    On the other hand, many aid projects, including the education projects you mention, are about engendering some kind of behavioural change. Surely we cannot expect behavioural change to have the same sort of sustainability as a bridge particularly as we are dealing with human beings and the myriad of unpredictable influences that impinge on their lives. Surely this kind of sustainability, if it is legitimate to seek it, is of a different order and needs different tools to understand and measure it.

    1. Robert Cannon
      Robert Cannon May 25, 2016 at 10:24 am

      Thanks Simon. Yes, I agree it is an issue that has not been given enough thought. Neglect might be a more accurate description. Your question goes to the heart of the problem with sustainability. As your observations imply, it is a complex matter and I remain to be convinced that the idea of sustainability is necessarily the most helpful concept here, particularly when the word seems to be used increasingly as a ‘feel good’ slogan in so many contexts now. Maybe continuing benefit or continuing impact?

      The central idea of sustainability in projects of all kinds has been the continuation of benefits which, in the ideal situation, will be clearly and realistically defined in the project design. I wonder if they are, especially in education. The behavioural change you refer to in education should, ideally, be strong enough to survive most of the unpredictable influences and support continuing change and improvement and not be limited to the implementation of (say) one unchanging teaching method or one specific management tool. Unlike bridges, an additional complication in education is that many projects operate at several interlocking levels and, further, can also include physical infrastructure such as school buildings.

      All this leads to the difficult matter of how you measure sustainability. We certainly need to do much better than a lot of past practice which has been little more than a quick judgement resulting from a very limited sample of cases (districts, schools, teachers) that have received assistance. Another curiosity is the contrast between the attention given to evaluating sustainability, which has been very limited, with the attention given to impact evaluation.

      As you correctly observe, it seems we have not yet grasped how to think about the issues properly. Curious!

  6. Dan Moulton
    Dan Moulton May 24, 2016 at 11:00 am

    Bob, Thank you for raising this important issue. I think all the points you raise are worthy of consideration for future development assistance for education. In my experience one of the ways to improve sustainability is to provide direct technical assistance to those who need to produce something. This often means sitting with the persons responsible for a product over relatively long periods of time introducing more effective and efficient tools that allows them to perform a required task. The skills the counterparts receive through this technical assistance methodology become well ingrained and longer lasting and thus more sustainable compared with one off training.

    1. Robert Cannon
      Robert Cannon May 25, 2016 at 10:34 am

      Yes, one of the saddest phases of projects is finalisation when so many enthusiastic people ask where continuing support, of even the most limited technical kind, will come from. Donors and governments alike are generally unwilling or unable to provide access to continuing help of this kind. Short term and one-off training is one of the most ineffective ways to bring about sustainable change.

Leave a Reply