Randomized evaluation of policies to create local accountability under free primary education in Kenya and Uganda

RESEARCH QUESTION

How can local monitoring institutions be harnessed to improve education quality under free primary education, where outside funding may undermine local ownership of schools and where communities are limited in their ability to raise funds for schools themselves?

PROJECT

The introduction of free primary education in Kenya and Uganda, in 1997 and 2003 respectively, has led to dramatic changes.  In Kenya, enrolments have risen by 30% over the past 4 years.  In Uganda, a 73% increase in gross enrolment ratios were observed in the first year alone.  However, this unprecedented and rapid expansion of the education sector has sparked concerns that the quality of education provision may be declining.  While teacher-student ratios have recovered to pre-1997 levels in Uganda, primary completion rates have declined in recent years.  To address this apparent decline in educational quality, the CSAE, together with Professor Germano Mwabu, Professor Mwangi Kimenyi, Lawrence Bategeka, Dr Frederick Mugisha, EPRC, have launched a research project in collaboration with the Ministries of Education in each country.  The projects in each country will assess the current level of educational achievement and design and evaluate policy interventions to improve educational outcomes.

Extensive consultations with these Ministries and focus group interviews in schools in both countries have identified a number of potential interventions with the potential to improve the delivery of education services.  In both countries, tools to strengthen management and monitoring through existing institutions at the community level – the School Management Committees (SMCs) – are being developed.  The proposed Community-Based Monitoring System (CBMS) is designed around the use of a school scorecard, to be completed by SMCs as a means of providing measures of performance both to parents and to District Education Officers (DEOs).  Testing of alternative designs for the CBMS will be explored as a way to assess the effectiveness of “top-down” versus “bottom-up” routes to accountability in school management.  In Uganda, unique data from a behavioural economics experiment will yield insight into the nature of social interaction in SMCs, in order to explain the scope and limitations of community-based monitoring.

In addition, a second intervention is being considered in each study country.  In Kenya, this will assist communities in hiring local contract teachers to address the shortage of centrally-hired teachers from the national Teacher Service Commission (TSC).  Currently, some schools with an insufficient number of TSC teachers manage to raise funds locally on the initiative of the SMC. For a variety of reasons – socio-economic status of the community, poor school management – many schools fail to fill teacher vacancies via this route and have empty classrooms. By offering partial funding for these initiatives from the Ministry and requiring local co-funding, the intervention attempts to design a mechanism which delivers external funds without undermining local accountability.  In Uganda, incentive systems that recognize the power of positive incentives – matching grants, prizes, and other forms of recognition – are being discussed to address the inability of schools to raise funds to provide meals for students.  In a context in which the absence of school fees is a critical condition for broad-based access to education, the use of positive incentives for fundraising (rather than the threat of excluding children who do not pay fees from school) may provide an equitable and cost-effective means to meet school needs.  Because local communities provide co-funding under these interventions, it is hoped that they will take greater ownership of school management.

A workshop was held in Kampala in May 2008, in conjunction with the Ministry of Education and Sports, to train District Education Officers in four project districts in the concepts both of policy evaluation and in the concepts of community-based monitoring.  This has resulted in the participatory design of a scorecard system to be implemented in these districts.  A workshop is currently being planned in Nairobi to train District Education Officers (DEO) from around Kenya in the principles of CBMS, who will then be charged with training SMC members in use of the scorecards.
 
The evaluation of these policy interventions will be based on a randomized controlled trial.  In Kenya the experimental group will comprise approximately 196 schools spanning multiple districts; in Uganda the comparable figure is approximately 100 schools. The studies will measure impacts by comparing differences across groups in outcomes of student and institutional performance before the intervention and up to one year after the start of the intervention.  The primary outcome of interest is student achievement, as measured by performance on national exams as well as literacy and numeracy exams, developed in collaboration with the Kenya Institute of Education (KIE), which formulates the national curriculum, and drawn in Uganda from the Uganda National Education Board’s National Assessment of Progress in Education testing system.  Secondary outcomes of interest include changes in rates of enrolment, absenteeism, and transition to secondary school. 

In Kenya we are currently finalizing the interventions and baseline data collation with the Ministry of Education and have scheduled the implementation to begin in September 2009. The next four months will be spent recruiting and training project staff. In addition we plan to hold further meetings to inform and sensitize community stakeholders.  In Uganda draft baseline survey instruments have been piloted and data collection is expected to begin in June 2008, with intervention implementation to follow. 

Our Kenyan research team has received approval from the Kenyan Minister and Permanent Secretary of Education and Minister of Planning to access a variety of administrative data sources including: the Education Management Information System (EMIS); Kenya Certificate of Primary Education (KCPE) exam data; Southern and Eastern African Consortium to Monitor Education Quality (SACMEQ) II; the Kenya Integrated Household Budget Survey (KIHBS). We have collated this data, which will serve as a baseline for the interventions.  Our Ugandan research team has conducted preliminary analysis of existing Ministry of Education and Sports administrative data to inform intervention design and targeting, and will collect purpose-designed survey data as a baseline for the interventions.