Words: 3064
The tutors test whether schools that received early schoolmates management interventions (training in school based management and direct funding for school-based reforms) attained higher average test scores than those that did not receive such inputs. The analysis uses schooled overall composite test scores (comprising all subject areas tested) and test scores in three separate subject areas: English, math, and science. Their preferred estimator, difference-in-difference with propensity score matching, shows that the average treatment effect of participation in school-based management was higher by 1. Percentage mints for overall composite scores, 1. 2 percentage points for math scores, 1. 4 percentage points for English scores, and 1. 8 percentage points for science scores. These results suggest that the introduction of school- based management had a statistically significant, albeit small, overall positive effect on average school-level test scores in 23 school districts in the Philippines. The paper provides a first glimpse of the potential for school-based management in an East Asian context based on available administrative data.
The authors suggest that the next order of research is to answer policy-related questions guarding the reforms: what aspects of the reform lead to desired results; are there differential effects across subpopulations; and what are the potential downsides to the reforms? The Philippines is embarking on a nationwide implementation of school-based management and the authors recommend that mechanisms for rigorous evaluations be advanced simultaneously.
Such evaluations should not only provide more accurate estimates of the effectiveness of the reforms, but also help answer policyholder questions regarding design and implementation of those reforms in different socio- cultural contexts. This paper?a joint product of East Asia Education Sector Unit, Independent Evaluation Group; and the World Bank Institute?of a broader Bank-wide effort to promote rigorous and analytical approaches to monitoring and evaluation. Policy Research Working Papers are also posted on the Web at http://con. Rollback. Org. The author may be contacted at nkhattri@worldbank. Org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the reservations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors.
They do not necessarily represent the views Of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team The Effects of School-based Management in the Philippines: Indri Coattail Keywords: school-based management, decentralization, evaluation, repression score matching, administrative data Indri Chatter, Senior Evaluation Officer, Independent Evaluation Group, The World Bank, Washington D.
C. Email: nkhattri@worldbank. Org; Christina Ling, Evaluation Officer, World Bank Institute Evaluation Group, The World Bank, Washington D. C. Email: cling@worldbank. Org; Sherries Shah, Adjunct Professor, The Will Family Department of Politics, New York University, New York. Email: sj46@nyu. Dude. We would like to thank the Department of Education of the Philippines and Professor H. Unique, University of the Philippines, for providing the data and information on SUB.
The authors also gratefully acknowledge the support of Dingdong Ho (Senior Operations Officer, PEAPOD, and Task Team Leader for the Third Elementary Education Project), Lunette Deal Cruz Perez (Senior Operations Officer, SASHES), Christopher Thomas (Manager, PEAPOD), and Eduardo Velvet Bustiest (SASHES). This work was partially supported by the Education Program Development Fund. We are indebted our reviewers for providing helpful comments and suggestions on previous drafts. All remaining errors are ours alone.
The opinions and conclusions expressed in this paper do not necessarily reflect those of the Executive Directors and officials of the World Bank or of its member governments. 1. Introduction Decentralization is a key feature of institutional reform throughout the world. 2 The main argument underpinning decentralization policies is that they empower people to be part of the local decision-making process -they improve government performance by alleviating information asymmetries and costs by bringing decision-making closer to the people concerned.
However, decentralization can also worsen the provision Of public goods in the presence of externalities, lack of technical capabilities by local overspent, or capture of lower-level administration by local elites-3 In the context of the education sector, decentralization typically includes one or more of the following features: decentralized revenue generation, curriculum design, school administration, and teacher hiring and management. Decommissioning authority for these types of functions is devolved to regional/ municipal governments or to schools themselves.
The policy of allowing schools autonomy in decisions in these areas is referred to as schoolmates management (SUB), school based governance, or school self management,5. Responsibility and decision-making over different types of school operations are transferred to individuals at the school level, who in turn must conform within a set of centrally or state-level determined policies. The popularity of SUB is evidenced by the large number of development agencies promoting it as a key component of the decentralization reforms and the growing number of countries that have adopted aspects of this approach.
SUB reforms began in the 1 9705 in Australia. Since then, a wide range of countries have experimented with or introduced SUB in all regions f the world, including Hong Kong (China), Indonesia, El Salvador, Nicaragua, Kenya, Gorky Republic, Nepal, Paraguay and Mexico. 6 Nevertheless, the impact of SUB on education quality, including student outcomes, remains a contentious issue, with some researchers arguing that SUB leads to enhanced educational outcomes, 7 while others contending that SUB leads to the deterioration of educational quality especially among the weakest schools. The range of SUB approaches and the contexts in which they are implemented makes the debate about SUB quality an intricate one. The evaluation of SUB is complicated by the diversity of approaches to and elements of decentralization that collectively constitute “SUB” and by the institutional and socio-cultural contexts in which they are implemented. Nonetheless, some studies in recent years have found that SUB reforms are associated with improved education outcomes and processes (e. . , 2 King and Osler, 1998. Giuliani and Crossroads, 2001 4 De Garage, 2004 5 Greeter, Patrons, and Coding, 2006 6 see “What is SUB? ” world Bank (2007) 7 e. G. , Greeter, Patrons, and Coding, 2006 8 Bernhard, 2002. 3 2 Sofia & Shapiro, 2006; Sawed & Ragtag, 2005; Gunderson et al. , 2004; Zealand & Filmier, 2002). However, rigorous evidence base for the effectiveness of SUB in boosting student performance is thin.
A recent review of the empirical literature on SUB since 1995 indicates that only 14 studies utilized rigorous methods to assess the impact of SUB, and only six reported positive impacts on students’ test scores (Barrier-Sorry et al. , 2009) . Eleven studies are country-specific from Latin America, one from Kenya, and two exploit data from multiple countries. No empirical evidence is available from East Asia. This paper contributes to the small but growing empirical literature on the effectiveness of SUB by extending the research to East Asia.
The paper revised an initial analysis of the potential of schoolmates management for improving educational outcomes in the Philippines, using aggregated school- level test scores and administrative data. The data are quite limited and do not allow for a thorough analysis of the processes and approaches through which SUB reforms affect outcomes. Nonetheless, the paper attempts to use existing data to answer an initial question: did the introduction of SUB lead, on average, to enhanced educational outcomes?
It also demonstrates how administrative data can be mined for exploratory assessments of potentially larger programs. The remainder of the paper is organized as follows: the next section provides a brief overview of the SUB program in the Philippines; section 3 outlines the methodological approaches used for the analysis; section 4 describes the data; section 5 discusses the empirical results; and section 6 provides a summary of our conclusions, discusses the limitations of the study, and highlights the implications for future, more rigorous SUB evaluations in the Philippines. . SUB Program in the Philippines SUB was implemented in between 2003 and 2005 in 23 districts participating in the Third Elementary Education Project (DEEP) supported by the World Bank. 9 The project provided funding for school infrastructure, training, curriculum development, and textbooks. SUB was introduced as an integrating framework for obtaining school-level project inputs and building school capacity for education planning and program implementation beginning in school year 2003-04.
Schools participating in SMS were required to design a five-year School Improvement Plan (SIP) in partnership with parents and the community using data such as student achievement and students’ learning needs assessments, with the school principal or head teacher leading the process. Based on the SIP, schools developed an Annual 9 Although some other school districts in the Philippines had ongoing SUB initiatives, they are not included in this analysis as insufficient information is available about the nature and timing of those initiatives.
Implementation plan (Alp) at the beginning of the school year and a report card to be shared with the community at the end of the school year. Project inputs for infrastructure, training, textbooks, and so forth, were partially based on the SIP. Principals and head teachers received training in leading the development and implementation of the SIP and the Alps in collaboration tit teachers and key members of the larger community.
SUB schools also received funds for maintenance and operating expenses directly in cash rather than in kind, as had been the case previously. These cash funds could be used by the schools based on their Alp, The cash allocation was based on a formula that provided each school with a flat amount of funds plus a prorated figure based on the number of student and teachers as well as other criteria, such as percentage of indigenous student population in the school.
Schools not participating in the SUB received no SUM-related training and no ash funds, and they were not required to develop SIPS and Pips. The SUB was designed to improve student outcomes through two main venues: by empowering the school community to identify education priorities and to allocate the school maintenance and operating budgets to those priorities (such as curriculum enrichment programs); and by en hanging transparency and accountability through the annual implementation plans and school report cards.
However, the SUB program articulated no explicit assumptions regarding the timeshare within which improvements in student achievement were expected to take place. Systematic data on the level of uptake and implementation Of the key features Of the reforms are also not available. The SUB training, funds, and requirements were rolled out in three batches and eventually covered almost all (84 percent) of the 8,613 schools in the 23 project districts.
The first batch comprised 1 ,666 schools in 2003-2004, largely because they were perceived to be more capable, although no explicit assignment mechanism was designed. 10 The next batch of 2,700 schools was targeted for SUB roll in 2004-2005, and another batch of 1,529 was included in 2005-2006. 3. Analytic Approach We assess the effects of SUB on student performance using average school- bevel test scores from all schools and school-level indicators between 2003 and 2005 from the 23 DEEP districts in the Philippines.
We base our analysis on a pipeline comparison strategy. Since we have student achievement data over a three year period, but schools were inducted into the program in three batches over time, we identify the 10 No written materials were available on the identification strategy. Project off calls interviewed as part of this paper mentioned that schools were chosen on the basis of the perceived street night of their capabilities. Treatment group as the first batch of schools that received SUB funds in 003-2004.
The treatment group consists of the schools that did not receive SUB funds and training in 2003-2004. Therefore, for this study, we personalized SUB batch one schools as the treatment group (n=1666) and all other schools (batches that received the SUB intervention later) as the control group 1 In our analysis, the treatment group (batch one) therefore had exposure to SUB for a period of two years, 2003-04 and 200405. Because selection into SUB was not voluntary, there is likely to be placement bias in our control and treatment groups.
We therefore use two non-experimental evaluation techniques to estimate program impact: preference-in-difference estimators and difference-in-difference with propensity score matching estimators. Using nonparametric kernel matching techniques, we correct for potential sources of bias. Specifically, we tested whether the composite test scores and test scores individually in three subject areas (math, science and English) were higher after SUB was introduced for schools in the treatment group compared to those in the control group. . Data and Statistical Specification We use school administrative data from all 67 schools in the 23 DEEP districts to examine the effect of SUB program on student test scores. The dataset includes information on school personnel, students, facilities, and enrollment and dropouts for all schools in the district over a 3-year period, 2002-2003 to 2004-2005. The data were collected by the Philippines Department of Education as part of the management of the Third Elementary Education Project (DEEP) of which SUB was one component.
Student achievement is measured using National Achievement Tests conducted in 2002-03 (year one – pre-intervention) in Grade 4, 2003-04 (year two) in Grade 5, and 2004-05 (year three, post-intervention) in Grade 6 in English, mathematics, science. However, only the school-level Mean Percentile Score (MSP) data are available in the dataset, limiting the analysis to the school level. We utilize the composite MSP computation (based on tests in all subjects), as well as the English, mathematics, and science MSP for 2002-03 (pre-intervention) and 2004-05 (post-intervention). 1 The number of schools in the analysis is not the same as the number of schools included in the SUB program due to missing information. The difference is 728 schools. Our method of analysis consists of two quasi-experimental evaluation techniques. We have data on test scores over three years for all schools. We also have time-invariant data on school characteristics. Once we determined the control and treatment groups as described in the preceding section, the preferred estimator to use in this context was the difference-in-differences (DID) estimator.
However, in order for the DID estimator to provide credible estimates of the program impact, it should be free of any biases inherited by pre-existing (pre-intervention) differences between treatment and control groups. Because of problems incurred with using the DID estimator due to way in which SUB program was rolled out (i. E. , no clear assignment Achaeans), we also estimate and discuss DID in conjunction with propensity score matching (GSM) to estimate average treatment effects. i) Difference-in-differences (DID) estimation The ideal experiment for identifying the effects of SUB on student test scores would have been to randomly assign schools to control and treatment groups. The SUB effect could then have been estimated by comparing schools that received SUB with their peers in the comparison group. Unfortunately, such a design is not feasible in this case because of the nature Of the SAM program roll. Therefore, we assigned the set of schools that achieved early intervention in 2003 as the treatment group. The control group comprised all other schools that did not receive SUB support in 2003 – i. . , schools that received the intervention later. The SUB effect on student test scores can be evaluated as follows: SUB effect = (Outscores SUB . POST CLC Outscores SUB , PRE ) 0 (Outscores MONISM , POST C] Outscores MONISM . PRE -ћ-CEQ. 1) Converting equation (1) above to an estimating model, we get: Yes (Year* Sibs)Dњsets њEDP]Acts where Y is the outcome measure. We include school-level composite test scores as well as test scores in math, science, and English as outcome assures. Year is a dichotomous variable that equals 1 for postmodernist and equals O for pre-intervention.
SUB is a dichotomous variable that is equal to 1 indicating that a school received SUB support during all of the treatment years (t =2002-2004). Z is a vector of school characteristics. CLC p is division- specific fixed effects intended to capture division-specific aggregate fixed effects correlated with schooling outcomes (demographic trends or changes in government, for example). 0 t, s is the school level error term that includes all the unobserved school characteristics that we assumed are uncorrelated with the explanatory variables for the time being. 2 The vector of school characteristics includes total enrollment, total dropout, student-teacher ratio, textbooks per student, teacher manual per teacher, school type, school head type, whether the school was an elementary leader school (a school considered to be a leader among others in the district) and whether the school had received SKIFF funds. 13 School type consists of four categories: complete combination and multi-grade schools; complete large and medium school; complete small schools; and primary schools.
School head type insisted of three categories: head principal, head teacher, and teacher in charge. Table 1 summarizes the descriptive statistics for all schools and by treatment status of the school. 0 0 is the intercept showing the average test scores of the SUB and non-SUB schools; CLC 1 shows the change between 2002-03 and 2004-05; C] 2 is the difference in the treatment and control group; and D 3 is the DID estimator capturing the differences between the control and treatment groups, before and after the intervention.
More specifically, it measures the SUB effect as the change in average school-level test scores among two groups before and after the 2003 SUB intervention. These groups included: the “first batch,” or the schools that received SUB first (treatment group) to schools that received the intervention later and those that did not receive the intervention at all (comparison group). In order to obtain the DID estimates, we created a panel dataset from our original dataset of 67 schools.
Since we had test scores for all three years, we simply replicated this data for every year to obtain a panel dataset with 15,501 observations. In this dataset, each school is observed in each of the three years (2002-03 to 2004-05). In order to use the DID estimates in this context, we need to be certain that the difference in postmodernist outcomes between SUB and non-SUB schools would have been identical in the absence of the intervention. However, this assumption is impossible to test because we do not actually observe the contractually.