Abstract
We offer a nationwide analysis of the initial years of Costa Rica’s PSA program, which pioneered environmental-services payments and inspired similar initiatives. Our estimates of this program’s impact on deforestation, between 1997 and 2000, range from zero to one-fifth of 1% per year (i.e., deforestation is avoided on, at most, 2 out of every 1,000 enrolled hectares). The main explanation for such a low impact is an already low national deforestation rate. We also consider the effect of enrollment. Predicted deforestation on enrolled versus nonenrolled hectares, and matching analyses suggest an enrollment bias toward lower clearing threat. Enrolling land facing higher threat could raise payments’ impact on deforestation. (JEL Q24, Q28)
I. Introduction
Payments for environmental services (PES) are one “textbook” approach to forest conservation. Those who value environmental services offer to pay more than the costs incurred in supplying those services. That such payments both create an incentive to supply services and compensate suppliers appears to be part of their lasting appeal (some early discussions of PES include those by Smith 1995; Chomitz, Brenes, and Constantino 1998; Ferraro 2001; Frank and Muller 2003; Miranda, Porras, and Moreno 2003; Rojas and Aylward 2003; Rosales 2003; Tikka 2003; and Echavarria 2004). However, we do not know if they have had any impact. Despite calls for empirical evaluation of conservation policy (Ferraroand Pattanayak 2006), ecopayment programs rarely have been rigorously evaluated (GEF 2010).
We examine the impact on deforestation rates of Costa Rica’s famous PES system. The Pagos por Servicios Ambientales (PSA) program has been one of the most advanced initiatives in the developing world (Pagiola 2002) and an inspiration for other PES programs. We evaluate its early years (1997-2000), emphasizing the importance of using appropriate control groups for rigorous impact evaluation. Evaluation is critical for establishing the conditions under which PES will increase forest relative to a business-as-usual scenario. Certainly, payments can have impact under the right conditions, yet our results indicate that one critical lesson Costa Rica has provided, as a leader in PES implementation, is that forest impact is not automatic. That reality should shift more of the energy dedicated to PES toward the design and targeting of programs.
In Costa Rica, deforestation decreased during the 1980s and 1990s (de Camino 2000; Sanchez-Azofeifa et al. 2003), and many have assumed that the PSA program contributed significantly to this trend (e.g., Friedman 2009). However, payments under PSA did not even start until 1997. Further, shifts in other factors that could have reduced deforestation occurred simultaneously: Forestry Law 7575 not only created the PSA program but also restricted post-1997 deforestation, ecotourism flourished during this period (Rojas and Aylward 2003), and cattle ranching faced variable prices—all factors that likely helped to lower deforestation rates (Pfaff and Sanchez-Azofeifa 2004). Given those other changes, it is difficult to know what role the PSA program played in reducing deforestation. Even if one embraces the logic of payments and the pioneering value of Costa Rica’s specific initiative, the PSA program’s impact on forest might be marginal. As the impact of perfectly enforced conservation is equal to the deforestation that it blocked, the PSA program’s impact could be zero if other changes eliminated all the deforestation pressure.
We provide the best estimate of PSA’s nationwide impact on parcels enrolled in the early years. Before, during, and after our analyses, others have studied specific regions and used larger land units that combine enrolled with nonenrolled lands (Sierra and Russman 2006; Sanchez-Azofeifa et al. 2007; Morse et al. 2009; Daniels et al. 2010; Arriagada et al. 2012). Many of these analyses combine time periods with different enrollment approaches. We examine a period with a single ("first come, first served") enrollment approach (Barton et al. 2004; Sierra and Russman 2006).
To evaluate the PSA program’s impact, we use observed deforestation rates on non-PSA lands to estimate what the deforestation would have been on the enrolled lands without the PSA program. We must estimate this counter-factual because it is not observed. Prior policy impact evaluations have assumed, as the counterfactual, either complete deforestation or a deforestation rate equal to that on average nonenrolled lands or on neighboring nonenrolled lands (Joppa and Pfaff [2010a] cite many such analyses). To reduce bias, we instead estimate this counterfactual by matching PSA observations to the non-PSA observations with the most similar characteristics. Because matching reduces bias, it raises estimates of policy impact when policies are in areas of higher deforestation pressure, but it lowers estimates when policies are in areas with lower pressure.
Given that all of the landowners who participate in such ecopayment programs do so voluntarily, we might expect PSA contracts to be on land facing relatively low pressure, at least on average. One reason is that those who volunteer may be those with lower profits from clearing their land, since they have less to lose by contracting not to clear. In addition, even if everyone participated, each person might offer their least profitable lands to the program.1 Still any agency could target threats by selectively accepting parcels from among all volunteered land, especially if a program is oversubscribed, as was the PSA program. Yet for a program’s first phase, in which the agency has to simply establish procedures, we might or might not expect such targeting. Further, if there were any targeting, it could have been based upon any number of rationales unrelated to threat.2
For Costa Rica during 1997-2000, however, the main point here is that deforestation was low. Clearing on all non-PSA lands occurred at a rate of 0.2% per year, that is, one-fifth of 1%. As a counterfactual, this low non-PSA rate of clearing suggests that deforestation was avoided annually on 2 out of every 1,000 hectares enrolled. We emphasize that this is not a consequence of PSA’s enrollment processes, as this initial impact estimate derives from countrywide trends.
We can evaluate the impact of the PSA program’s enrollment processes by studying where the PSA contracts were located. Despite low deforestation countrywide, selective approval among all volunteered land could have targeted high-threat locations to generate deforestation impacts (Pfaff and Sanchez-Azofeifa 2004). We find, instead, that the PSA land faced lower threats than average non-PSA lands. A regression for deforestation in the prior decade predicts lower 1997-2000 deforestation for lands under PSA contracts than for nonenrolled lands. Thus, the 0.2% annual impact estimate above might be an overestimate. Our application of matching methods supports this claim. On average, matching impact estimates are lower. They range from 0.2% annual deforestation impact (equal to our initial estimate) to zero (statistically insignificant).
II. Background And Framework
Costa Rica’s PSA Program
Costa Rica’s Environmental Law 7554, established in 1995, was followed by Forestry Law 7575, in 1996, and by the Biodiversity Law, in 1998. Taken together, they form a legal background for the PSA program’s payments to people who possess forested land. Officially, the compensation is said to be for providing some particular ecoservices (e.g., climate-change-mitigation services, hydroservices, scenic services, biodiversity services). Yet in practice, even though the different parcels surely provide different levels of environmental services, all lands are paid the same.
Initially, PSA acceptances simply followed the applications. For instance, size was not officially considered for acceptance. However, empirically, it appears characteristics such as farm size, human capital, and income did have an influence (Porras 2010). Larger landowners were, in fact, disproportionately represented in the program (Miranda, Porras, and Moreno 2003; Zbinden and Lee 2005).
Payments could not, of course, always out-compete all other potential land uses. Average returns from PSA contracts varied from $22 to $42/ha/year, before all related costs, while cattle ranching had returns from $8 to $125/ ha/year, depending on the location, land type, and ranching practices (Arroyo-Mora et al. 2005). One measure of the cattle-ranching returns is the cost of renting pasture. In the Cordillera Central within the heart of Costa Rica, which is near some PSA parcels, pasture rental ranged from $20 to $30/ha/year (Castro and Arias 1998).
Three contracts existed: forest conservation, reforestation, and sustainable forest management. Forest-conservation contracts required owners to protect existing (primary or secondary) forest for five years in order to receive $210/ha in equal annual installments. Reforestation contracts bound landowners to plant trees on abandoned cleared land and to maintain the resulting forest for 15 years. Forest-management contracts required a “sustainable logging plan" for conducting low-intensity logging while maintaining the provision of ecoservices from the logged forests.
Legally, any PSA contract creates a legal easement that remains with the property if it is sold. Owners transfer to the national government the rights to the climate-mitigation potential of the parcel. Fondo Nacional de Financiamiento Forestal (FONAFIFO), a public forestry-financing agency created under Forestry Law 7575, administers the program. Inspection responsibilities within the program’s implementation, however, have rested with the Sistema Nacional de Areas de Conservacion (SINAC), as well as with the Ministerio del Ambiente y Energia (MINAE).
Initially, the main funding source for the PSA program was a 15% consumer tax on fossil fuels established under the Forestry Law. Article 69 stated that FONAFIFO was to receive one-third of the revenue. The Ministry of Finance, however, rarely delivered that amount. In 2001, Article 69 was repealed, but the Ley de Simplificacion y Eficiencia Tributaria assigned 3.5% of tax revenue to the program (Camacho and Reyes 2002). This provided less for the PSA program in theory, but in fact, it increased funds transfers from the Ministry of Finance (Camacho and Reyes 2002). As of 2003, this approach had provided an average of $6.4 million/year (Pagiola, Landell-Mills, and Bishop 2002).
Funding for the PSA program also comes from voluntary contracts with private hydroelectric producers, who reimburse FONAF-IFO for payments provided to individuals such as upstream landowners in watersheds. These private agreements have been limited but could well grow.
Land Use Choice with Payment Option
We present a simple model where landowners choose land uses in order to maximize returns. This framework conveys how the profitability of private forest clearing determines impact and, further, how variation in private clearing choices can complicate estimation of payment impact. This model and its illustration are in the spirit of the classic von Thiinen (1826) land model.
Figure 1 orders all units of land by relative profitability of clearing. Thus, profit from agriculture, minus profit from land in forest, is higher to the right. Where this difference in profits is greater than zero (i.e., to the right of xN), the land will be deforested. Forest will remain in [0, xN].
Rent-Maximizing Land Use without and with Payments
Next, for environmental services produced by land in forest, we introduce a payment P for all participants. This payment adds to the profit for keeping land in forest, which, by raising the hurdle for clearing, helps forest to compete with the profit from land that remains in agriculture. As a consequence, now the private landowners sign up for payments within the range [0, xP], i.e., where P is larger than the relative profit earned from forest clearing. Forest beyond xP is cleared.
Critically, not all those who qualify for payments have to modify their land-use plans. Those in [0, xN] would not need to modify their plans, since with payments or without, they choose forest. In contrast, parcels within [xN, xP] are deforested in the absence of payment but not when there is a payment. Thus, impact depends on the fraction of enrollment from [xN, xP], denoted as α. If α = 1, that is, only land from [xN, xP] is enrolled, then all payments prevent deforestation. On the other hand, if α = 0, that is, only land from [0, xN] is enrolled, then payments have no impact.
We will estimate α by finding locations outside the program that are similar to the parcels in the program, then computing deforestation rates for those places. The percentage of those places that were cleared is an estimate of α. If all were cleared, then all were from [xN, xP], thus α = 1. Putting that result another way, the payments program could then be said to be 100% efficient.
We note that not all who may wish to apply on the basis of being in [0, xP] will apply, as some landowners maynot know about the PSA program or may face high costs of application. Further, not all of those who apply are guaranteed enrollment, due, for instance, to the limits on funding.
Assuming that all of [0, xP] applies, which parcels are enrolled affects not only program impact but also the accuracy of simplistic impact estimates. If α = 1, targeting is, in fact, fully effective. All of [xN, xP] will be enrolled, while no parcels are enrolled from [0, xN], such that the forested locations outside the program are those in [0, xN]. These locations are not similar to the enrolled: none will be cleared, while all of the enrolled would have been; and α is underestimated at zero, though all payments actually had impact. Conversely, if α = 0, impact is zero because none of the parcels in [xN, xP] are enrolled, while all of the land in [0, xN] is enrolled. Then, using the [xN, xP] unenrolled applicants as controls will overestimate, at 100%, the effectiveness of the payments.
Generally, accurate estimation of α requires that some parcels that are similar to those enrolled exist outside the program. We believe that this is the case in Costa Rica, in other words, both in and outside of the program, some parcels would be cleared without payments, while others would not. The potential for this to be so is increased by the fact that the PSA program was oversubscribed.
III. Data And Matching Methods
Data
Fifty thousand points were randomly drawn from Costa Rica’s 51,000 km2, that is, roughly one for every square kilometer of national territory. We eliminate some points where there are clouds in the satellite picture or if experts say that the picture was inconclusive (Pfaff and Sanchez 2004). We also drop all points in public protected areas and all points in the PSA before 1997, plus the points in non-forest-conservation PSA contracts. We use the forested locations for analyzing the forest-protection contracts. By 1997, there were 10,108 private-land locations covered by forest.
Deforestation
We obtained geographic information about the spatial distribution of forest in 1986,1997, and 2000 from the University of Alberta (Sanchez-Azofeifa et al. 2003). These maps allowed us to estimate the amount of forest in each of these years, namely, annual deforestation rates at the national level, as well as exactly what parcels were deforested between 1986 and 1997, or between 1997 and 2000. Kerr and Pfaff (2007), using the same maps, found annual deforestation between 1986 and 1997 to be about 1%, and annual deforestation between 1997 and 2000 to be around 0.20%.
Payments (PSA)
We obtained information about the PSA program from FONAFIFO. We focus our analysis on forest-conservation contracts. They account for most of the hectares enrolled and were the only contracts aiming to stop forest clearing.3 In 1997, over 88,000 ha were entered into such contracts, making up over 86% of all the new PSA contracts for that year. In 1998, over 47,000 ha were enrolled into such contracts, making up over 79% of the total new PSA contracts. Then in 1999, more than 55,000 ha were enrolled, again over 86% of all new enrollments.
Payment levels per hectare varied according to the year when contracts were signed (the totals to be paid were distributed in equal amounts across the five years that any of these contracts lasted). In Colones, the Costa Rican currency, the payment per hectare was 50,000 in 1997, then rose to 60,000 in 1998 and 1999. On average, that is about $210/ha, or $42/ha/ year. We obtained spatially specific information for all forest-conservation contracts for 1997, 1998, and 1999 throughout Costa Rica (Figure 2 conveys their distribution across the country). This precise location information allows us to examine what conditions are most highly correlated with the payment locations, in other words, to characterize the types of land where payments took place.
PSA Forest-Protection Contract Locations 1997-1999 Source: Authors, using information from FONAFIFO.
Other Deforestation Determinants
Additional maps with locations of the rivers, cities, national parks, schools, sawmills, national and local roads, and slopes came from the Ministry of Transport and the Instituto Tecnologico. We use a vegetation description based on Holdridge Life Zone criteria, which indicate that most of Costa Rica is in 11 “life zones”: humid pre-montane; humid lower-montane; tropical humid; very humid pre-montane; very humid lower montane; very humid montane; tropical dry; pluvial pre-montane; pluvial lower-montane; pluvial montane; and paramo. We also use the Ministry of Agriculture’s administrative divisions (Central, Huetar Atlantica, Huetar Norte, Brunca, Pacifico Central, and Chorotega) to generate regional dummies for use as controls within these analyses.
For each location in our sample, we find the distances to the closest national road, closest local road, closest sawmill, closest river, closest national park, closest school, and closest already-cleared area. We also find the distance to the national capital, San Jose, as well as the distances to the two main ports, Limon and Caldera, and finally also the distances to the county capitals. We also classify each location in terms of its Holdridge Life Zone and its administrative units. The full-sample means and standard errors for all of these variables are presented in Table 1.
Descriptive Statistics
Matching Methods
Defining “Similar” in Practice
We estimate the forest impact of PSA using untreated (i.e., unpaid) locations that are similar to paid ones. Specifically, we use deforestation rates on such “matched” unpaid lands as an estimate of the deforestation that would have occurred on paid lands without any payments. We apply two kinds of matching. Nearest-neighbor propensity-score matching (Rubin 1980; Rosenbaum and Rubin 1983; Hill, Brooks-Gunn, and Waldfogel 2003) uses a fixed number of matched control observations for each of the treated observations. Any propensity-score matching defines “similarity” based on the estimated probability of being treated. That metric is generated using a first-stage (or treatment) regression that explains, for all observations, where treatment did and did not occur. The second matching approach that we apply is a nearest-neighbors covariate-matching estimator (Abadie and Imbens 2006a), again using a fixed number of matches for each of the treated observations. Covariate-matching estimators define “similarity” without using any first-stage regression, instead using the distances between the treated and untreated points in the space of the matching covariates.
Computation of standard errors is another difference in these two kinds of matching approaches. Abadie and Imbens (2006b) show that bootstrapping standard errors is invalid with nonsmooth, nearest-neighbor matching estimators, such as propensity-score matching with a fixed number of matches (to be contrasted with kernel versions using smoothly declining weights for less-well-matched untreated observations). For propensity-score matching, we apply Hill, Brooks-Gunn, and Waldfogel’s (2003) weighted robust standard errors. They account for the fact that an unpaid observation might be the best match for multiple treated observations, but not for the fact that the propensity score is predicted. We lean upon the covariate matching’s standard errors (Abadie and Imbens 2006a).
Observably Most Similar Is Not Always Similar
As matching is attempting to control for the influences of differences—between the paid set of parcels and the unpaid set—in the observable factors that can affect the deforestation outcome, one might ask why ordinary least squares (OLS) could not achieve the same result. An important practical answer is that if the groups differ greatly in observables, the burden on an OLS specification is considerable.4 Thus, finding the most similar lands for comparisons can improve the policy-impact estimates.
However, the observable factors could still differ considerably, between paid and unpaid lands, even after the matching procedure. When the treated and the matched untreated groups do differ in their observable characteristics, OLS for the matched sample can indeed help to reduce bias from this imperfection in even the best possible matching. Another approach employed in this situation is to rerun analyses without the treated points that have poorest matches.5 Generally, one not only searches for, but also then checks for, observables’ balance after matching, in order to confirm whether this search for the greatest similarity actually yielded observable similarity.
Yet, matched pairs of observations also could differ significantly in their unobservables as well. The allocation of the treatment (payments) could be a function of factors that we do not observe within our data set, but to which, nonetheless, the local agencies have in fact reliably responded. One reason for suspecting that unobservables can differ is the nature of the voluntary process for payments. The owners of the treated lands were those willing and able to get their land enrolled, and they could differ from the owners of the untreated lands in ways that would affect not only their participation, but also what they would have done with their land had they not participated (Alix-Garcia, Shapiro, and Sims [2012] address this issue by comparing treated landowners to other applicants). Thus matching can improve comparisons considerably, but yet not address all possible factors.
IV. Results
To calculate the impact of the PSA program on deforestation rates, one approach would be to measure the rents for each parcel enrolled in order to determine the fraction of parcels for which the rents in nonforest uses are highest. Since that sort of information was not readily available, an alternative strategy was to calculate a counterfactual deforestation rate in order to estimate the fraction of land that would have been deforested if the PSA program had not been implemented.
Comparing Enrolled to All Untreated
One possible strategy for estimating the counterfactual deforestation rate for lands enrolled in the PSA program is to calculate the deforestation rate for all lands where the program had not been implemented. We obtain this estimate using the fraction deforested, by 2000, of all the untreated points that were forested in 1997. This estimation approach does not address any bias in the PSA program’s locations (as addressed in some prior parks evaluations; see Joppa and Pfaff 2010a).
In areas outside the program, we find that average deforestation, between 1997 and 2000, is low. Specifically, the three-year deforestation rate equals 0.6% (Table 1), less than 1% for the entire 1997-2000 time period we study. The implied annual deforestation during this period is 0.20%, that is, one-fifth of 1% per year. According to this impact estimate, very little of the land that was enrolled in the PSA program actually was protected from deforestation: 2 ha/year out of every 1,000 enrolled, or 10 ha per 1,000 (1%) over the full five years of a PSA contract. Thus, we can compute that these five-year contracts postponed deforestation within 880 of the 88,000 hectares enrolled in 1997, with similar fractions applying also for 1998 and 1999.
We emphasize that this simplistic impact-evaluation result literally has nothing to do with how the PSA program was, in fact, implemented. It provides an estimate of what the PSA program would have prevented had contracts been located randomly, as well as an upper (lower) bound on impact if locations of PSA contracts were biased away from (toward) deforestation threats.
Addressing Nonrandom Enrollment
Deforestation OLS
Our model suggests that, given the voluntary structure of the PSA program, enrolled land will not be a random subset of all lands but, instead, will feature lands lower in agricultural profits. Yet, knowledgeable agency experts could target either clearing threats or other specific goals. For a better estimate of PSA impact than the 0.20% result above, we must endeavor to control for the deforestation influence of any differences between enrolled and nonenrolled parcels.
A standard econometric approach is a simple OLS regression. Table 2 presents this analysis. The coefficient for PSA (— 0.003) implies a 0.11% annual impact when using OLS to control for other variables. To control for additional factors that we do not directly measure, such as the local population density and various unobservable elements of local development, we also include district6 fixed effects, which generates estimates of the PSA coefficient (— 0.005 and 0.004) that imply 0.17% and 0.13% annual impacts, respectively, with and without controlling for other factors. The estimates are a little bit lower than when not controlling for other variables. Further, as seen in Table 2, these estimates of PSA impact do not differ significantly from zero.
OLS Regressions (without and with Covariates)
Predicted Deforestation and Enrollment
Our first effort above to address nonrandom enrollments suggested a bias toward lower threats. Such an outcome could result from landowner choice or from the agency’s selection decisions. Agency selection would have to be based on past behavior, that is, on prediction of the future using the past. We run a regression to explain deforestation during 1986-1997, which is pre-PSA, and then use those coefficients to predict the deforestation after 1997, based upon parcel characteristics. Next, we examine whether PSA enrollment leans toward high or low predicted deforestation. We divide all of the parcels we consider into five ranges, based upon predicted rates of clearing. Figure 3 shows that, if anything, PSA enrollment draws more heavily on low predicted clearing. For points with higher predicted clearing, the fraction of all parcels enrolled in PSA is smaller. This suggests that the 0.20% annual deforestation rate in untreated areas overestimates impact.
Nonrandom Enrollment Suggests Lower Threat
Matching to Address Nonrandomness
Matching has been used to address nonrandom location of policies to reduce deforestation (regarding protected areas, see Andam et al. 2008; Pfaff et al. 2009; Joppa and Pfaff 2010b; regarding PES, see Arriagada et al. 2012; Alix-Garcia, Shapiro, and Sims 2012). We provide propensity-score-matching estimates, although we also include robustness testing using other strategies, including co-variate matching.
Propensity-Score Matching
Among the 10,108 private-land locations within our sample that were still in forest in 1997, 319 are within PSA polygons. Table 3 presents a probit analysis to determine the effects of location characteristics on the probability of being treated, that is, the probability of a parcel being under a PSA contract. We predict the probability of being treated for all of the 10,108 points, based on this regression. The predicted probabilities are an index of similarity that we can use to match.
Using only the best match, we have limited data for impact estimation. Thus we prefer to include more matches, up to six (below, we present results for both more and fewer matches). Of course, we must check the balance, particularly because a sixth-most-similar control will be less similar. Figure 4 emphasizes the need to check whether matching did result in similar untreated points. It shows that match quality is low for the relatively high propensity treated points, i.e., for treated points whose characteristics made it more likely that they would be enrolled in the PSA program. As poor matches lessen the value of matching, we test for robustness by dropping these points. Figure 4 shows that this could be done using a caliper, based on match quality (horizontal lines), or using propensity scores (vertical lines), since poor matches are correlated with higher scores.
Propensity Score Matching, First-Stage (Treatment) Regression
Match Quality Falls for Parcels Most Likely to Be Treated
Table 4 compares treated, all-untreated, and matched-untreated groups (for Table 5’s fifth row, i.e., matching using six matched-untreated points) by presenting the means for deforestation and all the covariates used in matching and as explanatory factors in the treatment regression. We see that matching helped to eliminate significant differences between treated lands and the set of all of the untreated or unpaid locations. Some of the variables showed significant differences in their means before the matching was done, but those differences were all eliminated by the matching.
Propensity Score Matching Yields Improvements in Balance (for Table 5’s Fifth Row)
Comparing Estimates of the PSA Program’s Impact on 1997-2000 Deforestation
That said, there remain nonzero differences between these means. In addition, even if the means were identical, treated and untreated points within each of the matched pairs surely would differ. For example, one matched control may be farther from a road than the treated point it matches, while a second match may be closer to a road than the treated point it matches. These differences can affect the results even if, on average, the set of all the matches have the same road distance, suggesting the need for post-matching regressions to help to control for remaining differences.
Comparing Estimators
Table 5 combines our simple estimate (from the subsection “Comparing Enrolled to All Untreated") with OLS (from the sub-subsection "Deforestation OLS”) and matching estimates. Mean deforestation for untreated parcels is shown in the first row and first column, as the very simplest impact estimate, with the OLS estimate of payment impact to its right. The fact that the OlS estimate is lower than the simple mean suggests bias in PSA location toward lower threats.
For matching, using Table 3’s estimated treatment probabilities, we carry out propensity-score matching using one, two, three, and six matched controls per treated location. Those results are presented in Table 5’s second, third, fourth, and fifth rows, respectively. All these estimates are very similar to, or lower than, the 0.6% first-row-first-column simple estimate based upon all the non-PSA lands. This matching conclusion is robust to a post-matching OLS regression to reduce influences of remaining point-by-point differences, as seen in the results in the second column.
Table 5’s sixth, seventh, and eighth rows present the results from propensity-score matching after dropping treated observations for which we do not find good matches. First, we drop the worst matches (see Figure 4). With and without the bias adjustment, this estimate is very similar to the prior propensity-score matching, and it is essentially identical to using a caliper. The impact estimate slightly increases when we focus on the very highest quality matches (again, see Figure 4). Yet these are all similar and, again, the estimates are, if anything, smaller than if payments were implemented randomly.
For an additional robustness check, Table 5’s ninth row presents results from covariate matching. With or without bias adjustment, these estimates are very similar to the other matching estimates and are lower than 0.6. However, neither covariate estimate is significantly different from zero (recall that these results feature the valid standard errors following Abadie and Imbens 2006a). Finally, we include propensity-score-matching and covariate-matching estimates after dropping observations with the 10% lowest and with the 10% highest propensity scores. Again, we find that none of these estimates are significantly different from zero (rows 10 and 11 in Table 5).
In sum, we find some suggestion that PSA locations were biased toward lower threats, as most matching estimates of PSA’s forest impact are below average clearing in all non-PSA locations. However, the dominant result is a very low forest impact that is due to the low rate of clearing.
V. Discussion
For the whole of Costa Rica, we find that little impact upon rates of deforestation resulted from the initial PSA forest-conservation-payments contracts that were initiated during 1997-2000. The clearing rate on all unpaid lands suggests that 0.20% of the enrolled locations would have been cleared annually without payment, or about 1% in a typical five-year contract. Examining the somewhat nonrandom allocation of payments that arose through the “first-come, first-served” approach employed during this time period suggests that this is, if anything, an overestimate.
This result is relevant for the design of new schemes motivated domestically or internationally. Considerable energy on the part of different institutions has gone into initiating such payments, though not nearly as much energy has been focused upon analysis of impacts. While Costa Rica may be an outlier in countrywide trends, still, its main lesson is quite general: payments to where average deforestation threats are low can be constrained in their forest impacts. Further, even if some significant deforestation threats exist, they may not be addressed, given enrollment biases. Thus, learning from Costa Rica’s initiative means identifying how payments can best influence use of the land, and devoting further energy to the design and targeting of ecopayment programs.
We emphasize that, while any single summary result (such as our national impact on paid lands) cannot represent all program outcomes across various choices of location and timing, there is an underappreciated commonality across the growing set of analyses of payments for environmental services: the finding that most forest enrolled in PES programs would not have been deforested (consistent with Pattanayak, Wunder, and Ferraro’s [2010] review of econometric studies of PES). In other words, even focused efforts to identify forest impacts of such payments have found, almost universally, that the great majority of paid standing forest would have remained standing without payments.
Consider, for example, the carefully done analyses by Arriagada et al. (2012). Their study area is described as being atypical—it features a seemingly unusual targeting of deforestation threats—and within that location they find statistically significant positive impacts. These results suggest that avoided deforestation was achieved, on the order of 1 in every 5 to 10 paid parcels. In light of this, it is worth asking how an area or an actor achieves relatively high impacts. Yet, it is also clear that impacts are far from automatic, even in such cases. Such a cautious conclusion is supported by recent rigorous evidence concerning PES in Mexico (Alix-Garcia, Shapiro, and Sims 2012). They report payment impacts roughly similar to those of Arriagada et al. (2012) in a setting with more clearing but, at least in their period of study, an apparently less strong focus upon the targeting of threats.
In general, any particular spatial scope may affect estimates of impact. Daniels et al. (2010), for example, acknowledge our results above and suggest that subnational studies outside of the PSA polygons can identify significant levels of impact upon agricultural abandonment, implying gains in forest. It is worth considering such spillovers, although we would emphasize, as per Arriagada et al. (2012), that an effect being found in one location does not imply that the same effect exists elsewhere, and that controlling for other factors is critical when inferring policy impact. Along the latter lines, among featured studies by Daniels et al. (2010), Morse et al. (2009) consider a site close to Arriagada’s, and the authors emphasize that any reduction in the loss of forest is, at least in part, also a product of the Forest Law and of the socioeconomic conditions. The issue of the spatial scope of impact analyses also arises very generally when considering policy leakage. Looking for impacts only within paid parcels can miss impact-offsetting or impact-augmenting responses to the payments on other lands, for reasons including effects on prices (Robalino 2007) and potentially also effects on landowners’ dominant motivations (see, e.g., Alpizar et al. 2012).
This concern highlights another difference between our study and Arriagada et al.’s (2012). Their analyses blend paid and unpaid lands through the use of units of analysis larger than paid parcels. That permits the empirical inclusion of any impacts upon the unpaid lands in those spatial units. The same is true for Sanchez-Azofeifa et al. (2007). For both analyses, this could help to explain findings of greater positive impact. On the other hand, as shown by Alix-Garcia, Shapiro, and Sims (2012) for Mexico, spatial spillovers also can include leakage that lowers the net impact of such payments.
It is important to highlight the institutionally innovative nature of the PSA program. In addition, effectiveness should be measured against all stated goals including distribution, for instance, to compensate owners for the benefits to society that their land uses provided (FONAFIFO 2006). Still, a focus on pressure could raise impact. If the PSA program remains oversubscribed, then admittances could target those who would clear forest in the absence of payments. We note that discussion of targeting appears to have increased since 1997. Also, the PSA program could use fewer higher payments to areas with higher profits, and thus higher threats of clearing, although this could shift away from poorer landowners and affect distributive impacts. Such adjustments would require information on profits, which very well may be imperfect yet could be acquired.
One final perspective on Costa Rica in particular is that the total impact of the creation of a PSA program could be higher than the impact of payments themselves. As noted, Forestry Law 7575 limited post-1997 deforestation. If the creation of a PSA per se made it politically more feasible to pass such regulations, then regardless of the impacts of the payments, the creation of the PSA program may have had a high forest impact. Yet, even if that is so within Costa Rica, we suspect that many countries will not pass such laws that greatly restrict private clearing. Thus, based on our results, we emphasize that to have greater impacts, payments may need to target threats.
Acknowledgments
For financial support we gratefully thank the Tinker Foundation; the Inter-American Institute for Global Change Research, the National Science Foundation (in particular, the Methods and Models for Integrated Assessment program); the National Center for Ecological Analysis and Synthesis; the Social Science and Humanities Research Council of Canada; the CEES within the Earth Institute, as well as the ISERP at Columbia University; and the Swedish International Development Cooperation Agency through the Environment for Development Initiative. We also benefitted greatly from Arturo Sanchez’s willingness to provide a lot of data. For helpful comments (without any implication for remaining errors), we thank participants at our presentations at Yale, Harvard, Stanford, Resource for the Future, ETH Zurich, ISTF, the World Bank, U.S. AID, and the EfD initiative.
Footnotes
The authors are, respectively, research fellow, Environment for Development Initiative, CATIE, Cartago, Costa Rica; and associate professor, Sanford School of Public Policy, Duke University, Durham, North Carolina.
↵1 We note that Miranda, Porras, and Moreno (2003) and Zbinden and Lee (2005) find PSA participants differ from nonparticipants.
↵2 From a variety of perspectives-many involving benefits but not costs and mostly ignoring deforestation threats-authors have long suggested reasons not to implement policies randomly across space, but instead to optimally target them (Tubbs and Blackwood 1971; Gehlbach 1975; Williams 1980; Cocks and Baird 1989; Church, Storms, and Davis 1996; Csuti et al. 1997; Polasky et al. 2000; Cammet al. 2002; and Polasky, Camm, and Garber-Yonts 2001).Where threat is not explicitly involved, such allocations still may create a positive or negative correlation with threat,which will matter within evaluations.
↵3 The other contracts are for regeneration and reforestation of nonforest areas and for timber production.
↵4 As not surprisingly this natural question long has arisen, here is one early reply: “Unless the regression equation holds in the region in which observations are lacking, covariance will not remove all the bias, and in practice may remove only a small part or it. Secondly, even if the regression is valid in the no man’s land, the standard errors of the adjusted means become large, because the standard error formula in a covariance analysis takes account of the fact that extrapolation is being employed. Consequently the adjusted differences may become insignificant merely because the adjusted comparisons are of low precision. When the groups differ widely in x, these differences imply that the interpretation of an adjusted analysis is speculative rather than soundly based” (Cochran, cited by Rubin 1984).
↵5 Crump et al. (2006) note that with a lack of covariate overlap, many common estimators are sensitive to specification (see also Heckman, Ichimura, and Todd 1997, 1998). Crump et al. (2006) characterize optimal subsamples for which treatment effects can be estimated most precisely. We do not apply them. However, we do examine results for robustness to dropping relatively high propensity-score treated observations that we find do not have good untreated matches.
↵6 There are 455 districts in the sample.
References
In this issue
Jump to section
Related Articles
Cited By...
- Alternatives for improving Payment for Ecosystem Services (PES) effectiveness on water resources
- Effective Targeting and Additionality: Evaluating the D.L. 701 Reforms for Afforesting Erodible Land in Southern Chile
- Additionality in Payment for Ecosystem Services Programs: Agricultural Conservation Subsidies in Maryland
- Cash for carbon: A randomized trial of payments for ecosystem services to reduce deforestation
- PES Impact and Leakages over Several Cohorts: The Case of the PSA-H in Yucatan, Mexico
- Global Food Demand and Carbon-Preserving Cropland Expansion under Varying Levels of Intensification
- Protected Area Effectiveness in European Russia: A Postmatching Panel Data Analysis