From Wikipedia, the free encyclopedia - View original article
Oportunidades (English: Opportunities) is a government social assistance (welfare) program in Mexico founded in 2002, based on a previous program called Progresa, created in 1997. It is designed to target poverty by providing cash payments to families in exchange for regular school attendance, health clinic visits, and nutritional support. Oportunidades is credited with decreasing poverty and improving health and educational attainment in regions in which it has been deployed.
Key features of Oportunidades include:
Oportunidades has become a model for programs instituted in other countries, such as a pilot program in New York City, the Opportunity NYC  and the Social Protection Network in Nicaragua. Other countries that have instituted similar conditional cash transfer programs include Brazil, Peru, Honduras, Jamaica, Chile, Malawi and Zambia.
Progresa-Oportunidades is designed to be a centrally run program that relies on a horizontal integration of programs and services among the various agencies and ministries within the executive branch. This required the establishment of a body with enough power to coordinate the participants in the program and monitor the budget. Instead of restructuring an old agency, it was decided to form a new agency with all of the appropriate powers and the backing from the president.
Although this provided for an easier and faster startup, it also meant that many of the related agency structures, which Progresa-Oportunidades would have to rely upon for its future sustainability, were incompatible with the new program. Officials in related structures such as the Ministry of Health and Education were not provided with the appropriate incentives to channel their work towards Progresa-Oportunidades. Many were individuals who had worked on earlier poverty programs and who now saw their resources shifting in a new direction. Also, officials often had more to gain politically from abandoning this program and starting a new one.
As a centrally administered program, Oportunidades allows for low operational costs and a greater level of efficiency in the transmission of benefits directly to the participants in the program. The program has sometimes been criticized for this completely “top down” approach. However, the lack of community participation in the identification of beneficiaries and the allocation of funding also helps to limit the opportunities for corruption at the local level, which has traditionally been a problem with such government-funded programs.
To effectively disseminate information about the program, Progresa-Oportunidades pursued a “three-pronged” information strategy. First, an extensive amount of information was made generally available through the Internet. Secondly, information also provided to Congress and other government officials at all levels in the form of detailed budget proposals, program evaluations, and other relevant documents. Finally, public relations campaigns were initially minimal to avoid raised public expectations. However, since 2006, the public profile of the program has been raised, particularly through extensive radio and television advertisements.
Traditionally, most anti-poverty programs in Mexico have relied upon presidential support to establish their funding and public profile. Although Oportunidades originally was no different in its dependence upon Zedillo to support its startup costs, much effort has since been made to establish an image of Progresa-Oportunidades as existing independently of the president and party politics, to heighten chances it would survive transfers of power in the executive branch. Several congressional provisions have helped to ensure this identity. Among these are several provisions that prevent the program being used for political proselytizing. Also, in 2000 and 2003, program officials were prohibited from signing up new beneficiaries within six months of national elections in order to prevent vote buying from taking place. In addition, budget provisions have been enacted that attempt to directly communicate with program beneficiaries to educate them about rights and responsibilities regarding the program.
The transition between the Zedillo and Fox administrations was a crucial test of the long-term sustainability of Progresa-Oportunidades. President Fox’s administration achieved early public recognition for its commitment to anti-poverty strategies as well as its willingness to continue with earlier initiatives that had already proved to be successful. Also, Fox ultimately decided to continue with the program because the independent image of the program meant that it had not become identified with the old ruling party (the PRI) or with former President Zedillo in the eyes of the public. In addition, Fox had received confirmation of the program’s benefits both from the program’s own impact evaluations and outside international financial institutions who all held the program in high esteem and hoped to see it continue.
Oportunidades has been hailed as a success by many actors both in Mexico and globally. The first round of evaluations were carried out by the prestigious International Food Policy Research Institute (IFPRI), between 1997 and 2000. The #IFPRI-Progresa partnership played a large role in shaping Mexican social policy, and in bringing the Randomized controlled trial (RCT) to the forefront of policy evaluation worldwide. While often praised as a model of successful program evaluation, IFPRI-Progresa suffered from a number of flaws in both design and practice.
Since 2002, the Instituto Nacional de Salud Pública (National Public Health Institute, INSP) and the Centro de Investigaciones y Estudios Superiores en Antropología Social (Center for Research and Higher Studies in Social Anthropology, CIESAS) have been responsible for carrying out ongoing evaluations of both program operations and impact. The Oportunidades evaluation website contains a relatively comprehensive listing of the documents and databases produced during all of the evaluation rounds.
Communications between IFPRI and Progresa's leadership commenced in late 1997, with the final contract ($2.5 million USD) signed during the third quarter of 1998. The IFPRI-Progresa research team's overall objective was to "determine if Progresa was functioning in practice as it [was] intended to by design." The evaluation was divided between 'operational' evaluation (is the program being carried out according to the design?) and 'impact' evaluation (is the program having the intended effect on the target population?). Without discounting the importance of the operational evaluation, this article focuses exclusively on the impact evaluation, which received far greater attention and had a much broader audience. The impact evaluation also involved the production of far more technical documents, reports, and later, published journal articles and books.
The results of IFPRI's initial evaluations began to form into reports in late 1999 and early 2000; most final reports (18 in total) were submitted before the end of 2000, with the executive synthesis report published on September 1st of that year. The thematic core of the IFPRI evaluations conformed to the three program components: health, education, and nutrition. In these areas, Progresa increased school enrollment, lowered incidence of disease in children, and increased growth for young children receiving nutritional supplements. In sum "in the central impact areas...the results [were] encouraging."
According to the bulk of literature describing IFPRI-Progresa, the project was the first large-scale social policy evaluation implemented in a ‘developing’ country context to use the Randomized controlled trial (RCT) research design. In the now perennial debates surrounding the RCT as a tool of social policy evaluation IFPRI-Progresa provided (and continues to provide) an important proof-of-concept, an example exalted by proponents and held aloft before critics. Early project documentation, however, suggests that this image is, at best, an oversimplification, and at worst, a willful manipulation.
Regardless of the true nature sampling procedure, IFPRI-Progresa also suffered from several other equally serious problems including selective attrition, sample contamination, and changing eligibility requirements. Collectively, these flaws (although impossible to empirically measure) significantly weaken the internal validity of the evaluation, and cast serious doubt on both the accuracy and the precision of the results.
According to one source, original sample selection took place in mid-1997. All available information indicates that personnel at SEDESOL undertook and directed the actual process prior to the arrival of most of the IFPRI research team. The procedure itself, however, remains unclear.
One of the most detailed accounts is as follows:
"The design of the impact evaluation of Progresa in communities and households is quasi-experimental...To undertake this component of the evaluation, a random sample of communities with 'high' or 'very high' degrees of marginalization were selected which would be incorporated into the program during Phase II (November 1997) and which would serve as the [treatment] communities...Another sample of communities with similar characteristics was randomly designated from those that could have been the object of later selection, and that could function as controls...the size of the sample was estimated starting from a universe of 4,546 localities to choose 330 base localities, and from a universe of 1,850 to choose 191 control localities, using a distribution proportional to the size of the locality.”
This source (the first chronologically to describe the procedure) inspires several questions: 1. Why is the procedure characterized as quasi-experimental? 2. Why were communities selected from the universes of 4,546 and 1,850 communities, rather than from the overall universe of eligible communities (which was much larger)? 3. Does the phrase “with similar characteristics” indicate that communities were aligned using a matching procedure (such as propensity score matching)?
The methodological guide does not provide a clear enough picture to allow a researcher to replicate the sample selection process. Worse, this source provides one of the most detailed accounts, while subsequent sources tend to gloss over important features. For example, the synthesis report describes the process as follows:
"From a set of rural communities in the same geographic region, localities were randomly selected for participation in PROGRESA (treatment localities) while the rest were introduced into the program at later phases (control localities).
There is no distinction made between RCT and quasi-experimental design, no mention of the intermediate universes from the original source, and no allusion to any matching process. Throughout the cloud of documentation produced by IFPRI-Progresa, mention of these details does reappear sporadically, but never with enough detail to answer the questions posed above.
Attrition bias is a well-known issue in quantitative study designs which can create a situation "analytically similar" to selection bias in that "attrition lead[s] to selective [read: systematically different] samples." "The IFPRI evaluation did not follow up migrants as part of the evaluation surveys," and yet by November 2000, 17% of households and 29% of individuals from the original November 1997 sample were no longer in the survey. Moreover "attrition differ[ed] significantly between treatment and control groups, even after controlling for household characteristics and the eligibility criteria."  The presence of nonrandom attrition indicates that even if the samples were experimentally selected and statistically equivalent to begin with, by the end of the experiment period they would have been significantly unequal.
As opposed to selection bias, which features prominently in the IFPRI-Progresa documentation, only two final reports mention attrition bias (which, recall, creates essentially the same problem as selection bias from an analytic point of view). Schultz (2000) carries out analyses of enrollment using both a "panel sample" (households with data across all five survey rounds) and "pooled sample" (households with data in at least one survey round). While admitting that "it is not possible to implement a satisfactory sample-selection-correction model," the word 'attrition' appears only once. The rest of the report frames the use of both samples as a form of robustness testing. This framing turns attention away from attrition bias, instead highlighting the dual-sample testing as a strength of the analysis. Behrman and Todd (1999) discuss attrition more extensively, but only hypothetically, never making any connection to the actual data collected.
Here, contamination refers to the possibility that "families or individuals from control localities or other localities [can] immigrate to treatment group localities in order to receive program services." (although the term can refer to several other concepts) As with attrition bias, the decision to not track down sample out-migrants makes any quantification of this bias impossible. For later survey rounds, however, the outlook is worrisome: "The rapid expansion of the program, even before households in the original control communities started receiving transfers, meant that control communities often literally were surrounded by other communities that were already receiving them. Under such circumstances, it is likely that households in the control communities may have expected to receive [Progresa] transfers before they actually started to receive them, which complicates interpretation of the estimated program effects."
The targeting mechanisms Progresa employed were the subject of much discussion and varied across time. To summarize, Progresa targeted beneficiaries using a three-tiered mechanism: (1) the selection of marginal communities according to specially-developed marginality index, (2) the selection of poor households using a multidimensional poverty line, and (3) community validation of the list of beneficiaries at a town meeting.
The weighting of different household characteristics in the calculations of household (not community) eligibility was modified during mid-1999, between the IFPRI-Progresa surveys, a process dubbed densificación or densification (in response to "the tendency of the [targeting] method to classify as beneficiaries households with more children and to exclude smaller households or older households that have no young children,"). These adjustments shifted the definition of 'poor'/eligible households and increased the number of eligible households in the sample from 53% to 78% in the treatment group, and from 50% to 78% in the control group. As of November 1999, when the final round of ENCEL surveys was conducted, "819 of the 3023 densified households [27% of those added through densification, 3% of the overall sample] had been incorporated." Densification confounds the clean interpretation of the results. Three of the final evaluation studies define 'eligible' households as only the group originally chosen for incorporation (pre-densification). Ten others employ the post-densification criteria, and finally two more experiment with both. Of course, since a significant of percentage of those granted eligibility post-densification had not received benefits by the final survey round, neither definition is truly accurate.
Despite its internal weaknesses, IFPRI-Progresa catalyzed a number of changes in Mexican governance and global international development agendas.
By the time Progresa began to form in the heads of Mexican social policy designers, the controversy surrounding the long rule of the Partido Revolucionario Institucional (Institutional Revolutionary Party, PRI) and its social policies was already entrenched. Under 'New Federalism,' president Ernesto Zedillo (1994-2000) was attempting to de-politicized social programs. One of the paramount objectives at the core of Progresa’s development, therefore, was to establish the program's apolitical status by sustaining it through the presidential switch in 2000. Vicente Fox Quesada, leader of the Partido Acción Nacional (National Action Party, PAN), won the presidential elections on July 2nd, 2000, and took office on December 1st. The IFPRI final reports were released primarily during this interval. Program supporters used the transition period to convince skeptical parties in the new administration of Progresa' effectiveness through personal contact, meetings, and media reports.
Due to the IFPRI evaluations, Progresa gained political hardiness and funding sources, while evaluation became a codified part of Mexican social programming. Fox not only continued the program, but expanded it significantly into urban areas. A loan for the amount of US$1B from the Inter-American Development Bank (IDB) for the expansion of the program was negotiated during 2001, and disbursed beginning in 2002. In 2003, the Ley General de Desarollo Social (General Law of Social Development, LGDS) explicitly codified the requirement for independent evaluation for all federally-run Mexican social programs and for this purpose established the Consejo Nacional de Evaluación (National Council on Evaluation, CONEVAL).
The excitement for the new program spread not only through Mexico, but through the IDB and World Bank. Both of these organizations extensively employed the example of IFPRI-Progresa to forward both CCTs and EBP (via 'rigorous' evaluation of social programs). The World Bank "featured the program as a model" in the 2004 Conference on Poverty in Shanghai, and in 2006 the former President James D. Wolfensohn commented "Progresa’s rigorous emphasis on evaluation has set a standard for poverty reduction programs in the developing world." The evaluation remains one of the programs' most celebrated features, having "inspired many others in Latin America and other regions." 
CCTs worldwide have drawn from the Oportunidades' model, often through direct consultation with its designers and managers. Numerous subsequent evaluations of both Oportunidades and other CCT programs have corroborated the positive effects found in the results found by IFPRI-Progresa.