This page gives an update on the Containment of Artemisinin Resistance in Eastern Myanmar project, which is being carried out by Population Services International (PSI), was recommended to Good Ventures by the Gates Foundation and is also being funded by the UK’s Department for International Development (DFID). Good Ventures contributed to this project as part of an effort to learn from major foundations by co-funding projects with them. We previously published updates on this project in April 2013 and November 2013.
Note: This update was written in July 2015 and reflects the progress of the project through July 2015.
Published January 2016
Our last update summarized major project developments through May 2013. This page summarizes developments through July 2015.
This project aims to increase use of artemisinin-based combination therapy (ACT) to treat malaria and reduce the use of artemisinin-based monotherapy (AMT) in order to prevent the development of drug resistance.
As of July 2015, the project appears to have had success in displacing a large portion of the AMT market, and ACT seems to be becoming widely available. PSI surveyed outlets that stock antimalarials and found that 54% stocked AMT in 2012, while only 10% stocked AMT in 2014. ACTs were available in 24% of outlets in 2012 and 80% in 2014. Outlets reported that AMTs accounted for 35% of the antimalarials that they sold in 2012, compared with 12% in 2014, and that ACTs accounted for 18% in 2012 and 53% in 2014. However, efforts to inform providers about best practices appear to have had mixed success: only 42% of providers named ACTs when asked what the most effective antimalarial is (up from 18% in 2012) and there is some evidence to suggest that the practice of prescribing partial courses of antimalarials has continued.
PSI has encountered several challenges in the course of the project, including unexpected complications with banning imports of AMT, difficulties managing the ACT supply chain due to changes in the prevalence of malaria, and conflict in some regions making them inaccessible to project staff.
1 Project progress
- PSI signed a contract with a second private-sector distributor, PolyGold, in July 2014, and PolyGold began distribution of its brand of ACT, Artel+, in August 2014.1 PSI reported that as of November 2014 it had sold around 1.3 million doses of ACT to its two private-sector partners.2 Sales of ACTs from PSI to distributors slowed in 2013 and 2014, which PSI attributes to declining malaria prevalence.3
- PSI now aims to encourage the use of rapid diagnostic tests (RDTs) for all fever cases before dispensing ACTs.4 PSI completed a pilot project to promote the use of RDTs.5 PSI has told us that it is interested in scaling up this program.6 RDT use is expected to have increased benefits in the context of decreasing malaria rates. In particular, appropriate diagnosis can prevent drug wastage, delay in treatment for other causes of fever, and the emergence of resistance to the non-artemisinin drugs in ACTs.7 Approval for the RDT roll out from the Ministry of Health was granted in February 20158 with the provision that PSI distribute the RDTs to the private sector free-of-charge.9 This means that PSI will not be able to use existing private sector supply chains and so distribution may be less efficient than it has been for ACT.10
- The government of Myanmar has banned the importation of most brands of AMTs, but sales of AMT are not banned and in some places sales have continued. It is unclear whether the AMTs on the market are mainly those imported prior to the ban, whether AMTs are being imported illegally, or whether the ban has failed to cover all brands of AMTs.11
- During the field visit for donors in February 2014, a DFID participant noticed that all ACTs encountered in the market were set to expire in March or April. Donors raised concerns that expired drugs could eventually trickle down to hard-to-reach areas, which led PSI to begin its replacement process slightly earlier than it had planned.12 PSI told us that by the end of April 2014, it had removed 100,000 doses of ACT from the market and that it planned to remove the other 100,000 packs by the end of May.13 We have not followed up to confirm that this happened.
- The project had spent less than it originally projected it would by early 2014, due to a delayed start and to a reduced need for ACTs relative to initial expectations.14 PSI therefore requested and received an 18-month no-cost extension (NCE), allowing it to extend its use of grant funds through March 2016. The project was originally scheduled to end in October 2014. PSI plans to conduct an additional round of surveys during the NCE period.15
- The Greater Mekong Sub-region (GMS; an area encompassing 2.6 million square miles in Cambodia, China, Laos, Myanmar, Thailand, and Vietnam)16 has shifted its strategy from resistance containment to malaria elimination due to concerns that the independent rise of artemisinin resistance in multiple locations indicates that efforts to eliminate the parasite in one region may have a limited impact on the emergence of resistance in neighboring regions. As such, GMS leaders have set a target of eliminating malaria from the region by 2030, and, in connection with that effort, PSI will expand the geographic scope of its own project.17
2 Monitoring and evaluation
In our introductory report on the project, we outlined the ways PSI planned to monitor and evaluate the project. At that time, PSI planned to collect data on the project from outlet surveys, household surveys, a supply chain assessment, mystery client surveys, exit interviews, focus group discussions, and monthly sales data.
Since our last update, we have seen results from:
- 2014 private outlet survey
- 2013 private outlet survey
- 2013 household survey
- RDT pilot project evaluation
- Mystery client surveys
- Site visit by donors in February 2014, which included visits to 10 drug outlets and 6 informal providers. GiveWell/Good Ventures participated in this visit.
In addition, DFID has commissioned an independent evaluation of the project, which Montrose International began in mid-2013. It has since produced several reports.18
2.1.1 Outlet surveys
We have not seen a detailed report on the 2013 outlet survey. PSI told us that the 2013 surveys used the same methodology as the baseline surveys (completed in 2012 and discussed in our November 2013 update) and the 2014 outlet survey (discussed below).19
PSI shared a detailed report on the 2014 outlet survey. This survey appears to have used methods that were broadly similar to those of the baseline survey.20
Townships were randomly selected for inclusion in the sample. Within each township, five urban and five rural areas were randomly selected, and all eligible outlets in those areas were surveyed (eligible outlets had malaria blood testing available or reported that they had had antimalarials in stock within the prior three months). The survey involved a provider interview and a structured questionnaire used to record information about all antimalarials and RDTs present at the outlet.21 Providers reported how many antimalarials and RDTs they had sold or distributed in the preceding week.22
The outlet surveys cover outlets in “intervention” and “comparison” sites: both are supplied with ACT by PSI’s distributors, but the comparison sites do not receive PSI’s promotional activities meant to promote uptake of ACT.23 The intervention areas are townships located along the eastern part of the country bordering with China and Thailand, and include the priority area for national artemisinin resistance containment efforts. The comparison area is near the intervention townships. We do not know how, specifically, the comparison townships were selected.24
In the results section below, we present data from both intervention and comparison groups. We find the change over time in the outlets surveyed more meaningful than differences between intervention and comparison groups. This is because (a) we are more interested in the impact of the program as a whole than the impact of the promotion component of the program, and (b) it is unclear whether the comparison townships are a suitable control group.
2.1.2 Household survey
Our November 2013 update noted that most of the people surveyed in the baseline household survey were unable to identify which type of medication they used during their most recent fever episode. The follow up household survey, completed in 2013, found that individuals were able to remember which medication they had used at much higher rates than in the baseline survey.25 It is unclear to us why the second survey was more successful than the first, and this raises concerns for us about the reliability of these results and their comparability to the baseline results. For that reason, we have not yet analyzed the results.
2.1.3 RDT pilot evaluation
PSI shared details of the methodology used to evaluate the RDT pilot project.26 Six townships were selected for the project and divided into three groups.27 In all three groups, outlets received subsidized RDTs. In group one, this is all they received. In group two, outlets received the subsidized RDTs plus financial incentives to use RDTs in the form of free ACTs or RDTs. In group three, providers received the subsidized RDTs plus intensive support visits involving education and counseling to encourage RDT use.28
Household surveys, mystery client surveys, data from RDTs and from stock resupply, and qualitative interviews with providers and staff were used to assess the project.29 For the household surveys, baseline and follow up surveys were conducted and all households that had had a member with a fever in the preceding three weeks who had taken a malaria drug or had symptoms consistent with malaria were included.30 Mystery client data was collected from randomly selected providers.31 Trained mystery clients visited outlets and said that they had a fever that they thought was like a malaria fever that they had had on a previous occasion. A researcher accompanied each mystery client, posing as a friend, and, after leaving the outlet, completed a survey form on the provider’s response.32
2.1.4 Mystery client surveys
In addition to the mystery client surveys conducted as part of the RDT evaluation, PSI has also provided some results from mystery client surveys that aimed to assess whether outlets sell full or partial courses of ACTs and provide clients with instructions for correct use.33 We have not seen the methodology or detailed results from these surveys.
2.1.5 Site visit
According to these reports:
- Availability of ACTs has increased substantially. In 2012, the Outlet Survey audit found Quality Assured ACT (QAACT) in only 24% of antimalarial-stocking outlets surveyed in the intervention areas. That increased to 70% in 2013 and 80% in 2014. In the comparison areas, QAACT was found at 30% of antimalarial-stocking outlets surveyed in 2012, 46% in 2013, and 50% in 2014.34 Meanwhile, oral AMT was found in 54% of antimalarial-stocking outlets surveyed in the intervention areas in 2012, falling to 37% in 2013 and 10% in 2014. For antimalarial stocking outlets in the comparison areas, oral AMT was found in 50% in 2012, 31% in 2013, and 24% in 2014.35
- Subsidy is largely passed on to consumers. It appears that the subsidy has been successful at keeping the price of ACT similar to that of AMT: the 2013 and 2014 outlet surveys found that, of surveyed outlets selling Supa Arte 4 (the QAACT PSI distributes through AA Medical Products),36 between 79% and 90% of outlets in both the intervention and comparison areas were selling it for less than 500 kyat,37 which PSI reports is the cost of one typical dose of AMT.38
- ACT market share has increased; AMT market share has decreased but AMT remains available. In each outlet survey, outlets were asked to report how much of each type of antimalarial they sold or distributed in the preceding week. The reported market share of QAACT as a percentage of antimalarials sold or distributed in the interventions zones was 18% in 2012, 62% in 2013, and 53% in 2014, while in the comparison zones it was 23% in 2012, 53% in 2013, and 23% in 2014. Meanwhile, the reported market share of oral AMT in intervention zones was 35% in 2012, 14% in 2013, and 12% in 2014, and in the comparison zones was 34% in 2012, 25% in 2013, and 31% in 2014.39
- Other antimalarials are frequently used. The 2014 outlet survey found that non-artemisinin antimalarials make up between 30% and 40% of the antimalarial market based on reported sales by outlets in the week preceding the survey.40 It is unclear whether these are being prescribed appropriately – while they are the correct course of treatment for one type of malaria, in cases where RDTs are not used, providers may be prescribing them inappropriately in place of ACTs.41
- Knowledge among sellers has increased but remains fairly low. The outlet surveys report the percentage of antimalarial providers that were able to correctly report the most effective antimalarial when asked. In 2012, 18% in the intervention zones and 24% in the control zones were able to respond correctly, while in 2013, 42% of intervention zone outlets and 32% of control zone outlets were able to do so, and in 2014, 50% of intervention zone outlets and 47% of control zone outlets were able to do so.42
- RDT pilot found modest gains in RDT use. The household survey conducted to evaluate the RDT pilot found that financial incentives increased the percentage of fever patients who reported using an RDT from 2.7% at baseline to 11.9%, and that education and counseling increased the percentage from 5.4% at baseline to 13% in the townships studied, while RDT usage in the townships with just the subsidy increased from 3% at baseline to 6.4% after the roll-out.43 The “mystery client” survey that PSI conducted at a random sample of outlets in the three intervention arms44 found that 65% of providers proposed performing an RDT test for malaria when a “mystery client” reported that s/he had malaria-like symptoms.45. 40% of providers both proposed performing an RDT test and also performed all 5 involved steps correctly.46
- Results from mystery client surveys are difficult to interpret. PSI reports that the year 3 mystery client survey found that only 10% of priority outlet providers prescribed a mystery client with malaria-like symptoms a full course of ACT and provided instructions for correct use, down from 29% in the year 2 survey. PSI hypothesizes that this may be due to the fact that malaria is becoming less common and so providers may be less likely to assume that a fever indicates malaria or to stock ACTs.47 We have not seen additional results or methodology from the mystery client surveys, and do not know how to reconcile the low percentage of providers that apparently distribute full courses of ACT with the higher market share of ACT. One possibility is that providers may be selling partial courses of ACTs; we have not seen data on how many providers sold mystery clients partial doses.
- During the 2014 site visit, DFID expressed a “very positive” assessment of the project’s progress.48
3 Risks to the success of the project
In our initial report on the project, we discussed several risks that we believed could prevent PSI from executing the project as planned. Here we provide updates on those initial risks:
- Targeting of subsidies: In our initial report on the project, we noted the possibility that PSI’s price subsidy could be captured by suppliers or retailers instead of reaching consumers. As noted above, it does not appear that this is occurring.
- Patient compliance with full treatment course of ACT: We noted initial concerns that patients might not purchase or complete a full course of ACT. The 2013 household survey results indicate that a substantial percentage of patients with fevers either buy or consume a partial ACT dose,49 although as discussed above we have concerns about the reliability of the data from this survey.
- Partnership with large supplier: We stated in our original report that a successful relationship with AA Medical Products would be crucial for the success of the project. To date, we are not aware of PSI having any difficulties working with AA, and, as discussed above, PSI has entered into a partnership with a second supplier, which will mitigate this risk further.
- Inherent challenges of behavior change: In our initial report, we noted that changing behavior through public advocacy seemed inherently difficult. We noted, however, that PSI sought to change behavior using both advocacy and other tools: in particular, a ban on existing AMTs and a subsidy for the replacement. As discussed above, while ACT seems to have gained market share relative to AMT, it remains the case that many providers do not prescribe it or use it correctly.
In addition, there were several risks we did not foresee. Issues that PSI has encountered which we did not note in our first report include:
- Unanticipated complexity of banning AMTs: As we noted in a previous update, PSI first believed that the government had banned AMTs and then discovered that only one brand of AMT had been banned. PSI’s advocacy to the government led to a second brand being banned. It remains unclear whether all AMTs have been banned.50
- Supply chain management under conditions of limited information: Because of declining malaria rates and a lack of information about the initial state of the market, PSI’s first order of ACTs was larger than necessary. This led to drugs expiring before they could be sold and PSI needing to replace drugs throughout the supply chain.
- Conflict-affected areas: Parts of Eastern Myanmar are inaccessible to PSI’s program and evaluation due to ongoing conflict within the regions. PSI believes that this area has high malaria rates.51 It is possible that ACT resistance could continue to develop and spread from these areas even if the program is successful elsewhere. Some anecdotal evidence indicates that these regions are receiving some subsidized ACTs.52
|Asia Development Bank Overview of the Greater Mekong Subregion||Source (archive)|
|Chris White, email to GiveWell, April 28, 2014||Unpublished|
|Chris White, email to GiveWell, May 27, 2014||Unpublished|
|DFID Annual Review (February 2014)||Source|
|GiveWell PSI Myanmar site visit notes 2014||Source|
|GiveWell Reflections on a Site Visit in Myanmar (Burma)||Source|
|Henrietta Allen, email to GiveWell, September 25, 2015||Unpublished|
|Montrose Independent Evaluation Case Study 2 Private Providers (June 2014)||Source|
|Montrose Independent Evaluation Inception Report and Evaluation Framework (August 2013)||Source|
|Montrose International Project Evaluation Report (June 2014)||Source|
|PSI AMTR Baseline Household Survey||Source|
|PSI AMTR Baseline Outlet Survey||Source|
|PSI Myanmar 2014 Outlet Survey Report||Source|
|PSI progress report (March to September 2013)||Source|
|PSI Progress Report (Oct 2014 to March 2015)||Source|
|PSI Year 3 Progress Report (Oct 2013 to Sept 2014)||Source|
|RDT Pilot Phase 1 Final Report (PSI and UCSF)||Source|
|Updated Risk Matrix (June 2015)||Source|