Published: April 2016
Note: Prof. Hawken reviewed this page prior to publication.
This page gives both (a) the original rationales behind two grants made in 2014 to support projects led by Professor Angela Hawken, and (b) updates on the progress of those two grants since then, as of December 2015. Note that we have an existing write-up for the first grant, but hadn’t previously written about the second, as responsibility for the grant changed hands several times (a brief page with the details of the grant is here).
Prof. Hawken reviewed a draft of this page prior to publication.
- Grant to BetaGov
- Grant to support research on the use of marijuana and other drugs in Washington State
- Reflections and lessons learned from both grants
- Our plans going forward regarding Prof. Hawken’s work
Grant to BetaGov
Original rationale for the grant and proposed activities
Between late 2013 and early 2014, we had not yet chosen focus areas within U.S. policy, and were conducting a very early exploration of potential focus areas. We saw grantmaking as a key part of our explorations.1 We were referred to Dr. Hawken by two of the people we had been working with to explore the space of criminal justice reform. We made a $200,000 grant to seed her work on BetaGov, which we described at the time as follows:
BetaGov aims to generate knowledge about what works in the public sector (in areas including but not limited to criminal justice) by serving as a repository for practitioners’ ideas to be tested, serving as a database of results to facilitate learning across studies, and providing a toolkit (including web-based training, webinars, assessment tools, and an RCT call-in hotline) so that practitioners can conduct their own RCTs.
Details (including the above quote) are at our original write-up.
Prof. Hawken provided us with a long-form write-up on her project that she had submitted to the Department of Justice (which did not provide funding). This document is not public, and the project’s name and vision changed between that proposal and our grant. Prof. Hawken stated to us that she was ideally hoping to raise a total of $2 million over four years; the $200,000 we provided was approximately half of what would be needed for the first year. We requested but did not receive a more detailed budget. As discussed in more detail below, the update on the project we received in December 2015 stated that BetaGov had made progress in initiating trials, and had only spent around $130,000 of our original grant.
Progress to date
We spoke with Prof. Hawken about the project’s progress in March 2015, and received a written update on BetaGov’s progress in December 2015.2 As of that time:
- BetaGov had expanded to include trials in the area of health and human services (HHS), as well as criminal justice.
- The project had been successful in initiating a number of trials in both criminal justice and HHS. The most recent list that we received included 78 trials, although the details of many of these are not yet public.
- The BetaGov website had been launched, and now includes a virtual map of some of the trials, each with a very brief description, status, and location.
- The project had grown to a team of 12 from across several fields.3
- About $130,000 of the original $200,000 grant had been spent; Prof. Hawken had not raised further funding to support the project specifically, though she had been able to effectively support the project further via donated time (hers and others’). 4
Overall, our impression is that BetaGov is going well in that it is successfully initiating large numbers of small-scale policy trials. However, we currently don’t know when we’ll see the results of these trials, and have generally found it difficult to stay up to date on the project, verify its impact (due to the lack of public content, coupled with our limited capacity for investigation) and share information about it. The addition of the map of trials to the website is helpful, but does not include any information about when trials are likely to be completed or results released.
Grant to support research on the use of marijuana and other drugs in Washington State
Original rationale for the grant and proposed activities
In March of 2014, Prof. Hawken contacted us about a research opportunity separate from BetaGov. She stated that due to changes in Washington State law, the Washington State Department of Corrections was planning to stop testing people under supervision for THC (the active ingredient in marijuana). Prof. Hawken believed that this change presented an opportunity to study the potential impact of marijuana legalization on use of both marijuana and other illicit drugs. By gathering all urine samples collected by the Dept. of Corrections, and then testing them for THC (for academic rather than disciplinary purposes), she hoped to understand how the policy change affected use of both marijuana and of other illicit drugs that were already being tested for.5 The sudden nature of the policy change (in our view and Prof. Hawken’s) presents an opportunity for an unusually rigorous study because it provides an unusual opportunity to isolate the effect of removing a penalty for marijuana use, with relatively low risk of getting this effect confounded with other changes going on around the same time.
Prof. Hawken believed she had a good chance of being funded by the NIH, but that any funding wouldn’t be committed in time for her to start sample collection immediately after the policy change - which could be important for the creation of a compelling “natural experiment” that would isolate the impact of the policy change. She requested funding from us in order to collect samples until she had a decision from the NIH. She estimated costs of about $30,000 per month, with most of this paying for THC test strips (~$6,000 per month), shipping (~$6,000 per month) and a tester (~$10,000 per month). She estimated that she would need four months of support before hopefully receiving funding from the NIH.
We did a cursory review of the literature to confirm our impression that this could be an unusually rigorous study on a question directly relevant to a rapidly-shifting policy area. We were still early in our work on U.S. policy, and hadn’t yet determined whether marijuana policy would be a focus area (this determination was still ongoing as of March 2015). However, because we were considering it as a focus area and were putting weight on the positive referrals we’d had to Prof. Hawken previously, we provided $150,000. The grant was made in early June of 2014.
Progress to date
When we spoke with Prof. Hawken about both BetaGov and this project in March 2015, she had not ended up receiving funding from the NIH, and did not believe funding would be forthcoming. In addition, there had been delays in beginning sample collection. However, costs of the project had been lower than expected (Prof. Hawken stated that this was due to her ability to test more samples per hour than anticipated), and only about $80,000 of the funds we’d provided had been spent. Prof. Hawken believed she had already collected enough data to make a compelling statement, with strong statistical significance, about the impact of removing the penalty for marijuana use (we are not revealing the conclusion here because Prof. Hawken has not yet published her results). At that time, we did not receive clearance to publish supporting documents we received, or notes from the conversation.
In December 2015, Prof. Hawken provided us with a written update and backward-looking budget for the project, both suitable for publishing; according to the budget, the final costs of the project were around $120,000 (from our original grant of $150,000).6
As far as we know, the key result from this project has not yet been published; we have seen a preliminary paper that we believe Prof. Hawken plans to publish.
Reflections and lessons learned from both grants
These grants were among the first we made, and came well before we had set focus areas for U.S. policy. We didn’t yet have strong frameworks or stable staff allocations for following grants, and these have caused us to have a difficult time following and providing updates on this grant. In particular:
- Our interest in criminal justice reform was very preliminary. We hadn’t yet mapped the space or thoroughly investigated which aspects of criminal justice reform we wanted to focus on.
- Responsibility for these grants changed several times. Holden Karnofsky (Executive Director of the Open Philanthropy Project) was the initial point of contact with Prof. Hawken, leading the investigation process for the BetaGov grant and initiating the process for the Washington THC grant, but he did not have the capacity to continue following them. More than once, responsibility was passed to another staff member whose duties would later change; thus, responsibility changed hands multiple times.
- We didn’t yet have a standard grant agreement or formalized internal grant process, and we did not set clear enough expectations with Prof. Hawken regarding public updates about the grants.
In addition, Prof. Hawken has - in some cases - not provided the information (and clearance for publication) we’ve requested on the timelines that we’ve agreed to. We have generally encouraged Prof. Hawken to prioritize programmatic work over providing us with updates or approving our materials for publication, so we bear some responsibility for this.
Partly due to waiting on clearance to publish materials and partly due to other issues noted above, we are writing up the Washington THC grant almost two years after the grant was made. We believe we will be better able to avoid issues like this in the future due to a more formalized grant process (which includes clearly communicating and agreeing with the grantee about expectations for public updates) and a better-developed overall strategy for U.S. policy.
With that said, we do have the impression that both projects have progressed well while using a relatively small amount of the funding we’ve provided, and we expect eventually to have public information that will give a clearer idea of their progress.
Our plans going forward regarding Prof. Hawken’s work
We do not believe that Prof. Hawken has a pressing need for additional funding. It is possible that we might provide more support to BetaGov in the future, depending on what we learn about completed trials and the project’s future plans. At present, we don’t plan to prioritize further investigation into the progress of these two grants.
|BetaGov, December 2015 Financials||Source|
|BetaGov, December 2015 Update||Source|
|Washington THC project, December 2015 Financials||Source|
|Washington THC project, December 2015 Update||Source|
For details, see:
- Near-term grantmaking on how we were thinking about grants at the time.
- Deep investigations of new causes on our decision to use grantmaking as a tool for investigating causes.
- The “In a nutshell” (top) section of our 2014 criminal justice reform cause report for why we prioritized criminal justice reform.
- 2. BetaGov, December 2015 Update
- 3. “GoodVentures/GiveWell provided the initial core funding to the creators of BetaGov, which has grown into a 12-person interdisciplinary research team (PhD-level policy analysts, economists, psychologists, statisticians, a professional communications analyst/writer, and support staff; see BetaGov.org).” BetaGov, December 2015 Update
- 4. See BetaGov, December 2015 Financials
“Washington legalized the recreational use of marijuana in 2012. As of June 1, 2014, to align with state policy, the Washington Department of Corrections (WADOC) discontinued testing for THC. An important unknown clouding the policy change in Washington State is whether this reform will lead to an increase in marijuana use and, if so, whether this will result in a change—increase or decrease—in the use of other drugs and violating behavior. The policy change in Washington State presents a unique opportunity for an unprecedented natural experiment.” Washington THC project, December 2015 Update pg. 1