Policy Evaluation of Michigan’s Statewide Naloxone Standing Order & Evaluation Toolkit
Background and Purpose
In 2017, Michigan implemented a statewide naloxone standing order. This standing order allows people to get naloxone, a drug that can prevent death in the case of an opioid overdose, at participating pharmacies without an individual prescription from their doctor. Our team at DataWise conducted a policy evaluation of the standing order. The evaluation focused on Kent County, Michigan, and involved seven steps:
Identify indicators of success for the standing order through interviews and focus groups.
Create a flowchart to illustrate how the standing order works and where the eight indicators of success fit into the implementation of the standing order.
Conduct a survey of stakeholders to validate and refine the flowchart and indicators of success.
Where possible, access existing data sources for each indicator of success.
Develop and implement new evaluation tools to measure the remaining indicators of success. These evaluation tools included a community survey, pharmacist survey, and pharmacy secret caller study.
Compile data measuring the indicators of success into a scorecard.
Make policy recommendations to increase naloxone access through the standing order and other distribution methods.
Policy Recommendations
We proposed several policy recommendations to improve the standing order and increase naloxone access through other means.
Standing Order Flowchart and Scorecard
We created a flowchart to show how the standing order works and where the indicators of success fit. This flowchart became the cornerstone of the evaluation because it can identify steps that are going well and steps that should be improved. As detailed in he scorecard, six of the seven indicators of success can now be measured using a combination of previously existing data sources and data from newly developed evaluation tools.
Evaluation Tools
Infographics, key finding reports, materials, and instructions for each evaluation tool are available in this online evaluation toolkit.
-
The evaluation of Michigan’s statewide naloxone standing order started through individual interviews with key informants and online focus groups with stakeholders. Information from these interviews and focus groups were used to (a) identify indicators of success for the standing order, (b) document how the standing order works, and (c) identify barriers that prevent the standing order from having its full potential impact.
-
It is important to engage stakeholders throughout an evaluation to ensure that their ideas were heard accurately and to validate the direction of the evaluation at key milestones in the evaluation process. We invited stakeholders from Kent County, Michigan, to provide feedback on the flowchart and indicators of success. Stakeholders also reported their level of collaborations with each other, which we used to create a network map of organizations addressing the opioid crisis in Kent County, Michigan.
Method and key findings (.pdf)
Download survey file (.qsf; can be opened only in Qualtrics)
Survey export (unformatted) (.pdf)
Interactive data visualizations (link to Tableau Public)
Network map of organizations in Kent County, Michigan (link to Kumu)
-
We designed and implemented a survey of community members living in Kent County, Michigan. This survey measured public awareness, public stigma, and prevalence of community members who keep naloxone on hand in their everyday life.
Method and key findings (.pdf)
Download online survey file (.qsf; can be opened only in Qualtrics)
-
We designed and implemented a survey of Michigan pharmacists to measure pharmacists’ training, awareness, stigma, and frequency of recommending naloxone. We thank Victoria Tutag Lehr of Wayne State University and Claire Nolan of the Michigan Center for Clinical Systems Improvement (Mi-CCSI) for their partnerships with survey design. We thank the Kent County Pharmacists Association and the Michigan Pharmacists Association for their partnership with survey distribution and recruitment.
Method and key findings (.pdf)
Download online survey file (.qsf; can be opened only in Qualtrics)
Survey export (unformatted) (.pdf)
Interactive data visualizations (link to Tableau Public)
The pharmacist survey was adapted in part from:
Wayne State University Center for Urban Studies (n.d.). 2021 Pharmacist Survey on Buprenorphine for Opioid Use Disorder (OUD).
Wayne State University Center for Urban Studies (n.d.). Pharmacist Survey on Overdose Prevention and Naloxone.
Meyerson B.E., Agley J.D., Jayawardene W., Eldridge L.A., Arora P., Smith C., Vadiei N., Kennedy A., Moehling T., and the PharmNet Research Team. (2020). Feasibility and acceptability of a proposed pharmacy-based harm reduction intervention to reproduce opioid overdose, HIV and hepatitis C. Research in Social and Administrative Pharmacy, 16(5), 699-709. doi: 10.1016/j.sapharm.2019.08.026
-
Although surveys are an efficient way to collect data, self-reports have limitations. Further, potential barriers to the standing order’s success are lack of pharmacy staff awareness and concern about what others, including pharmacist staff, will think about people who request naloxone. For these reasons, we designed and conducted a pharmacy secret caller study of pharmacies in Kent County, Michigan. Evaluation team members called pharmacies, posing as community members seeking naloxone.
Method and key findings (.pdf)
Online data collection form preview (link)
Download data collection form file (.qsf; can be opened only in Qualtrics)
Acknowledgements
This evaluation was funded by the Michigan Department of Health and Human Services (MDHHS) Michigan Overdose Data to Action (MODA) program (Agreement #s: E20203597-00, E20212442-00, and E20220674-00) with funds from the Centers for Disease Control and Prevention (CDC) Overdose Data to Action (OD2A) grant. We thank the Kent County Opioid Task Force (KCOTF) leaders and members for their involvement throughout the evaluation.
Questions, Feedback, and Requests for Support
Please reach out if you have questions or would like to use or adapt these evaluation tools Please email info@wearedatawise.com or laura.luchies@wearedatawise.com.