Performance improvement CME in psychiatry: implementing black box warning informed consent


Performance improvement CME in psychiatry: implementing black box warning informed consent

David Mintz1, Henry Tulgan2, Mikaela Mintz3 and Danna G. Muir4

1Psychiatric Education, Austen Riggs Center, Stockbridge, MA, USA
2Department of Medicine, University of Massachusetts Medical School, Worcester, MA, USA
3Austen Riggs Center, Stockbridge, MA, USA
4Department of Continuing Medical Education, Harvard Medical School, Boston, MA, USA


Objective. Performance improvement continuing medical education (PI CME) is a recent educational methodology designed better to link educational content and outcomes in the context of limitations in the effectiveness of traditional CME. This study examines the ease of use and effectiveness of psychiatric CME in a small hospital. Methods. All staff psychopharmacologists assessed their performance in providing informed consent of black box risks of prescribed psychiatric medications in a 3-month period. Staff were educated regarding black box risks of all commonly prescribed psychiatric medications, and their performance in the following quarter was reassessed. Significance of change following the educational intervention at 3 and 6 months was determined by Chi square analysis. Results. PI CME was clearly successful in supporting behaviour change, χ2(1, N=60) =20.86, p=0.000, far outstripping traditional CME efforts. Changes in behaviour persisted over time, χ2(1, N=61)=4.04, p=0.044. This PI CME event received the highest possible rating by CME participants, and took few staff resources to implement. Conclusions. PI CME is an educational technology that can be carried out, without significant burdens to participants or educators, in psychiatric departments of small hospitals. PI CME can be much more effective than traditional CME in bringing about desired behaviour change in psychiatrist behaviours. Hospital-based PI CME may have other benefits, such as meeting criteria for Maintenance of Certification.

Keywords: Performance improvement, continuing medical education, informed consent, black box warning, psychiatry

Received: 31 March 2014 Accepted: 6 June 2014 Published online: 27 June 2014

Journal of European CME (JECME) 2014. © 2014 David Mintz et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 Unported (CC BY 4.0) License (, allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Citation: Journal of European CME 2014, 3: 24501 -


Since 2000, a series of reports from the Institute of Medicine (IoM) have called into question the traditional principles that physicians have traditionally relied on in pursuit of continued competency.14 Though physicians now have a nearly endless supply of information at their fingertips through the Internet, the sheer volume of information and the rapidity of change have complicated the dissemination of new knowledge and adoption of new practices. A “quality chasm”2 has emerged, where average clinical practice does not meet its potential based on available knowledge and technologies.

One remedy is the development of greater linkages among education, accreditation, certification, and licensure.3 In addition, policy initiatives are increasingly promoting models of education that are not knowledge-centred, but rather focused on competencies or clinical outcomes.5,6 To this end, new educational formats have been developed and approved for continuing medical education (CME).7,8 Internet point-of-care models allow physicians to receive educational credit for learning “just in time” to provide competent and appropriate care. This focused learning contrasts with more traditional models of “just in case” learning where large amounts of information are learned and stored for possible future use. Integration of performance improvement (PI) with CME is another strategy for linking education directly with outcomes.

This study reports on one small institution’s efforts to educate staff, promote behaviour change, and foster patient-centred care through PI CME focused on the enhanced provision of informed consent. Educational results show that PI CME can be easy to implement, effective where other educational interventions have failed, and highly appreciated by psychiatric learners.

Performance improvement and continuing medical education

CME has long been deemed a core element in ensuring on-going competency in the practice of physicians. In order to promote competent care that integrates new knowledge and contemporary standards of care, physician participation in CME has become a requirement of most state licensing boards for continued licensure. Likewise, specialty boards establish CME requirements as prerequisites for Maintenance of Certification (MOC). However, as educational and research technology has advanced, it has become apparent that the most prevalent methodologies for providing CME do not effectively achieve the most fundamental goals of CME. While commonly applied didactic learning formats (e.g., lectures) may help physicians achieve personal goals of accumulating required CME credits, there is mounting evidence that such passive learning formats do little to change medical behaviours.911

Following the recommendation of two national task forces promoting more effective CME, the American Medical Association recognised, in 2005, a novel learning format: performance improvement CME (PI CME).7 PI CME is an effort explicitly to join CME with measures of change in performance and with the American Medical Association Physician’s Recognition Award (AMA PRA) Category 1 Credit system. PI CME represents a shift from the traditional time-based credit to a model of value-based credit.12 In recognition of the potential superiority of this type of educational activity and in order to further promote implementation of this learning format, the AMA structured the credit system such that completion of accredited PI CME events would earn 20 Category 1 credits, though time spent in the CME activity could be substantially less than 20 hours.

PI CME, as defined by the AMA PRA system, is a three-part educational event that involves elements of performance assessment coupled with education intended to promote positive change in physician behaviours. Stage 1 entails learning from a performance assessment of current practice. Stage 2 entails learning from the application of PI to patient care. In Stage 3 of PI CME, learning is consolidated by follow-up performance assessment to evaluate changes in practice as a result of the PI CME event.

PI CME and psychiatry

There is, in psychiatry, as in other medical fields, a need to integrate quality improvement into individual practice and systems of care.13 In 2010, the American Board of Psychiatry and Neurology responded to these needs by revising its requirements for MOC for psychiatrists who were certified after 1 October 1994.14 Beginning in 2013, psychiatrists who wish to maintain board certification must complete three Performance-in-Practice (PIP) Clinical Modules every 10 years, which involves assessing practice in some important clinical domain, and reassessing practice for evidence of improvement. PI CME also carries the potential benefit of meeting board re-certification criteria for PIP.

Medical institutions that offer CME credits can support the educational and professional needs of physicians in those systems by incorporating PI CME into the range of learning formats that are offered. PI CME programmes also benefit patients by enhancing positive outcomes and/or reducing negative outcomes. Health care systems that join PI and CME are benefited by enhanced outcomes, reduced medico-legal risk, and satisfying requirements of regulatory bodies (e.g., JCAHO).15 To date, however, there are no articles in the peer-reviewed medical literature that describe institutional implementation of psychiatric PI CME programmes. A search on PubMed for “Performance Improvement CME” and “psychiatry” yields only six results,1621 all of which are themselves journal-based PI CME for individual practitioners.

Despite the myriad benefits of integrating PI with CME, there is a perception of significant impediments to implementation of PI CME. Already overburdened physicians may prefer passive learning, and may balk in relation to the perception that PI is an onerous mandate. Physicians may also demur from the experience of being evaluated. CME providers, too, may be slow to adopt this new learning model because of the perception that planning PI CME is more labour-intensive than planning other CME,15 and that “PI CME required unprecedented collaboration between CME planners and QI stakeholders to enable change of clinical practice.”22

These impediments may exist more in anticipation than in actual practice. The authors describe briefly the implementation of a PI CME programme at a small psychiatric hospital. The results of this study suggest that not only is PI CME a superior modality for effecting behaviour change in physicians, but it is also relatively easy to implement and highly appreciated by staff.

Informed consent: a case study of PI CME

Informed consent is a basic procedure in the practice of medicine. Ethically, it is important that patients be empowered to weigh the potential risks against potential benefits when choosing among treatment options. Informed consent is a key element in providing patient centred care, and has become a current IoM focus,23 given evidence that as few as 9% of patients are provided adequate informed consent for treatment.24 Beyond this, there are also important clinical reasons for providing thorough informed consent. There is mounting evidence that aspects of the doctor–patient relationship such as “good communication” can promote improved medication outcomes.25 Careful and thorough disclosure of risks, contrary to common perception, appears to promote medication adherence.26 Such alliance-based interventions may be especially important in patients who present with a history of “treatment-resistance.”27,28

The Austen Riggs Center (ARC) is a small, not-for-profit psychiatric facility specialising in the care of complexly co-morbid patients often labelled as treatment resistant. With an overarching psychodynamic orientation, alliance-based interventions and strategies aimed at maximising the authority of patients are often at the core of efforts to address treatment resistance, including pharmacologic treatment resistance.

A system-based practice gap was identified in relation to informed consent regarding black box warnings for psychiatric medications. In 2004, the U.S. Food and Drug Administration (FDA) extended its black box warnings to all antidepressant agents.29 This was the beginning of a proliferation of warnings associated with psychiatric drugs. A review of practice patterns at the ARC revealed that informed consent about black box risks was not documented in a majority of cases for patients newly admitted (already on those drugs) or newly prescribed drugs associated with black box warnings. Consultation with the facility’s insurer confirmed that failure to provide informed consent is a significant source of successful lawsuits against psychiatrists. Given an institutional commitment to fully authorising patients as partners in medical decision-making, enhancing informed consent was a natural target for PI.

More traditional efforts to affect behaviour change produced no significant improvement. These efforts included education about medico-legal risks of failure to provide informed consent, regular reminders at pharmacology staff meetings and reviews of the overall compliance of pharmacology staff. A prompt was also built into the digital medication ordering system, identifying medications associated with a black box warning. This also did not produce significant improvement in performance by physicians.

The Director of Medical (Psychiatric) Education (DME), in conjunction with the Center’s PI Committee, determined that PI CME could be a more effective approach to improve the response for this identified practice gap while also serving to provide the prescribers with education that could satisfy state requirements for Maintenance of Licensure, (MOL) and fulfil criteria for MOC.


The DME and the Education Committee at ARC reviewed patient records following traditional education activities and confirmed that the gap between the actual and ideal practice of provision of informed consent was still present. This was sufficient to identify the learning need as both a performance and a patient outcome issue. Subjects consisted of all 10 of the Center’s credentialed psychopharmacologists. Following AMA guidelines, an educational activity was devised which included an assessment by each participant of his or her performance in documenting informed consent about black box warnings for psychiatric medications. They subsequently participated in a lecture format educational session followed by an evaluation by each participant of his or her performance. Reluctances to engage were ameliorated by a firewall between these performance assessments, done for educational purposes, and performance measures that might affect professional standing within the organisation.

For Stage A, the DME devised a simple algorithm within the electronic medical record for identifying patients prescribed medications associated with black box warnings within the preceding 3 months. Less than 2 hours of DME time was spent on developing the search algorithm.

During a regular pharmacology staff meeting, prescribers were instructed in this procedure and were also provided a hand-out identifying all black box warning associated with commonly prescribed psychiatric drugs, and a data collection sheet to assess their performance on this measure. A regular meeting time was used both because it was easiest to gather all the prescribers at this time, and because this format did not require any additional time commitment on the part of the prescribers. Participants were able to identify all applicable patients, review their pharmacology notes for clear documentation of informed consent, and record performance within the one hour allotted for the pharmacology staff meeting. For this activity, participants were awarded five CME credits for their completion of Stage A.

Stage B consisted of a one-hour multimedia presentation reviewing black box warning associated with psychiatric medications and ethical and medico-legal implications of failure to provide informed consent. Research and preparation for this presentation required less than 6 hours of DME time.

Stage C occurring 3 months after Stage B used a procedure identical to Stage A. Prescribers assessed their performance on providing and documenting informed consent and compared their pre-and post-CME performance for evidence of behaviour. Stage C required no additional DME time and 1 hour of regularly scheduled meeting time on the part of each participating physician. Participants who completed all three stages earned a total of 20 CME credits.

The protocol was reviewed and approved by the Center’s Institutional Review Board.

After data collection by participants, the Center’s PI Committee and Education Committee gathered data for subsequent analysis. Pre and post-CME data were subjected to statistical analysis (Chi square) in SPSS, to determine whether this intervention effected significant change. Measures were taken again at the end of the subsequent quarter to determine whether the PI CME event produced changes that persisted over time. The DME and Center’s Education Committee collected participant ratings of this educational event for routine assessment. Participant ratings for this event were compared to participant ratings of every CME event in the same academic year, with means and standard deviations calculated in SPSS.


In the 6 months prior to the PI CME intervention, average compliance with documenting informed consent about black box warnings was only 11.6%, despite staff education regarding black box warnings associated with common psychiatric medications, and repeated reminders to prescribing staff of the importance of documenting informed consent. In the following quarter, average compliance rose to 70.6%. PI CME was clearly much more successful in supporting behaviour change than traditional CME efforts, χ2(1, N=60)=20.86, p=0.000. Average compliance as measured at 6 months following the intervention had fallen to 55%, though this still represented a significant change with regard to the period prior to the PI CME event efforts, χ2(1, N=61)=4.04, p=0.044. This suggests that significant change from PI CME can be sustained over time.

Analysis of participant ratings (n=6) of the educational event revealed that this PI CME event was highly rated, with an average overall rating of 5.0 (on a scale of 1 to 5, with 5 indicating highest marks). The average overall score for this CME provider was 4.55 (SD=0.36), suggesting that this form of CME was easily palatable, and likely preferable, for its intended audience.

The demonstrated success of this activity was presented within the intrastate re-accreditation survey of the Center in January 2011 by The Massachusetts Medical Society (MMS) Committee on Accreditation Review, and recognised as a potential model of PI CME within the state.


PI CME holds promise as an innovative educational technology which can promote effective improvements in clinician behaviours in ways that are unparalleled by more traditional CME methodologies. This study provides further support for the rationale behind PI CME: this intervention, embedded in a PI CME learning format, successfully closed a specific learner gap when other traditional educational methods proved ineffective. PI CME was clearly more effective than traditional formats at bringing about and sustaining desired behaviour change.

Since the incorporation of PI CME into the AMA credit system, there have been several reports of successful institutional examples which include, among others, primary care,21 cardiology,30 and gynaecology.31 However, in the psychiatric literature, published examples have been limited to online CME.1621 This study is, to our knowledge, the only contribution thus far to the peer-reviewed literature on implementing local institutional PI CME in psychiatry. The general paucity of examples of institutional PI CME may reflect resistances to integrating new educational models, particularly when there are concerns that these new models will impose added burdens on DMEs and educational participants alike.

This study shows that PI CME can be implemented without imposing significant burdens on planners or participants. From start to finish, this event required fewer than 8 hours of DME time. Participants spent 3 hours each on the activity, but this was during time that was already committed to regularly scheduled meetings. It is likely, in fact, that this methodology will prove to be a significant time-saver for most physicians. Given the increasing complexity of health care systems and regulatory requirements, PI CME may integrate, into a single event, processes that satisfy requirements for MOL, MOC, and credentialing.12

In addition to the 20 Category 1 credits awarded for participation in this activity, participants reaped numerous other rewards. At the conclusion of this event, participants carried less medico legal exposure in relation to failure to document informed consent discussions. Institutions providing PI CME support their physicians in other important ways. Participants in this educational event benefited by participation in PIP self-assessments, potentially meeting state board requirements for MOL. This format also has the potential to help participants meet board requirements for MOC. Far from finding the process onerous, participants rated the event very highly.

Patients also benefited by becoming more informed participants in medical decision making. Though no specific effort was made to correlate this study with broader outcome measures, it is likely that this intervention also enhanced clinical outcomes in a population that was generally treatment refractory. This activity addressed a number of factors in the doctor-patient relationship and prescribing process that are IoM priorities and which are known to enhance pharmacologic treatment outcomes. These factors include involvement in medical decision making,3234 good communication,3537 and providing information about side effects.26

There are a number of potential limitations with regard to this research. Most significantly, the setting, as a small, long-term psychoanalytic hospital, is generally unique. The combination of small size and treatment philosophy based on communication and patient authority may limit the generalisability of these results to other settings. This event looked only at documentation of informed consent. There was no assessment of the effectiveness of the informed consent itself (i.e., evidence that the patients understood and made use of the information provided to them).


PI CME is an educational technology whose time has come. It may be particularly effective in ameliorating practice gaps when more traditional CME has failed to bring about adequate behaviour change. In the context of increasing requirements for PIP assessments in the field of psychiatry, there will likely soon be a proliferation of vendors with a national scope who offer PI CME as an aspect of PIP and MOC. PI CME need not be complicated, however, and can easily be implemented by small intrastate accredited providers in ways that address local practice gaps or which focus on unique institutional concerns while addressing physician need for PIP assessments. PI CME need not strain staff resources, and is rated highly by learners. This PI CME activity designed to increase informed consent regarding black box warnings was highly successful, both in improving physician performance and in regard to overall ratings by participants. The psychiatric literature could benefit from more examples of PI CME, as educational experimentation is undertaken and PI CME begins to enter the mainstream.


The authors would like to thank their colleagues who participated in this study.

Declaration of funding
This study was unfunded, and the authors have no commercial affiliations that could represent a possible conflict of interest.


  1. Institute of Medicine (IOM). To err is human: building a safer health system. Washington, DC: National Academy Press; 2000.
  2. Institute of Medicine (IOM). Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.
  3. Institute of Medicine (IOM). Health professions education: a bridge to quality. Washington, DC: National Academy Press; 2003.
  4. Institute of Medicine (IOM). Redesigning continuing education in the health professions. Washington, DC: National Academy Press; 2009.
  5. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med 2002;77:361–367. PubMed Abstract | Publisher Full Text
  6. Calhoun JG, Davidson PL, Sinioris ME, Vincent ET, Griffith JR. Toward an understanding of competency identification and assessment in health care management. Qual Manag Health Care 2002;11:14–38. PubMed Abstract | Publisher Full Text
  7. The Physician’s Recognition Award and credit system. 2010 revision:7–8.
  8. Aparicio A, Willis CE. The continued evolution of the credit system. J Cont Edu Health Professions 2005;25:190–196. Publisher Full Text
  9. Cantillon P, Jones R. Does continuing medical education in general practice make a difference? BMJ 1999;318:1276–1279. PubMed Abstract | PubMed Central Full Text | Publisher Full Text
  10. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behaviour or health care outcomes? JAMA 1999;282:867–874. PubMed Abstract | Publisher Full Text
  11. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274:700–705. PubMed Abstract | Publisher Full Text
  12. Dorman T, Miller BM. Continuing medical education: the link between physician learning and health care outcomes. Acad Med 2011;86:1339. PubMed Abstract | Publisher Full Text
  13. Oldham JM, Golden WE, Rosof BM. Quality improvement in psychiatry: why measures matter. J Psychiatr Prac 2008;14(Suppl 2):8–17. Erratum in J Psychiatr Pract 2008;14:411. Publisher Full Text
  14. Faulkner LR, Tivnan PW, Winstead DK, Reus VI, Andrade NN, Brooks BA, et al. The ABPN Maintenance of Certification Program for psychiatrists: past history, current status, and future directions. Acad Psychiatry 2008;32:241–248. PubMed Abstract | Publisher Full Text
  15. Lowe MM, Aparicio A, Galbraith R, Dorman T, Dellert E, American College of Chest Physicians Health and Science Policy Committee. The future of continuing medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest 2009;135;69S–75S. PubMed Abstract | Publisher Full Text
  16. Gelenberg AJ, Thase ME. Performance improvement CME: improving outcomes in depression. J Clin Psychiatry 2010;71:e19. PubMed Abstract | Publisher Full Text
  17. McIntyre RS, Wagner KD. Performance improvement CME: long-term treatment of bipolar disorder. J Clin Psychiatry 2011;72:e09. PubMed Abstract
  18. Adler LA, Barkley RA, Newcorn JH. Performance improvement CME: adult ADHD. J Clin Psychiatry 2011;72:e15. PubMed Abstract | Publisher Full Text
  19. Kane JM, Correll CU. Performance improvement CME: managing schizophrenia. J Clin Psychiatry 2011;72:e31. PubMed Abstract | Publisher Full Text
  20. Shelton RC, Trivedi MH. Performance improvement CME: algorithms and EMRs in depression. J Clin Psychiatry 2011;72:e29. PubMed Abstract | Publisher Full Text
  21. Kane JM. Performance improvement CME: schizoaffective disorder. J Clin Psychiatry 2011;72:e23. PubMed Abstract | Publisher Full Text
  22. Shershneva MB, Mullikin EA, Loose AS, Olson CA. Learning to collaborate: a case study of performance improvement CME. J Cont Edu Health Prof 2008;28:140–147. Publisher Full Text
  23. Institute on Medicine (IOM). Committee on the health professions education summit. Health professions education: a bridge to quality. Washington, DC: The National Academies Press; 2003.
  24. Braddock CH, Edwards KA, Hasenberg NM, Laidley TL, Levinson W. Informed decision making in outpatient practice: time to get back to basics. JAMA 1999;282:2313–2320. PubMed Abstract | Publisher Full Text
  25. Mintz DL, Flynn DF. How (not what) to prescribe: non-pharmacologic aspects of psychopharmacology. Psychiatr Clin North Am 2012;35:143–163. PubMed Abstract | Publisher Full Text
  26. Bull SA, Hu XH, Hunkeler EM, Lee JY, Ming EE, Markson LE, et al. Discontinuation of use and switching of antidepressants: influence of patient–physician communication. JAMA 2002;288:1403–1409. PubMed Abstract | Publisher Full Text
  27. Mintz DL, Belnap BA. A view from Riggs: treatment resistance and patient authority – III. What is psychodynamic psychopharmacology? An approach to pharmacologic treatment resistance. J Am Acad Psychoanal Dynam Psychiatry 2006;34:581–601. Publisher Full Text
  28. Mintz DL, Belnap BA. What is psychodynamic psychopharmacology? An approach to pharmacological treatment resistance. In: Plakun EM (Ed.), Treatment resistance and patient authority: an Austen Riggs Reader (pp. 42–65). New York: Norton; 2011.
  29. Leon AC. The revised black box warning for antidepressants sets a public health experiment in motion. J Clin Psychiatry 2007;68:1139–1141. PubMed Abstract | Publisher Full Text
  30. Cannon CP, Hoekstra JW, Larson DM, Carter RD, Cornish J, Karcher RB. Physician practice patterns in acute coronary syndromes: an initial report of an individual quality improvement program. Crit Pathways Cardiol 2010;9:23–29. Publisher Full Text
  31. Wachtel MS, Hatley WG, de Riese C. Using Poisson regression to compare rates of unsatisfactory pap smears among gynecologists and to evaluate a performance improvement plan. Acta Cytol 2009;53:160–164. PubMed Abstract | Publisher Full Text
  32. Loh A, Simon D, Wills CE, Kriston L, Niebling W, Härter M. The effects of a shared decision-making intervention in primary care of depression: a cluster-randomized controlled trial. Patient Edu Counsel 2007;67:324–332. Publisher Full Text
  33. Woolley SB, Fredman L, Goethe JW, Lincoln AK, Heeren T. Hospital patients’ perceptions during treatment and early discontinuation of serotonin selective reuptake inhibitor antidepressants. J Clin Psychopharm 2010;30:716–719. Publisher Full Text
  34. Clever SL, Ford DE, Rubenstein LV, Rost KM, Meredith LS, Sherbourne CD. Primary care patients’ involvement in decision-making is associated with improvement in depression. Med Care 2006;44:398–405. PubMed Abstract | Publisher Full Text
  35. Lin EH, Von Korff M, Katon W, Bush T, Simon GE, Walker E. The role of the primary care physician in patients’ adherence to antidepressant therapy. Med Care 1995;33:67–74. PubMed Abstract | Publisher Full Text
  36. Bultman DC, Svarstad BL. Effects of physician communication style on client medication beliefs and adherence with antidepressant treatment. Patient Edu Counsel 2000;40:173–185. Publisher Full Text
  37. Ciechanowski PS, Katon WJ, Russo JE, Walker EA. The patient–provider relationship: attachment theory and adherence to treatment in diabetes. Am J Psychiatry 2001;158:29–35. PubMed Abstract | Publisher Full Text

Correspondence: Dr. David Mintz, Austen Riggs Center, Stockbridge, MA 01262, USA. Tel: +(413) 931 5315. Fax: +(413) 298 4020. E-mail:;
Dr. Henry Tulgan, 25 Palomino Drive, P.O. Box 962, Pittsfield, MA 01201, USA. Tel: +(413) 442 4395. Fax: +(413) 499 1874. E-mail:

About The Authors

David Mintz

United States

Henry Tulgan
University of Massachusetts Medical School
United States

Mikaela Mintz

United States

Danna G. Muir

United States

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Related Content