key: cord-0800381-pr6c2cyq
authors: Abella, Anna Davidson; Landers, Monica; Ismajli, Flandra; Carmona, Yaritza
title: Stakeholder Perspectives on Implementing a Police-Mental Health Collaborative to Improve Pathways to Treatment
date: 2022-01-09
journal: J Behav Health Serv Res
DOI: 10.1007/s11414-021-09782-0
sha: 8cac0b7e2dfe33f904aad8f8a1497733f758136b
doc_id: 800381
cord_uid: pr6c2cyq

High rates of criminal justice involvement among individuals with mental illness have led to collaborative efforts between law enforcement agencies and mental health providers to improve crisis responses and pathways to treatment. The development and implementation of these police-mental health collaborations (PMHCs) have received little attention in the literature, but these processes are crucial in understanding feasibility and sustainability. The PMHC discussed here is an interagency effort to identify individuals involved with law enforcement who have unmet behavioral health needs and engage them in services. Perspectives from leaders, service providers, and clients highlight the importance of developing PMHCs that support individuals with serious mental illness at multiple points, from initial crisis to independent management of treatment. In an environment where police responses to individuals with mental health and substance use disorders are increasingly scrutinized, it is critical to highlight and evaluate ways that behavioral health and law enforcement agencies work together to collaboratively address these problems.

The overrepresentation of individuals with serious mental illness (SMI) in the criminal justice system has been well documented. Recent estimates suggest that the prevalence of serious mental illness is at least three times higher among incarcerated individuals than in the general population. 1, 2 Police continue to act as default responders to emergencies involving individuals with mental illness, due in large part to an inadequate infrastructure for meeting the full spectrum of mental health needs, from crisis response, to access to treatment, and to sustainably funded comprehensive services. [3] [4] [5] [6] To harness this extant relationship, law enforcement and behavioral health agencies have developed a number of collaborative responses over the past few decades, with the aim of diverting individuals with serious mental illness from the justice system. [7] [8] [9] [10] [11] Some scholars have identified key system junctures where coordinated responses can help prevent further justice system involvement among individuals with SMI and co-occurring disorders, such as the sequential intercept model 12 or integration of police in development of the crisis care continuum. 13 There has been particular interest in further developing "Intercept 0", 14 points in the justice system trajectory where early intervention and immediate coordinated responses may be especially beneficial for individuals with mental illness.

One prominent example of an early coordinated response model involves officer and mental health professional "co-response" units who work together to provide crisis or post-crisis intervention and connect individuals with mental health resources. Spurred by the crisis intervention team (CIT) model developed in the late 1980s, 15 co-response models have gained popularity over the past few decades, though they vary significantly in structure and design and are typically not utilized as a "one size fits all" approach due to differences in agency capacity, partnership approaches, and funding. The majority of available studies on co-response models assess outcomes related to reducing criminal justice and health system burden by examining indicators such as reducing utilization of crisis stabilization services or hospital emergency departments, and decreasing financial costs and organizational burdens to jails and courts in managing cases involving mental illness. 11, 13 Many of these outcomes are mixed, though there has been some consistent indication of PMHCs showing reductions in arrests and costs to the criminal justice system. [9] [10] [11] Less is known about what happens after individuals are diverted from criminal justice involvement or involuntary commitment, such as what kinds of connections are made to services after referral and what kinds of mental health outcomes clients have; in many cases, outcomes are not reported beyond referral or transportation to mental health services. 16 Importantly, the research literature includes very little documentation or analysis of the implementation of co-response models and is missing crucial voices from stakeholders and clients, with the exception of some studies that have assessed officer perspectives on mental illness. 11, 17, 18 Insights from leaders, service providers, and clients are especially critical in understanding what factors facilitate or inhibit effectiveness, and in understanding the extent to which those most directly involved in these interventions perceive them to be a feasible solution to the problems they aim to address.

The purpose of this paper is to advance knowledge of the collaborative processes that inform police-mental health co-response models and highlight systems-change opportunities by centering key stakeholders' experiences with implementing an intervention in West Central Florida. The coresponse model described here involves a direct, post-crisis referral mechanism to a team of system coordinators who support clients in identifying and engaging in behavioral health services until they are able to manage their treatment more independently. The authors conducted an evaluation of the implementation of this model, which includes assessments of system-level factors that influence the implementation environment, client experiences with services, and leadership and frontline perspectives on the barriers and facilitators to implementing the model. Implications of this model are discussed, with a focus on collaborative efforts between behavioral health and law enforcement agencies to improve responses to individuals with mental illness.

The co-response model discussed here was developed as a key component of a broader initiative to improve behavioral health service coordination and access in Pinellas County, FL. The intervention was designed to address unmet mental health need among individuals with regular crisis service use and law enforcement interaction, but with less intensive needs than the highest system utilizers (for which a separate intervention was being implemented). By establishing a system coordination team that received referrals directly from law enforcement, system leaders hoped to improve pathways to treatment and alleviate the burden to the local criminal justice system, which is not equipped to provide mental health treatment or related services. As shown in Fig. 1 , a steering committee (tier 1) was organized by leaders from four local government and health services agencies who had an established history of coordinating and funding services: Pinellas County Human Services, Central Florida Behavioral Health, the Pinellas County Sheriff's Office, and the Florida Department of Health in Pinellas County. The steering committee contracted with four mental health provider agencies (tier 2) who each dedicated staff to form a system coordination team (tier 3), with one of the agencies providing facilities and administrative oversight. The team, referred to as the Pinellas Integrated Care Team (PIC Team), included nine system coordinators, a supervisor, and a certified recovery peer specialist (CRPS). Funding for this model was supported by a combination of tier 1 and tier 2 staffing or financial allocations as well as a private grant secured by the steering committee. This team worked in conjunction with a six-person mental health unit (MHU) involving officers with specialized mental health training from the Pinellas County Sheriff's Office (PCSO) and clinical social workers from a mental health provider agency. Because of this partnership, services were only available to individuals in the PCSO jurisdiction, which serves smaller cities and unincorporated areas in Pinellas County.

To refer individuals for system coordination services, a senior officer from the MHU reviewed daily arrest reports, involuntary mental health exam initiations, and frequent 911 calls (see Fig. 2 ). Any cases that were deemed to potentially involve individuals in need of mental health services were placed on a list shared with MHU co-response teams, who typically followed up within 24 hours, after the event (though in cases where individuals were difficult to reach the timeframe was longer). Paired officer and social worker response teams engaged individuals by going to their places of residence, or in some cases, hospitals or jails, and informing them of system coordination services. If individuals agreed to services, they were assigned a system coordinator from the PIC Team who worked closely with clients on treatment goals and on facilitating mental health, substance use, medical, housing, employment, and other related services (tier 4 in Fig. 1 ). Clients were engaged in services anywhere from several weeks to more than 1 year, with the majority of clients (72%) receiving services between 1 and 6 months. The MHU and PIC Team held weekly PMHC model meetings to review cases and discuss ongoing efforts to engage clients. While an analysis of client outcomes such as reductions in arrests and involuntary commitments was used for the overall evaluation (see 19 for a technical report describing outcomes), this paper focuses on findings from the implementation analysis.

A qualitative design was used to evaluate the implementation of the PMHC and to understand the extent to which the behavioral health environment supported the goals of the intervention. 19 The evaluation was guided by a systems change framework originally developed to assess child welfare initiatives but which has clear applicability to overlapping systems, such as behavioral health. 20 This framework is oriented around five key elements for implementing sustainable systems change: (1) leadership/commitment, (2) vision and values, (3) organizational capacity and infrastructure, (4) stakeholder involvement, and (5) environment. The evaluation took place over a 3-year period between July 2018 and April 2021. Due to the COVID-19 pandemic, evaluation activities shifted from mostly in-person the first 2 years to fully virtual during the final year. Though the pandemic led to some changes in behavioral health service provision, such as restrictions in in-patient facilities and an increase in telehealth use, at the time of the evaluation staff did not report a significant interference with meeting clients' needs. The evaluation was exempted from oversight by the host institution's Institutional Review Board (IRB), though evaluation team members maintained certification in human subjects research and ensured that ethical guidelines were adhered to.

Data collected for the qualitative analysis involved interviews, focus groups, and meeting observations with stakeholders at multiple levels, including leadership, staff, and clients. Attempts were made to elicit feedback from tier 4 ancillary providers, but the response was too small to be included in the analysis. The evaluation activities and timeframe for each stakeholder group is shown in Table 1 and described below. 

The evaluation team conducted semi-structured interviews with the four steering committee members and the project manager during the first 2 years using a protocol derived from the five key elements for implementing sustainable systems change outlined above. 20 For the third year, the evaluation team conducted a focus group with the same five members, during which outcomes of the evaluation were shared to elicit responses on participants' impressions of initiative effectiveness. Interviews and focus groups were conducted via phone and videoconference, and were recorded with permission and professionally transcribed. The evaluation team conducted observations of monthly steering committee meetings throughout the 3-year period to examine evidence of collaborative activity and efforts to address implementation barriers. Evaluation report findings were shared on a semi-annual basis during these meetings to discuss outcomes that may be used to guide ongoing implementation. Independent, unstructured notes were taken and analyzed along with official meeting minutes.

To gather insights from direct service staff, focus groups were conducted with members of the MHU and the PIC Team during the second and third years. A focus group protocol for year 2 was developed to assess staff understanding of their role in the initiative, facilitators, and barriers to engaging clients, gaps in the service environment, and perceptions of the initiative's success. The third year focus group was modeled after the steering committee focus group protocol, in which the evaluation team provided prompts based on significant outcomes of the initiative to elicit responses.

Focus groups were open to all members, with 8 PIC Team and 5 MHU staff participating in year 2, and all 11 PIC Team members and 5 MHU staff participating in year 3. Focus groups were conducted in person during the second year and via videoconference in the third year. Groups were recorded with permission in order to produce transcripts for analysis. Monthly observations of joint MHU and PIC Team meetings were conducted, resulting in unstructured meeting notes to include for document analysis.

To understand clients' perspectives about the implementation process, the research team conducted semi-structured interviews with clients enrolled with the PIC Team within 1 month of program enrollment and 3 months post-discharge. Initial interviews consisted of 14 questions about clients' previous involvement with behavioral healthcare, their ability to access services, how services offered by the PIC Team compared to services offered by other organizations in the county, and post-discharge plans for care. Follow-up interviews consisted of eight questions about length of enrollment, preparation for discharge, service impact, and recommendations for improvement. During the evaluation period, 382 clients were enrolled in PIC Team services. The primary reason for referral for the majority of engaged clients was mental health (71.5%), followed by substance misuse (20.7%); for 7.9% of clients, the primary reason for referral was medical, developmental disability, or domestic violence.

During each of the three evaluation years, a key contact from the PIC Team randomly selected 10 client IDs to be invited to take part in interviews. In some cases, clients were unable to participate because their conditions were not stable, they were being hospitalized, or they were in jail, and the evaluation team was unable to reach them without undue burden. The same clients who completed initial interviews were asked to take part in follow-up interviews after their discharge; however, in some cases, clients could not be reached after they were discharged, so additional clients were invited to participate in discharge interviews, even if they did not complete an initial interview. The evaluation team interviewed 34 clients total, 19 of whom completed both interviews (initial and follow-up) and 15 of whom completed one interview (either initial or follow-up). Interviews typically lasted 30 min or less and were audio-recorded, with verbal permission from participants. Participants were compensated for their time with a $25 cash disbursement after completing each interview.

Transcripts from interviews and notes from meeting observations were initially analyzed in Atlas. ti using a deductive coding scheme. The five key elements of sustainable systems change were used as primary groups into which 16 codes were organized. A conclusion group was added to capture three summary codes: strengths, weaknesses, and recommendations. Multiple evaluation team members coded the materials and conducted an inter-coding agreement exercise until an overall agreement rate of 80% or higher was achieved. Some variation in interpretation was expected given that interviews and focus groups were semi-structured in nature and interviewers sometimes went "off script" when further elaboration was warranted; in these instances, final determinations were made through discussion and collaborative analysis. A secondary inductive analysis was conducted across all stakeholder feedback to determine emergent cross-cutting themes. Table 2 provides a summary of key findings, which are described by theme below.

Evidence across data sources showed a relatively cohesive understanding of the purpose and goals of PICA, though steering committee feedback alluded to some early challenges to having multiple agencies at the table, such as sharing the same vision for the initiative: "In the beginning, we had to establish what we meant by change, and what we meant by collaboration, and… we really had to kind of start from before the beginning." One aspect of the vision that was shared across all steering committee members was that the initiative was addressing a significant "void" that was not being met by the existing system of behavioral health care providers, despite many previous efforts to facilitate improvements. One member noted, "In discussions that I had over a long period of time, everybody, I mean everybody, acknowledged the need, but nobody was willing to collabo-rate…" The gap in services that stakeholders saw as going unaddressed was that many individuals with SMI were receiving repeated crisis intervention involving law enforcement-usually through involuntary mental health examinations, referred to as Baker Act exams in Florida-but with little to no follow-up care. Stakeholders consistently pointed to problems like a severe lack of funding for mental health in the state and the tendency for providers to operate in silos as contributing to the previous lack of collaboration.

Strategic collaboration was central to the model's development and is reflected in the interagency make-up of the decision-making and service provision entities (the steering committee, the PCSO MHU, and the PIC Team 1 ). This design was seen as ensuring collaboration on a structural level, in part to quell historical tensions among providers around competition for resources. One stakeholder Strategic interagency collaboration A cohesive vision across partners contributed to success in filling "void" in fragmented and siloed system Cross-system funding and leadership was crucial to commitment and buy-in across partners and providers Interagency service structure provided greater insights into gaps in service coordination and alignment of outcome indicators and policies across the system Environment of services Existing community resources were well-utilized with staff dedicated to system coordination The model highlighted gaps in psychotherapy, in-patient facilities, specialty services, and case management Clients reported many difficulties historically navigating behavioral health services and insurance allowances and greatly valued this support from system coordinators Engagement Frequent staffing meetings and open communication among system coordinators and MHU staff contributed to a sense of unity among direct service staff Flexibility in frequency and length of interactions were seen as facilitators of successful client engagement Some clients struggled to maintain progress independently after discharge, highlighting need for ongoing case management Many clients reported a positive difference in treatment by the PIC Team compared to previous care Role of law enforcement in behavioral health Some clients previously had negative experiences with law enforcement, which may impact willingness to engage MHU officers perceived their "softer," more specialized approach as important in engaging people with mental health needs Stakeholders emphasized the importance of mental health training for officers and discussed evolving strategies to ensure more officers received training suggested that the interagency makeup of the initiative would increase buy-in and commitment: "Everybody's got skin in the game, so we've all put forth resources-financial resources and, in some cases, human capital resources-and have collaboratively insisted that there be accountability, that it be metric-driven, that there be defined outcomes, and that the people we are contracting with hold themselves and their people accountable." Stakeholders also attributed the interagency nature of the model to success in problem-solving: "…I think that's why this is successful, is because everyone comes with a different asset…whether it's law enforcement, whether for us to hear from a health provider, whether it's somebody that is going to help with food resources, you know, or housing, these folks are working together." Because of the interagency make-up of the model, stakeholders also saw it as a "beta site" for identifying areas where service coordination, outcome indicators, policies, and contracts were not in alignment, and for beginning to develop strategies for addressing and streamlining these components across the behavioral health system.

Feedback from interviews and focus groups with steering committee members, direct service staff, and clients provided insights into whether the environment of services was conducive to successfully implementing the co-response model. The PIC Team was widely perceived as a "barometer" of how well the behavioral health system of care functioned, as system coordinators needed to connect clients with SMI and co-occurring conditions to numerous services. Feedback on whether the environment of services was sufficient for meeting the needs of individuals enrolled in the PIC Team was mixed. Respondents from the steering committee and the PIC Team emphasized how well existing behavioral health services were utilized by the system coordinators and saw this as an example of how effectively the system can function when staff are in place to identify and coordinate services. In fact, incidental funds were made available to the team to provide financial assistance to clients when there were not existing resources to meet concrete needs, but only a very small portion of the funds were accessed. Several stakeholders suggested that, in terms of availability, the service environment was in some ways more supportive than they initially thought and found system coordination to be a key component of more effective utilization.

However, some gaps in behavioral health services were made obvious through the model. System coordinators discussed barriers they encountered, such as a lack of licensed clinical social workers and psychotherapists in the area, insufficient capacity at substance use treatment and in-patient psychiatric care facilities, inadequate services specializing in intellectual disabilities, and challenges processing Medicare claims. System coordinators also emphasized the need for more in-home services and for targeted and intensive case management in general, which was seen as a gap in care on the part of public and private insurance providers. Wait times were also problematic across the system, with 1 to 2 months being a typical length of time to wait for an appointment for psychotherapy, for instance. In some cases, it was reported that clients initiated their own involuntary commitments in order to be prioritized in the system and receive treatment sooner. Some system coordinators saw their roles as invaluable in addressing this pattern because they served as a resource for clients in the interim between clients' intake appointments and treatment appointments-which could be several additional weeks after waiting for the initial appointment-and during which clients would otherwise be without needed care if they were not enrolled in the PIC Team.

Clients expressed mixed views on how easy it was to access mental health services in the region. Many reported that difficulties in navigating mental health care led them to go without care or to have gaps in care. This was caused by a lack of sufficient insurance coverage, lack of awareness of local resources, or inability to reach providers. These clients had similar complaints, saying they "didn't even know where to go", or that they were "always on the phone trying to get help" but did not receive answers. Many felt their lack of awareness of resources or processes was because this type of knowledge was not made easily available. Many became frustrated, feeling shut out at different turns. Some went without care for extended periods of time, such as one client who shared that her family was unable to connect to appropriate services for four and a half years: "My daughter had to be admitted into [the] emergency [room] just to try and get our family help that she didn't get." Several added that their interactions with their system coordinators were a key component of their treatment. They were provided direct access to resources and developed skills to manage their health and well-being in the future, such as healthy coping mechanisms. As one client said, "I think I'd still be stuck in the same spot if [my system coordinator] didn't show up." Some clients, however, described having success in finding the help they needed through counseling centers on college campuses, referrals from medical care or welfare services, or from being connected to care shortly after a voluntary or involuntary psychiatric examination.

Many stakeholders suggested that heightened engagement among PIC Team members was unique to this team and led to better problem solving and resource sharing. Weekly staff meetings between the PIC Team and MHU demonstrated the collaborative nature of this model, where system coordinators and officers offered mutual support in responding to challenges with client engagement or identifying appropriate courses of action. System coordinators appreciated the fluid nature of the model which allowed them to work together on cases where clients' needs were especially intense. In contrast with their previous case management experiences, many welcomed the lack of restrictions on timeframes or number of contacts that the private grant funding for the project allowed. Clients, too, shared that if the coordinator they were assigned to was not a good fit, they were able to transfer to another. Stakeholders at all levels pointed to an overall sense of unity among the PIC Team, which they felt was rare among behavioral health case management in general (see. [21] [22] [23] Clients generally reported very positive relationships with their system coordinators, describing them as respectful, helpful, and knowledgeable. System coordinators were described as listening to their clients, talking about "real problems," and being solution-oriented. System coordinators were perceived as proactive and in frequent communication with clients to check on how they were doing, which was especially helpful for those who had trouble initiating contact, as one client noted, "I don't have to reach out so much. Because when you're in depression, you're tryin' to not reach out as much." System coordinators managed referrals, appointments, and transportation, while also providing services in-home. They referred clients to a wide range of appropriate services and worked with clients to individualize services to their needs. One client commented, "[My system coordinator] helped me decide on what program is best suited for me." System coordinators were seen as persistent, and as encouraging clients through feelings of helplessness to achieve their goals.

After discharge, a majority of clients tried to continue treatment independently. Some felt more confident in their ability to find providers, continue treatment, and prioritize finances. Some clients had difficulties making and meeting appointments, adhering to psychotropic medication use, arranging transportation, and requesting financial assistance. Their responses suggest a drawback of the intensive engagement provided by system coordination services, and that support for service navigation and treatment scheduling is not replicated elsewhere in the behavioral health system but may be necessary for clients with high levels of need who are targeted by the model. This difficulty of post-discharge service continuation was highlighted in other stakeholder feedback which suggested that most service providers lack the level of engagement and coordination offered by the PIC Team, and that there is a need to better understand referral patterns during the 3-month period after discharge. Complaints related to system coordination were not common, but those that were shared often related to a lack of decision-making ability or confusion about treatment or service involvement. Some examples include disdain at system coordinators' unscheduled home visits, or on the contrary, wishing for more frequent contact during care. Overall, however, clients' experiences with system coordination services spoke to a high level of engagement that was more often instrumental to their overall care. Frequent contact during and after care was cited by many as important in helping clients manage their responsibilities, developing rapport, and ensuring clients felt listened to and respected by providers. "It's nice to have a friend, someone to talk to… the thing I miss most is just conversations and throwin' ideas back and forth." Some clients shared that the psychoeducation about diagnosis and treatment they received was particularly important, especially for those who may have never had their illness explained to them. One client felt grateful for these conversations with their care coordinator and described one such interaction: "[My system coordinator] said, 'Do you have an understanding of what you've been diagnosed with?' And I'm, like, 'No one ever asked me that. Thank you so much for asking me because I don't.'"

All stakeholder groups shared perspectives on how involvement from law enforcement impacted the model. Feedback from clients, MHU respondents, and system coordinators indicated that officer engagement with individuals with SMI could be problematic. Several clients reported previous negative interactions with officers during crisis engagements, describing officers as coercive or domineering and as misunderstanding symptoms of their conditions. These instances were separate from clients' interactions with the MHU, but they clearly left a negative impression on clients, one of whom expressed fear of calling the police for help in the future. This distinction between patrol officers and MHU officers was reinforced by numerous stakeholders, as a comment from one MHU respondent illustrates: I feel like [MHU deputies] take a softer approach...the deputies that work in the mental health unit, they know how to speak with the people that have gone through crises and what not. And, you know, we do wear a [more] dressed down uniform than the rest of the patrol deputies, too. So I think that helps, too, because it kind of sets us aside from, "Oh, there's the patrol deputy from last week that just Baker Acted me," compared to, "Oh, here's a deputy that's wanting to help me now." Still, members from the MHU acknowledged that when officers initially approach individuals through identification for PIC Team services, many individuals are hesitant or skeptical. This skepticism, coupled with unscheduled nature of the visits, can make engagement challenging: "I think a great barrier, aside from law enforcement presence is, you know, these are unwarranted…they're not asking to make contact with us…The contacts aren't long, so we're trying to gather a lot of very delicate and personal information, very intimate information in a short amount of time." Some respondents discussed strategies to ease tensions upon initial engagement, such as using humor and referring to officers as "glorified Uber drivers." One officer said that the tone often changes by the end of the initial engagement: "I know a lot of the clients, when we engage them, they do get really nervous. But by the time we're done with our conversations, they always thank us for coming out. They typically love us when we come out after that." MHU respondents also spoke of doing ongoing observation and outreach with individuals whom they engaged, but who ultimately chose not to enroll in system coordination services, to continue attempts to divert them from arrests.

There was wide acknowledgement among stakeholders, particularly from the Sheriff's Office, that mental health and crisis intervention training was sorely lacking for officers outside of the MHU and for other law enforcement agencies, and that CIT training, in particular, had significantly decreased over the past several years, despite efforts to have more officers trained. In addition to CIT training, the Sheriff's Office is trying to broaden the scope of officers who can identify mental health problems and will provide a shorter 8-h training, Mental Health First Aid, to all officers, which some respondents saw as a positive change for law enforcement:

It doesn't make them mental health professionals, but it gives them a little more knowledge of what they're seeing when they go on a call. So, if they see a behavior that they would have seen prior, they probably might have Baker Acted, or they might have arrested them. But now... they're able to identify it a little more, and then realize it's more of a mental health concern than a criminal concern.

Along with these changes to expanded (but less intensive) training, stakeholders discussed hopes or plans to expand the work of the MHU to other municipalities or to potentially partner with other types of agencies, such as housing or substance use service providers, and to sustain the work of the PIC Team long-term.

At the end of the twentieth century, most regions in the USA were still piecing together service structures that could adequately meet the needs of individuals with mental illness following decades of deinstitutionalization policies that resulted in slow and fragmented implementation. 6, 24 Scholars widely attribute this period to the increase in police involvement in mental health response, largely because appropriate mental health crisis and treatment options were limited, and jails offered a simpler pathway for addressing mentally ill individuals with anti-social, violent, or criminal behavior. 17, 25 In 1992, Henry Steadman highlighted the important role that law enforcement and mental health agencies play as "boundary spanners," suggesting that this framework of cross-system collaboration-which has existed informally for many decades-is a key component in effectively addressing criminal justice-mental health interactions. What progress has been made in formalizing these collaborative efforts and identifying effective components of them in the past two decades? Many PMHC models have been developed during this time, yet there is significant variability in them, and studies on them have often left out crucial perspectives from stakeholders, particularly from leaders responsible for implementing them and from clients who are the focus of interventions. Stakeholder perspectives on the PMHC model discussed here provide some insight into how collaborations between police and mental health agencies are experienced across different groups, which is especially important to understand in this pivotal moment when practitioners, activists, and policymakers are calling for new visions for public service funding and structuring. [26] [27] [28] There is little dispute about the problems that leave individuals with SMI and criminal justice involvement in the lurch: law enforcement agencies are first responders to crises but lack training and capacity to address behavioral health, 13 crisis services are temporary and do not support longterm treatment intended to prevent recurrence, 29 and behavioral health systems are fragmented and underfunded and lack coordinated responses across agencies. 30 Initiative leaders, direct service staff, and clients involved in the PIC Team all had favorable views of the initiative's capacity to facilitate change against these pervasive barriers, and much of this was credited to strategic and intentional collaboration and successful system coordination. By the end of the 3-year implementation period, each steering committee member asserted their continued commitment to and investment in sustaining, if not expanding the model. The committee has also been implementing other initiatives to bolster the behavioral health system infrastructure as a result of barriers identified through the PMHC, such as developing a process to enhance client referral processes and information sharing; working to align contract elements across providers; establishing an optimal data set for assessing behavioral health outcomes system-wide; building partnerships with hospitals; and increasing MHU staff to serve additional areas. These activities across multiple areas of the system, along with the interagency makeup of the PIC Team, help to address the problem of what Wolff 24 refers to as "single-problem typology" inherent in public system funding and service provision that "is in direct conflict with the variable needs of persons with multiple co-occurring problems and diagnoses" (p. 134).

Client perspectives on their experiences with system coordination services have shown that this more engaged, individualized case management model provides numerous benefits that were often lacking in their previous experiences with behavioral health services. Clients' generally positive regard of system coordinators and their client-focused approach resonates with the steering committee's vision of providing a "warm embrace" through the PIC Team. Most behavioral health systems lack a central coordination entity, and clients are typically expected to independently manage care for multiple needs (e.g., mental health, substance use, employment, or housing), across numerous providers (e.g., psychologists, psychiatrists, detoxification programs). This client feedback highlights the importance of system coordination or navigation models, which are rapidly emerging across numerous medical and behavioral health systems to better coordinate access to care and provide enhanced client support in areas that have long-standing barriers to care. 31 System coordination in PMHCs is especially important given that diversion-focused initiatives are often focused on the front end of law enforcement engagement and lack a "cushion" of support that ensures individuals with SMI have sufficient care after a crisis. High rates of readmission for crisis services demonstrate that initial stabilization is not enough, and that individuals who use these services need integrated care across a continuum of behavioral health services. 13 MHU respondents expressed a deep sense of commitment to improving pathways to treatment for mental illness, while at the same time their acknowledgment of barriers to trust in the community reveal a need to consider the complexity of law enforcement agencies' role in behavioral health. Conversations about defunding the police have surfaced in the wake of several highly publicized instances of police violence and abuse of power throughout 2020, some of which involved individuals with mental illness. 32 Many people have questioned whether police should be involved at all in cases of mental illness, [33] [34] [35] and some models of crisis response that explicitly exclude law enforcement have been highlighted in popular news media as potential alternatives. 36 However, many existing behavioral health systems rely heavily on law enforcement agencies to respond to crises, and to move away from that mode of operation, new infrastructures are needed, which require a significant amount of funding, development, and testing. In the PMHC model described here, a law enforcement stakeholder repeatedly emphasized that if there are not sufficient resources to respond to individuals with mental illness, they will continually end up with law enforcement. Feedback from this initiative may help demonstrate that in many cases, law enforcement agencies are not asking to be responsible for addressing mental illness, but they are offering to be a bridge to appropriate service provision.

Insights from system leaders, staff, and clients from the PIC Team described here speak to a number of important implications for the behavioral health field. Perhaps most pressing is the need to adequately fund integrated behavioral health systems-a nationwide project which, arguably, was never sufficiently developed following the movement from deinstitutionalization to communitybased mental health services. Despite public perceptions that police are overstepping into behavioral health territory, many law enforcement agencies are, themselves, asking for more robust behavioral health services that can alleviate their crisis response burden and diversion efforts. If communities' needs were met through more comprehensive public services at multiple points along the behavioral health spectrum, from prevention and early intervention to intensive treatment and follow-up care, then there would be much less pressure on the justice system to intervene in behavioral health.

There is a clear need for greater case management and system navigation services for clients with behavioral health conditions. A key finding from our evaluation was that many behavioral health services that clients needed existed in the region, and these services were well utilized by the PIC Team, yet clients have many barriers to identifying, accessing, and initiating them on their own. These services should theoretically be made available through insurance providers, but reports of both clients and system coordinators from our evaluation who tried to access these services indicate that they fall critically short in providing any real value. Longer-term support for individuals with SMI is also warranted, including a longer period of follow-up care from providers; fewer restrictions by insurance providers on how many contacts may be made with clients, how many visits they may have, and how long clients may remain in services; and better mechanisms for facilitating a sense of connection between clients and providers so that clients are more likely to maintain contact. Additionally, any efforts to improve behavioral health service coordination should include positions-ideally centrally or independently staffed-that can help clients coordinate their care. These coordination efforts should involve law enforcement partnerships to ensure that criminally involved individuals with SMI and/or co-occurring conditions can have their behavioral health needs prioritized through appropriate services.

More conclusive research is needed on police-mental health collaborative models and their outcomes in general, but specifically with regard to feasibility and sustainability of various models. It is important to understand how multi-agency funding structures can be sustainable. PMHCs across the nation have a variety of funding sources, and outcomes on the extent to which they are ultimately cost-saving when compared to the funds needed to staff them are unclear. The funding framework depicted here, where multiple agencies allocate funding to an initiative, should be replicated when possible and should be braided with grant funding in order to avoid sustainability problems that arise when initiatives are fully funded by one-time grants. More research is needed to understand the long-term impact of the models at the population level, as the vast majority of studies on PMHCs assess short-term outcomes with a relatively small regional or agency-wide focus, making it difficult to know whether the impact seen at the intervention level leads to change at a broader level.

Improving access to care and reducing involvement in the criminal justice system for people with mental illness

Is crisis intervention team (CIT) training evidence-based practice? A systematic review

The police and mental health

Persons with severe mental illness in jails and prisons: A review

Persons with serious mental illness in the criminal justice system: Background, prevalence, and principles of care

Improving police interventions during mental health-related encounters: Past, present and future

Bureau of Justice Assistance (BJA)

Emerging partnerships between mental health and law enforcement

Envisioning the next generation of behavioral health and criminal justice interventions

Interagency collaboration models for people with mental ill health in contact with the police: a systematic scoping review

Coresponding police-mental health programs: a review. Administration and Policy in Mental Health and Mental Health Services Research

Use of the Sequential Intercept Model as an approach to decriminalization of people with serious mental illness

Police responses to persons with mental illness: Going beyond CIT training

Revising the paradigm for jail diversion for people with mental and substance use disorders: Intercept 0

The Crisis Intervention Team model of police response to mental health crisis: A primer for mental health practitioners. Best Practices in Mental Health

Comparing outcomes of major models of police responses to mental health emergencies

Police perspectives on responding to mentally ill people in crisis: Perceptions of program effectiveness

Officer perceptions of police responses to persons with a mental illness

Pinellas Integrated Care Alliance (PICA) evaluation final report

Key elements for implementing sustainable systems change

Care coordination as imagined, care coordination as done: Findings from a cross-national mental health systems study

What is case management? A scoping and mapping review

Models of community care for severe mental illness: A review of research on case management

Interactions between mental health and law enforcement systems: problems and prospects for cooperation

The criminalization of mentally disordered behavior: Possible side-effect of a new mental health law

The public health funding paradox: How funding the problem and solution impedes public health progress

Defund the police: Moving towards an anti-carceral social work

Police funding

High utilizers of psychiatric emergency services

Moving beyond parity-mental health and addiction care under the ACA

Care integration in the Patient Protection and Affordable Care Act: Implications for behavioral health. Administration and Policy in Mental Health and Mental Health Services Research

s killing started a movement. 9 months later, what's changed?

Should police be first to respond in a mental-health crisis? BBC News

issues/ posit ion-state ment-59-respo nding-behav ioral-health-crises

Unbundle the police. The Atlantic

Removing cops from behavioral crisis calls: 'We need to change the model'. National Public Radio

The authors would like to thank the Pinellas Integrated Care Alliance steering committee representatives, the Pinellas Integrated Care Team, and the Pinellas County Sheriff's Office Mental Health Unit staff for their insights and collaborative partnership around this evaluation.Funding This evaluation was funded through a sub-contract with Central Florida Behavioral Health as part of a grant from the Foundation for a Health St. Petersburg.

Ethics Approval The University of South Florida Institutional Review Board (IRB) classified the evaluation as not being human subjects research and therefore exempt from IRB oversight.

The authors declare no competing interests.