key: cord-0035455-5anxfesa authors: Liu, Yufen; Bussell, Scottie; Mi, Guodong title: Quantitative Performance Monitoring of China’s HIV Response date: 2019-05-20 journal: HIV/AIDS in China DOI: 10.1007/978-981-13-8518-6_23 sha: ad51af30326fcabf8b69e6ca813657af8259f474 doc_id: 35455 cord_uid: 5anxfesa China has made tremendous progress in advancing its national HIV program. Key to this progress was setting targets to motivate implementers to work effectively. Indicators were established using an iterative process with national stakeholders while also meeting international pressures. A plethora of demands may have distracted from ownership of the initial national program but, in the end, strengthened a focus on marginalized groups. China gradually relied less on donors and developed a uniquely Chinese HIV control program. The result was remarkable enlargement of prevention and treatment services that led to a reduction in mortality. Setting targets, and measuring performance against those targets, was vital to gauge continued progress and identify gaps in programing and implementation. Continuous monitoring and evaluation (M&E) is an important component of any health system. Setting meaningful targets is a first step toward improving the effectiveness of an HIV/AIDS response, including the health system itself, as well as new interventions and programs. A crucial second step is establishing indicators, also sometimes called metrics or benchmarks, for the purpose of assessing progress (or lack thereof) toward the selected targets. Best practice is to construct these indicators within the so-called "SMART" format (Doran 1981) , ensuring the indicators are: • Specific: detailed and explicit, avoiding overly general aspects, and logically linked with the given objective • Measurable: capable of objective assessment in a manner that is consistently repeatable, comparable, and auditable • Attainable: achievable in the sense that the target can be met, but difficult enough such that the indicator is meaningful • Relevant: logically linked to the objective as well as resulting in information that is actionable • Time-bound: having predictable, repeated timelines for assessment Health system indicators may assess broad, and sometimes overlapping, program aspects including availability, coverage, quality, and impact (World Health Organization et al. 2009 ). Availability indicators primarily examine health service or intervention delivery or offering to target populations and must be tailored to the unique context of each delivery modality. For example, availability of condoms in vending machines, at pharmacies, at community-based organizations, or from outreach workers must be measured differently. Coverage indicators measure the extent to which a target population is actually provided with a particular health service or intervention. These kinds of indicators can prove challenging, since an accurate measure requires a carefully estimated total target population for its denominator. Furthermore, frequency of contact-either successfully reaching a target individual or a target individual successfully accessing a service-is an important factor in assessing coverage for some types of health services. Quality indicators cover a broad range of measures including the scope and completeness of interventions or programs, their effectiveness and efficiency, and their safety profile. Although often challenging to measure, best practice for quality indicators is measurement of performance in these categories against standards set by independent, international organizations. Impact indicators primarily include disease incidence and prevalence, measurable by traditional epidemiological tools and mathematical modelling, but may also include prevalence of disease risk behaviors (e.g., sharing of drug-injecting equipment, consistent condom use; World Health Organization et al. 2009 ). There has been growing recognition that monitoring the success of programs should be performance-based rather than resource-based (i.e., measures of success should consider the differing levels of resources in different settings). Increasingly, international donors have been monitoring indicators for performance and requiring evidence of progress as a part of the application process for extended or renewed funding. For example, the Global Fund to Fight AIDS, Tuberculosis and Malaria (hereafter the Global Fund) began requiring information on progress toward achieving performance-based indicators in order to drive accountability and transparency by grant recipients and to inform future funding decisions. In September 2003, at the International Conference on AIDS and STIs in Africa, representatives from Joint United Nations Programme on HIV/AIDS (UNAIDS), the Global Fund, attending African nations, and a variety of NGOs, funding agencies, and other organizations came to an agreement on three core principles for country-level HIV/AIDS responses. Called the "Three Ones," these principles were one, agreed-upon framework for coordinating all partners, one national coordinating authority with support from multiple sectors, and one comprehensive, countrylevel monitoring and evaluation system (Joint United Nations Programme on HIV/ AIDS 2004). Achievement of the third of the "Three Ones" principles was expected, at the time, to be a significant challenge-countries' HIV M&E systems needed to meet the needs of the global community engaged in fighting HIV/AIDS, national and subnational stakeholders from multiple sectors, public health workers coordinating efforts "on the ground," and funding agencies, both domestic and international, looking to examine the returns on their investments. Furthermore, these M&E systems needed to be built, tested, and launched quickly, yet be robust enough to achieve their purpose, and at the same time be flexible, scalable, and adjustable so that they could meet the needs of the future as well. It was in this context that China embarked on its journey toward developing an M&E system for its HIV response. In November 2003, China sent a contingent from the Chinese Center for Disease Control and Prevention (China CDC) to Bangkok, Thailand, to attend a workshop entitled "Strengthening Monitoring and Evaluation of National AIDS Programs in Asia," presented by UNAIDS, the US Agency for International Development (USAID), the World Health Organization (WHO), the World Bank, the Global Fund, the US Centers for Disease Control and Prevention (US CDC), and others. One of the primary objectives of the workshop was to get country representatives together with multiple international funding agencies to discuss coordination of the HIV response across the Asia region and to grow M&E capacity through the sharing of experiences, challenges, and plans. Although these workshops had already been held several times in Africa, leaders stressed that M&E systems needed to be tailored to the local context in order to successfully achieve their purpose and that there was no one-size-fits-all set of targets and indicators. Although already halfway through two major initiatives, the China's Mediumand Long-Term Plan for AIDS Prevention and Control (1998-2010; Ministry of Health et al. 1998) Furthermore, several international aid projects were underway already, and more were already on the horizon. Hence, while it was recognized that there was a need for detailed and high-quality information on the HIV response, China's HIV response itself was still in its infancy. As noted in a 2003 assessment of prevention, treatment, and care for HIV/AIDS in China jointly prepared by the Ministry of Health and the United Nations Theme Group on HIV/AIDS in China, there was yet "no overall plan for the gathering, storage and usage of HIV/AIDS information" (Ministry of Health and the United Nations Theme Group on HIV/AIDS in China 2003). Thus, a strategy for the development of an HIV M&E system that would standardize data collection, storage, audit, analysis, and exchange and that would simultaneously be tailored to the unique needs of China as well as meet the needs of the international community had become a high priority (see Chap. 20 for more information). At the Bangkok workshop, China CDC representatives sought to operationalize an M&E system based on five core principles: (1) the system needed to be practical, realistic, and simple; (2) each province was responsible for quality assurance with higher-level, centralized verification; (3) focus needed to be placed on prioritization of indicators; (4) epidemic areas needed to be identified; and, finally, (5) multiple sectors needed to be involved. At the start of the workshop, attendees from China CDC had a draft of 10 core, and 14 additional, indicators to share with the group, seek feedback on, and further evaluate based on learning from the workshop ( Fig. 23 .1). At the conclusion of the Bangkok workshop, attendees from the China CDC had created a 12-month action plan that included six high-level activities: (1) formation of M&E teams to do the work of M&E plan development and coordination, (2) development of an M&E information management system, (3) strengthening of case reporting activities and sentinel surveillance systems, (4) initiation of Draft "Additional" Indicators behavioral surveillance among key populations, (5) implementation of M&E training and capacity building, and (6) encouragement of community participation as well as participation by PLWH. By the end of 2003, the M&E plan was not completely centralized-program cites still developed their own plans with assistance from expert groups, monthly and quarterly reports were generated, and progress toward targets was generally self-assessed. Furthermore, the importance of tying indicators to performance was not mentioned among official China CDC documents, perhaps because China's efforts toward curbing the HIV epidemic were too new to detect change. However, there were subtle hints that China's HIV programs would be judged by their impact and that stakeholders would be held accountable for generating success. For example, in a directive on HIV/AIDS in 2003, Premier Wen Jiabao stated (Shen and Yu 2005) : Furthermore, State Council documents that described national HIV response policy at that time explicitly stated that those leading HIV response efforts would be held accountable for their actions through annual evaluations of their job performance and that those who engaged in irresponsible conduct would be punished (Shen and Yu 2005 ) (see Chap. 18 for more information). In 2004, several international agencies launched efforts aimed at increasing the capacity of the Chinese Government's HIV response. Principal among these was the Global Fund, and the China CDC was the largest recipient of grant moneys worldwide in the 2000s. Over a 10-year engagement, from 2004 to 2014, the Global Fund appropriated over 320 million USD to HIV response activities in China. Although the Global Fund was ready to begin deploying funds as early as 2002, China was not ready, both politically and ideologically, to meet the Global Fund's requirements for transparency, inclusiveness, destigmatization, and community (i.e., civil society) participation (Huang and Ping 2014) . Before 2003, China was largely in denial of its HIV epidemic, and societal stigma and discrimination against PLWH and their families was common and severe (Huang 2005) . Participation of civil society in the fight against HIV/AIDS was minimal at this time, and Chinese Government power was substantial and focused on suppressing behaviors deemed undesirable. Not surprisingly, harm reduction was still very controversial in China during this timegiving condoms to sex workers or offering clean needles to drug users was viewed as supporting illegal activity. Thus, China CDC's first application to the Global Fund in 2002, for grants to support HIV prevention programs for drug users, was rejected partly because it lacked a harm reduction component (Huang and Ping 2014) . However, beginning in late 2003, sweeping political changes occurred at the highest levels of China's government, and national HIV response policy took a major turn for the better with the announcement in December of that year of the "Four Frees and One Care" policy. Additionally, China tackled the outbreak of severe acute respiratory syndrome (SARS) virus quickly and effectively in 2003, which gave the international community new confidence in China's public health response capabilities. And finally, China's eagerness to compete for, and win, Global Fund grants to help with the costs associated with implementing and scaling up a range of initiatives also contributed to major changes in the way it was willing to conduct its HIV response efforts (Minghui et al. 2015) . China adapted to global norms and included harm reduction objectives when applying for the fourth round of Global Fund grants in 2004. Hence, the Global Fund's involvement in HIV/AIDS programs in China began in September 2004, with a first phase that boosted the China CARES program, which was just getting off the ground at that time (see Chap. 25 for more information). This first award of 11 million USD from the Global Fund to finance the China CARES program helped to shape many of China's HIV/AIDS program indicators. Pre-Global Fund China CARES targets and indicators differed from Global Fund targets and indicators (Table 23 .1). As an example of how targets and indicators changed with Global Fund involvement, in the area of prevention and treatment, a pre-Global Fund target of 90% of PLWH receiving free antiretroviral therapy (ART) measured by an indicator of the rate of ART coverage for eligible AIDS patients evolved into a post-Global Fund target of universal access measured by an indicator of the proportion of adults and children with HIV still alive 12 months after ART initiation. Minghui and colleagues described the important, yet difficult-to-quantify, effect of the Global Fund's involvement in China's HIV response (Minghui et al. 2015) : The way China's leaders think about development has progressed, making the public health system more transparent, open, and accountable. This quantum advance in governance derived partly from procedural and reporting requirements that came with China's alliance to the Global Fund. However, the influence of international donors on target setting and monitoring of performance against those targets did have some drawbacks. Projects often collected similar information, but the information could not be shared because of different definitions and coding used by different funding agencies. Staff were inundated with paperwork. And, with an increasingly fragmented system attempting to meet the needs of so many different M&E requirements, it was difficult to grasp the true, overall performance of China's HIV programs. In this way, the national HIV response was weakened and slowed by the involvement of so many international donors (Wu et al. 2011 ). (2006 State Council 2006) . The plan, as described by Rou and colleagues, "called for multi-sectorial cooperation in the HIV response, including all of society, and emphasized the importance of prevention programmes and their monitoring and evaluation" and included "specific work goals and indicators according to practical measurements" ) (see Chap. 18 for more information). To assess the impact of this increased government support, specific targets were developed. The aim was to create a performance-based approach to monitor the progress of large-scale, government-driven HIV programs. Setting targets and determining indicators to measure progress against those targets was viewed as means to make national HIV/AIDS program spending more efficient on two levels. First, on a national level, it would provide evidence for key policy and budgetary decision-making processes. Second, on a local level, it would facilitate the internal performance management of health agencies. Initially, indicators served as a tool for assessing the progress of program implementation. Performance assessment against these targets was also especially helpful in setting goals and determining indicators for the following year. The next step toward M&E for China's HIV response was to conduct a pilot. Therefore, in 2006, the National Center for AIDS/STD Control and Prevention (NCAIDS) of the China CDC selected two indicators-open 300 community-based MMT clinics and enroll a cumulative total of at least 30,000 PLWH in the National Free ART Program by the end of 2006 . Both these targets were met. This quick, early success demonstrated to authorities that setting targets and collecting data to measure performance against those targets could effectively promote the implementation of HIV/AIDS programs. They were convinced. And, since the beginning of 2007, NCAIDS has annually set incremental, realistic, quantitative targets for performance and used those targets to hold implementers of China's national HIV/AIDS programs accountable for their work ). NCAIDS, China CDC, was leading the way for HIV M&E, and thus, it made sense that accountability was assigned to leaders within this organization. So, for ease and convenience, the different indicators were quickly categorized and assigned based on the organizational structure of NCAIDS-the directors of each technical division were held accountable for indicators that fell within their department. Early each year, specific indicators and targets for each division were written into a contract, which was then signed by the director of NCAIDS and the directors of each of the technical divisions. At the end of year, an external review panel, consisting of NCAIDS retirees (emeritus professors), China CDC experts, and academics from the School of Public Health at Beijing University, and clinicians from Peking Union Hospital, was invited to judge the performance of each division against their agreedupon targets . Accountability for achieving each target was also assigned to each province. At the beginning of each year, representatives from all provincial-level administrative regions attended a national annual working meeting to review the prior year's performance and to set new targets for the current year. A ranking system was usedthe top five provinces were awarded a National AIDS Program Quality Performance Medal, while the bottom several provinces were required to improve their performance for the next year . For the following year, provincial and local leaders aimed to implement the best practices of top-performing provinces. This innovative approach helped enabled the staff responsible for program implementation to carry out their daily missions in a way that made their targets more achievable. Additionally, NCAIDS carries out regular field visits and supervision activities, which serve to remind local CDC staff of their accountability for the year's targets. These visits and activities are usually led by the NCAIDS technical directors, but occasionally the responsible chief of HIV/AIDS Division at the National Health Commission is invited to facilitate coordination for HIV/AIDS programs across different sectors. In August and September of each year, NCAIDS also organizes monitoring activities to supervise the data quality assurance activities of at least three provinces. This monitoring aims to guarantee the authenticity of field work and reliability of the data it generates. Early in this process of building an M&E system for China's HIV response, data collection, storage, extraction, and use were a very laborious process. Among a myriad of issues, systems were fragmented, data collection forms had both gaps and overlaps, and data fields were not standardized, making it difficult to use existing data with confidence. However, in 2008, China's National HIV/AIDS Comprehensive Response Information Management System (CRIMS) was launched. This system integrated eight previously existing systems and one new system; standardized data collection; moved it to a real-time, web-based platform; and provided a user-friendly interface where data entry and extraction were simple and fast. This new tool dramatically changed the information landscape for China's HIV response and helped the HIV epidemic and response M&E effort to take a huge leap forward ) (see Chap. 24 for more information). Old core indicators were carried forward into 2007, and new core indicators were introduced (Table 23. 2; Liu et al. 2010) . All targets and indicators fell into categories of availability, coverage, quality, and impact. Over time, China's indicators became more consistent, sensitive to the changing epidemic, and more mature. The data required to measure performance against them also improved over time as CRIMS, which was launched in 2008, began to contain more and better-quality data covering a more comprehensive expanse of information required . Targets also became more ambitious. As China's national HIV response program matured, technical expertise grew, and consistent and adequate funding continued to be applied, NCAIDS leaders and technical directors became more focused and confident in what the health system, the public health system, the CDC network, and partnering NGOs and communitybased organizations (CBOs) could achieve. Thus, annual achievements against those ever-more-ambitious targets were able to keep pace. One example is the scale-up of the National Free ART Program. Annual indicators related to improvement of ART coverage included, for example, the cumulative number of PLWH on ART, the total number of PLWH currently on ART, and the proportion of PLWH on ART who remained on ART and survived 12 months. As ART coverage (i.e., the proportion of ART-eligible PLWH who were on ART) grew, mortality among PLWH fell (Fig. 23.2 ; Pisani and Wu 2017) . Another example of how indicators have changed over time and of how targets have become more ambitious is the UNAIDS 90-90-90 targets, which were set out in 2014. These targets were meant to cause a refocusing of efforts and a swell of momentum for the further expansion of treatment on a global scale. The targets are 90% of all PLWH know their status, 90% of all diagnosed PLWH are on treatment, and 90% of all PLWH on treatment achieve viral suppression by 2020 (Joint United Nations Programme on HIV/AIDS 2014). China retrospectively examined its performance against these targets (Fig. 23.3 ; Ma et al. 2018 ) and decided to incorporate Developing meaningful indicators and setting annual targets for performance against those indicators has been crucial for motivating implementers to do the work of scaling up massive HIV response programs at local levels nationwide. The monitoring and accountability framework built around this M&E system has rigorously and successfully tracked HIV response program inputs, processes, outputs, outcomes, and impacts. Monitoring progress against priorities has emphasized the need to focus tactical work at the local level on national strategic objectives. More importantly, annual activities around HIV response M&E have become institutionalized-it is now routine work accepted as the norm in China's public health institutions. This supports China's goal of eventual elimination of HIV/AIDS and sets the nation up for success into the future, when the next major public health challenge appears. There's a S.M.A.R.T. way to write management's goals and objectives The politics of HIV/AIDS in China. Freeman Asian Studies Symposium on China "Changing perspectives on China: new debates, approaches, and challenges in 21st-century Asian studies The global fund's China legacy Three Ones" key principles Joint United Nations Programme on HIV/AIDS. 90-90-90-an ambitious treatment target to help end the AIDS epidemic Quantitatively monitoring AIDS policy implementation in China The human immunodeficiency virus care continuum in China Development of a unified web-based national HIV/AIDS information system in China Ministry of Health, State Development Planning Commission, Ministry of Science and Technology, Ministry of Finance. China's medium and long-term plan on prevention and control of HIV/ AIDS. Beijing: Ministry of Health, State Development Planning Commission, Ministry of Science and Technology, and Ministry of Finance, People's Republic of China; 1998. Ministry of Health, United Nations Theme Group on HIV/AIDS in China. A joint assessment of HIV/AIDS prevention, treatment and care in China. Beijing: Ministry of Health and the United Nations Theme Group on HIV/AIDS in China People's Medical Publishing House Scaling up prevention programmes to reduce the sexual transmission of HIV in China Governmental policies on HIV infection in China China's action plan for reducing and preventing the spread of HIV/AIDS (2001-2005). Beijing: State Council, People's Republic China Notice on strengthening AIDS prevention, treatment and care programs Action plan on HIV/AIDS prevention and containment Thirteenth five-year action plan for controlling HIV/AIDS in China State Council AIDS Working Committee. Framework for monitoring and evaluation of China's National AIDS Program. Beijing: State Council, People's Republic of China The Global Fund to Fight AIDS, Tuberculosis and Malaria. Monitoring and evaluation toolkit: HIV, tuberculosis and malaria and health systems strengthening. Geneva: The Global Fund to Fight AIDS, Tuberculosis and Malaria United Nations General Assembly Special Session on HIV/AIDS. Monitoring the declaration of commitment on HIV/AIDS guidelines on construction of core indicators Monitoring the 2001 declaration of commitment on HIV/AIDS Joint United Nations Programme on HIV/AIDS. A guide on indicators for monitoring and reporting on the health sector response to HIV/AIDS. Geneva: World Health Organization, United Nations International Children's Emergency Fund, and Joint United Nations Programme on HIV/AIDS The integration of multiple HIV/ AIDS projects into a coordinated national programme in China The authors would like to thank Jennifer M. McGoogan for her input and editorial assistance.