Impacts of Distance Education on Agricultural Performance and Household Income: Micro-Evidence from Peri-Urban Districts in Beijing sustainability Article Impacts of Distance Education on Agricultural Performance and Household Income: Micro-Evidence from Peri-Urban Districts in Beijing Jianxin Guo 1,*, Songqing Jin 2,3,*, Lei Chen 1 and Jichun Zhao 1 1 Institute of Agricultural Information and Economics, Beijing Academy of Agriculture and Forestry Sciences, Beijing 100097, China; chenl@agri.ac.cn (L.C.); zhaojc@agri.ac.cn (J.Z.) 2 China Academy for Rural Development, Zhejiang University, Hangzhou 310058, China 3 Department of Agricultural, Food, and Resource Economics, Michigan State University, East Lansing, MI 48824, USA * Correspondence: guojx@agri.ac.cn (J.G.); jins@msu.edu (S.J.); Tel.: +86-10-51503298 (J.G.); +1-517-353-4522 (S.J.) Received: 22 September 2018; Accepted: 26 October 2018; Published: 30 October 2018 ���������� ������� Abstract: Information communication technology (ICT) has changed the traditional agricultural extension service mode worldwide. This paper examines the effects of the Rural Distance Education Project (RDEP) on the household income, agricultural productivity, and off-farm employment of farmers in peri-urban areas in Beijing. Using the survey data of 783 randomly selected farm households from 54 villages in three Beijing peri-urban districts in 2014, and the propensity score matching method (PSM), we find that the RDEP has a significant and positive effect on agricultural productivity and input use. Meanwhile, the program’s effects are heterogeneous across districts and households. For example, the RDEP has significant impacts on several outcome indicators, such as agricultural labor productivity (at a 5% level of significance), agricultural land productivity (at a 10% level), and input use intensity (at a 1% level) in Tongzhou (an agriculturally more important district, with a more intensive RDEP usage), but none of these effects is significant in Pinggu district. Furthermore, the RDEP is found to have bigger, and statistically more significant effects, for households with junior high school education than for those with either lower or higher than junior high school education. Furthermore, the RDEP is more effective for households with more assets than those with fewer assets. These results point toward the importance of using a rural distance education program as an effective extension service, and the need to take community and individual characteristics into account in the implementation and design of future programs. Keywords: impact evaluation; distance education program; propensity score matching; agricultural productivity 1. Introduction Information communication technology (ICT) has changed the mode of production and dissemination of information, and has the potential to overcome the traditional obstacles of economic differences, geographical distance barriers and the unequal distribution of knowledge [1]. Recognizing that the ICT-based extension service has a greater potential than the traditional extension system, many governments in the developing world have made huge efforts to establish comprehensive ICT-based agriculture extension systems. The Chinese government has already invested massively in information infrastructure and ICT-based education programs to assist farmers in obtaining knowledge, information, and skills to enhance the level of their livelihood [2]. However, despite decades of investment in ICT-related infrastructure and education programs, evidence of the impact of such Sustainability 2018, 10, 3945; doi:10.3390/su10113945 www.mdpi.com/journal/sustainability http://www.mdpi.com/journal/sustainability http://www.mdpi.com http://www.mdpi.com/2071-1050/10/11/3945?type=check_update&version=1 http://dx.doi.org/10.3390/su10113945 http://www.mdpi.com/journal/sustainability Sustainability 2018, 10, 3945 2 of 19 investment on farm households’ welfare is still mixed, and sparse from China, which is possibly due to the variety of ICT technologies, different types of information provided, and data availability [3]. There are two active strands of literature regarding the evaluation of impacts of the ICT-based programs affecting rural farmers. The first strand of literature evaluates the impact of ICTs on access to market and price information. The second strand evaluates the effects of ICTs as a means of enhancing farmers’ knowledge about improved agricultural practices and technologies. The results are mixed in each of the two strands of research. Related to the first strand of research, considerable attention has been focused on the effects of accessing mobile phone services on market efficiency and agricultural prices with mixed findings. On the one hand, several studies found that mobile phone services significantly improved market efficiency and reduced information asymmetry. For example, an influential study by Jensen found that the introduction of mobile phone coverage improved market efficiency and significantly reduced the local sardine market price dispersion [4]. Megumi Muto et al. found that an expansion of mobile phone coverage significantly increased banana sales but not maize sales in remote rural communities in Uganda [5]. On the other hand, evidence from other African countries has demonstrated that accessing mobile phones did not significantly improve farmers’ market participation and spatial arbitrage, suggesting that the lack of relevant information or the existence of other market failures needs to be addressed [6,7]. Shimamoto et al. argued that it is not the mobile phone but rather the market information via a mobile phone that increases farmers’ market arbitrary power [8]. Outside developing countries, evidence from Ireland indicates that the digital divide is not a problem of access but rather a problem of engagement, and farm business characteristics determine whether farmers use computers for their agriculture business [9]. The second strand of literature focuses on the effects of ICT-based extension services on farmers’ knowledge about new agricultural practices and technologies and the ensuing effects on adoption behaviors, productivity, and welfare. It is perceived by many that ICT-based extension services are more cost-effective and efficient in disseminating new knowledge and technologies than the “outdated” traditional extension system. The empirical findings on the impacts of ICT-based agricultural extension services are also mixed. For instance, two recent studies about impacts of mobile phone-based information services (MIS) using data from two African countries demonstrate that MIS have positive and significant impacts on farmers’ practices in Kenya [10] and on pesticide use and food security in Ghana [11]. In contrast, a recent study by Cole and Fernando [12] showed that a mobile phone-based agricultural consulting service had no significant effect on the agricultural knowledge of Indian cotton farmers. There are two main reasons why the empirical evidence on the impacts of the ICT-based extension services has been inconclusive. First, ICT technologies and the information delivered by the ICT technologies are heterogeneous and complex. Among the ICT-based extension programs, the service information is not the same as MIS. Compared to MIS, other extension services, such as digital video instruction or online consultation, are much more complicated and difficult to be transmitted to and shared with other farmers independent of equipment or networks. Second, there are well-known empirical challenges associated with the evaluation of extension programs that are not based on experimental data [13,14]. Despite the relatively active micro-level studies on the impacts of rural ICT-based programs in developing countries with focus in Africa or Asia, there is a paucity of research on the impacts of ICT applications by farmers in rural China, despite being a country with more than one-fifth of the world’s farmers and a rapid expansion of ICT infrastructure, networks and telecommunication services in recent years (To our knowledge, there is only one earlier micro-level empirical study [15] related to the impact evaluation of ICT-based service on Chinese farmers. That paper explores the impact on farmers’ participation in market and sales prices. No one has studied the effect of ICT-based service on Chinese farmer’s agricultural production efficiency and household income). Therefore, we are among Sustainability 2018, 10, 3945 3 of 19 the first few examining the impacts of ICT programs in the context of rural China where most people have access to mobile phone service and the internet [15]. The impacts of the Rural Distance Education Project (RDEP) on farm households were assessed using survey data of 783 randomly selected farm households from both the RDEP and the non-RDEP villages in peri-urban areas of Beijing in 2014. Propensity score matching methods (PSM) were adopted to deal with the program placement/selection biases. For two of the outcome variables for which panel data are available, the combination of a PSM and difference-in-difference (DID) was adopted to further control for potential biases due to unobserved heterogeneity. We found that RDEP increases farm households’ agricultural productivity significantly, although effects on their gross income is not confirmed. There exist obvious heterogeneous impacts across districts and household characteristics. While the RDEP is found to significantly enhance agricultural land productivity, labor productivity, and input use intensity in Tongzhou district, this was not the case in Pinggu district. Moreover, while the productivity effects are positive and statistically significant for households with junior high school education, the effects are insignificant for those with either primary school or senior high school education. Furthermore, the productivity effects are more effective for households with more assets than for those with fewer assets. 2. RDEP Background There were 3950 administrative villages in suburb Beijing, which is the home of 1.16 million farm households and 2.68 million farmers, according to the rural Hukou registration data. An average farm household owns 0.19 hectares of arable land. There is a large income gap between urban and rural residents. In 2010, the per capita income of an average farmer in Beijing area was 13,262 yuan (about $1959, based on the 2010 average exchange rate), which was 45.62% of that of an urban resident. The income disparity among farmers was also large in suburban Beijing. The level of average income for the bottom 20% farmers was 7251 yuan (about $1071) per capita, which was only 23.81% of that of the top 20% of farmers. The gap between urban and rural public service is an important potential cause for the considerable urban–rural income gap. To help solve this problem, the Beijing municipal government initiated the RDEP to improve the scientific and technology support for rural area through information. The RDEP was put into operation in 2010, and about 2000 distance stations were constructed in 10 suburban districts. The RDEP in Beijing consists of a municipal platform (to develop and distribute training courseware and information) and distance education stations located in villages to educate and train the grass roots villagers. The distance education stations were established by district governments and situated in village public places (meeting rooms, classrooms, offices, etc.). These RDEP stations are administrated by local College Graduate Village Officials or the village committee members. The station administrators help villagers learn agriculture technologies via courseware, or search for information through the RDEP website. Furthermore, at least two of the collective training events are arranged to train farmers. Training details such as the training time and learning content are recorded by the platform information system. Details on trainees’ participation and learning intensification are also recorded by the administrators of the local distance stations. The RDEP provides more than 10,000 coursework and consultant services, including innovative farming technology, vocational skills, small business operations, daily wholesale and retail market prices, and health education courses. Farmers participate in the public training classes organized in the village station or search for needed courses and information through the station terminal, and consult experts regarding farming problems through an online system on demand in the station, in order to gain the information and knowledge needed to improve their product efficiency. The survey was approved by the Academic and Ethics Committee of the Institute of Agricultural Information and Economics, Beijing Academy of Agricultural and Forestry Sciences. All of the data were processed in an anonymized manner. A detailed explanation of the respondent’s rights and authorization of the survey was provided during the interview. All of the participants gave consent to participate in the survey prior to each interview. Sustainability 2018, 10, 3945 4 of 19 It is also important to note that the impact of the ICTs measured in this paper is a combination of the impacts of accessing ICT services (computer/mobile/internet) and the impacts of the delivered information through ICT, which was not made clear in the literature. Unfortunately, our data do not allow the disentanglement of the two separate effects. 3. Sample and Data 3.1. Sample and Data Collection A total of 783 households were drawn from three peri-urban districts based on a three-stage stratified random sampling technique. In the first stage, three districts in 10 rural districts were selected as showing in Figure 1, based on criteria regarding the functional zone planning, economic conditions, and program intensity (learning time) of an average station. Tongzhou, Pinggu, and Huairou belong to a plain, a semi-mountainous area, and a mountains area, respectively. In the Beijing municipal development planning, Tongzhou district belongs to the development zone, while Pinggu and Huairou belong to the ecological conservation zone. The learning intensity of stations in Tongzhou was 100.41 h per year, and those in Pinggu and Huairou were 34.88 h and 67.37 h, respectively. In the second stage, nine program (treatment) villages in each district were randomly sampled, and nine villages that had similar agro-ecological and socio-economic characteristics but had no distance education station at the time of the survey were selected as control villages. In order to increase the probability of having households in the control group to match the households in the treatment group, more control households than treatment households were selected. As a result, 324 participating households (12 households per treatment village) and 459 non-participating households (17 households per control village) were randomly sampled. To minimize the possible spillover or confounding effects from other ICT projects, two strategies were adopted. First, we purposively kept the treatment and control villages within the same district, with certain distance. Second, villages implemented with other ICT-based service projects were removed before sampling. The geographical distribution of the three peri-urban districts in Beijing is displayed in Figure 1. Purposive questionnaires were designed and used to collect data at both the village and the household level. Interviews were conducted by experienced enumerators in the months of August, September, and October in 2014. At the village level, information on village basic demographic characteristics, economic conditions, land assets, and non-land assets was collected from village officials. For the treatment villages, information on the RDEP implementation was also collected. At the household level, data were collected on household demographic characteristics, land endowment, non-land asset holdings, agriculture production, off-farm employment, business income, and non-labor income. Each completed questionnaire was rechecked and validated for accuracy by a fieldwork supervisor, and then again by a data administrator before data entry. The survey’s response rate was 100%. An unfortunate mistake that occurred at the beginning of the survey was that no data on household fixed assets was collected for nine villages in Pinggu. For this very reason, the value of fixed assets for the Pinggu district is missing in Table 1. Also, the fixed asset variable was excluded from the logit model regression that was used to estimate the propensity scores for the Pinggu subsample. Sustainability 2018, 10, 3945 5 of 19 Sustainability 2018, 10, x FOR PEER REVIEW 5 of 20 Figure 1. Sample districts in Beijing municipal city. The three districts with shading marks are Huairou, Pinggu, and Tongzhou, respectively, from top to bottom. Purposive questionnaires were designed and used to collect data at both the village and the household level. Interviews were conducted by experienced enumerators in the months of August, September, and October in 2014. At the village level, information on village basic demographic characteristics, economic conditions, land assets, and non-land assets was collected from village officials. For the treatment villages, information on the RDEP implementation was also collected. At the household level, data were collected on household demographic characteristics, land endowment, non-land asset holdings, agriculture production, off-farm employment, business income, and non-labor income. Each completed questionnaire was rechecked and validated for accuracy by a fieldwork supervisor, and then again by a data administrator before data entry. The survey’s response rate was 100%. An unfortunate mistake that occurred at the beginning of the survey was that no data on household fixed assets was collected for nine villages in Pinggu. For this very reason, the value of fixed assets for the Pinggu district is missing in Table 1. Also, the fixed asset variable was excluded from the logit model regression that was used to estimate the propensity scores for the Pinggu subsample. Table 1. Household (HH) and village characteristics. Variable Total (n = 783) Tongzhou (n = 261) Pinggu (n = 261) Huairou (n = 261) Treatment (n = 324) Control (n = 459) Treatment (n = 108) Control (n = 153) Treatment (n = 108) Control (n = 153) Treatment (n = 108) Control (n = 153) Demographic Characteristics HH size in 2010 (#) 3.40 *** (1.35) 3.04 (1.18) 2.70 (0.92) 2.75 (1.07) 4.38 *** (1.39) 3.4 (1.44) 3.11 (1.08) 2.92 (0.86) Figure 1. Sample districts in Beijing municipal city. The three districts with shading marks are Huairou, Pinggu, and Tongzhou, respectively, from top to bottom. Since the information services of the RDEP are multifaceted, and farmers may benefit from the program in a variety of different ways in terms of economic opportunities, resource allocation efficiencies, production, income, and welfare, the impacts of the RDEP program are measured by the following performance or impact indicators: (1) household (HH) gross income, which was calculated as the sum of agricultural income, off-farm employment income, business operation, and transferred income (including rent, dividends, and subsidies); (2) HH gross agricultural income, which was calculated as the value of crop and livestock production (including non-marketed produce valued at market prices) minus variable production costs (including purchased inputs, hired labor, land rent, etc.); (3) agricultural labor productivity (agricultural income per agricultural worker per month), which was calculated as the annual agricultural income divided by the total labor months working on the farm; (4) agricultural land productivity (crop income per mu), which was calculated as the annual crop income divided by land area (mu), where annual crop income is calculated as the value of crop and production (including non-marketed produce valued at market prices) minus variable production costs (including purchased inputs, hired labor, land rent, etc.); (5) agricultural input use intensity (fertilizer, pesticide, seed use per unit of land); (6) HH labor months working on farm; (7) off-farm labor productivity (off-farm income per worker per month); and (8) HH off-farm employment (person months on off-farm). The first indicator measures the overall income effect; indicators (2)–(6) measure the agriculture productivity and intensification effects; indicators (7)–(8) measure the off-farm labor productivity effects, and indicators (6) and (8) also capture HH labor allocation effects. Sustainability 2018, 10, 3945 6 of 19 Table 1. Household (HH) and village characteristics. Variable Total (n = 783) Tongzhou (n = 261) Pinggu (n = 261) Huairou (n = 261) Treatment (n = 324) Control (n = 459) Treatment (n = 108) Control (n = 153) Treatment (n = 108) Control (n = 153) Treatment (n = 108) Control (n = 153) Demographic Characteristics HH size in 2010 (#) 3.40 *** (1.35) 3.04 (1.18) 2.70 (0.92) 2.75 (1.07) 4.38 *** (1.39) 3.4 (1.44) 3.11 (1.08) 2.92 (0.86) Distance from RDEP station in village (m) 415.03 (481.43) 453.01 (554.30) 349.54 (208.21) 332.19 (307.91) 578.89 (720.91) 483.86 (788.08) 316.68 *** (308.85) 542.99 (577.68) HH head age (year) 54.18 ** (10.91) 55.86 (10.77) 55.69 (10.34) 56.58 (10.27) 56.23 ** (12.31) 59.07 (10.60) 50.63 (9.02) 51.93 (10.28) Gender of HH head (Male = 1, Female = 0) 0.86 (0.35) 0.88 (0.33) 0.89 (0.32) 0.92 (0.28) 0.81 (0.40) 0.85 (0.36) 0.88 (0.33) 0.88 (0.33) Marriage status of HH head 0.96 (0.19) 0.94 (0.24) 0.98 ** (0.14) 0.90 (0.30) 0.95 (0.21) 0.96 (0.19) 0.95 (0.21) 0.95 (0.21) Education level of HH head (year) 10.12 *** (2.87) 9.52 (3.13) 9.87 ** (2.55) 9.04 (2.86) 10.13 (3.01) 9.48 (3.05) 10.36 (3.02) 10.04 (3.41) 1 Education level of head’s father 0.23 (0.42) 0.19 (0.39) 0.25 *** (0.44) 0.12 (0.32) _ _ 0.44 (0.50) 0.44 (0.50) Maximum education of HH members (year) 12.81 ** (3.12) 12.30 (3.45) 11.51 (3.26) 11.45 (3.27) 13.96 *** (2.59) 12.37 (3.59) 12.95 (2.99) 13.07 (3.31) Share of HH members (<14 or >60) in 2010 0.21 ** (0.28) 0.25 (0.36) 0.26 (0.36) 0.25 (0.38) 0.22 *** (0.23) 0.35 (0.40) 0.14 (0.22) 0.16 (0.27) Party membership of HH head 0.58 *** (0.49) 0.57 (0.50) 0.50 (0.50) 0.41 (0.49) 0.81 ** (0.39) 0.91 (0.29) 0.42 (0.50) 0.41 (0.49) Village committee cadre membership of HH head 0.19 (0.40) 0.15 (0.36) 0.13 (0.34) 0.08 (0.28) 0.25 (0.44) 0.26 (0.44) 0.18 (0.38) 0.12 (0.33) Household Assets Land owned in 2010 (Mu) 4.45 (9.22) 5.06 (10.89) 3.80 (3.56) 4.85 (7.13) 2.93 (7.24) 2.63 (4.98) 6.64 (13.56) 7.69 (16.40) Proportion of cultivated land (%) 32.75 (50.60) 26.47 (79.63) 25.35 (49.94) 20.59 (59.95) 38.84 (53.76) 32.42 (45.42) 34.07 (47.42) 26.42 (46.07) 1 Gross value of assets in 2010 (log) 7.76 (5.85) 7.71 (5.74) 11.70 ** (2.25) 11.08 (2.44) _ _ 11.57 * (2.65) 12.04 (1.76) 1 HH income per capita in 2010 6.22 (4.46) 6.14 (4.47) 9.08 (0.85) 8.96 (1.49) _ _ 9.57 (0.62) 9.46 (1.01) Village Characteristics Collective assets (10,000 yuan) 712.64 (753.30) 638.99 (669.74) 768.92 (643.38) 778.14 (939.81) 995.92 (976.23) 774.41 (558.83) 373.08 (381.39) 394.40 (255.17) Collective assets in 2010 (10,000 yuan) 538.54 (632.01) 541.43 (676.56) 529.60 (440.73) 615.89 (822.30) 1038.37 ** (1122.52) 646.38 (549.12) 361.16 (441.76) 431.99 (524.75) Gross collective income in 2010 (10,000 yuan) 410.18 *** (1370.02) 140.22 (174.53) 819.51 *** (1969.37) 223.39 (228.30) 92.33 (99.18) 104.33 (111.17) 68.88 (60.07) 69.02 (51.58) Collective dividend per capita in 2010 (yuan) 355.09 *** (761.59) 628.57 (1512.13) 111.11 *** (315.73) 766.67 (1660.06) 276.67 *** (396.82) 0.00 (0.00) 625.20 (1036.16) 700.00 (1562.89) Distance from county center (km) 16.83 ** (13.99) 19.65 (19.15) 22.72 *** (6.09) 30.22 (19.30) 9.11 (4.47) 10.00 (4.89) 18.67 (20.87) 18.72 (22.39) Distance from Beijing Center (km) 60.79 (22.98) 60.06 (25.84) 48.14 (13.52) 48.89 (30.08) 67.56 (65.28) 68.89 (6.23) 66.67 (32.03) 62.39 (29.32) Notes: Mean values are shown, standard deviations are shown in parentheses. T-test results for mean difference between treatment and control groups: *, **and *** denote 10%, 5% and 1% significance level; 1 Data are missing in Pinggu. Source: Authors’ own calculation based on data from own survey conducted in 2014. 3.2. Descriptive Analysis Table 1 reports descriptive statistics for the participating and non-participating households. The descriptive analysis suggests the presence of noticeable differences between the RDEP participants and non-participants in their observed characteristics. There is a total of nine variables that are significantly different between households in the treatment and control villages. Of the nine variables, Sustainability 2018, 10, 3945 7 of 19 three are village-level variables, and six are household-level variables. Compared to a typical household in the control villages, a typical household in a typical treatment village appeared to be bigger in household size (3.40 versus 3.04), was headed by a younger and more educated member, and possessed more assets. According to our data, the village collective dividend per capita in 2010 of an average control village was much larger than that of an average treatment village. However, the differences in these characteristics between treatment villages and control villages vary considerably across districts. Table 1 shows obvious differences in demographic and socio-economic conditions across the three districts. For example, an average household in Pinggu had almost one member (or one and half members) more than those in Huairou (or Tongzhou). The landholding of an average household in Huairou was 1.5 times (or two times) bigger than that of an average household in Tongzhou (or Pinggu). Additionally, while an average treatment household had a higher value of total assets than an average control household in Tongzhou, the opposite was true in the case of Huairou. As for geographical location, Tongzhou is closer to Beijing, and achieved a much higher gross collective income than the other two districts. These considerable differences across districts collectively remind us to conduct additional analysis to assess the effects of the project separately for each district as well. 4. Methods In this paper, the impacts of the RDEP on crop productivity, input use intensity, labor allocation between on-farm and off-farm employments, and household income were evaluated using the PSM method in each of the three districts. The PSM, which was pioneered by Rosenbaum and Rubin (1983), was mainly developed as a way to match participants to non-participants according to the relevant pretreatment characteristics (X). The problem is that matching participants to non-participants is unmanageable when the number of relevant covariates is large, which is known as the “curse of dimensionality” problem. Rosenbaum and Rubin (1983) proved that if outcomes Y1 (Y0) (Y1 standing for potential outcomes under treatment and Y0 standing for potential outcomes under control) are independent of treatment status conditional on X, then they are also independent of treatment status conditional on P(X), where P(X) is the propensity score of being treated given X. Thus, a multi-dimensional matching exercise is then reduced to a single dimensional matching problem: matching each participant to one or more non-participants according to their estimated propensity scores. A standard logit or probit model can be used to estimate the propensity scores. Under PSM, the average treatment effect on the treated (ATT) is equal to the expected difference in the observed outcomes between participants and matched non-participants, appropriately weighted by the propensity score distribution of participants [16]: ATTPSM = EP(X)|D=1{E[Y1|D = 1, P(X)]− E[Y0|D = 0, P(X)]} (1) where D is a dummy variable indicating treatment status (= 1 if treated, 0 otherwise). The underlying identifying assumption behind the PSM approach is the conditional independence assumption (CIA), which means that after conditioning on observables (X), participants would have the same potential outcome Y0 as non-participants in the absence of the treatment, or more formally: E(Y0|X, D = 1) = E(Y0|X, D = 0) (2) To meet CIA, the researcher should observe all of the variables simultaneously influencing the participation decision and outcome variables. Besides the CIA condition, another important assumption to ensure that the PSM works in practice is the common support or overlap condition. It ensures that farmers with the same X values have a positive probability of being both participants and non-participants. The PSM estimation of program effects involves a number of standard steps [16–18]. The first step is to calculate the sample farmers’ propensity scores based on the estimated coefficients of a logit or probit regression. The next step is to match each household in the treatment group to one or more Sustainability 2018, 10, 3945 8 of 19 households in the control group according to their estimated propensity scores. The third step is to calculate the program effects (ATTPSM) based on Equation (1). There are several matching techniques (e.g., nearest neighbors K (NNK, k = 1 or other positive integer), radius caliper matching, kernel matching, local linear regression matching, and spline matching), with each having its own pros and cons. It is generally recommended to try a few different matching methods. If different matching methods give similar results, then the results are robust. The quality of matching needs to be checked by a balance test, the purpose of which is to determine whether the PSM procedure has served the purpose of making participants and non-participant groups sufficiently similar in terms of observed characteristics. The standard balance test is to test the degree to which the standardized bias or normalized differences in means are reduced via PSM matching. In the present study, Imben’s rule of thumb is also applied, which suggests that the balance is achieved if the percentage of bias is reduced to below 25% [19,20]. Another method to obtain covariate balance that was proposed by Hirano et al. [21] is to weight the observations with their respective estimated propensity scores in a weighted sample. Morgan and Todd [22] also showed that the average treatment effect of the treated (ATT) can be efficiently estimated by inverse probability weighted least square regression, as in Equation (3): Y = α̂ + δ̂OLS D + Xβ̂ + ε (3) where we assign weight ω(t, x) = P(X)× ( t P(X) + 1−t 1−P(X) ) to each observation, t is an indicator for treatment status (=1 if treated, 0 otherwise), and P(X) is the propensity score of being treated. A key assumption for the validity of the PSM method is that selection to participate in the RDEP is conditional on a set of observed characteristics (known as “selection on observables”). If this assumption is invalid, then other methods must be used to determine the project’s effects. For example, to take advantage of the panel information for two key outcome variables (income per capita and total asset), the combination of PSM and the difference-in-difference method (PSM-DID) can be used to evaluate the effects of RDEP on these two variables. Alternatively, a PS-weighted DID regression method proposed by Hirano and Imbens [19], as well as Morgan and Todd [22], can also be employed. Based on an assumption that any selection bias due to unobserved factors is time-invariant, the time-invariant selection biases can be differenced out in both the PSM-DID non-parametric approach and the PS-weighted DID regression method. Accordingly, the PSM-DID estimated ATT is given by: (∑i∈T(Yi1 − Yi0)− ∑j∈C Wij ( Yj1 − Yj0 ) )/NT , where T denotes the set of treated households, C is the set of control households, NT is the number of treated households, and Wij is the associated weight given to the jth non-treated household in comparison with the ith treated household. Similarly, the PS-weighted DID estimator of ATT is given by regressing the change in outcome on the treatment indicator: ∆Yit = α + βDi + ξit, with weights similar to those in Equation (3). 5. Results 5.1. Quality of Different Matching Methods Numerous factors have been documented to influence farmers’ training participation and technology adoption decisions [23–26]. In this paper, the choice of covariates is guided by previous studies in the literature and the data availability. Four sets of characteristics are used as covariates in the logit models: household demographic characteristics, asset ownership, village characteristics, and access to market. Table 2 reports the balance test results for five matching methods with the propensity scores estimated from a logit model. The Hotelling’s T2 indicate that the covariance matrices are the same in the treated and the control groups after matching for all five methods, suggesting that the quality of matching is high for all of the methods. The balance test results in Table 2 further encourage us to choose the kernel matching, radius caliper matching (caliper = 0.25σ’), and NN matching (k = 5) Sustainability 2018, 10, 3945 9 of 19 as the three final matching methods used to estimate the impacts of RDEP (according to the results in Table 2, the Rubin bias is less than 25%, and the covariate biases are reduced to less than 10% for all three methods). To check the overlap region of the matching results, the probability density distributions of propensity scores for the RDEP participants and non-participants is illustrated in Figure 2. The graphic evidence shows that the probability density of the estimated propensity scores between the participants and non-participants change from “very different” before matching to “very similar” after matching for all the three chosen matching methods. Sustainability 2018, 10, x FOR PEER REVIEW 12 of 20 Figure 2. Estimated propensity scores before and after matching by different matching methods: (a) before matching; (b) kernel matching; (c) radius caliper matching; and (d) NN matching. 5.2. Overall Impacts of the RDEP Table 3 reports the estimated impacts of the RDEP on the eight outcome variables based on the three selected matching methods (columns 4–6) and the PS-weighted least square regression (column 3). For comparison purpose, the simple mean differences for all of the outcome variables between the treatment and control groups are also reported (column 2). While the results are generally robust across different matching methods, as well as between the matching methods and the PS-weighted regression approach, they are very different from those based on a simple mean comparison between participants and non-participants. Specifically, the simple mean comparison tends to overestimate the impact on total income, but underestimate the effects on agriculture productivity, including both the labor productivity and the land productivity. Table 3. Estimated effects of the Rural Distance Education Project (RDEP) (all samples). PS: propensity score. Outcome TTEST PS-Weighted Regression (LSR) Radius Caliper (0.25sd) (T = 321, C = 459) NN (k = 5) (T = 321, C = 459) Kernel (BW = 0.04) (T = 321, C = 459) HH gross income in 2013 (yuan) 19,471.02 ** 15,902.88 13,898.58 14,502.61 14,126.17 (9191.86) (11,176.77) (10,530.58) (10,898.43) (10,512.76) HH gross agricultural income (yuan) 14,513.38 ** 14,484.43 14,909.96 16,942.68 * 14,754.48 (8677.44) (10,675.45) (10,445.71) (10,207.95) (10,442.46) Agricultural labor productivity (yuan/person*month) 785.24 ** 767.49 840.05 * 1084.13 ** 823.68 * (393.64) (528.81) (448.61) (469.41) (447.74) Agricultural land productivity (yuan/mu) 302.97 ** 289.49 * 377.67 ** 404.10 ** 383.79 ** (170.13) (160.92) (188.28) (200.24) (189.10) Agricultural input intensity (yuan/mu) 187.55 ** 157.79 ** 188.49 ** 199.38 ** 185.36 ** (83.03) (76.33) (78.49) (92.87) (78.19) Off-farm labor productivity (yuan/person*month) 103.09 23.76 −3.23 −41.25 11.59 (178.49) (155.29) (148.54) (182.60) (148.59) Figure 2. Estimated propensity scores before and after matching by different matching methods: (a) before matching; (b) kernel matching; (c) radius caliper matching; and (d) NN matching. Sustainability 2018, 10, 3945 10 of 19 Table 2. Estimated coefficient and balance test for matching. NN: nearest neighbor. Variable Before Matching After Matching Logit Coefficient Mean NN(1) with Replacement NN(5) with Replacement Radius Cali (0.25sd) (T = 321, C = 459) Radius Cali (0.005) (T = 310, C = 459) Kernel (BW = 0.04) (T = 321, C = 459) Treated Control %bias %bias %bias %bias %bias %bias HH size in 2010 (#) 3.40 3.04 28.3 14.0 −3.7 −2.9 −3.7 −3.7 0.35 *** (0.07) Distance from distance education station in village (m) 415.08 453.01 −7.3 10.8 6.7 4.1 5.1 4.4 −0.00009 (0.0002) Gender of HH head 0.86 0.88 −6.7 −2.8 −4.1 −3.8 −3.0 −4.2 −0.23 (0.23) Age of HH head (log) 3.97 4.00 −15.4 −13.3 −4.5 −3.7 −1.0 −4.2 −0.60 (0.48) Education level of HH head (1 if > nine years, 0 otherwise) 0.44 0.32 25.2 7.1 3 2.2 −0.9 2.6 0.56 *** (0.19) Marriage status of HH head (1 if married, 0 otherwise) 0.96 0.94 11.0 1.4 −1.7 −0.3 −1.9 −0.2 0.31 (0.38) Share of HH members younger than 15 or older than 59 0.21 0.26 −15 −6.0 −2.4 −0.2 −2.4 −0.5 −0.16 (0.28) Education level of head’s father (1 if > nine years, 0 otherwise) 0.23 0.18 10.8 5.4 2.9 3.8 4.7 3.7 0.35 (0.23) Maximum education level of HH members (1 if > nine years, 0 otherwise) 0.41 0.39 4.7 −0.6 6.4 2.4 4.0 2.8 −0.28 (0.18) Party membership of HH head 0.58 0.57 1.2 −4.4 7.2 4.2 3.7 5.0 −0.06 (0.19) Village committee cadre membership (1 if yes, 0 otherwise) 0.19 0.16 8.3 0.8 0.5 −1.3 −3.4 −0.3 0.01 (0.23) Land owned in 2010 (mu) 4.44 5.06 −6.1 2.1 5.6 3.1 6.5 3.8 0.01 (0.02) Square of land owned in 2010 104.60 143.95 −5.3 −0.8 3.5 1.7 4.8 1.9 −0.0002 (0.0003) Proportion of cultivated land (%) 32.85 26.47 9.6 −1.3 5 −0.6 11.5 −0.2 0.002 (0.001) Gross value of assets in 2010 (log) 7.75 7.71 0.7 −2.1 −1.4 −2.5 −1.9 −1.8 −0.26 * (0.14) Sustainability 2018, 10, 3945 11 of 19 Table 2. Cont. Variable Before Matching After Matching Logit Coefficient Mean NN(1) with Replacement NN(5) with Replacement Radius Cali (0.25sd) (T = 321, C = 459) Radius Cali (0.005) (T = 310, C = 459) Kernel (BW = 0.04) (T = 321, C = 459) Treated Control %bias %bias %bias %bias %bias %bias Square of gross value of assets in 2010 94.12 92.22 2.6 −3.3 −1.8 −2.7 −2.3 −2.1 0.02 ** (0.01) HH income per capita in 2010 (log) 6.21 6.14 1.7 0.0 −0.5 −1.3 0.2 −0.7 0.54 *** (0.21) Square of HH income per capita in 2010 58.36 57.61 1.7 0.2 0 −0.7 0.2 0.1 −0.04 ** (0.02) Resident population of village in 2010 (log) 6.57 6.68 −12.4 −2.4 −3.9 −5.8 −3.8 −6.0 −0.39 *** (0.12) Collective assets of village in 2010 (log) 4.28 4.32 −1.7 6.6 3.2 0.6 3.1 1.2 −0.05 (0.06) Collective revenue of village in 2010 (log) 3.09 3.156 −2.7 5.5 4.1 −0.2 2.0 0.7 −0.04 (0.07) Collective dividend per capita in 2010 (log) 1.51 1.36 5.2 −1.4 −0.4 0.3 0.8 0.8 0.10 *** (0.04) Distance from district center (km) 16.79 19.65 −17 −2.2 0.6 −0.6 −0.6 0.5 −0.045 *** (0.009) Distance from Beijing Center (km) 60.85 60.06 3.3 3.9 3.9 2.0 −0.8 2.1 0.028 *** (0.01) Overall balance Hotelling’s T2 92.660 *** 11.350 7.663 5.238 12.114 5.035 Ps R2 0.084 0.016 0.008 0.005 0.012 0.006 LR chi2 89.03 14.42 6.84 4.42 10.07 4.95 P > chi2 0.000 0.954 1.000 1.000 0.996 1.000 Mean Bias 8.2 4 3.2 2.1 3 2.2 Med bias 6.1 2.4 3.2 2.2 2.4 1.9 Rubin B 70.3 * 29.9 * 20.7 16.6 25.6 * 17.6 Rubin R 0.82 1.34 1.22 1.74 1.4 1.63 Var 50 33 33 22 22 22 Notes: 1. Kernel matching by Gaussian kernel; 2. *, ** and *** denote 10%, 5% and 1% significant levels, respectively in column of logit coefficient, and standard deviations in parentheses; 3. In overall balance, the row of Rubin B, * denotes bias is significant above 25%; 4. Authors’ own calculation based on data from own survey conducted in 2014 and results from Stata commands (psmatch2 and ptest). Sustainability 2018, 10, 3945 12 of 19 5.2. Overall Impacts of the RDEP Table 3 reports the estimated impacts of the RDEP on the eight outcome variables based on the three selected matching methods (columns 4–6) and the PS-weighted least square regression (column 3). For comparison purpose, the simple mean differences for all of the outcome variables between the treatment and control groups are also reported (column 2). While the results are generally robust across different matching methods, as well as between the matching methods and the PS-weighted regression approach, they are very different from those based on a simple mean comparison between participants and non-participants. Specifically, the simple mean comparison tends to overestimate the impact on total income, but underestimate the effects on agriculture productivity, including both the labor productivity and the land productivity. Table 3. Estimated effects of the Rural Distance Education Project (RDEP) (all samples). PS: propensity score. Outcome TTEST PS-Weighted Regression (LSR) Radius Caliper (0.25sd) (T = 321, C = 459) NN (k = 5) (T = 321, C = 459) Kernel (BW = 0.04) (T = 321, C = 459) HH gross income in 2013 (yuan) 19,471.02 ** 15,902.88 13,898.58 14,502.61 14,126.17 (9191.86) (11,176.77) (10,530.58) (10,898.43) (10,512.76) HH gross agricultural income (yuan) 14,513.38 ** 14,484.43 14,909.96 16,942.68 * 14,754.48 (8677.44) (10,675.45) (10,445.71) (10,207.95) (10,442.46) Agricultural labor productivity (yuan/person*month) 785.24 ** 767.49 840.05 * 1084.13 ** 823.68 * (393.64) (528.81) (448.61) (469.41) (447.74) Agricultural land productivity (yuan/mu) 302.97 ** 289.49 * 377.67 ** 404.10 ** 383.79 ** (170.13) (160.92) (188.28) (200.24) (189.10) Agricultural input intensity (yuan/mu) 187.55 ** 157.79 ** 188.49 ** 199.38 ** 185.36 ** (83.03) (76.33) (78.49) (92.87) (78.19) Off-farm labor productivity (yuan/person*month) 103.09 23.76 −3.23 −41.25 11.59 (178.49) (155.29) (148.54) (182.60) (148.59) HH labor months working on farm (person*month) 1.07 −0.52 −0.075 0.17 −0.35 (0.71) (1.09) (1.10) (1.05) (1.15) HH off-farm employment (person*month) 4.72 *** 2.71 *** 2.00 * 1.86 2.15 * (0.98) (1.00) (1.09) (1.19) (1.11) Notes: 1. Kernel matching with Gaussian kernel; 2. Matching variables include HH size in 2010, district dummy, distance from the distance education station in the village, gender of HH head, age of HH head, education level of HH head, HH head’s marriage status, education level of head’s father, maximum education level of HH members, party membership, village committee cadre membership, share of members younger than 15 and older than 59 in 2010, land owned in 2010, square of land owned in 2010, proportion of cultivated land in 2010, gross assets in 2010 (log), square of gross asset in 2010, HH income per capita in 2010, resident population of village in 2010, collective assets of village in 2010 (log), collective revenue of village in 2010 (log), collective dividend per capita in 2010 (log), distance from district center, distance from Beijing center; 3. *, ** and *** denote 10%, 5% and 1% significant levels, respectively, and bootstrap standard errors with 500 repetitions in parentheses; 4. Source: calculation based on authors’ own survey data collected in 2014, and results from psmatch2. Results from all of the matching methods and PS-weighted regression indicate positive and significant impacts of RDEP on agricultural labor productivity, agricultural land productivity, and agricultural input use intensity. Participation in the RDEP increased agricultural labor productivity by 824–1084 yuan per month (equivalent to $133–$175 per month), agricultural land productivity by 289–404 yuan per mu ($701–$979 per ha., 1 hectare = 15 mu) and inputs of seeds, fertilizer, and pesticide by 158–199 yuan per mu ($382.03–$482.90 per ha.). The gains in land and labor productivity are also consistent with the household agricultural income being higher for the participating households than for the non-participating households (although significant only in one of the three matching methods, and at 10%). The total household income is also higher for participating households than non-participating households, although none is statistically significant at 10%. With regard to the effects of RDEP on labor allocation between farm and non-farm activities, while participating in the RDEP significantly increased household’s off-farm labor time by about two ‘person months’ (which is also statistically significant except for the case of NN matching), the RDEP had no significant effect on the time spent on farming activities. The RDEP also had no Sustainability 2018, 10, 3945 13 of 19 significant effect on off-farm labor productivity (i.e., earnings from off-farm employment per person month). These combined results tend to suggest that farmers might have benefited from the off-farm employment information rather than from the training of off-farm job skills of the RDEP. Meanwhile, the increase in off-farm employment time alone is not large enough to be translated into a significant increase in household income. For farmers in the peri-urban areas in Beijing, wage income and transfer income (collective dividends, agricultural subsidies, and property income, etc.) account for a major proportion of the gross household income. Assisting households in accessing off-farm employment information and improving household members’ job skills are equally important. 5.3. Heterogeneous Effects of RDEP across Crops To explore whether the impacts of RDEP on crop productivity vary by crops, we obtained PSM estimates of the RDEP’s impacts on crop productivity for two types of crops separately: grain crops and fruit trees, with the former including wheat, corn, soybean, potato, and other grains, and the latter including apple, pear, grape, peach, Chinese chestnut, cherry, and other fruit trees. Vegetables and the specialty crops are not included in the analysis, because they are insignificant in terms of crop areas, and because it is impossible to collect production data for them. The results based on kernel matching are reported in Table 4. The results show that the impacts of the RDEP on land productivity and input use intensity are consistent across the two types of crops and in line with the overall results in Table 3 in the sense that the RDEP significantly increased both outcomes. While the impacts of RDEP on crop income are different between the two types of crops, it is not significant for either type of crop, which is again consistent with the results in Table 3. Therefore, there is no strong evidence to support the heterogeneous effects across crop types, at least based on our data. Table 4. Estimated effects of the RDEP across different crops. Outcome Grain Crops (T = 89, C = 88) Fruit Trees (T = 49, C = 85) Net income from crop production (yuan) 1382.17 (1151.43) −2505.56 (2460.71) Crop land productivity (yuan/mu) 747.42 *** (276.80) 789.03 ** (385.29) Input use intensity (yuan/mu) 345.29 *** (100.99) 461.07 (580.78) Notes: 1. Kernel matching with Gaussian kernel; 2. Matching variables include HH size in 2010, district dummy, distance from the distance education station in the village, gender of HH head, age of HH head, education level of HH head, HH head’s marriage status, education level of head’s father, maximum education level of HH members, party membership, village committee cadre membership, share of members younger than 15 and older than 59 in 2010, land owned in 2010, square of land owned in 2010, proportion of cultivated land in 2010, gross asset in 2010 (log), square of gross asset in 2010, HH income per capita in 2010, resident population of village in 2010, collective assets of village in 2010 (log), collective revenue of village in 2010 (log), collective dividend per capita in 2010 (log), distance from district center, distance from Beijing center; 3. *, ** and *** denote 10%, 5% and 1% significant levels, respectively, and bootstrap standard errors with 500 repetitions in parentheses; 4. Source: calculation based on authors’ own survey data collected in 2014, and results from psmatch2. 5.4. Heterogeneous Effects of RDEP across Districts The descriptive evidence that the three districts differ distinctly in agro-ecological and socio-economic characteristics and in the level of RDEP involvement suggest the importance of evaluating the effects of the RDEP separately for each district. The results based on the kernel matching method from the individual districts are reported in Table 5. It shows that the effects of the RDEP vary considerably across districts. For example, the effects of the RDEP on agriculture land productivity and agricultural inputs intensity are positive and significant in both Tongzhou and Huairou, but neither is significant in Pinggu. Meanwhile, the RDEP significantly increased agricultural labor productivity in Tongzhou, but not in Huairou. In Pinggu district, the only outcome indicator that is Sustainability 2018, 10, 3945 14 of 19 significantly impacted by the RDEP is the person months spent on off-farm employment, and the effect is substantial, as the time spent on off-farm employment by an average participating household is four person months more than that of an average non-participating household. In Tongzhou, the RDEP significantly increased labor productivity by 632 yuan per person month (or $102 per person month), land productivity by 932 yuan per mu (or $2257 per hectare), and input intensity 248 yuan per mu ($599.74 per hectare). Table 5. Estimated effects of the RDEP across different districts. Outcome Tongzhou Pinggu Huairou (T = 84, C = 153) (T = 100, C = 153) (T = 97, C = 153) HH gross income in 2013 (yuan) 6911.19 (8733.02) 39701.78 (33,585.41) −4567.51 (3894.07) HH gross agricultural income (yuan) 6854.45 (5037.54) 35044.53 (33,535.26) 947.73 (1003.00) Agricultural labor productivity (yuan/person*month) 632.07 * (357.82) 887.35 (1115.65) 171.93 (388.53) Agricultural land productivity (yuan/mu) 931.78 ** (412.08) 61.83 (403.43) 380.49 ** (163.91) Agricultural input intensity (yuan/mu) 247.62 *** (87.61) 362.84 (243.16) 76.65 * (41.98) Off-farm labor productivity (yuan/person*month) 374.58 (398.86) −483.52 (389.44) −548.55 (364.49) HH labor months working on farm (person*month) −1.06 (1.93) 1.09 (1.96) 1.13 (1.14) HH off-farm employment (person*month) 2.75 (2.53) 4.31 ** (1.19) 0.21 (1.24) Notes: 1. Kernel matching with Gaussian kernel; 2. Matching variables include HH size in 2010, district dummy, distance from the distance education station in the village, gender of HH head, age of HH head, education level of HH head, HH head’s marriage status, education level of head’s father, maximum education level of HH members, party membership, village committee cadre membership, share of members younger than 15 and older than 59 in 2010, land owned in 2010, square of land owned in 2010, proportion of cultivated land in 2010, gross asset in 2010 (log), square of gross asset in 2010, HH income per capita in 2010, resident population of village in 2010, collective assets of village in 2010 (log), collective revenue of village in 2010 (log), collective dividend per capita in 2010 (log), distance from district center, distance from Beijing center; 3. *, ** and *** denote 10%, 5% and 1% significant levels, respectively, and bootstrap standard errors with 500 repetitions in parentheses; 4. Source: calculation based on authors’ own survey data collected in 2014, and results from psmatch2. 5.5. Heterogeneous Effects across Household Characteristics Finally, to explore the potential heterogeneous effects of the RDEP across household characteristics, all of the households were divided by the head’s level of education (primary, junior, or high school level), as well as by the level of household’s total asset value (above or below the median level of assets). The results for the different levels of education and assets are reported in Table 6. Table 6 suggests the existence of a considerable heterogeneity of the effects of the RDEP across the level of education and the value of assets. In terms of the heterogeneous effects across the level of education, our results show that the RDEP benefits households with a junior high school level of education to a more extensive and significant effect. While the RDEP significantly increased agricultural labor productivity, agricultural input intensity, and farm and off-farm employment, none is significant in the subgroups of primary and high school education levels. One possible explanation for the significant effects of the RDEP on a number of variables for junior high but not for the senior high school level education is that those with higher than a junior high school level education are likely to have more alternative sources to learn about technologies and information, and therefore, the marginal benefit from the RDEP is minimal. That the RDEP is also not effective for those with only primary school education or lower suggests that there exists an education threshold in order for a household to benefit from the RDEP program. Sustainability 2018, 10, 3945 15 of 19 As for the differing effects across assets, the results indicate that the RDEP mainly benefits households with higher levels of assets. Participating in the RDEP significantly increased agricultural labor productivity by 751 yuan per month (or USD $121.18 per month), agricultural land productivity by 933 yuan per mu (USD $2258.02 per hectare) and input use intensity by 250 per mu (USD $603.57 per hectare) for the households with higher levels of assets. The RDEP also significantly increased the gross agricultural income by 7027 yuan per year (USD $1134.58 per year) for the households with higher levels of assets. Although the effects on these outcomes are also positive for the households with lower levels of assets, they are largely insignificant. Table 6. Estimated effects of RDEP across household characteristics. Outcome HH Head’s Education Level HH Assets Level Primary Junior High School High School and Above Low High T = 40, C = 84 T = 133, C = 229 T = 146, C = 125 T = 108, C = 147 T = 100, C = 153 HH gross income in 2013 (¥ yuan) 774.631 (8590.091) 28,216.85 (21261.2) 5347.196 (14,426.45) −5769.666 (5650.171) 3781.995 (5648.524) HH gross agriculture income (¥ yuan) −2358.306 (5282.46) 24,395.41 (22,020.17) 10,855.85 (14,318.97) 2441.011 (3089.184) 7026.655 ** (3572.667) Agricultural labor productivity (yuan/person*month) −1210.353 (1449.343) 811.709 * (472.317) 1145.64 (1007.168) 666.430 (883.412) 750.508 ** (291.508) Agricultural land productivity (yuan/mu) 699.001 (571.745) 237.022 (237.140) 274.947 (319.822) 64.387 (206.987) 932.292 *** (304.855) Agricultural input intensity (yuan/mu) −81.662 (363.014) 290.495 ** (114.390) 171.252 (141.923) 75.606 (46.708) 249.199 *** (58.052) Off-farm labor productivity (yuan/person*month) −204.667 (392.527) 82.910 (219.999) −115.138 (257.239) −143.378 (190.762) −303.586 (457.129) HH labor months working on farm (person*month) −1.549 (5.116) 1.964 * (1.118) −1.166 (1.188) −0.461 (1.024) 1.216 (1.127) HH off-farm employment (person*month) 5.469 (3.589) 3.210 ** (1.264) 0.205 (1.254) 1.540 (1.425) 0.504 (1.322) Notes: 1. Estimated by Kernel matching Gaussian kernel, BW = 0.06. Matching variables for different education levels include: HH size in 2010, HH head age, party membership, village committee cadre membership, land owned in 2010, square of land owned 2010, proportion of cultivated land, HH income per capita 2010, square of HH income per capita 2010, collective dividend per capita in 2010, distance from district center, distance from Beijing center. Matching variables for different asset levels include: HH size in 2010, HH head age, HH head education level, HH head marriage status, HH head’s father education level, HH member maximum education level, ratio of child and ages in 2010, land owned in 2010, square of land owned in 2010, proportion of cultivated land, HH income per capita in 2010, collective dividend per capita in 2010, distance from district center; 2. *, ** and *** denote 10%, 5% and 1% significant levels, respectively, and bootstrap standard errors with 500 repetitions in parentheses; 3. Author’s calculation based on survey 2014 by psmatch2. 5.6. Impact of the RDEP on Income Per Capita and Assets Using the DID-PSM Approach, 2010–2013 During the field interview, recalled data were also collected for income per capita and the total value of fixed assets in 2010, which are the two most important outcome variables of the study. This additional valuable information allows us to use a combination of the PSM and difference-in-difference (DID) approaches (in short, DID-PSM) to assess the impacts of the RDEP on these two key outcome indicators. As discussed in the method section, PS-weighted DID was also used to estimate the impact. The results based on the DID-PSM (kernel regression-based PSM) and PS-weighted DID regression are reported in Table 7. At the 10% level of significance, the RDEP significantly increased the income per capita by approximately 10% when the data from Tongzhou and Huairou are pooled together. However, the effects, vary across districts. While the RDEP increased income per capita by 13.6–16.8% in Tongzhou and the impact is significant at 5% and 11% based on DID-PSM and PS-weighted DID regressions, respectively, it had no significant effect on income in Huairou. The effects estimated by PS-weighted DID on the total asset value of the pooled sample and Tongzhou sample were both significant and positive. However, for the Huairou sample, it was insignificant no matter whether the evaluation was based on DID-PSM or PS-weighted DID regressions. The results further support the earlier findings that the impact of the RDEP was bigger and more Sustainability 2018, 10, 3945 16 of 19 statistically significant in Tongzhou than in the other two districts. Had we had access to more panel data, we would have evaluated the RDEP impacts more comprehensively and accurately. Table 7. Effects of the RDEP on income per capita and HH assets 2010–2013 (difference-in-difference (DID) estimation). Outcome Pooled sample Tongzhou Huairou (T = 216, C = 296) (T = 84, C = 153) (T = 97, C = 153) PS Kernel Matched PS-Weighted PS Kernel Matched PS-Weighted PS Kernel Matched PS-Weighted Household income per capita (log) 0.11 * 0.11 * 0.17 ** 0.14 0.04 0.09 (0.05) (0.06) (0.08) (0.08) (0.09) (0.09) Household assets (log) 0.17 0.21 * 0.16 * 0.19 * 0.19 0.28 (0.11) (0.13) (0.09) (0.10) (0.21) (0.23) Note: 1. Data on HH income and HH asset were not collected in Pinggu, so the pooled regression (column 1) includes data from Tongzhou and Huairou only; 2. Matching methods and parameters are based on Table 6; 3. *, ** and *** denote 10%, 5% and 1% significant levels, respectively, and robust standard errors in parentheses; 4. Source: calculation based on authors’ own survey data collected in 2014. 6. Discussion Since the beginning, the rapid expansion of ICT-based agricultural extension services in developing countries has been accompanied by a wide skepticism regarding its uncertain impact, sustainability, and scale [13]. This paper provides encouraging evidence to support the ICT-based extension service, which indicates that RDEP increases farm households’ agricultural productivity. Notable heterogeneity exists across different characteristics at the village and household levels, which raises questions regarding the mechanism of how ICT improves agriculture extension efficiency and famers’ welfare. Unlike the early phase of mobile phone expansion in developing countries, which has shown to have significantly reduced the costs of obtaining information on market and new technology [4,7,13], the RDEP was implemented in peri-urban areas with large-scale coverage of mobile phones and internet services, where the ICT access effect began to fade away. What has been estimated in this paper is the impact of an alternative information channel and its transmitting content, but not the ICT access effect per se. While the identified overall positive effects of the RDEP on agricultural productivity and input use intensity are encouraging, the estimated heterogeneous effects across different groups are more telling and important for future policy design. In this regard, two points deserve our attention. The first one is the possible role of “digital engagement”. It has been shown earlier that the RDEP stations in Tongzhou were more intensively utilized (100 h per year) than those in Pinggu and Huairou (35 h and 67 h, respectively), and the effects of the RDEP on agriculture outcomes in Tongzhou were also the highest among all three districts. Previous studies have shown that the digital engagement divide is becoming more prominent with the narrowing of the digital access divide [27]. Digital applications themselves can become a habit or adoption decision, and if they are not sufficiently inclusive or inconvenient for the vulnerable group, they may become a new medium of inequality. Thus, it is important to implement inclusive public ICT-based services in rural areas. Relatedly, the finding that the productivity effects of the RDEP are positive and statistically significant for households with junior high school education, but not for those with a higher level of education or lower level of education, implies that targeting those with a medium level of education but more limited alternatives to acquire information is likely to be more effective. Second, it should also be kept in mind that ICT is not a ready-made panacea. An ICT-based extension service could potentially exacerbate income inequality in rural areas because of the different capacities of utilizing the new information channel between farmers with more assets and those with fewer assets. Practically, farmers’ awareness of improved technologies and techniques may increase through ICT-based extension projects; however, such increased awareness does not automatically translate into behavioral changes such as an increased adoption of improved practices or modern input. Sustainability 2018, 10, 3945 17 of 19 ICT-based extension projects can only solve some of the barriers faced by farmers; others depend on complement factors, such as farmers’ endowment conditions, market participation threshold, infrastructures, and institutional or cultural context [28–30]. If the aim of the project is to narrow the gap in agriculture productivity between poor and wealthy farmers, technology extension services packaged with skill training, input support, and other assistance could be a consideration in the future. 7. Conclusions and Policy Implications ICT-based extension services have been widely viewed as the future mode of extension. There have been active studies of the effectiveness and impacts of access to ICT-based technologies with a focus on mobile phones in Asia and Africa [26]. Nevertheless, the results of the impacts are mixed. While the literature on the impact of ICT-based technology and extension/education programs in developing countries is emerging rapidly, the rigorous empirical evidence on China is disproportionately scarce, despite China’s importance in the global economy and its vast stock of ICT-based infrastructure and internet. This study aims to complement the emerging literature by evaluating the impacts of a rural distance education program through using data from peri-urban areas of Beijing. Specifically, this study utilizes survey data from treatment and control villages, and the propensity score matching method to evaluate the short-term impacts of the RDEP on farmers’ agricultural productivity, input use, labor allocation, and overall welfare. This study is among the first few rigorous evaluations of the RDEP on agricultural performance, household income, and off-farm employment, and therefore provides important policy implications on the design and implementation of effective rural distance education programs. First, the findings that the RDEP had a positive and significant effect on agricultural productivity and input use reassures that a rural distance education program can be an effective method to train farmers to acquire information and new agricultural technologies. Second, the existence of a considerable variation in RDEP effects across districts and household characteristics implies that an effective RDEP should take the local community conditions and households’ characteristics into consideration in formulating, implementing, and scaling up the RDEP programs. It is important to note that it is the households with junior high education that benefit the most from the RDEP program, so they should be the prioritized targeting groups for receiving the RDEP projects. Our results also imply that it is important to identify the factors that prevent those with lower than a junior high education and those of lower assets from benefiting from the RDEP projects. There are also a few caveats to our study. We focus on relatively short-term effects, yet the adoption of new technologies or services is likely to take time to be effective. On the other hand, there are also studies showing that the effects of a training program may decay over time. We would like to explore the long-term versus short-term effects in our future research. Also, while the PSM is a popular method to evaluate training and extension programs, the PSM’s inability to control for selection on unobservable variables is always a concern. We would like to use alternative evaluation methods that better account for selection on unobservable variables in the future research. Finally, in the future, we would also like to pursue the evaluation of similar programs regarding outcomes that are related to food safety and the environment. As of today, the safety of agriculture products contributed from the overuse of agro-chemicals is among the top public concerns in China, and it is considered one of the most effective ways to spread environment-friendly agricultural technology to farmers through ICT-based services. Author Contributions: Conceptualization, J.G. and S.J.; Methodology, S.J.; Investigation, L.C. and J.Z.; Formal analysis, J.G. and S.J.; Writing—Original Draft Preparation, J.G.; Writing—Review & Editing, J.G. and S.J. Funding: This research was funded by Beijing Natural Science Foundation, 9172009 and Beijing Social Science Foundation, 17SRB008, and National Natural Science foundation of China (71673244 and 71773110). Acknowledgments: Guo specially thanks the financial support from the China Scholarship Council for staying at the Michigan State University to undertake the data analysis and draft paper writing. Jin gratefully acknowledges Sustainability 2018, 10, 3945 18 of 19 support from Michigan State University’s AgBioResearch. Comments by anonymous reviewers on this paper were also very helpful. All remaining errors are our own. Conflicts of Interest: The authors declare no conflict of interest. References 1. Grimes, S. Rural areas in the information society: Diminishing distance or increasing learning capacity? J. Rural Stud. 2000, 16, 13–21. [CrossRef] 2. Sun, C.; Yang, H. Empirical research on rural information service for farmers’ income—Based on the analysis of panel data in Harbin. J. Agric. Mech. Res. 2014, 10, 58–62. [CrossRef] 3. Anderson, J.R.; Feder, G. Agricultural extension: Good intentions and hard realities. World Bank Res. Obs. 2004, 19, 41–60. [CrossRef] 4. Jensen, R. The digital provide: Information (technology), market performance, and welfare in the south indian fisheries sector. Q. J. Econ. 2007, 122, 879–924. [CrossRef] 5. Muto, M.; Yamano, T. The impact of mobile phone coverage expansion on market participation: Panel Data Evidence from Uganda. World Dev. 2009, 37, 1887–1896. [CrossRef] 6. Tadesse, G.; Bahiigwa, G. Mobile phones and farmers’ marketing decisions in Ethiopia. World Dev. 2015, 68, 296–307. [CrossRef] 7. Aker, J.C.; Ksoll, C. Can mobile phones improve agricultural outcomes? Evidence from a randomized experiment in Niger. Food Policy 2016, 60, 44–51. [CrossRef] 8. Shimamoto, D.; Yamada, H.; Gummert, M. Mobile phones and market information: Evidence from rural Cambodia. Food Policy 2015, 57, 135–141. [CrossRef] 9. Hennessy, T.; Läpple, D.; Moran, B. The digital divide in farming: A problem of access or engagement? Appl. Econ. Perspect. Policy 2016, 38, 474–491. [CrossRef] 10. Ogutu, S.O.; Okello, J.J.; Otieno, D.J. Impact of information and communication technology-based market information services on smallholder farm input use and productivity: The case of Kenya. World Dev. 2014, 64, 311–321. [CrossRef] 11. Al-Hassan, R.M.; Egyir, I.S.; Abakah, J. Farm household level impacts of information communication technology (ICT)-based agricultural market information in Ghana. J. Dev. Agric. Econ. 2013, 5, 161–167. [CrossRef] 12. Cole, S.A.; Fernando, A.N. The Value of Advice: Evidence from Mobile Phone-Based Agricultural Extension. In Harvard Business School Working Paper; No. 13–047; Harvard Library: Cambridge, MA, USA, 2012. 13. Aker, J.C. Dial A for agriculture a review if information and communication technologies for Agricultural Extension in Developing Countries. Agric. Econ. 2011, 42, 631–647. [CrossRef] 14. Labonne, J.; Chase, R.S. The Power of Information: The Impact of Mobile Phones on Farmers’ Welfare in the Philippines; Impact Evaluation Series No. 33, Policy Research Working Paper No. 4996; The World Bank: Washington, DC, USA, 2009. 15. Hou, J.; Huo, X. Can informatization improve farmers’ market participation? Micro evidence frommain apple production areas in China. J. Financ. Econ. 2017, 43, 134–144. 16. Caliendo, M.; Kopeinig, S. Some practical guidance for the implementation of propensity score matching. J. Econ. Surv. 2008, 22, 31–72. [CrossRef] 17. Dehejia, R.H.; Wahba, S. Propensity score-matching methods for nonexperimental causal studies. Rev. Econ. Stat. 2002, 84, 151–161. [CrossRef] 18. Abadie, A.; Imbens, G.W. Bias-Corrected matching estimators for average treatment effects. J. Bus. Econ. Stat. 2011, 29, 1–11. [CrossRef] 19. Imbens, G.W.; Wooldridge, J.M. Recent developments in the econometrics of program evaluation. J. Econ. Lit. 2009, 47, 5–86. [CrossRef] 20. Wooldridge, J.M. Econometric Analysis of Cross Section and Panel Data; The MIT Press: Cambridge, MA, USA, 2010. 21. Hirano, K.; Imbens, G.W.; Riddr, G. Efficient estimation of average treatment effects using the estimated propensity score. Econometric 2003, 71, 1161–1189. [CrossRef] 22. Morgan, S.L.; Todd, J.J. A diagnostic routine for the detection of consequential heterogeneity of causal effect. Sociol. Methodol. 2008, 38, 231–282. [CrossRef] http://dx.doi.org/10.1016/S0743-0167(99)00027-3 http://dx.doi.org/10.13427/j.cnki.njyi.2014.10.014 http://dx.doi.org/10.1093/wbro/lkh013 http://dx.doi.org/10.1162/qjec.122.3.879 http://dx.doi.org/10.1016/j.worlddev.2009.05.004 http://dx.doi.org/10.1016/j.worlddev.2014.12.010 http://dx.doi.org/10.1016/j.foodpol.2015.03.006 http://dx.doi.org/10.1016/j.foodpol.2015.10.005 http://dx.doi.org/10.1093/aepp/ppw015 http://dx.doi.org/10.1016/j.worlddev.2014.06.011 http://dx.doi.org/10.5897/JDAE12.143 http://dx.doi.org/10.1111/j.1574-0862.2011.00545.x http://dx.doi.org/10.1111/j.1467-6419.2007.00527.x http://dx.doi.org/10.1162/003465302317331982 http://dx.doi.org/10.1198/jbes.2009.07333 http://dx.doi.org/10.1257/jel.47.1.5 http://dx.doi.org/10.1111/1468-0262.00442 http://dx.doi.org/10.1111/j.1467-9531.2008.00204.x Sustainability 2018, 10, 3945 19 of 19 23. Duflo, E.; Kremer, M.; Robinson, J. Nudging farmers to use fertilizer: Theory and experimental evidence from Kenya. Am. Econ. Rev. 2011, 101, 2350–2390. [CrossRef] 24. Verhofatadt, E.; Maertens, M. Smallholder cooperatives and agricultural performance in Rwanda: Do organizational differences matter? Agric. Econ. 2014, 45, 39–52. [CrossRef] 25. Chen, S.; Mu, R.; Ravallion, M. Are There Lasting Impacts of Aid to Poor Areas? Evidence from Rural China; Policy Research Working Paper Series, No. 4084; The World Bank: Washington, DC, USA, 2006. 26. Becerril, J.; Abdulai, A. The Impact of improved maize varieties on poverty in mexico: A propensity score-matching approach. World Dev. 2010, 38, 1024–1035. [CrossRef] 27. World Bank Group. World Development Report 2016: Digital Dividends; World Bank Publications: Washington, DC, USA, 2016. 28. Goyal, A. Information, Direct Access to Farmers, and Rural Market Performance in Central India; Policy Research Working Paper No. 5315; The World Bank: Washington, DC, USA, 2010. 29. Deichmann, U.; Goyal, A.; Mishra, D. Will Digital Technologies Transform Agriculture in Developing Countries? Policy Research Working Paper 7669; The World Bank: Washington, DC, USA, 2016. 30. Jensen, R. Information Efficiency and Welfare in agricultural market. In Proceedings of the 27th International Association of Agricultural Economists Conference, Beijing, China, 16–22 August 2009. © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). http://dx.doi.org/10.1257/aer.101.6.2350 http://dx.doi.org/10.1111/agec.12128 http://dx.doi.org/10.1016/j.worlddev.2009.11.017 http://creativecommons.org/ http://creativecommons.org/licenses/by/4.0/. Introduction RDEP Background Sample and Data Sample and Data Collection Descriptive Analysis Methods Results Quality of Different Matching Methods Overall Impacts of the RDEP Heterogeneous Effects of RDEP across Crops Heterogeneous Effects of RDEP across Districts Heterogeneous Effects across Household Characteristics Impact of the RDEP on Income Per Capita and Assets Using the DID-PSM Approach, 2010–2013 Discussion Conclusions and Policy Implications References