key: cord-0794010-jkdjgjp7 authors: Omrani, Hashem; Shamsi, Meisam; Emrouznejad, Ali title: Evaluating sustainable efficiency of decision-making units considering undesirable outputs: an application to airline using integrated multi-objective DEA-TOPSIS date: 2022-03-30 journal: Environ Dev Sustain DOI: 10.1007/s10668-022-02285-8 sha: b898898f162c04c83a658e758273fcb8a8c4101c doc_id: 794010 cord_uid: jkdjgjp7 Sustainable development has gained significant attention in the literature due to the increased global awareness of environmental sustainability during the last decade. Sustainable development has three aspects, including economic, social, and environmental. The challenge of sustainable development is to establish a balance between these three aspects. Assessing the efficiency of a company contributes comprehensive information to improve its overall performance. Despite numerous studies in this field, the literature lacks studies that simultaneously consider all three aspects of sustainable development, especially the social aspect. The main objective of this paper is to calculate the technical, social, and environmental efficiency scores. We also introduce a new efficiency called sustainable efficiency that merges all three sustainable development aspects in one efficiency score. This study applies two existing data envelopment analysis (DEA) models to evaluate technical, social, environmental, and sustainable efficiencies. These models, namely the three-step method and the modified three-step method, are computationally intensive. Also, this paper introduces two new DEA models, namely the common weight goal programming DEA and the common weight DEA, to assess the efficiencies with much fewer computations. Each model produces results that are different from one another. Therefore, the TOPSIS approach is applied to provide an overall result by integrating the results obtained from the four presented models. For this purpose, the implementation of four TOPSIS models is required. To illustrate the capability and validity of the developed models in efficiency calculation, a case of Iranian airlines is presented. The selected airlines are evaluated in different aspects, and final results are obtained by applying TOPSIS. The findings show that using TOPSIS to combine the results of several DEA models leads to a fully ranking of airlines in four aspects of technical, social, environmental, and sustainable efficiencies. Also, it is recommended to managers to probe pairwise comparison between different efficiencies of airlines in order to find and improve the weak ones. We live in a world where communication and the development of its civilizations require a transportation network. The transportation industry reflects the economic state and industrial development. Hence, this industry should be considered as one of the crucial factors in any society's economic, cultural, and social development. As a part of the transportation network, the airline industry plays a fundamental role in different countries' relations, exchanging culture, displaying economic and military powers, and accelerating a country's vital affairs. Many people choose air transportation every day for various purposes. The airline industry has a growing share of freight and passenger traffic compared to other transportation modes in terms of ease of access, high transportation speed, reasonable cost, and high reliability. The air transport system facilitates access to global markets and services. Air transport has developed a global supply chain in which companies can export their products to anywhere in the world in a matter of hours rather than a few days. In 2019, the International Air Transport Association (IATA) announced that airlines enabled the freedom to do business globally by transporting 64 million tonnes of cargo to markets worldwide. 1 According to the IATA annual review 2019, this activity supported a third of global trade by value, generated 65 million jobs, and contributed $2.7 trillion in Gross Domestic Production (GDP). In addition, according to the IATA forecast, it is expected to increase these values to 105 million jobs and $6 trillion in 2034. 2 Iran has a vital role in international transportation regarding its particular geographical and strategic location, connecting East Asia to West Asia. Therefore, employing scientific methods to evaluate Iranian airlines' performance to improve their productivity is considered an important issue as the main subject of this research. In recent years, sustainable development has attracted the attention of researchers in various fields of study. One topic addressed by researchers in this field is to analyze the relationship between a company's financial performance and its social and environmental responsibilities. Aktaş and Demirel (2021) stated that sustainable corporates seek environmental and social improvements besides corporates' profitability. Sustainable development refers to the condition in which society can provide life satisfaction using existing resources without jeopardizing the sustainability of the natural ecosystem for future generations while balancing the economic, social, and environmental criteria. In particular, two primary elements that must be considered in sustainable development are environmental and social standards. Environmental sustainability focuses on reducing environmental pollution, and social sustainability emphasizes increasing human well-being (Long, 2021) . Nowadays, due to population growth and declining natural resources, governments and various international agencies expect manufacturing and service organizations to update their procedures under the principles of sustainable development (Lee & Lam, 2012) , which leads to economic growth and environmental preservation (Park et al., 2018) . Besides, adhering to sustainable development protocols can improve the organization's reputation due to its attention to social responsibility. This study attempts to provide an integrated approach to investigate the companies regarding social and environmental practices. As a major component of the aviation industry, airlines can affect environmental and social indicators. Each airline has its own aircraft. These aircraft can emit different amounts of pollutants according to various criteria such as aircraft age and engine modifications. On the other hand, each airline has, directly and indirectly, created several job opportunities, which can be considered as a social factor. Thus, the importance of making airlines sustainable is essential in this regard. Although some scholars have strived to investigate sustainable development within airlines' efficiency measurement, the implementation is still not adequate, and there are still gaps in the literature. In today's competitive environment, organizations consider increasing efficiency as their primary goal. Efficiency demonstrates how the organization manages available resources to provide a product or service (Chen, 2005) . Various efficiency measurement methods are used to evaluate the performance of a company or organization. Measurement of efficiency and effectiveness enables organizations to identify the causes of inefficiencies, enhancing efficiency, and ultimately increasing the organization's productivity. Efficiency measurement methods can also help policy-makers formulate and implement appropriate policies to improve efficiency and increase productivity in the transportation industry. Efficiency measurement methods can be categorized into two main groups: parametric models such as stochastic frontier analysis (SFA) approach and nonparametric models such as DEA. DEA is one of the most used nonparametric models for measuring efficiency which was introduced by Charnes et al. (1978) and hence named CCR. They developed a model that was able to convert the fractional linear measure programming to a linear programming model to calculate efficiency scores by performing a series of linear programming optimizations separately for each decision-making unit (DMU). DEA is a mathematical programming model capable of calculating efficiency for multiple similar DMUs with various inputs and outputs without assigning former weight to the input and output variables. In this method, the efficiency frontier is calculated based on efficient DMUs previously defined for the DEA model to calculate other units' efficiency (Shirazi & Mohammadi, 2019) . More specifically, DEA defines a production function for each DMU, and then, it calculates the efficiency of that single DMU by comparing it with the efficiency of other DMUs. The DMUs are divided into two groups in the DEA method: efficient DMUs with a score of 1 (100%) and inefficient DMUs with a score of less than 1. This classification enables the standard DEA model to calculate each dataset's efficiency without ranking all DMUs (Aldamak & Zolfaghari, 2017 ). In the standard DEA approach, the DMU's under evaluation efficiency can be improved by decreasing or increasing outputs. However, in real-world conditions, organizations encounter undesirable outputs that need to be reduced. Standard DEA models lack the power to reduce undesirable outputs. To solve this issue, we need to use some advanced DEA models. Researchers have modeled undesirable outputs in the DEA process, and several approaches have been proposed based on various techniques categorized into four methods. In the first method, the undesirable outputs are ignored. Ignoring undesirable outputs means that these outputs do not affect the evaluation process, which leads to inaccurate results (Yang & Pollitt, 2009 ). The second method offers to consider undesirable outputs as normal inputs. The DEA approach seeks to decrease inputs and increase outputs. Given that undesirable outputs should also be reduced, so it is considered as inputs. In the third method, the undesirable outputs are treated in their actual format same as desirable outputs. In this approach, the undesirable outputs are modeled through nonlinear modeling methods. This method assumes that undesirable outputs are weakly disposable, which means the value of undesirable outputs cannot be increased without influencing the values of other desirable outputs (Färe et al., 1989) . The final method suggests that necessary transformations should be applied to undesirable outputs. For example, Koopmans (1951) indicates that undesirable outputs could be converted to desirable outputs by multiplying them to minus one. The DEA's main challenge is simultaneously considering inputs, desirable outputs, and undesirable outputs in the evaluation process. Various techniques have been proposed to overcome this issue. Korhonen and Luptacik (2004) and Zhang et al. (2008) introduced three types of efficiency, including technical efficiency, environmental efficiency, and eco-efficiency, that should be calculated for each DMU. It should be noted that technical efficiency refers to operational, resource, and technology efficiencies. They proposed TEf = Desirable Outputs∕Inputs to estimate the technical efficiency and EEf = Desirable Outputs∕Undesirable Outputs to estimate the environmental efficiency. They combined these two ratios, introduced a new efficiency called eco-efficiency, and then proposed various approaches to measuring it. Korhonen and Luptacik (2004) proposed to treat undesirable outputs as inputs in this approach to calculate eco-efficiency. This approach is called "the three-step methodology," which calculates technical, environmental, and eco-efficiency scores. The three-step methodology analyzes the efficiency of DMUs from three different aspects separately and offers a practical comprehension for the efficiency of DMUs. However, this method has its drawbacks. First, the three-step methodology runs three models to calculate technical, environmental, and eco-efficiency scores for each DMU; hence, it is computationally intensive. Second, according to Mahdiloo et al. (2015) , in this method, it is not necessary to calculate the eco-efficiency score because the eco-efficiency value is equal to the maximum values obtained for environmental and technical efficiency. Mahdiloo et al. (2015) modified the three-step methodology and proposed a single multiple objective linear programming (MOLP) model. We call this model the "modified three-step model." The developed model run once instead of running three separate models for each DMU to calculate the eco-efficiency score by determining non-dominated weights for inputs and outputs of each DMU. However, we believe that the modified three-step methodology proposed by Mahdiloo et al. (2015) is still computationally intensive, and there is still room to enhance the model. Furthermore, these two models do not take into account the social aspect of sustainable development. Most of the previous studies consider only two aspects of sustainable development, namely technical and environmental. Thus, in this study, we incorporate the social aspect of sustainable development in the model. In this study, we have tried to contribute to the DEA application development by extending the three-step methodology and the modified three-step model and also by proposing two methods to simultaneously calculate airlines' technical, social, and environmental scores by finding common weights for all inputs and outputs. Also, a new type of efficiency called sustainable efficiency is introduced, which combines economic, social, and environmental efficiency to evaluate organizations' efficiency in terms of sustainable development criteria. The developed models are applied in a dataset obtained from (Iran Civil Aviation Organization). The first proposed model is called "common weight goal programming DEA" which is developed by integrating and extending the three-step methodology (Korhonen & Luptacik, 2004) and the modified three-step method (Mahdiloo et al., 2015) into a MOLP model based on goal programming approach. Furthermore, another linear programming (LP) model called "common weight DEA" is also proposed. The two proposed models determine common weights for all inputs and outputs rather than optimal weights in one run. The current study introduces four types of efficiency (technical, social, environmental, and sustainable efficiency). All four efficiencies are calculated with four discussed models. Thus, four technical scores, four social scores, four environmental scores, and four sustainable scores are obtained for each DMU. To provide a comprehensive result, the obtained efficiency scores are integrated with the TOPSIS method. Figure 1 show a general process of the explained procedure. First, we feed the collected data to four different DEA models to calculate four types of efficiency scores. Then, each type of calculated efficiency score is fed to its own TOPSIS model. Therefore, four TOPSIS models are implemented to calculate the integrated efficiency scores. The main contributions of this study are as follows: The three-step model Fig. 1 The process of calculating the integrated efficiency scores by four different DEA models • Extending the two existing well-known DEA models (three-step DEA and modified three-step DEA) in the literature to evaluate more than two types of efficiency of airline performance. • Implementing a goal programming approach to modify the two existing DEA models in a way to reduce the complexity of the calculations. • Considering the social aspect of the airlines' efficiency in the DEA model as well as considering undesirable outputs in the calculations. The remainder of this paper is organized as follows: Sect. 2 reviews the related literate. In Sect. 3, four DEA models are described. It shows how four different DEA models calculate technical, social, environmental, and sustainable efficiency scores. The proposed common weight goal programming DEA model and common weight DEA model are also described in Sect. 2. A case study is presented in Sect. 4. Four different DEA models are applied to calculate efficiency scores in this section. The implementation of the TOPSIS method to calculate integrated efficiency scores is also described in Sect. 4. Section 5 demonstrates the achieved results. In Sect. 6, we present a summarized conclusion of the current study. A review of previous studies confirms that the employment of the DEA method as a powerful and reliable tool to evaluate the efficiency of companies has expanded rapidly since its introduction by Charnes et al. (1978) . Cui and Li (2017) reported that several studies have attempted to calculate the efficiency of airlines using the DEA method. Kiani Mavi et al. (2010) introduced a new method for forecasting in DEA by integrating the group analytic hierarchy process (GAHP) into DEA. The developed model was superior to conventional methods due to having the advantages of both approaches. Alinezhad et al. (2011) combined the multiple criteria decision-making (MCDM) approaches with the DEA model to introduce a novel approach for the selection of the best advanced manufacturing technology candidates based on a number of attributes. Zhu (2011) measured the efficiency of 21 airlines using a centralized two-stage network DEA model. Chang et al. (2014) studied the economic and environmental efficiency of 27 airlines by applying the slack-based measure DEA (SBM-DEA) model. They categorized inefficient airlines into two major groups: the airlines with fuel inefficiency and the airlines with less diversified revenue structure. Amini et al. (2016) proposed a DEA model to evaluate the efficiency of a green supply chain with undesirable outputs. Cui and Li (2016) introduced a new two-stage operating framework, including operations and carbon abatement stages. Then, they developed a network SBM model with weak disposability to assess the efficiency of 22 international airlines. Barak and Dahooei (2018) have proposed a novel integrated hybrid fuzzy DEA and fuzzy Multiple Attribute Decision-Making (MADM) model to assess the safety efficiency of airlines. They employed the DEA model only to determine the criteria weights. Then, six MADM approaches are utilized to rank the alternatives. Sakthidharan and Sivaraman (2018) applied an input-oriented DEA model to examine how efficiently each airline used its operating cost relative to other airlines in India between 2013 and 2014. Yu et al. (2019) have measured the airline efficiency in China and India using a dynamic network DEA approach during the period between 2008 and 2015. Wang et al. (2019) have presented a hybrid model which includes a DEA window model and grey theory approach to estimate the present and future efficiency of 16 major Asian airlines. Kuljanin et al. (2019) implemented a fuzzy theory-based DEA model to determine the efficiency of airlines located in Central and South-East Europe. Rapposelli and Za (2020) In their study, Chen et al. (2021) developed a two-stage undesirable SBM network DEA model to investigate the operational and environmental efficiency of nine Chinese airlines from 2013 to 2018. They combined the undesirable SBM approach to include the CO 2 emissions into the calculation. Zhang et al. (2021) declared that most existing researches only consider operational dimensions in the performance and efficiency evaluation while disregarding stock market indicators in their methodological approaches. Hence, they proposed and utilized a two-stage network DEA to consider operational and stock market indicators in evaluating nine major international airline companies from 2006 until 2016. Kim and Son (2021) implemented a DEA approach to investigate the environmental efficiency for airlines belonging to each continent. They considered fuel consumption, operating cost, the number of employees, and the number of fleets as the input and total revenue, revenue passenger kilometers, revenue ton kilometers, passenger load factor, cargo load factor, and CO 2 reduction as the output. Various DEA models have been proposed in the literature to evaluate the performance of airlines in Iran. Tavassoli et al. (2014) introduced a SBM-DEA model to assess the technical efficiency and service effectiveness of airlines in Iran for the first time. The developed model was able to convert both non-storable features of transportation service and production technologies into a unified framework. Omrani and Soltanzadeh (2016) proposed a dynamic network DEA structure to assess the performance of eight interconnected airlines in Iran. The proposed model was able to identify and improve airlines' inefficient processes by analyzing their internal structure. Frontier-type models such as DNDEA models cannot handle uncertain data. Thus, by integrating the DNDEA and fuzzy logic, Soltanzadeh and Omrani (2018) proposed a new model that could deal with fluctuations in data. In this method, the efficiency scores are represented by membership functions rather than definite values, which provide more information about DMUs. Shirazi and Mohammadi (2019) introduced a novel robust DEA model based on the SBM approach to evaluate the efficiency of 14 airlines in Iran. They included undesirable outputs and inaccurate data in the proposed model to simulate the realworld situation. Heydari et al. (2020) presented a fully fuzzy network DEA-RAM model to measure Iran airlines' efficiency scores. First, they designed a network structure for airlines, and then, they proposed a DEA-RAM model to measure airlines' performance. They applied a fully fuzzy method to the developed model to control the uncertain data. Tavassoli et al. (2020) developed a stochastic super-efficiency DEA model to estimate the efficiency score of Iranian airlines with both stochastic and zero inputs and outputs. A number of recent studies have addressed performance assessment in terms of one or more aspects of sustainability. Gatimbu et al. (2020) examined small-scale tea processors' environmental efficiency in Kenya using the DEA approach. They considered process waste, level of greenhouse gas (GHG) emissions, and wastewater as the environmental outputs. Pais-Magalhães et al. (2021) measured the waste sector's eco-efficiency in European countries by employing the DEA method. The introduced eco-efficiency indicator was combined with energy and environmental efficiency. An SFA approach was developed by Bibi et al. (2020) to measure the technical and environmental efficiency of the agricultural sector in South Asia. You et al. (2021) introduced an eco-policy efficiency score using the DEA method to evaluate the economic efficiency and social efficiency of Kuwait. They selected primary students' enrollment as the output corresponding with social efficiency and per capita gross domestic product (GDP) as the output related to economic efficiency. The results were used to change Kuwait's policy from sustaining economic growth to sustainable development. Here, we discuss four methods to evaluate four types of efficiency scores. First, the three-step model developed by Korhonen and Luptacik (2004) is presented. Then, the modified threestep model introduced by Mahdiloo et al. (2015) is described. Finally, we introduce our two proposed models. The overall structure of the presented methodology is illustrated in Fig. 1 . The notations are listed in Table 1 . Here, we extend the three-step model to calculate four types of efficiencies. Suppose that the three-step methodology is applied to a system with n DMUs, s desirable technical outputs, t desirable social outputs, and p undesirable outputs. In this method, the desirable outputs are divided by inputs to calculate the efficiency scores. Here, we have two types of desirable outputs, including social outputs and technical outputs. Models (1) and (2) calculate the technical and social efficiency, respectively. The above models generate a value between 0 and 1 for the objective function. Value 1 is the highest efficiency that can be obtained for each DMU. Models (1) and (2) are in nonlinear form, which can be converted into a linear programming model as follows (Charnes et al., 1978) : The simplex method can be applied to solve Models (3) and (4). Models (3) and (4) seek to find the optimal weights of inputs, desirable technical outputs, and desirable social outputs to optimize the technical and social efficiency scores, respectively. In Model (4), the maximum possible value of the objective function is equal to 1, To calculate the technical and social efficiency of n DMUs, Models (3) and (4) should be solved separately for each DMU. Hence, there are n × 2 models that should be solved. Furthermore, to calculate the environmental efficiency, the desirable outputs are divided by undesirable outputs. The achieved values indicate how efficiently the DMUs add worth in return for their environmental implications. Model (5) calculates the environmental efficiency based on technical and social outputs. The above model generates a value between 0 and 1 for the objective function. Value 1 is the highest efficiency that can be obtained for each DMU. Model (5) is in nonlinear form, which can be converted into a linear programming model: We can now combine Models (3), (4), and (6) to calculate sustainable efficiency. In this model, all the inputs, desirable technical, social, and undesirable outputs, are (3), (4), (6), (7) are employed to calculate technical, social, environmental, and sustainable efficiency scores, respectively. Another model was developed by Mahdiloo et al. (2015) , which is less computationally intensive compared to conventional three-step methodology. The proposed model is based on a goal programming approach. In this method, deviation variables are defined for each DMU. The number of deviation variables for each DMU is equal to the number of objective functions. Thus, three deviation variables corresponding to technical, social, and environmental objective functions are defined. Assume ∑ s are three objective functions that should be optimized. The aspiration level for each objective function is equal to 1. d o tec , d o soc , and d o env are deviation variables of DMU o from aspiration level of technical, social, and environmental efficiencies, respectively. The goal is to find the lowest value for deviation variables. Therefore, a linear model based on a goal programming approach is presented. Model (8) calculates the optimal values for weights to maximize technical, social, environmental, and sustainable efficiency scores for each DMU. The following equations calculate efficiency scores of DMUo: Here, we develop two new models that can assess technical, social, environmental, and sustainable efficiency scores of all DMUs with only one run by finding the common weights for inputs and outputs. The developed models are based on a goal programming approach. Therefore, we first discuss the goal programming approach, and then, we will present our proposed models. Charnes et al. (1955) introduced the goal programming for the first time, and then, Ijiri (1965) extended it in more detail. Goal programming is one of the fundamental models in which the decision-maker seeks to achieve several goals simultaneously. Goal programming consists of three main components: goal, aspiration level, and deviation variables. The goal is a general expression that reflects the demands of the decision-maker in the form of mathematical relations. For instance, in the present study, the goal is to maximize efficiency scores. The aspiration level is a specific numerical value with an acceptable or desirable level of success for the given goal. Achieving the aspiration level of the goal is constrained by a set of factors such as facilities and resources. In many cases, the aspiration level of the objective function cannot be achieved in practice, and there is always a difference between the value of the goal and the achievable aspiration level. This difference is measured by a variable called the deviation variable. The general form of the goal programming is as follows: where m is the number of goals, d + j , d − j are the deviation variables, and f j (x) is the objective functions of the problem. The D can be assigned with various combinations of d + j and d − j according to the needs of the decision-maker. D is set to D = d − j to achieve a minimum level of goal. To be sure that the objective function does not exceed the specific level, we set the D to d + j . If we want the objective function to be as close to a certain level of aspiration, then D = d + j + d − j . To maximize the obtained value relative to a certain level of the aspiration, we assign d − j − d + j to the D. Finally, D = d + j − d − j means that the obtained value relative to a certain level of the aspiration should be minimized. In the proposed model, three objective functions need to be optimized under their constraints for DMU o . These objective functions are Teff = ∑ s where Teff is technical efficiency, Seff is social efficiency, and Eff is environmental efficiency. Given that each objective function's maximum optimal value is known in advance and equal to one, the linear goal programming method can be applied simultaneously to optimize these objective functions. In fact, goal programming attempts to achieve a common optimal solution for all objective functions by minimizing the sum of all objective functions' deviation values from their ideal value. Therefore, a deviation variable is defined for each of the objective functions. d j tec , d j soc , and d j env are a deviation of the DMU j from the aspiration level of technical, social, and environmental efficiency, respectively. Here, a mathematical linear programming model is developed that calculates the common weight for inputs, desirable, and undesirable outputs while minimizing all deviation (13) The constraints (1) to (3) of the model (14) are related to technical, social, and environmental efficiencies. Also, in the constraints (4) to (7), the weights are normalized. In Model (14), is a very small positive constant that prevents the weights of variables be equal to zero. Model (14) is a DEA model that is developed based on the MOLP approach. Model (14) runs only once for all DMUs to find non-dominated common weights for inputs and outputs. Then, the efficiency scores are calculated for each DMU by Eqs. (9) to (12). It should be noted that the efficiency scores calculated by Model (14) In goal programming, a deviation variable from the ideal value is defined for each objective function. Hence, the technical objective function can be rewritten as ∑ s (14) is not able to provide optimal weights and instead will produce non-dominated common weights. For more details, readers are referred to Li and Reeves (1999) . We also propose another common weight DEA model to calculate technical, social, environmental, and sustainable efficiency scores of all DMUs with only one run. The present study attempts to calculate sustainable efficiency by combining three types of efficiency: technical, social, and environmental efficiencies. The technical efficiency of DMU j is equal to ∑ s The same inference applies to social and environmental efficiencies. Therefore, in the proposed common weight DEA model, the goal is to maximize the total summation of ∑ s We also showed that the efficiency value is always between zero and one. Thus, The exact inference applies to social and environmental efficiencies. By these definitions, Model (15) is developed. The objective function of Model (15) maximizes the summation of technical, social and environmental efficiency, simultaneously for all DMUs. The first three constraints indicate that these efficiencies cannot be more than one. Here, to obtain final efficiency scores for DMUs, the TOPSIS method is applied. TOP-SIS method is one of the most well-known Multi-Attribute Decision-Making (MADM) techniques introduced by Hwang and Yoon (1981) . TOPSIS is a method used to select the best alternative based on a number of criteria. In this method, m alternatives are analyzed by n criteria. The underlying logic of this method defines the positive ideal point and the negative ideal point. The best alternative is the one that has the shortest distance from the positive ideal point and the farthest distance from the negative ideal point. The mathematical procedure for TOPSIS is as follows: 1. The first step is to form a decision matrix. The decision matrix evaluates i alternatives based on j criteria. In other words, each alternative is scored based on a number of criteria in the decision matrix. 2. The second step is to normalize the DM matrix to eliminate the scale of the data. n ij = a ij ∕ � ∑ m i=1 a 2 ij is used to normalize the data. The normalized DM matrix is shown as: 3. In the third step, the Shannon entropy approach is applied to calculate each criterion's weights. Then, the estimated weights are multiplied by the normalized matrix. The normalized weighted matrix is shown as: 4. In this step, each alternative's relative proximity to the positive and negative ideal point is calculated by the following formulas. Assuming that all criteria are good, v + j is equal to the maximum value of v j , and the v − j is equal to the minimum value of v j . 5. The fifth step is to calculate the ideal solution. In this step, the relative proximity of each alternative to the ideal point is calculated. To do this, we use the following formula: 6. In the final step, the alternatives are ranked based on the calculated CL values. The same decision matrix created in the TOPSIS method is used to calculate weights by the entropy method. The first step is to normalize the DM matrix to eliminate the scale of ij is used to normalize the data. Each element of the normalized matrix is shown with p ij . The next step is to calculate the entropy of each criterion with Next, the value of deviation degree is calculated by d j = 1 − E j . The deviation degree of each criterion indicates how much useful information it provides for the decision-maker. The final step is to determine the weight for each criterion w j = d j ∕ ∑ n j=1 d j . Iran Civil Aviation Organization, known as CAO, was established in 1946. This organization is responsible for planning, setting policies, formulating rules and regulations, and concluding international contracts in the Iran aviation industry. Iran Civil Aviation Organization joined International Civil Aviation Organization (ICAO) in 1949. By providing aviation security, the organization will help increase public satisfaction with the country's aviation and ultimately contribute to developing a sustainable economy. "Iran Airway" is one of the oldest airlines in Iran, which has been known as one of the leading and most active domestic airlines since 1946. "Persian Airway" is another Iranian private airline that was established in 1954. The mission of this airline was to transport cargo from Iran to European countries. Due to various reasons, including Iran's particular geographical position, increasing people's desire to use airplanes as a means of public transportation, and expanding relations with other countries caused the government to establish a national airline. Therefore, in 1962, an international airline called "Iran Air" was established by merging "Iran Airway" and "Persian Airway." Following the advancement of technology and societies' development, the need for airlines has increased, and today 16 airlines are operating in Iran, responsible for passenger and cargo transportation. "Iran Air tour" is an Iranian private airline that was launched in 1972. The main hub of this company is Tehran Mehrabad Airport and Mashhad International Airport. Iran Air tour is one of the companies with a license for aviation training in Iran from the National Aviation Organization. Iran Air tour is a member of the International Air Transport Association (IATA) and holds the IATA Operational Safety Audit (IOSA). Iran Air tour Airlines consists of more than 1300 specialized and skilled personnel. "Aseman Airlines" was established in 1980 by integrating four airlines: Air Taxi, Pars Air, Air Service, and Hoor Aseman. Aseman Airlines currently has one of the broadest flight networks to domestic cities, emphasizing flying to deprived areas of the country. Aseman Airlines has established three flight centers in Tehran, Shiraz, and Mashhad, respectively, to cover its flight network better. Each of the three centers has its own flight crew and independent maintenance hangars, so that daily flights can be sufficiently independent while attracting and training local specialists. Aseman Airlines, like other airlines in the world, pays special attention to the safety of its flights before any other indicator. Therefore, by creating a safety management system called S.M.S, it applies and monitors the latest global safety approaches in its organization to provide a safe and comfortable flight. "Saha Airlines" was established in 1990 following the standards of the CAO and started serving customers. Saha is one of the oldest airlines with the most extended history in the domestic aviation industry. One of the most significant advantages of this airline is the use of the latest technologies in the world in the aviation industry along with young and motivated personnel to satisfy customers. Simultaneous development and alignment of the country's aviation fleet and the growth of technical, operational, and safety capabilities have been essential features of this airline in its goals. Ongoing staff training and the development of Saha Aviation's vast infrastructure are among the steps taken by the relevant authorities to advance the airline's goals. "Kish Airlines" was established in 1991 to develop the activities of Kish Island by building connections with the centers of the provinces of Iran and the countries near the Persian Gulf. The company is currently one of the IATA members companies and the first domestic IOSA-certified company with all the essential features and international standards. "Mahan Air" is another Iranian airline that was established in 1991. A year after, "Katun Airlines" was launched with only five airplanes. The two main objectives of these two airlines are: (1) safe, efficient, and efficient transportation of passengers, cargo, and mail; (2) providing superior quality services by a professional and trained team of pilots, flight attendants, and ground crew in line with international standards. Caspian Airlines was established in 1993 as one of the private airlines in the aviation industry and has started its activity with 4 Yak-42 aircraft. "Qeshm Airlines" started its flight and cargo activities in 1993 by renting one airplane. Since 1995, by concluding a lease agreement with the owners of wide-body aircraft, it has operated its flights on Tehran-Qeshm, Tehran-Dubai, and other cities in Iran. "Zagros," "Taban," "Ata," "Meraj," "Puya," "Sepehran," and "Varesh" airlines are relatively new airlines established during 2005 and 2017. From now on, we indicate each airline by a number: Mahan (1), Aseman (2), Caspian (3), Iran air tour (4), Zagros (5), Ata (6), Iran Air (7), Kish (8), Qeshm (9), Taban (10), Karun (11), Sepehran (12), Varesh (13), Meraj (14), Saha (15), and Puya (16). The current paper selects three inputs and four outputs to calculate Iranian airlines' efficiency, based on previous studies conducted by Heydari et al. (2020) and Cui and Li (2019) . Fleet size v 1 , available seat-kilometer v 2 , and available ton-kilometer v 3 are considered as inputs. The number of employees (w) is considered as social output. Passenger-kilometer performed u 1 and ton-kilometer performed u 2 are considered as technical outputs. The amount of emitted CO 2 ( ) is considered an undesirable output which is the main undesirable output of the transport sector . Fleet size is equal to the number of active airplanes operating in each airline. Ton-kilometer is the fare for transporting one ton of cargo per kilometer. Seat-kilometer is the fare for transporting one passenger per kilometer. The number of employees is equal to the number of people working in the airline. It should be noted that the required data is available only for 2019, which has been obtained from the Iran CAO website (https:// www. cao. ir/). The dataset is displayed in Table 2 . The values of inputs and outputs have diverse measurement units that can adversely affect the analysis results. Therefore, the data must be normalized. To normalize the data, we divide the values of inputs and outputs of each DMU o by the summation of the values of the DMU j . Using the normalized data to calculate the efficiency scores produces the same results as non-normalized data (Wang et al., 2009) . First, we calculate the efficiency values using the three-step method. To do this, we run Models (3), (4), (6), and (7) separately for each DMU. A total of 4 × n = 64 linear programming models are implemented to calculate all technical, social, environmental, and sustainable efficiency scores. To prevent giving zero values to weights, we assume that all weights are greater than or equal to a very small value called epsilon ( = 10 −6 ). The efficiency scores equal to 1 are bolded in all Tables. The ranking of all 16 airlines and their technical, social, environmental, and sustainable efficiency scores are presented in Table 3 . For example, the technical, social, and environmental efficiency scores of DMU 1 are 0.87, 0.56, and 0.49, respectively, while the sustainable efficiency score is equal to 0.92. Airlines 2, 3, 4, 14, and 15 are technically efficient, and airlines 2, 7, and 14 are socially efficient. Airlines 9, 11, 13, and 16 obtained the full environmental efficiency score of 1. According to Table 3 , all airlines are sustainable efficient except airlines 1, 5, 6, 8, and 12. The sustainable efficiency score is calculated by integrating technical, social, and environmental efficiency. Airlines are ranked based on their sustainable efficiency scores. The three-step methodology identified 11 airlines sustainably efficient, which is more than half of the DMUs. Furthermore, airline 10 is recognized as a sustainable, efficient DMU, although it is not technically, socially, and environmentally efficient. Thus, the sustainable efficiency scores obtained from the three-step methodology are not accurate and need to be modified. The following discussion also shows how the three-step method produces inaccurate results by determining unreliable weights for inputs and output. The achieved values for the weights of the inputs and outputs determined by Model (7) are shown in Table 4 . It can be seen that Model (7) gives values almost equal to zero ( 10 −6 ) to some of the weights leading to disregarding the efficiency score corresponding to these weights. Table 5 shows the efficiency type ignored by Model (7) for each DMU. It should be noted that the minimum possible value obtained for weights is equal to 10 −6 imposed by constraints v i ≥ 10 −6 , u r ≥ 10 −6 , w q ≥ 10 −6 , and k ≥ 10 −6 to the model. Without these constraints, the value 10 −6 obtained for the weights will be replaced by zero. As a result, the factors corresponding to these weights are eliminated and completely ignored in the efficiency calculations process. Thus, the three-step approach provides inaccurate results by determining unreliable weights for inputs and output. Mahdiloo et al. (2015) improved the three-step methodology due to its calculation complexity and inaccurate weighting system. The modified three-step methodology is a MOLP DEA model that runs only once for each DMU to determine the non-dominated weights for inputs and outputs. The non-dominated weights are used in Eqs. (9) to (12) to assess the technical, social, environmental, and sustainable efficiency scores. The achieved results for efficiency scores are shown in Table 6 . In the developed goal programming DEA model by Mahdiloo et al. (2015) , the maximum sustainable efficiency score calculated for each DMU can be equal to 0.5. Therefore, the results presented in Table 7 are normalized so that the values calculated for sustainable efficiency scores are between 0 and 1. It can be seen that the technical, social, environmental, and sustainable efficiency scores of each airline obtained from the modified three-step methodology are less than or equal to their scores calculated by the three-step methodology, because the modified three-step methodology finds the non-dominated weights for inputs and outputs instead of optimal weights for each DMU. According to Table 6 , airlines 2, 3, 4, and 14 are technically efficient, airlines 2, 7, and 14 are socially efficient, and airlines 9, 11, and 16 are environmentally efficient. Although airline 16 is only socially efficient, Model (8) recognized it as a sustainable, efficient DMU. Airlines 2 and 14 are technically and socially efficient, while their sustainable efficiency scores are less than 1. Also, airline 14, with its technical and social efficiency of 1 and having the highest average of technical, social, and environmental efficiency, is ranked 5th. Airline 2 has the second-highest average of efficiency and is technically and socially efficient while ranked 6th. It can be concluded that the extended, modified three-step model cannot generate accurate results when calculating four types of efficiency scores. As previously mentioned, implementing the three-step and the modified three-step methodology faces two significant difficulties. For the case of the Iranian airlines, which has 16 DMUs, running 64 and 16 models is required for the three-step and the modified three-step methodology, respectively. Apart from being computationally intensive, these two models cannot produce accurate, sustainable efficiency scores. Therefore, a new common weight goal programming DEA and common weight DEA models are developed to find technical, social, environmental, and sustainable efficiency scores. Our developed model finds the common non-dominated rather than optimal weights for all inputs and outputs in one run. The achieved results for common weight goal programming DEA and common weight DEA models are shown in Tables 7 and 8, respectively. As discussed before, in Model (14), the efficiency scores can reach values higher than 1. Hence, the results shown in Table 7 are normalized scores to bring the calculated efficiency scores to the range between zero and one. The rankings determined by Model (14) are more acceptable than rankings determined by the three-step and the modified threestep methodology. Airline 3 is technically efficient, airline 7 is socially efficient, and airline 11 is environmentally efficient. Also, airline 14 is sustainably efficient and ranked as the top DMU. In these methods, the sustainably efficient airline is not necessarily technically, socially, and environmentally efficient. In this paper, four different methods have been applied to calculate four types of efficiency scores, including technical, social, environmental, and sustainable efficiency scores for 16 airlines. Therefore, airlines are ranked with four different results. The TOPSIS method is used to provide an overall ranking by integrating the results obtained from the four described methods. For this purpose, the implementation of four TOPSIS models is required. Each TOPSIS model integrates one type of efficiency score. The entropy weight method is used to determine the weights. The results are shown in Tables 9 and 10. It can be seen that the first rank of each efficiency is assigned to a different airline. Airlines 3, 7, 11, and 13 rank first in technical, social, environmental, and sustainable efficiency. In the proposed TOPSIS approach, the airlines are ranked based on four aspects: technical, social, environmental, and sustainable development. The diverse rankings help managers to investigate DMUs based on different criteria. For example, if the country's macro-strategy is to pursue environmental sustainability, airline 11 can be a good role model for other airlines. According to the obtained results, the key findings of the present study can be expressed as follows: • The three-step DEA model needs complex calculations to evaluate efficiency scores. • The three-step DEA model is not able to produce accurate results due to the errors in determining weights. • The modified three-step DEA model cannot generate accurate results when calculating four types of efficiency scores. • The common weight goal programming DEA and common weight DEA models presented in this paper are able to provide more accurate results when calculating four types of efficiency scores by determining the common non-dominated rather than optimal weights for all inputs and outputs in one run. Here, we present a pairwise comparison between achieved efficiency scores for better analysis. However, providing this type of analysis for all the approaches is too detailed to be pursued here; thus, the pairwise comparison analysis is only presented for the integrated efficiency scores obtained from TOPSIS. We assume that efficiencies above 0.5 are satisfactory and DMUs with efficiencies below this value need to be enhanced. It should be noted that choosing the minimum acceptable value depends on management decisions, and in this study, we chose the value of 0.5 at will. Airlines having satisfactory efficiency scores in both examined aspects are considered the "star" DMUs (Boussofiane et al., 1991) . The airlines determined as the star DMUs are benchmarks for other DMUs. Inefficient DMUs can emulate the star airlines to reach an ideal efficiency. Airlines with an efficiency score lower than 0.5 in both examined aspects are known as weak DMUs. Weak DMUs need special attention to determine the cause of their inefficiency and to enhance their efficiency. The calculated sustainable efficiency and technical efficiency scores are illustrated in Fig. 2 . Six DMUs are specified as star DMU, which means they are both technically and sustainably efficient. DMU 3 , DMU 6 , DMU 9 , and DMU 10 are socially inefficient, yet they are considered as sustainable, efficient DMUs. The reason behind this is illustrated in Fig. 6 . As we can see in Fig. 6 , DMU 3 , DMU 6 , DMU 9 , and DMU 10 are characterized as star DMUs in terms of technically and environmentally. Hence, the social aspect of sustainability has been dominated, and these DMUs are specified as star DMUs. Sustainable efficiency scores versus social efficiency scores are presented in Fig. 3 . Airline 3 obtained the highest social efficiency and sustainable efficiency simultaneously, which can be considered the social benchmark for other DMUs. Although DMU 1 is technically efficient, according to Fig. 7 , this DMU lacks social and environmental efficiency; therefore, it is specified as a weak DMU. Figure 4 presents sustainable efficiency versus environmental efficiency. It can be seen that more than half of the DMUs are environmentally sustainable. DMU 4 is the only DMU with desirable sustainable efficiency, which is not environmentally efficient. As shown in Fig. 5 , DMU 4 is identified as a star DMU in terms of social and technical efficiency; thus, its environmental sustainability aspect is dominated and ignored. Providing such pairwise comparison analysis can help managers to recognize where improvement should be investigated with priority. It also can prevent misunderstanding the results. For instance, based on Figs. 2, 3 and 4, airline 7 has been introduced as a sustainable DMU, but it should not be assumed that this One of the factors affecting the development of countries is the quality of the air transportation industry. The air transportation system accelerates a country's economic processes by linking various sectors of the economy. Besides, as a critical component of the air transportation network, airlines significantly influence any country's economic, social, and environmental issues. Therefore, evaluating airlines' efficiency given sustainable development aspects can be a vital issue for policy-makers. DEA is one of the most well-known approaches applied to assess the efficiency of DMUs. However, the literature lacks studies This study extended the existing DEA models, including the three-step methodology introduced by Korhonen and Luptacik (2004) and the modified three-step method proposed The weight for ith input u r The weight for rth desirable technical output w q The weight for qth desirable social output k The weight for kth undesirable output by Mahdiloo et al. (2015) , to be applicable to calculate four types of efficiency. We also proposed two models called "common weight goal programming DEA" and "common weight DEA" to calculate efficiency scores. The extended three-step model and the modified three-step model calculate optimal weights and non-dominated weights for each DMU, respectively. In contrast, the common weight goal programming DEA model and the common weight DEA model determine non-dominated common weights for all inputs and outputs. This paper showed that the existing DEA models, including the three-step methodology and modified three-step methodology, require complex calculations since for a case with n DMUs, the three-step model needs to run 4 × n models and modified three-step method needs to solve n linear programming models. We showed that the three-step model is not able to provide accurate, sustainable efficiency scores for some DMUs due to its unreliable weighting system. The three-step methodology specified DMUs as sustainable efficient if they are technically, socially, or environmentally efficient. This method also identified airline 10 as a sustainable, efficient DMU, while its all three efficiency scores were less than 1. We also proved that the modified three-step model was not able to provide accurate results for some DMUs. The current paper introduced two approaches to deal with undesirable outputs in the DEA model and used them to evaluate the technical, social, environmental, and sustainable efficiency scores of 16 airlines in Iran based on their attention toward sustainable development principles. Our proposed models integrate four different technical, social, environmental, and sustainable efficiencies into a single model and only run once to calculate the common weights instead of calculating optimal weights for inputs, desirable and undesirable outputs. In the current study, airlines are ranked based on four types of efficiency scores obtained from four different models. Hence, each DMU has four technical efficiency scores, four social efficiency scores, four environmental efficiency scores, and four sustainable efficiency scores. We applied the TOPSIS approach to integrate the four values of each efficiency score for each DMU. The developed models in this study extend the DEA approach's capability to assess airlines' efficiency based on sustainable development criteria. This study is the first effort to calculate Iranian airlines' efficiency considering all three aspects of sustainable development: technical, social, and environmental. Also, our proposed models are the first attempt to calculate common weights for all inputs, desirable outputs, and undesirable outputs by running only one model. Various models have been proposed in the literature to evaluate the performance of airlines. However, due to some limitations, a few of these models have considered social and environmental criteria. One of the main limitations of this study is that we do not have access to other undesirable outputs' data, such as the amount of CO 2 caused by other airlines' activity other than aircraft's emissions due to data unavailability. On the other hand, we could not personally inspect airlines' activities to determine more than one social factor due to security reasons. We offer the following topics for future researchers to consider in their studies. In classical DEA models, the input and output data are specified with precise and definite values. In contrast, in real-world problems, the data are usually fluctuated, uncertain, or dependent on different scenarios and cannot be collected in a precise way. Therefore, one of the most challenging issues related to DEA studies is the uncertainty correlated with the data. Thus, developing approaches to handle data uncertainty in DEA models can be an exciting topic for researchers. Based on the findings of this study, future researchers can also implement similar models employing other techniques of contributing undesirable outputs and undesirable inputs. Furthermore, the developed model in this study can be implemented in other similar cases. In other words, the proposed approach in this study is applicable to other sectors and industries where technical, social, environmental, and sustainable efficiency is required. A hybrid framework for evaluating corporate sustainability using multicriteria decision making. Environment, Development and Sustainability Review of efficiency ranking methods in data envelopment analysis An MCDM-DEA approach for technology selection Development of data envelopment analysis for the performance evaluation of green supply chain with undesirable outputs A novel hybrid fuzzy DEA-Fuzzy MADM method for airlines safety evaluation Technical and environmental efficiency of agriculture sector in South Asia: A stochastic frontier analysis approach Applied data envelopment analysis Evaluating economic and environmental efficiency of global airlines: A SBM-DEA approach Optimal estimation of executive compensation by linear programming Measuring the efficiency of decision making units Assessing eco-performance of transport sector: Approach framework, static efficiency and dynamic evolution Measuring super-efficiency in DEA in the presence of infeasibility Exploring the operational and environmental performance of Chinese airlines: A two-stage undesirable SBM-NDEA approach Airline energy efficiency measures considering carbon abatement: A new strategic framework Will airline efficiency be affected by "Carbon Neutral Growth from 2020" strategy? Evidences from 29 international airlines Investigating the impacts of the EU ETS emission rights on airline environmental efficiency via a Network Environmental SBM model Multilateral productivity comparisons when some outputs are undesirable: a nonparametric approach Environmental efficiency of small-scale tea processors in Kenya: An inverse data envelopment analysis (DEA) approach. Environment Sustainability of Chinese airlines: A modified slack-based measure model for CO2 emissions A fully fuzzy network DEA-Range Adjusted Measure model for evaluating airlines efficiency: A case of Iran Methods for multiple attribute decision making. Multiple attribute decision making A forecasting method in data envelopment analysis with group decision making Analyzing the environmental efficiency of global airlines by continent for sustainability An analysis of production as an efficient combination of activities. Activity analysis of production and allocation Eco-efficiency analysis of power plants: An extension of data envelopment analysis A comparative efficiency and productivity analysis: Implication to airlines located in Central and South-East Europe Managing reverse logistics to enhance sustainability of industrial marketing A multiple criteria approach to data envelopment analysis Eco-efficiency and effectiveness evaluation toward sustainable urban development in China: a super-efficiency SBM-DEA with undesirable outputs Technical, environmental and eco-efficiency measurement for supplier selection: An extension and application of data envelopment analysis Dynamic DEA models with network structure: An application for Iranian airlines Scoring method of eco-efficiency using the DEA approach: Evidence from European waste sectors Environmental efficiency assessment of U.S. transport sector: A slack-based data envelopment analysis approach Efficiency evaluation of Brazilian airlines operations considering the Covid-19 outbreak Quality and Efficiency Evaluation of Airlines Services Impact of operating cost components on airline efficiency in India: A DEA approach Evaluating efficiency of airlines: A new robust DEA approach with undesirable output Dynamic network data envelopment analysis model with fuzzy inputs and outputs: An application for Iranian Airlines Developing a new super-efficiency DEA model in the presence of both zero data and stochastic data: a case study in the Iranian airline industry Efficiency and effectiveness in airline performance using a SBM-NDEA model in the presence of shared input Performance evaluation of major asian airline companies using DEA window model and grey theory Ranking decision making units by imposing a minimum weight restriction in the data envelopment analysis Evaluation of carbon emission efficiency in China's airlines Evaluating the environmental efficiency of the US airline industry using a directional distance function DEA approach Incorporating both undesirable outputs and uncontrollable variables into DEA: The performance of Chinese coal-fired power plants Effects of eco-policy on Kuwait based upon data envelope analysis. Environment, Development and Sustainability Developing an innovation capital index of global airlines using a hierarchical data envelopment analysis approach A comparative study of airline efficiency in China and India: A dynamic network DEA approach Eco-efficiency analysis of industrial system in China: A data envelopment analysis approach Using operational and stock analytics to measure airline performance: A network DEA approach Airlines performance via two-stage network DEA approach Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations The authors would like to thank Professor Luc Hens, the Editor-in-Chief of Environment, Development and Sustainability journal and two anonymous reviewers for their insightful comments and suggestions. As a result, this paper has been improved substantially.