key: cord-0781315-twjaq5do authors: Diwekar, U.; Amekudzi-Kennedy, A.; Bakshi, B.; Baumgartner, R.; Boumans, R.; Burger, P.; Cabezas, H.; Egler, M.; Farley, J.; Fath, B.; Gleason, T.; Huang, Y.; Karunanithi, A.; Khanna, V.; Mangan, A.; Mayer, A.L.; Mukherjee, R.; Mullally, G.; Rico-Ramirez, V.; Shonnard, D.; Svanström, M.; Theis, T. title: A perspective on the role of uncertainty in sustainability science and engineering date: 2020-09-09 journal: Resour Conserv Recycl DOI: 10.1016/j.resconrec.2020.105140 sha: dada186e3575e90765d2ac97e30ad109dfffc96d doc_id: 781315 cord_uid: twjaq5do The Trans-Atlantic Research and Development Interchange on Sustainability Workshop (TARDIS) is a meeting on scientific topics related to sustainability. The 2019 workshop theme was "On the Role of Uncertainty in Managing the Earth for Global Sustainability." This paper presents the perspectives on this topic derived from talks and discussions at the 2019 TARDIS workshop. There are four kinds of uncertainties encountered in sustainability ranging from clear enough futures to true surprises. The current state-of-the-art in assessing and mitigating these uncertainties is discussed. The demands of humans on the earth have grown considerably with the human population and the increase in per capita consumption of all kinds (fuels and energy, food, water, minerals, etc.). Consider that while the earth and its energy budget from the Sun have remained roughly constant over the past 4000 years, the human population has risen from approximately 7 million around 4000 BCE to more than 7 billion (Worldmeters, 2020) at present. The value of the global economic activity, as measured by the World Gross Product, a proxy for consumption, had risen from approximately US$1 billion around 4000 BCE to about US$78 trillion (Wikipedia 2020; Worldbank 2015) by 2014, both in 1990 International US Dollars. About 94% of the population increase and almost 100% of the rise in the World Gross Product have happened since 1500 BCE. These conditions create an unprecedented planet-wide situation where humanity has the imperative of managing the earth within very tight constraints with little margin for error. This requires that humans be able to: (1) accurately assess the current condition of the earth, (2) forecast future trends as accurately as possible, and (3) develop effective management strategies. Uncertainty poses significant difficulties for all three efforts. The concept of sustainability is closely related to that of sustainable development, but the two are different. Sustainability is about ensuring that the earth can meet the material and energy needs to support complex systems including humanity over the long term. This has economic dimensions because the rate of resource consumption heavily depends on economics, and it has social dimensions because the ultimate goal is to support human society. Sustainable development provides a vision of how human society could develop sustainably, and it adds a stronger focus on social dimensions such as intergenerational equity. Sustainability is, therefore, a basic concept oriented to making the survival of civilized human existence on earth feasible. The focus of TARDIS is mostly on sustainability. However, we did not attempt to create our own formal definition of sustainability. We merely used the aforementioned framework as a working definition of sustainability to focus on discussions. The Trans-Atlantic Research and Development Interchange on Sustainability (TARDIS) Workshop is a three-day study session on scientific topics related to sustainability, stimulated by presentations, and discussion. The workshop involves participants from the United States, Europe, and other parts of the world in the multidisciplinary field of sustainability. The 2019 TARDIS group included the participation of experts from engineering, mathematical sciences, physics, ecology, economics, decision sciences, and political science fields. The theme of the 2019 workshop was "On the Role of Uncertainty in Managing the Earth for Global Sustainability." This paper presents the summary perspectives on this topic derived from talks and discussions at the 2019 TARDIS. Sustainability and Uncertainty: The problems involved in understanding, modeling, assessing, and managing for sustainability, fall under what Rittel and Weber (1973) call "wicked problems". For example, a recent study by the World Bank (Damania et al., 2019; Steffen et al., 2015) that focuses on threats to global water quality concludes that the uncontrolled release of reactive nitrogen (Nr) into rivers, lakes, and estuaries presents us with the "largest global externality" that threatens human health and ecosystem stability at a scale that is not fully recognized by the governments of the world. As often happens with such "wicked" problems, solutions are confounded by interconnected positive and negative impacts that are difficult or impossible to unravel. For instance, reactive nitrogen in the environment is intertwined with several of the UN Sustainable Development Goals: Zero Hunger (2), Good health and Well-Being (3), Clean Water and Sanitation (6), Affordable and Clean Energy (7), Industry, Innovation, and Infrastructure (9), Sustainable Cities and Communities (11), Responsible Consumption and Production (12), Climate Action (13), Life Below Water (14), and Life on Land (15). Actions taken to address the role of Nr in any one of these goals may exacerbate Nrrelated impacts in the others. Multiple stakeholders and their conflicting perspectives also generate wicked sustainability-related problems. One of the major challenges in the sustainability system, which results in wicked problems is the problem of uncertainties. Some examples of uncertainties in sustainability include definition and quantification of various objectives, impact assessment methods and models, forecasting future, and unexpected events. In a broad sense, uncertainty may be defined simply as limited knowledge about future, past, or current events (Walker et al., 2003) . Fig. 1 shows four different quadrants of information leading to uncertainty. This diagram is the focal point in business strategy and planning literature (Marshall et al., 2019) . Sustainability systems and issues extend to all four quadrants of information, whether the problem is of community sustainability, manufacturing sustainability, sustainable economics, regional or global sustainability, as described in the various sections below. The paper is divided into the following sections. Section 2 presents types of uncertainties followed by a section on systemic risks, precautionary principle, and normative risks. Section 4 talks about the effect of uncertainties on the environmental policy, followed by a section related to sustainable economics and uncertainty. Section 6 is devoted to community sustainability and uncertainty. Sustainable manufacturing and sustainability assessment sections follow that. Decision making under deep uncertainty is Section 9, followed by Global sustainability and stochastic processes. Section 11 is devoted to the communication of uncertainties in sustainability education. The summary section is last. The literature related to uncertainty defines two kinds of uncertainties, namely epistemic uncertainty and aleatory uncertainty. Epistemic uncertainty is a lack of knowledge on underlying fundamentals and characterized by alternative models. Aleatory uncertainty refers to the inherent uncertainty due to the probabilistic variability. These uncertainties are characterized by a probability distribution. Sustainability problems are difficult because they involve deep uncertainties. A recent open book on decision making under deep uncertainty divides uncertainties into four types. • Level 1 uncertainty represents situations in which one admits that one is not absolutely certain, but one does not see the need for or is not able to measure the degree of uncertainty in any explicit way (Hillier and Lieberman 2001) . These are generally short-term uncertainties (clear enough futures) where historical data can be used to predict the future. Sensitivity analysis can be used to address these types of uncertainties. • Level 2 uncertainty can be described in terms of probability distributions, probability bounds, interval methods, or fuzzy information. These uncertainties are useful where system models are reasonably closer to the real world. Most of the manufacturing uncertainties fall into these categories. • Level 1 and 2 uncertainties are observed for known unknowns. According to Knight (1921) , Levels 1 and 2 uncertainties come under the category of "risks" and are often associated with optimality, unlike Levels 3 & 4, which involve "robustness." • Level 3 uncertainties involve situations in which there are a limited set of plausible futures, system models, outcomes, or weights, and Types of Information: known-knowns are known to exist, and information is available, known-unknowns are known to exist, but no information is available, unknown-knowns are not included in the analysis but the information is possible if they did, and unknown-unknowns are not known to exist, and no information is available if they did (Marshall et al., 2019) . probabilities cannot be assigned to them-so the tools of neither Level 1 nor Level 2 are appropriate. In these cases, traditional scenario analysis is usually used. Long-term assessment of sustainability (also comes under level 4 uncertainty) comes under this category. • Level 4 uncertainty represents the deepest level of recognized uncertainty. This is called deep uncertainty. There are two types of deep uncertainties: the uncertainty where there are many possible futures and outcomes, and there are unknown futures and unknown outcomes. Deep uncertainties are related to decreasing confidence in our ability to anticipate correctly future technological, economic, and social developments, future changes in the system we are trying to improve, or the multiplicity and time-varying preferences of stakeholders regarding the system's outcomes, or handling low probability but high impact events (black swans (Taleb 2010) ) like a natural disaster, a pandemic, a financial crisis, a terrorist attack, or truly novel events for which there is no historical experience. (unknown knowns and unknown unknowns, true surprises). Systemic risk, ontological uncertainty, and time-dependent uncertainties come under this category of level 4 uncertainty. Dealing with these kinds of uncertainties involve monitoring the present and adaptations over time. The precautionary principle and options theory are two widely used methods to deal with these uncertainties. Uncertainty, as well as the precautionary principle, take a prominent place within the scientific debate on sustainable development in general and on managing systemic risks specifically (Renn 2008; Becker 2014) . Against the backdrop of the many meanings of systemic risks within the scientific discourse, we argue that the precautionary principle is only a negative or threshold criterion and that it is worthwhile to add positive normative criteria to inform management of systemic risks. However, we introduce the concept of normative uncertainty adding up to epistemic uncertainty in dealing with systemic risk. Systemic risks are a relatively new type of risk concept (Renn, 2016) . According to predominant understanding, the occurrence of an event would not simply lead to harm but to a breakdown of the system (Kaufman and Scott 2003; Smaga, 2014) . In other words, the system would go beyond its resilience level and would no longer be able to execute its functions. With some probability the "event" climate change can, for example, lead to a breakdown of the earth's life-supporting functions; or a global financial crisis can be accompanied by major banks going bankrupt which in turn would severely undermine their functions supporting the economy potentially leading to a breakdown of the economy. Such risks display new characteristics in comparison to traditional risks such as droughts, hailstorms, a car traveling, or smoking. Whereas the latter are computable, systemic risks are not. It is neither possible to estimate the probability of occurrence of the triggering events, nor is it possible to assess the related harm (and to bring that into a cost-benefit analysis). These come under level 3 and level 4 types of uncertainties. Risk assessments are always accompanied by epistemic uncertainties. The results of measuring frequencies of events or modeling the complexity of systems together with assessing potential harm are normally given within ranges expressing possible limitations. However, uncertainties related to systemic risks do not express a given range of likelihood. They represent the mode "we do not know," due to its character as not being computable. We can, for example, not give a reasonable likelihood estimation for whether the financial crisis in 2008 and the thread of big banks potentially collapsing would have led to a breakdown of the economic system. The issues are just too complex to come up with estimations. The possibility that it could and would bring about a huge amount of harm by a breakdown of the economy was seen as sufficiently reasonable to take action. The motivating principle, according to actions, is known as the precautionary principle (Gardiner 2006) . It does not claim that you should not take risks. It rather says (in case of the financial crisis in 2008): We do not know whether the collapse of system-relevant banks will lead to a breakdown of the economic system. However, the level of risk resulting from such a breakdown is sufficient to intervene and not to let those banks go bankrupt. The precautionary principle is an action-guiding principle where the occurrence of events could have the effect that a threshold (or resilience) level of a major system supporting societal functions will trespass. However, the precautionary principle only indicates not to take a specific path. In contrast, it cannot inform positively decision making for "Doing X," e.g., deciding that saving exposed banks whose activities brought about the crisis is the best decision to take. Additional criteria are needed in order to make a positive decision. According to relevant theories in political economy and philosophy (Renn 2009; Stirling 2007; Kaul and Mendoza 2003) , public decisions should (ideally) primarily be motivated by taking public or common goods as well as the overall goal of human well-being into account. Hence, the negatively oriented precautionary principle should be accompanied by a positive criterion on public goods and human wellbeing. Together they build a sound basis for decision making regarding systemic risks, nowadays prominently expressed by the Agenda 2030 and its 17 Sustainable Development Goals. When tackling systemic risks from a perspective of societal goals, alternative options to deal with, as well as different outcomes of the crisis, come into the picture. Specifically, it substantiates and rationalizes so-called 'pro-active intervention strategies' to systemic risks, such as adaptation and transformation (IRGC 2018), scenario discovery approach (Bryant and Lampert, 2010) , etc. However, although such a positive approach offers a productive evaluative basis for assessing first what systemic risks are and what goods have to be taken into account, in strategies for dealing with (avoiding) systemic risks, this opens up a broad space of interpretation. This happens because there is not one single understanding of what constitutes public goods or belongs to human well-being. This points to another field of uncertainties, i.e., normative uncertainty (e.g., Mazouz 2003) . It is by far not the case that anything goes when talking about justice or sustainable development. Philosophy, social sciences, and economics provide a number of well-established approaches to frame justice (cf. Miller 2017). We have a variety of options producing a range of normative claims and assessments (cf. Burger 2018 for types of sustainability conceptions). As we have to learn to deal with epistemic uncertainties, the sciences additionally need to improve their understanding of normative uncertainties when dealing with systemic risks like climate change. Lastly, in this overview of uncertainty types, we consider the role of ontological uncertainty (Nielsen and Ulanowicz 2011) , which refers to the emergence of new unpredictable or un-"pre-statable" types (Kauffman 2019). Any system that exhibits requisite complexity will have novel behaviors from the interactions of the agents within the system. For example, simple combinatorics tells us that the interactions of 80 distinguishable components will offer approximately 80! =~7 × 10, 118 possible configurations. Yet, given the age of the universe, estimates are that there have only been~10, 110 quantum level events (Elsasser and Staude 1978) . In other words, there will be surprise events and other events that will never materialize. This level of uncertainty, which challenges the old school determinism, has manifest itself in many 20th century theories as given in Table 1 (examples are taken from Nielsen et al., 2020, p. 30) . The realization of ontic uncertainty is not inherently negatively consequential, but rather has benefits as it implies ontic openness that potentially brings new opportunities to any complex adaptive system, including humans striving for sustainable development. Of course, this also reinforces the importance of applying the precautionary principle as prudent guidance into an unknown future. As in most areas of policy, environmental and sustainability policy must often contend with scientific uncertainty when prescribing actions and identifying targets (Bradshaw and Borchers 2000) . Depending on the problem, scientific uncertainties in policymaking can be of any one of the four types described earlier. Policymakers often grant implementing agencies considerable discretion to modify their actions in response to new information or understandings, but there are limits to how much discretion can be accommodated before policies must be rewritten (Woods and Morey 2008; Lind-Riehl et al., 2016) . At some point, when the language and concepts used in laws, regulations, and policies no longer relate to the state of knowledge in the field, these prescriptions must be amended. The US Endangered Species Act or ESA is one such example of science moving beyond the legal terminology. As implied by its title, the ESA is a law that protects species from extinction. Enacted in 1973, the ESA is one of the most stringent environmental laws in the world, saving thousands of species from extinction and enabling the recovery of dozens of well-known and unique species (Suckling et al., 2016) . When faced with a species in decline, the two enforcement agencies, the US Fish and Wildlife Service and the National Marine Fisheries Service, must first determine whether the species meets the ESA's criteria for being listable. One of these criteria includes the distinctiveness of that species, subspecies, or population from other more common species. Advancements in genetic technology and taxonomic science have complicated the boundaries between species, making it more difficult to use the ESA to protect biodiversity from threats faced by land-use change, pollution, invasive species, and climate change (Ritchie et al., 2018 ). An increasing number of lawsuits and petitions call on wildlife agencies to delist or downlist species from the ESA, based on new information that calls the species' distinctiveness into question. One such example is the coastal California gnatcatcher (Polioptila californica), a small, nonmigratory songbird that inhabits the rapidly urbanizing region of Southern California and northern Baja California Peninsula in Mexico (Mayer, forthcoming) . The protection of this northern-most subspecies of California gnatcatcher has been constantly challenged by development industries in California, mainly over the scientific uncertainty of its distinctiveness from other subspecies. Although ecological and morphological evidence continues to indicate that the coastal California gnatcatcher is substantially different from California gnatcatchers further south in Mexico, genetic evidence has been more equivocal and contentious (McCormack and Maley 2015; Patten 2015; Zink et al., 2013 Zink et al., , 2016 . Difficulties in determining the clarity of boundaries between species and subspecies are not unique to the California gnatcatcher, but rather are inherent in the concept of a species (Zachos 2018) . Species are continuously evolving over time and can and do hybridize with closely related species as they evolve, blurring their genetic and morphological distinctiveness. The difficulty of species delineation, and ongoing disagreements over concepts and methods among scientists, translates into difficulty for policy decision-making, even with considerable discretion for implementing agencies. Insight into the roles of ambiguity, uncertainty, and discretion in the interaction between the scientific process and policy process can improve policy implementation and outcomes (Doremus 1997; Bradshaw and Borchers 2000; Cairney et al., 2016; Wilhere 2017) . Perhaps the first work applying financial decision-making models to environmental issues relevant to sustainability was by Weisbrod (1964) . Weisbrod considered the case of the Sequoia National Park, for which demand is infrequent and uncertain (i.e., many people consider going, but few actually go). He imagines a private owner who is able to price discriminate in charging entrance fees and thus capture the entire consumer surplus for himself. Even under these conditions, however, entrance fees do not cover the full costs of running the park. Ignoring all non-recreation benefits of the ecosystem in question, a rational owner might choose to irreversibly convert the area to other more profitable uses (e.g., chop down the forests and build condominiums). However, when demand is infrequent and uncertain, there is a certain value people give to retaining the option to visit the park even if they never do so. This value is a pure public good: as long as the park is there, the option itself is non-excludable and non-rival, and there are no non-coercive mechanisms for charging consumers for this option value. If enough people benefit from this option value, it would be socially efficient to keep the park open even if privately inefficient. The solution may be public ownership or subsidies. Cicchetti and Freeman (1971) have shown that Weisbrod's option value will be positive when there is uncertainty in either demand or supply, and it has the same effect on decisions as risk aversion. Arrow and Fischer (1974) examined a slightly different situation, in which there is an opportunity to irreversibly convert an ecosystem to some other use, such as damming a river or converting a forest to housing developments. They look at the situation in which decisions this period change the expected values of conversion and/or non-conversion in the next period. For example, society might learn about a valuable service provided by the ecosystem, or a new medicinal plant, if it is left unconverted. They concluded that these unknown values created a quasi-option value that again increases the value of avoiding irreversible outcomes, acting the same as risk aversion. Conrad (1980) later showed that the quasi-option value is equivalent to the expected value of information, while the option value is equivalent to the expected value of perfect information. However, in financial decision making, and in the models described above, unknown outcomes are of a very restricted variety: possible outcomes are known, as are the probabilities of each. Knight (1921) referred to this type of unknown outcome as a risk. In the highly complex ecological-economic systems that humans want to sustain, it is often impossible to make objective assessments of the probabilities of different outcomes, and therefore impossible to model expected values. In many cases, we cannot even guess at possible outcomes. For example, human activities change ecosystems in ways that generate new evolutionary pressures on the species within them. Humans react to their self-created changes through cultural evolution, including the development of new technologies. Both evolution and technological advances can lead to entirely unpredictable outcomes, in which not even possible outcomes can be objectively assessed. This type of unknown outcome has been referred to as surprise (Schneider et al., 1998; Casti, 1994; Faber et al., 1992; Kates, 1985) novelty (Faber and Proops,1998) and ignorance (Faber et al., 1992; Daly and Farley, 2003) . How to deal with ignorance and uncertainty in modeling sustainability is a serious challenge. This is within the area of deep uncertainty. In reality, due to the aforementioned uncertainties, we suffer Table 1 Examples of uncertainty and indeterminacy in the modern understanding of nature (Nielsen et al., 2020) . Creative destruction as part of the cyclic process Jørgensen Heisenberg extended to ecosystems Kauffman Continuous evolution into the possible adjacent states U. Diwekar, et al. Resources, Conservation & Recycling 164 (2021) 105140 profound ignorance about the most serious problems we currently face in our rapidly evolving system. It is a characteristic of complex systems that if we ramp up the value of a critical parameter beyond some critical point, the system can "flip" into an alternate regime about which we know very little and from which it can be difficult or impossible to return to the previous state (Hughes et al., 2013; Lenton et al., 2008; Lenton and Williams, 2013; Pearce, 2007) . We are currently ramping up the value of innumerable critical parameters, both ecological and economic, simultaneously. Climate change, biodiversity loss, nutrient flows, and land-use change are a few of the most egregious ecological examples (Rockstrom et al., 2009; Steffen et al., 2015) . Financial systems regularly experience explosive growth followed by crisis. Stocks are now held on average for some 30 s, down from 8 years in the 1960s (Hudson 2011) . Over half the value of stocks in the US are owned by the richest 1% of the population (Wigglesworth 2020). The S&P 500 increased by 260% from 2009 to 2019, vs. a 35% increase in GDP, and by more than 12 times as much as GDP in 2019, contributing to an explosive increase in inequality. Since over the counter foreign exchange trading first appeared in the early 1980s, transactions have soared to $6.6 trillion per day nearly 30 times greater than global GDP, with purchases also held for only seconds on average (Hudson 2011) . We are often uncertain precisely which parameters are critical, can only guess when they will reach critical thresholds, and remain profoundly ignorant about new regimes into which our system may flip. As has been eloquently (Taleb 2010) pointed out, the most consequential events in history have been Black Swans: extremely rare events with very high impacts that could not have been predicted ahead of time. The previous experience provides little to no guidance in predicting novel events. Another major source of uncertainty is the reflexive nature of social systems and social science (Soros, 2013) . Societies develop beliefs and norms that affect their members' behavior. For example, social scientists develop theories to explain how people behave and how social systems function, but it is widely believed, the theories themselves can affect human behavior. Theories can be self-fulfilling or self-negating. For example, even prior to the COVID-19 crisis, many pundits were predicting a recession. If producers believe predictions, they will stop investing in new productive capacity, reducing the creation of new jobs if consumers believe that as well, they will start saving their money, reducing aggregate demand, leading producers to lay off workers, who respond by saving even more money in a positive feedback loop. These are the actions that can trigger a recession. In contrast, before the financial crisis that exploded in 2007, Ben Bernanke and other influential economists claimed we had achieved a 'great moderation' and no longer had to fear financial crisis (Bernanke 2004; Stock and Watson 2002) . This very likely stimulated excessive risk-taking, and that precipitated the crisis, as predicted by Minsky's Financial Instability Hypothesis (Minsky 1977) . Several community environmental quality problems fall under wicked problems because depending on the stakes one has, the problem formulation -different characterizations of the problem that emerge when viewed from the perspectives of different stakeholders -can be very different: industry stakeholders whose primary objective is usually economic optimization within environmental regulation standards; communities that seek to balance socioeconomic welfare through access to jobs while reducing associated environmental contaminants -if they are dependent on jobs from the contaminating industries; more affluent communities that would rather have polluting industries removed or distanced from their communities to optimize environmental quality and public health, and government agencies whose primary objective is to administer regulations at the nexus of these competing demands. The triple bottom line does come into play here, but the relative importance of the different types of capital pertaining to the problem situation can be very different -sometimes reflected in long drawn out battles over community environmental contamination issues (Brackett, 2019) . These multiple objectives bring in uncertainties as some objectives are well defined and can be quantified reasonably as others cannot be quantified accurately. Use of Multi-objective Programming (MOP) under uncertainty (Fu et al., 2001; Diwekar, 2012) or many objective robust decision making (MORDM) (Kasprzyk et al., 2013; Singh et al., 2015) methods are useful in this context. The methods are suitable for level 2 as well as level 3 & 4 types of uncertainties. Environmental contamination in local communities is a pervasive problem that affects sustainable development and public health at the local level, judging from the news and social media reporting. The popular media has intermittently reported on environmental contamination in various communities around the country where residents have been unaware of such contamination and its impact on their health for extended periods of time. Examples include ethylene oxide emissions in the states of Georgia and Illinois, and the Flint Water Crisis in Michigan (Goodman and Miller, 2019; Lutz, 2019; 11 Alive News, 2019; ABC-7 Chicago, 2019; Scalia, 2019; Bowles, 2018) . Communities that have been unknowingly exposed to environmental contaminants for relatively extended periods tend to develop a distrust of the public officials responsible for environmental quality (Goodman and Miller, 2019; Lutz, 2019; 11 Alive News, 2019; ABC-7 Chicago, 2019) . Notable uncertainty exists concerning community exposure to environmental contaminants in the air, water, and soil, and there are several synthetic compounds in use today in various consumer products that have not been reviewed for safety by any government agency (ASCE, 2006) . Therefore, uncertainties abound with local environmental contamination, both known unknowns and unknown unknowns. The emergence of lower-cost environmental quality sensors offers a growing opportunity for communities to engage in crowd science, partnering with universities and government officials, to develop a better understanding of their local environmental risks and begin to monitor and contribute to managing the most critical of them. How can emerging sensor technologies be adapted to monitor various critical contaminants in a community's risk profile in a cost-effective manner, and what is the feasibility and cost of large-scale deployments? The news media report that thousands of people around the country are beginning to measure air quality as climate change reports become increasingly dire, wildfires tear across the American West, and trust in the federal government's air quality oversight fades (UN, 2015) . The PurpleAir Map (2020), for example, is available to the public via the Internet, and air quality monitoring network built on a new generation of "Internet of Things" sensors being used by individuals and communities. Developed on the concern that local air contamination was higher than indicated, PurpleAir monitors are now being used by several individuals and communities around the world with air particulate data being fed in real-time over Wi-Fi to a public visual display online. Coupling standard public domain reporting with organized crowd science efforts to monitor critical environmental contaminants, such technology can allow communities to initiate a new level of data-driven engagement with public officials. This can go a long way in fostering bottom-up approaches to balance the top-down approaches to environmental quality management at the local level. Wicked problems that have persisted across years, decades, and centuries may have possible solution pathways that begin with the education of local communities to understand the profile of risks they are contending within their communities. It can then continue with communities partnering with universities, government officials, and other stakeholders to address such long-standing problems -leveraging emerging technologies and gathering relevant data to engage in smart governance. A recent case study in Colorado presents a stake-holder-focused approach to local government sustainability planning to explicitly link U. Diwekar, et al. Resources, Conservation & Recycling 164 (2021) 105140 bottom-up and top-down approaches to environmental management (Hopton et al., 2010; Dubinsky and Karunanithi, 2017a, b) . The deliberative process was intertwined with different types of uncertainties. During the stakeholder engagement phase, social values, competing interests, and points of view, trade-offs and choices would constitute different forms of uncertainties. The participation of community members introduced local context and local knowledge, which provided clear roadmaps for the construction of robust, meaningful, and realistic scenarios. It also facilitated the knowledge transfer process and ensured that locally defined sustainability benchmarks would be emphasized. It is highly recommended that the stakeholder-centric methodology be used in any future local governance sustainability projects. In a second case study, a multidisciplinary team evaluated a suite of six metrics of urban performance (Ecological Footprint, Green Net Product, Net Energy, Human Wellbeing Index, Fisher Information, Emergy) linked to an agent-based simulation model to explore current conditions and trends in greater Chicago. It was found as Cronin (1991) suggested, that Chicago and other urban systems are not static but constantly evolving. This poses the question of evolving uncertainties. The question for sustainability then is not whether an urban system (or any system) is sustainable at a given point in time, but rather whether that system has the capability and capacity to successfully evolve and adapt to the conditions that it will encounter in the future. Manufacturing sustainability has become a global issue, especially in developed and fast-developing countries. An improved corporate sustainability performance does not automatically lead to an improved sustainability performance of the systems within which a company is embedded. A comprehensive assessment framework is needed to analyze whether an improved corporate sustainability performance can positively contribute to sustainable development on a global level. This includes first-order and second-order performance measurements (Baumgartner, R. J., & Korhonen, J., 2010; Baumgartner and Rauter, 2017) . The former assesses the direct sustainability impacts and usually focuses on narrow issues of efficiency, while second-order sustainability performance focuses on systemic effectiveness and covers sustainability impacts on society and nature in total, i.e., the entire system. This problem is usually posed as a multi-objective optimization problem (Moradi and Huang 2016 , Liu and Huang 2012 , 2013 , Z. 2015 . However, there are not only uncertainties associated with each performance measure, but deciding the weights for different objectives presents significant uncertainties in solutions. There are both the aleatory and epistemic uncertainties that appear in various types of manufacturing sustainability problems. The uncertainties in secondorder performance measures are much higher than in first-order measures, and this needs to be balanced. This is because of the long-term nature of the performance objectives. This is obvious in the example of plastics (Plastics Europe, 2017; Plastics Europe 2020; Geyer et al., 2017; Shonnard et al., 2019; Benavides et al., 2017; Gracida, 2019; Closed loop partners, 2019), and environmental impacts by the disposal of plastic from municipal solid waste. Landfilling and leakage to the environment (as opposed to recycling or re-use) are thought to be the largest end-of-life fates for plastics that are used in packaging, which is the most common use of plastics. Plastic packaging exhibits the shortest lifetime between its production and appearing at its end-of-life as waste in need of management. A circular economy for these polymeric materials, rather than a linear economy, is proposed as a solution to the responsible management of waste plastics. However, large-scale implementation of a circular economy for plastics is not well-understood with respect to economic costs and benefits, environmental impacts, and societal effects relative to business-as-usual linear management. A systems analysis for sustainability for a circular economy of plastics is difficult because of several problems related to uncertainties related to the definition of goals and knowledge gaps on current and future recycling processes. Sustainable manufacturing often uses life cycle analysis for sustainability assessment. Given the importance of life cycle assessment in sustainability, we have devoted a special section below on life cycle assessment and uncertainty. Increasing awareness of environmental degradation and resource depletion has led to the incorporation of environmental sustainability considerations in engineering design and analysis. One such widely popular technique is the environmental life cycle assessment (Graedel and Allenby, 2019) , which aims to model and quantify the environmental impacts of products and systems over their entire life cycle (Rebitzer et al., 2004) . By assessing the entire life cycle of the system of interest, one can evaluate the environmental impact of released emissions or of materials and energy used during various stages (material extraction, manufacturing, usage, disposal) of the product or system of interest. LCA methods tend to focus heavily on emissions and their impact and resource use. It is also important to include ecosystem goods and services as well as societal benefits and impacts in LCA to move towards sustainable development. However, ecosystem services and societal implications are either not accounted for or not well represented in most life cycle methods. Some recent LCA approaches have been developed for the purpose of assessing the role of ecosystem services in process and input-output life cycles (Eco-LCA) (Zhang et al., 2010) . However, these have been limited in their scope, and placeholders exist for many ecosystem goods and serviced. These placeholders in LCA (known unknowns) can lead to wrong policies. In "Frames in toxicity controversy," Arnold Tucker (Tucker, 1999) presents a case study where researchers had to advise the Dutch government about the choice of incineration technology. However, LCA at that time lacked fate modeling and could not provide any certainty about the choice. The researchers used the precautionary principle and asked for more research. However, the politicians did not have time or money to carry out the research, and it resulted in the wrong policy. These placeholders represent known unknowns and are part of Level 3 uncertainties. As an example, consider the critical ecosystem service of pollination provided by insects. A lack of this service would be a significant detriment to not only ecosystem biodiversity and function, but also to various industrial sectors, agriculture, and the world economy (Potts et al., 2016; Klein et al., 2007) . In the United States, insectmediated pollination of crops accounts for between US$ 14-23 billion of pollination-dependent crop production alone, making this ecosystem service a highly valuable asset to the nutritional and economic welfare of the agricultural sector (Chopra et al., 2015) . In addition, there are upstream and downstream industry sectors that rely upon pollinationdependent crop production (fertilizers, pesticides) as well as nonagricultural industrial sectors (pharmaceutical, fuel) that share linkages with crop production and agricultural sectors, leading to intricate indirect dependence upon pollination service mediated by insects. Although this example is specific to insect pollination service, there are numerous other ecosystem goods and services that are being exploited at unsustainable rates and facing severe degradation (Carpenter et al., 2006) . While the economic valuation of ecosystem goods and services provides an elegant framework highlighting their importance for society and human welfare, there is a need to explicitly account for their contribution when designing and developing products and services. Data and models are needed at multiple scales to account for the contribution of ecosystem goods and services in environmental sustainability assessments and engineering design. At the process scale, recent work has attempted to account for the role of nature and specific ecosystem goods and services for the sustainable design of engineered U. Diwekar, et al. Resources, Conservation & Recycling 164 (2021) 105140 systems (Carpenter et al., 2006; Gopalakrishnan et al., 2016; Charles et al., 2020) . From the LCA perspective, new environmental impact categories will need to be developed for a comprehensive accounting of ecosystem goods and services. For example, the use of process-based models such as InVEST (Sharp et al., 2018) could be integrated into life cycle frameworks. Environmentally-extended Input-Output models have shown promise for an accounting of ecosystem goods and services, but they are also fraught with uncertainty associated with data and aggregation, which can hinder detailed assessments (Zhang et al., 2010) . Some of these challenges could be addressed with hybrid approaches (Suh et al., 2004) . A more accurate valuation of ecosystem goods and services and their inclusion in human and engineering decision-making has the potential to motivate and guide conservation, revitalization efforts, and policy decision-making. Recent efforts have focused on explicitly accounting for the role of ecosystems in supporting human activities by the framework of Techno-Ecological Synergy (TES) . The goal of this framework is to determine the extent of ecological overshoot from specific activities for each ecosystem service and to encourage mutually beneficial designs of technological and ecological systems. Several case studies indicate the potential environmental and economic benefits of establishing such synergies (Bakshi, 2019; Hanes et al., 2017; R.J. 2018; Gopalakrishnan et al., 2016 Gopalakrishnan et al., ,2017 Gopalakrishnan and Bakshi, 2018; Martinez-Hernandez et al., 2017; Hernandez et al., 2019) . However, these studies are based on simplified models that do not account for the dynamics of technological and ecological systems, which can be quite different. Technological systems are usually designed to have low variability and predictable behavior. Thus, such systems tend to stay around a "set point" and exhibit homeostasis. In contrast, ecosystems tend to be intermittent and are not predictable. They exhibit "homeorhesis." Benefiting from TES requires designs that combine the desired homeostasis of technologies and human-designed systems with the homeorhesis of nature. Our hypothesis is that TES designs are more resilient than conventional techno-centric designs. Exploring this hypothesis requires approaches for quantifying resilience. Various approaches may be relevant for this task, depending on the nature of information available about the designed systems. Ecological network analysis relies on information about the flow between nodes in a network. It relies on information-theoretic metrics that are useful for understanding the character of ecological and economic systems (Ulanowicz et al., 2009) . If time-series data from the system are available, methods such as Fisher information (Eason et al., 2014 ) may be used. If more details such as dynamic models of the system, then mathematical approaches based on viability theory (Béné, and Doyen, 2018 ) may be applied. Various approaches have been developed to account for the effect of uncertainties in network analysis (Guesnet et al., 2015; Hines et al., 2018) The effect of uncertainties may be included in the stochastic models used in these methods (De Lara et al., 2009 ) and by the use of robust and stochastic optimization methods (Sahinidis, 2004; Diwekar, 2010) . The Merriam-Webster dictionary defines decision making as the act or process of deciding something, especially with a group of people. Decision making under uncertainty methods are around for some time and is the topic of several books (e.g., Morgan and Henrion 1990; Bedford and Cooke 2001) . However, these methods do not deal with deep uncertainties that are present in regional as well as global sustainability models. DMDU methods are a topic of intense research in recent literature. DMDU methods are not based on the 'predict-then-act" paradigm, but that aims to prepare and adapt. According to a recent book on DMDU , there are four methods to deal with decision making under deep uncertainties. These include robust decision making (RDM), dynamic adaptive planning (DAP), dynamic adaptive policy pathways (DAPP), and engineering options analysis (EOA). While the first three involves hybrid approaches and described briefly below, EOA is quantitative in nature and is discussed in next section. For details of these methods, please refer to Marchau et al., 2019. RDM draws from scenario analysis the concept of organizing information about the future into a small number of distinct cases that help people engage with, explore, and communicate deep uncertainty. However, RDM is different from scenario analysis that it uses computer simulations to generate large ensembles of future states of world as compared to a small number of cases in scenario analysis. RDM involves four steps 1 Consider diverse plausible futures 2 Seek strategies which are robust across a wide range of plausible futures 3 Employ adaptive strategies to achieve robustness 4 Use humans and computers alternatively to test each other's conclusions about futures and strategies. DAP is a DMDU approach for designing a plan that explicitly includes provisions for adaptation as conditions change and knowledge is gained. DAP approach involves 1 Specifying a set of objectives and constraints 2 Designing short-term actions initial plan 3 Establish a framework to guide future (contingent) actions. DAPP explicitly includes decision making over time. It is proactive and dynamic planning. It involves adaption pathways for multiple futures. DAPP supports the design of a dynamic adaptive strategy that includes initial actions, long-term options, and adaptation signals to identify when to implement the long-term options or revisit decisions. Also, central to the approach is the concept of Adaptation Tipping Points. The Adaptation Tipping Point (ATP) approach was developed in the Netherlands in response to a desire of the national government for a planning approach less dependent on any particular set of scenarios. DMDU approaches have been used for regional sustainability assessment and policy analysis. For example, Molina-Perez et al. (2019) studied how to design adaptive plans (RDM) for water supply infrastructure under deep uncertainty to ensure sustainable development in Montreal, Mexico. Groves et al. (2019) presented a case study of the application of RDM to long-term water resources planning for the Colorado River Basin. Vincent et al. (2019) used DAP to derive innovative traffic safety technology in the Netherlands. In a chapter, Lawrence et al. (2019) describes how flood risk managers at a regional level in New Zealand applied the DAPP approach to managing deep uncertainty around flood frequency associated with changing climate, and examines the lessons learned from taking DAPP theory into practice. Hamarat et al. (2014) applied multi-objective simulation-based optimization with RDM to study how the European Union can achieve its carbon reduction target under deep uncertainty. Trindade et al. (2019) proposed deeply uncertain pathways for multicity regional water supply infrastructure investment and portfolio management. Shi et al. (2019) proposed two adaptive screening procedures to identify when comprehensive decision analysis methods, such as DMDU methods introduced here, should be implemented. Mathematical models featuring the critical components of a real ecosystem can aid in the formal study of sustainability. The first such comprehensive model for studying global sustainability is the model from the Club of Rome (Meadows et al., 1972; Meadows, 2014) called the World Model. This model has components like the human U. Diwekar, et al. Resources, Conservation & Recycling 164 (2021) 105140 population, natural resources, pollution, industrial production, and agricultural land use. The results of this model established growth patterns and limits for the first time. Models accounting for processes such as resource extraction, waste assimilation, recycling, and pollution in an integrated ecological, economic framework have also been proposed in the literature (van den Bergh, 1996) . In recent years, the united global metamodel of the biosphere (GUMBO) (Boumans et al., 2002) was developed to include dynamic feedbacks among human technology, economic production and welfare, and ecosystem goods and services. The model was calibrated using time series data for 14 key variables from 1900 to 2000. Recently, the model predictions were compared with data from 2000 to 2018 (Boumans, 2019) . There are a number of planetary integrated assessment models available in the literature (Wilson et al., 2017) to study global phenomena such as global climate change. A model that deals with food-energy nexus for global sustainability is recently proposed by Kotecha et al. (2013) . Since these models are dealing with long-term sustainability, they face the problem of time-dependent uncertainties (deep uncertainty). For example, Fig. 2 shows how the global temperature change over the time span of 400 years. We can see that there is significant uncertainty in global warming and is changing with time. We cannot describe these uncertainties using static probability distributions. These are time-dependent uncertainty. Financial literature abounds with such uncertainties and hence, as a result, provided a way to deal with such uncertainties using real options theory. Real options theory uses stochastic processes to represent the time-dependent uncertainties. A stochastic process is one that evolves over time in an uncertain way. Stochastic processes do not have time derivatives in the conventional sense, and, as a result, they cannot be manipulated using the ordinary rules of calculus as needed to solve the stochastic optimal control problems. Ito (1951 Ito ( , 1974 provided a way around this by defining a particular kind of uncertainty representation based on the Wiener process as a building block. The Wiener process, also known as Brownian motion, is a continuous limit of the random walk and is a continuous-time stochastic process. A Wiener process (continuous or discrete) can be used as a building block to model an extremely broad range of variables that vary continuously and stochastically through time. Examples of Ito processes are simple Brownian motion, geometric Brownian motion, mean-reverting process, etc. In the real options analysis (ROA), Dixit and Pindyck (1994) used stochastic calculus (Ito's Lemma) coupled with dynamic programming to study investment decisions under uncertainty. This approach uses Ito's Lemma, which is called the Fundamental Theorem of Stochastic Calculus and allows us to differentiate and to integrate functions of stochastic processes. However, for deep uncertainties involved in sustainability, we need to consider the engineering options analysis. The difference between EOA and ROA is that EOA considers multiple options with a range of possibilities. However, the mathematical methods used for ROA and EOA are similar. Recently, Diwekar (Diwekar, 2008; Rico-Ramirez et al., 2003; Rico-Ramirez and Diwekar, 2004) presented basic concepts for dealing with time-dependent uncertainties modeling. These concepts are derived from the financial and economics literature and engineering optimal control theory. These concepts could be extended to studying sustainability as presented in the recent literature (Shastri and Diwekar, 2008; Shastri et al., 2008a,b; Diwekar and Shastri, 2010; Diwekar, 2012b Diwekar, , 2015 Doshi et al., 2015 , Rodriguez-Gonzalez et al., 2019 . Although the TARDIS workshop of 2019 focused on the role of uncertainty in sustainability science and engineering, it was also discussed whether attempts to accurately assess the current condition of the earth and forecast future trends as accurately as possible are in fact in themselves symptoms of the underlying problem and reflects an anthropocentric view of human exemption, dominance, and omnipotence. Effective management strategies might, therefore, focus also on fostering new approaches that reflect a more humble and flexible human-nature relationship. A recent article in PNAS (Steffen et al., 2018) argued that the "social and technological trends and decisions occurring over the next decade or so could significantly influence the trajectory of the Earth System for tens to hundreds of thousands of years and potentially lead to conditions inhospitable to current human societies and to many other contemporary species." Despite the best efforts of scientists to qualify their interventions and acknowledge that their scenarios are "mediating tools to communicate vulnerabilities, perceptions of risk or possible consequences, they often become taken as statements of the future" (Yusoff and Gabrys, 2011) . Transformation is a way of reflecting on existing practices and imagining new ways of doing and knowing (Duncan et al., 2018) , the capacity to collectively envision and meaningfully debate realistic and desirable futures (Milkoreit, 2017) . Transformations, however, involve "systems change and because of the political nature of change are subject to contestation" (Hebinck et al., 2018) . Norgaard (Norgaard, 2018) suggests that while we have made significant progress in developing an ecological imagination through advances in climate science, we have made much less progress in actually changing course. The gap between responses to climate change and the socio-political realities of contemporary life have led to inaction, but the further problem is that we do not know and cannot know what changes are in motion (Nightingale et al., 2019). Beck's world risk society idea and its attendant uncertainty capture the sense that the speed and complexity of interactions blur cause and effect, creating the escalating potential for cascading catastrophic risks and a vision of the future characterized by chaos (Rickards et al., 2014) . Living with uncertainty is characterized "as the continuous emergency of its own emergence" (Rickards et al., 2014) captured in the idea of the 'long emergency' (Kunstler, 2005) . There is growing experimentation with participative and deliberative designs adapted from innovative governance mechanisms, e.g., citizens assemblies for national dialogues on climate change as a method for engaged research. The focus is very much on facilitating the active participation of local communities and civil society actors to explore the relationship between probable, preferable, and plausible futures (Ellyard, 2011) . Faced by growing societal demand for urgent action on climate change, e.g., Extinction Rebellion and the Global Climate Strikes inspired by Greta Thunberg, there is an opportunity for more meaningful engagement between science and society. Yet this is not straightforward and poses very particular challenges in preparing U. Diwekar, et al. Resources, Conservation & Recycling 164 (2021) 105140 present and future generations of researchers and professionals with the tools that are fit for the purpose of addressing these challenges. It is increasingly claimed that we are living in a world that is so complex, interconnected and uncertain, and that many of the situations that professionals, for example, engineers, have to deal with should be seen and treated as 'wicked problems' (an early description of such problems was provided by Rittel & Webber in 1973 . Andersson et al. (2014 talk about societal systems as wicked, featuring properties that make them a combination of complicated and complex, using their terminology. Farrell & Hooker (2013) talk about all realworld science and engineering problems as wicked and describe three sources of wickedness: (1) finitude of cognitive abilities, (2) complexity of the problems, and (3) normativity in problem understanding and resolution. Again, we can take the example of reactive nitrogen to describe the wickedness of problems. Reactive nitrogen encompasses biologically and radiatively active, and chemically reactive nitrogen compounds. On a global scale, human activities now create approximately two-fold more Nr than natural continental ecosystems. In the United States, Nr creation by human activity is about 5-fold larger than natural processes by: (1) the Haber-Bosch process to generate ammonia (NH3) for synthetic nitrogen fertilizer and industrial feedstocks, (2) the enhancement of biological nitrogen fixation (BNF) in crop cultivation (e.g., legumes), and (3) the combustion of fossil fuels. The first two anthropogenic activities form Nr on purpose; the last one forms Nr as an unwanted pollutant. Comparing these three sources, Haber-Bosch Nr is by far the largest (EPA et al., 2011) . Anthropogenic creation of Nr provides essential benefits for humans-first and foremost, in meeting human dietary needs. A large fraction of the human population of the earth could not be sustained if fertilizers containing Haber-Bosch (Nr) did not augment food production significantly. There are, however, costs associated with these benefits. Essentially all of the Nr created by human activities is lost to the environment, often with negative, unintended consequences. There it circulates between, and accumulates within, environmental systems and contributes to a number of adverse public health and environmental effects. In an educational context, the collaboration of Haber (a chemist) and Bosch (a chemical engineer) is often presented as a triumph of science, and the ideal model for technological advancement. Few at the time could foresee the unintended consequences of Nr proliferation; indeed, perhaps these consequences were not foreseeable-the fields of environmental science and engineering, ecology, and environmental chemistry were at best immature, and the widely perceived vastness of earth lent it an aura of immutability. As Hardin points out in his seminal essay "The Tragedy of the Commons" (Hardin, 1968) , the concept of "waste" is relatively modern; humankind evolved on earth where almost all natural cycles were closed-loop systems-wastes simply did not exist. In many ways, our education paradigm is just emerging from its siloed past in which insular disciplinary approaches reigned supreme. The need for a new paradigm of convergent education is now widely recognized (NSF, 2017) . So, what approaches are more useful for dealing with wicked problems? Porter & Córdoba (2009) describe three different approaches to systems thinking. Functionalist approaches are made from a scientific, systems analytic perspective and with a reductionist worldview. Interpretive approaches, on the other hand, view systems as "mental constructs of observers" and explicitly explore normative assumptions behind the problem definition. Complex adaptive systems approach instead recognize systems as not only complex, but also "adaptive," characterized by self-organization, emergence, and bottom-up change and thereby suggest that conflicts may be ultimately unresolvable and that there may not be definitive solutions to sustainability problems -or even a "best way of getting things done." If real-world problems are actually wicked, the third approach should be more useful, and it is then a problem that professionals are not properly trained to deal with such contexts. Jonassen (2000) , for example, argues that engineering students are primarily trained to solve "story problems", using functionalist approaches. This is likely a result of the predominant culture in science and engineering based on objectivism, positivism, and reductionism and with a lack of recognition and discussions of the usefulness of alternative views. In an attempt to guide engineering education towards approaches that recognize the wickedness of real-world problems, Lönngren et al. (2017; ) developed a description of and educational elements for developing such approaches. The description contains five 'structural' aspects related to the ability to deal with wicked problems, that focus on the interconnection between problems parts, improvement measures and secondary problems, but also five 'referential' aspects that focus on local contexts and stakeholders and their spheres of influence as well as the importance of lack of information and uncertainties. These latter five try to capture a new type of relationship to problem-solving, or rather to the management of situations, as wicked problems do not really have solutions. Our relationship to what we know -and more importantly, what we do not know -requires mechanisms and tools that prepare researchers and professionals to foster adaptive relationships to the dynamics of complex socio-ecological systems. The challenges addressed here are akin to those identified by Bai et al. " to find ways to live and act without knowing the future…science needs to have closer and different relations with practice, …where science not only informs practice but also learns from practice" (Bai et al., 2016) . At its most basic core, sustainable development is an effort by humans to make institutions and choices to ensure that the earth can support our continued presence for the indefinite future. This is a difficult challenge given the size of the human population, the level of human activity, and the finite size of the earth. This brings humanity to a situation where it has to learn to manage the earth out of simple necessity. This is a difficult challenge because: (1) there are multiple aspects that must be considered and acted on, including policy, economics, manufacturing, environmental impacts, and education to name but a few, and (2) the knowledge necessary to effectively conduct the management action comes with substantial uncertainty which can be specific to some aspect or systemic and is often time-dependent. Further, the challenges to the Earth system can fall into any of the four known categories, where the most difficult one is that unknown unknowns, i.e., issues that are not even known to exists. Here we have explored the challenges and made an effort to frame approaches that can be reasonably expected to lead to effective solutions or least management approaches. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The Vision for Civil Engineering in 2025. Based on The Summit on the Future of Civil Engineering -2025 Sustainable Engineering: Principles and Practice Techno-Ecological Synergy: a Framework for Sustainable Engineering Strategic perspectives of corporate sustainability management to develop a sustainable organization Sustainability science: Managing risk and Resilience For Sustainable Development Probabilistic Risk analysis: Foundations and Methods From Resistance to Transformation: a Generic Metric of Resilience Through Viability Life-cycle analysis of fuels from post-use non-recycled plastics The Great Moderation Modeling the dynamics of the integrated earth system and the value of global ecosystem services using the GUMBO model Revising GUMBO Model Flint Water Crises -Five Years Later: there's Still No Trust Uncertainty as information: narrowing the sciencepolicy gap Thinking inside the box: a participatory, computerassisted approach to scenario discovery Sustainability, Sustainability Assessment, and the Place of Fiscal Sustainability To bridge the divide between evidence and policy: reduce ambiguity as much as uncertainty Millennium ecosystem assessment: research needs Complexification: explaining a Paradoxical World through the Science of Surprise Connecting air quality regulating ecosystem services with beneficiaries through quantitative serviceshed analysis Economic dependence of US industrial sectors on animal-mediated pollination service Option demand and consumer surplus: further comment Quasi-Option value and the expected value of information Nature's Metropolis: Chicago and the Great West Ecological Economics: Principles and Applications Quality Unknown: The Invisible Water Crisis. The World Bank Group Multi-criteria dynamic decision under uncertainty: a stochastic viability analysis and an application to sustainable fishery management Introduction to Applied Optimization Optimization under Uncertainty in Process Systems Engineering A Systems Analysis Perspective of Green Design Green Engineering and Sustainability: a Systems Analysis Perspective. Sustainability: Multidisciplinary Perspectives 273 Perspective on pursuit of sustainability: challenges for engineering community Green process design, green energy, and sustainability: a systems analysis perspective Investment Under Uncertainty Listing decisions under the Endangered Species Act: why better science isn't always better policy Maximizing sustainability of ecosystem model through socio-economic policies derived from multivariable optimal control theory Greenhouse gas accounting of rural Agrarian regions: the case of San Luis Valley Transformation is 'experienced, not delivered': insights from grounding the discourse in practice to inform policy and theory Managing for resilience: early detection of regime shifts in complex systems On the polarization of young stellar objects Global Sustainable Society Environmental Protection Agency Science Advisory Board. Reactive Nitrogen in the United States: an Analysis of Inputs, Flows, Consequences, and Management Options -A Report of the EPA Science Advisory Board Humankind and the Environment: an Anatomy of Surprise and Ignorance Evolution, Time, Production and the Environment Design, science and wicked problems A Core Precautionary Principle Production, use, and fate of all plastics ever made Assessing the capacity of local ecosystems to meet industrial demand for ecosystem services Ecosystems as Unit Operations for Local Techno-Ecological Synergy: integrated Process Design with Treatment Wetlands Biosolids Management with Net-Zero CO2 Emissions: a Techno-Ecological Synergy Design towards sustainable production of chemicals and fuels from the fast pyrolysis of waste polyolefin plastics, PhD Dissertation In: Robust Decision Making (RDM): Application to Water Planningand Climate Policy in Decision Making under Deep Uncertainties An exploratory approach for adaptive policymaking by using multi-objective robust optimization Synergies and trade-offs in renewable energy landscapes: balancing energy production with economics and ecosystem services Including nature in the food-energywater nexus can improve sustainability across multiple ecosystem services The Tragedy of the Commons Imagining transformative futures: participatory foresight for food systems change Techno-ecological synergies of solar energy for global sustainability Introduction to Operations Research Uncertainty analyses for ecological network analysis enable stronger inferences Development of a multidisciplinary approach to assess regional sustainability Higher Taxes on Top 1% Equals Higher Productivity. The Real News Network Multiscale regime shifts and planetary boundaries Guidelines For the Governance of Systemic Risks On stochastic differential equations On stochastic differentials Toward a design theory of problem solving Success, strain, and surprise A World Beyond physics: the Emergence and Evolution of Life What Is Systemic Risk, and Do Bank Regulators Retard or Contribute to It? Providing global public goods: Managing globalization Importance of pollinators in changing landscapes for world crops Risk, Uncertainty, and Profit Model Based Approach to Study the Impact of Biofuels on the Sustainability of an Integrated System Many objective robust decision making for complex environmental systems undergoing change The Long Emergency: Surviving the Converging Catastrophes of the 21st Century Dynamic Adaptive Policy Pathways (DAPP): From Theory to Practice in Decision Making under Deep Uncertainties Tipping elements in the earth's climate system On the origin of planetary-scale tipping points Hybridization, agency discretion, and implementation of the U.S. Endangered Species Act Sustainability Enhancement under Uncertainty: a Monte Carlo Based Simulation and System Optimization Method Sustainable Distributed Biodiesel Manufacturing under Uncertainty: an Interval-Parameter-Programming-Based Approach Technology Evaluation and Decision Making for Sustainability Enhancement of Industrial Systems under Uncertainty Avoid, control, succumb, or balance: engineering students' approaches to a wicked sustainability problem Scaffolding strategies in a rubric-based intervention to promote engineering students' ability to address wicked problems Residents, officials outraged over toxic emissions from Cobb Plant Forthcoming. The Bird and the Bulldozer Unsicherheit der Normativität und Normativität der Unsicherheit in den Diskursfeldern "globaler Wandel" und "Nachhaltigkeit Decision Making Under Deep uncertainty: From theory to Practice Forecasting unknownunknowns by boosting the risk radar within the risk intelligent organisation A Framework for Modeling Local Production Systems with Techno-Ecological Interactions Interpreting Negative Results with Taxonomic and Conservation Implications: another Look at the Distinctness of Coastal California Gnatcatchers The Limits to Growth Limits to Growth: The 30-year Update Imaginary politics: climate change and making the future The Financial Instability Hypothesis: an Interpretation of Keynes and an Alternative to "Standard Developing a Robust Water Strategy for Monterrey, Mexico: diversification and Adaptation for Coping with Climate, Economic, and Technological Uncertainties Multistage Optimization for Chemical Process Sustainability Enhancement under Uncertainty Uncertainty: A guide to Dealing With Uncertainty in Quantitative Risk and Policy Analysis Ontic openness: an absolute necessity for all developmental processes 2020. A New Ecology: Systems perspective The Sociological Imagination in a time of climate change Subspecies and the philosophy of science With Speed and Violence: Why Scientists Fear Tipping Points in Climate Change Three views of systems theories and their implications for sustainability education Summary for policymakers of the assessment report of the intergovernmental science-policy platform on biodiversity and ecosystem services on pollinators, pollination and food production Risk Governance. Coping With Uncertainty in a Complex World Precaution and the governance of risk Systemic risks: the new kid on the block, Environment: science and Policy for Sustainable Development Opening and closing the future: climate change, adaptation, and scenario planning Real option theory from finance to batch distillation Stochastic Maximum Principle for Optimal Control under Uncertainty Species definitions shape policy Dilemmas in a General Theory of Planning A New Approach to Solving Stochastic Optimal Control Problems Life cycle assessment: part 1: framework, goal and scope definition, inventory analysis, and applications A safe operating space for humanity Optimization under uncertainty: state-of-the-art and opportunities Imaginable surprise in global change science Invest: integrated valuation of ecosystem services and trade-offs Optimal control of lake pH for mercury bioaccumulation control Optimal control theory for sustainable environmental management Is sustainability achievable? Exploring the limits of sustainability with model systems When can decision analysis improve climate adaptation planning? Two procedures to match analysis approaches with adaptation problems Systems analysis for PET and olefin polymers in a circular economy Many-objective robust decision making for managing an ecosystem with a deeply uncertain threshold response The concept of systemic risk Fallibility, reflexivity, and the human uncertainty principle Planetary boundaries: guiding human development on a changing planet Trajectories of the Earth System in the Anthropocene Deliberate futures: precaution and progress in social choice of sustainable technology Eradication of tephritid fruit fly pest populations: outcomes and prospects The Black Swan: The Impact of the Highly Improbable Deeply uncertain pathways: integrated multicity regional water supply infrastructure investment and portfolio management Frames in the Toxicity Controversy. Risk Assessment and Policy Analysis Related to the Dutch Chlorine Debate and the Swedish PVC Debate Quantifying sustainability: resilience, efficiency and the return of information theory In: Ecological economics and sustainable development, chapter A multisectoral growth model with materials flows and economic-ecological interactions Dynamic Adaptive Planning (DAP): The Case of Intelligent Speed Adaptation. in Decision Making under Deep Uncertainties Defining uncertainty: a conceptual basis for uncertainty management in model based decision support Collective consumption services of individual consumption goods The role of scientists in statutory interpretation of the U.S. Endangered Species Act In: Evaluating Process-based integrated assessment models of climate change mitigation. Laxenbur, Austria. International Institute for Applied Systems Analysis Working Paper Uncertainty and the Endangered Species Act The World Bank DataBank Climate change and the imagination New) species concepts, species delimitation and the inherent limitations of taxonomy Accounting for ecosystem services in life cycle assessment, part II: toward an ecologically based LCA Phylogeography of the California Gnatcatcher (Polioptila californica) Using Multilocus DNA Sequences and Ecological Niche Modeling: implications for Conservation Geographic Variation, Null Hypotheses, and Subspecies Limits in the California Gnatcatcher: a Response to McCormack and Maley Foundation under the Grant CBET 1854519.