The emergence of nanotechnology has coincided with an increased recognition of the need for new approaches to understand and manage the impact of emerging technologies on the environment and human health. Important elements in these new approaches include life-cycle thinking, public participation and adaptive management of the risks associated with emerging technologies and new materials1. However, there is a clear need to develop a framework for linking research on the risks associated with nanotechnology to the decision-making needs of manufacturers, regulators, consumers and other stakeholder groups2,3. Given the very high uncertainties associated with nanomaterials and their impact on the environment and human health, research resources should be directed towards creating the knowledge that is most meaningful to these groups. Here, we present a model (based on multi-criteria decision analysis and a value of information approach) for prioritizing research strategies in a way that is responsive to the recommendations of recent reports on the management of the risk4,5 and impact of nanomaterials on the environment and human health6.
The considerable uncertainty associated with nanomaterials complicates product-development decisions that must be made when multiple materials or synthesis technologies are available, but the consequences of each technological choice are only incompletely understood. Value of information (VoI) explores how decision preferences might change if new information becomes available (for example, through research) before a decision is made7. However, the application of VoI to decisions concerning the environment or human health is further complicated by the need to evaluate multiple (and often conflicting) criteria. Multi-criteria decision analysis (MCDA) enables us to compare the outcomes of different decisions when it is not possible to reduce the relevant competing criteria to a single criterion8,9. Traditional approaches to VoI in MCDA can be straightforward when the relative satisfaction of all outcomes, also known as the utility, can be clearly enumerated. However, the high uncertainty associated with nanomaterials makes it impossible to obtain detailed utility functions, especially where the views of multiple stakeholders must be considered simultaneously10. Here we present a model for the application of VoI using MCDA outranking methods that allow us to make judgements and trade-offs across different decision criteria, even under conditions of high uncertainty.
Figure 1 presents a flowchart showing how this approach could be applied to the selection of a technology for the synthesis of single-walled carbon nanotubes. Although this decision might be made by a privately owned company in the context of materials production, the company is nevertheless held accountable to several different stakeholder groups (which are hypothetical in our case study), including manufacturers who might use nanotubes in their products, consumers who might buy these products, regulators and environmental groups. Consequently, the decision problem must be understood from the perspectives of these different stakeholders, who all place different weights on the importance of various decision criteria: cost, material efficiency, energy consumption, life-cycle environmental impacts, and risks to human health (Supplementary Fig. S1). Experts must assess the various synthesis technologies on offer—arc discharge (arc), chemical vapour deposition (CVD), high-pressure carbon monoxide (HiPCO) and laser vaporization (laser)—relative to the different decision criteria selected by the different stakeholders. Because these assessments are likely to be uncertain, experts must be permitted to provide probability distributions that bound a range of possible outcomes. These can be based on testing of material properties, evaluating synthesis technologies or on other results of research or experience.
The core of the decision recommendation is a comparative analysis of the expert assessments using MCDA outranking methods11 (see Supplementary Information). Here, we first use Monte Carlo simulation to sample point estimates from the probability distributions supplied by the experts. Each technology is compared with all others and ranked on each criterion according to which performs best. The results are then summed across criteria (suitably weighted for each stakeholder) to determine an overall score (the ‘net flow’) for each technology in a given simulation. Finally, all technologies are ranked in order, from most preferred (that is, highest net flow) to least preferred (lowest net flow) for each stakeholder group, and these rankings are compared to a ‘balanced’ perspective in which all criteria are weighted equally. To ensure a robust sampling of all possible combinations, the Monte Carlo simulation is carried out thousands of times and the results are reported as a probabilistic rank ordering of technologies in terms of which is most often preferred by each stakeholder.
To illustrate the approach, we use the hypothetical weights and triangular probability distributions previously reported12, with life-cycle data extracted from published literature13,14,15. In the case of health risks, the uncertainty is so high that expert judgements are represented with a ‘low–medium–high’ ordinal scale. (See Supplementary Information for further information, and ref. 16 for a discussion of regulatory considerations.) The MCDA results (Fig. 2a) show that HiPCO is most likely to be the preferred synthesis technology for most stakeholders, although a stakeholder with a preference for material efficiency over cost may prefer laser synthesis. CVD is clearly the least preferred alternative, with almost no chance of ranking first. For all stakeholders, no single technology choice dominates in all cases, but additional research could help differentiate between technologies and improve confidence in a final selection. These rank orderings are referred to as the ‘base case’.
The purpose of the VoI model is to explore the remaining uncertainties to determine which types of new information might influence the likely rank ordering for each stakeholder. To simplify the analysis, we created two classes of hypothetical research projects that could reduce uncertainty: ‘manufacturing research’ provides new information on cost, material efficiency, energy consumption and life-cycle environmental impact; ‘health research’ provides new information related to the risks to human health. New information that improves confidence in the eventual rank ordering is represented as a higher net flow, relative to the base case. These anticipated increases in decision confidence are useful for prioritizing research most relevant to the decision at hand.
Figure 2b shows how the net flows calculated by the MCDA outranking algorithm change as more information on the four different synthesis technologies becomes available through research. The bottom portion (in blue) of each bar in the chart represents average net flow for the highest-ranked technology in the base case, whereas the increases in these net flows that result from new information produced by the different research programmes are shown in different colours above the base case. As might be expected, new information related to underweighted criteria leads to little improvement in net flow. For example, even if it could be established through research that the health risks associated with HiPCO are unequivocally higher than those of the alternatives, manufacturers would continue to prefer HiPCO and, moreover, their confidence in their decision would not be undermined. However, both regulators and environmental groups would benefit from new information obtained from health research (shown in green in Fig. 2b), and to a lesser extent all the stakeholders would benefit from the results of manufacturing research (shown in red).
Although the four stakeholders have different views on which type of research is most useful, some stakeholders show an added value from completing the whole research programme in addition to its separate parts (shown in purple in Figure 2b). That is, information regarding criteria that are not highly valued will nevertheless make information relating to criteria that are highly weighted even more valuable for improving decision confidence. Even though in this case the synergetic impact is small relative to the base-case net flow, and is appreciable only for environmental groups and for the balanced weighting, this demonstrates the value of a formal stochastic investigation of the performance table from multiple perspectives, as intuition alone is insufficient to understand the interaction between different uncertainties.
The integrated MCDA/VoI model proposed in this Letter suggests that the highest-priority research should relate to questions that may modify the rank ordering of preferred technologies in a specific decision context. In the present case study, no single synthesis technology dominates under all weight scenarios, but some inferior technologies are identifiable (such as CVD). HiPCO is preferable for most of the hypothetical stakeholders based on the distribution of life-cycle and risk-to-health parameters, but when the uncertainties related to cost, materials efficiency, energy and life-cycle impact are reduced, the laser approach to synthesis may be preferable for some stakeholders. Because it is not realistic to undertake research that would reduce all uncertainties, it is useful to know which uncertainties are most influential to stakeholders that lack confidence in their preference ordering. Typically, this information will not dictate a single correct research direction. Rather, it will inform decisions about what type of research should be undertaken first.
The development of a comprehensive decision–analytic framework incorporating both uncertainty and sensitivity to new information provides an actionable intellectual agenda for further nanomaterials research. In practice, in addition to informing research strategies, this integrated MCDA/VoI approach could also facilitate the participation of various stakeholder groups. Although it is unlikely to achieve consensus among different stakeholders on what research is most important, it may help orient discussion of disagreements towards those issues that have practical relevance. This is especially urgent given ongoing efforts to revise federal research strategies to focus attention on both the commercialization of nanomaterials and research into the impact of nanomaterials on the environment and human health17. Adoption of this type of decision–analytic approach will require substantial institutional changes and expertise from the social sciences18. Traditionally, information has flowed from research scientists to risk managers, but there is growing recognition of the need for two-way interactions between decision analysis and risk analysis19.
The information analysis presented in this Letter uses an outranking decision model to rank-order technologies for the synthesis of single-walled carbon nanotubes, as reported in ref. 12. Uncertainties in input parameters are represented as probability distributions, whereas contrasting weight sets are illustrative of hypothetical stakeholder groups. In total, 10,000 rank orderings were generated via Monte Carlo sampling of new information from the supplied distributions20, resulting in a probabilistic distribution of net flows for each alternative and stakeholder.
Preferred alternative evaluation
Mathematically, let i, j, k denote indices corresponding to the stakeholders, decision criteria and synthesis technologies, respectively, and wij denotes an array expressing stakeholder i's weight on criterion j, and xjk denotes the technology k's score on criterion j. For each scenario, we calculate what is called a ‘weighted net flow’, ϕi(k) = (Σjwij ϕ′ij(k))/(i−1) for each technology using the PROMETHEE II algorithm11, where ϕ′ij(k) results from pairwise comparison of xjk to all other technologies on all criteria. ϕ′ij(k) ranges from –1 to 1, where –1 means technology k is worse than every other technology on criterion j, and +1 means it is better than all other technologies on criterion j. (See Supplemental Information for a detailed discussion and example of net flow calculations.) The technology that averages the greatest net flow over all 10,000 trials is most preferred.
Value of Perfect Information
In classical approaches7, VoI is defined as the increase in average value (or utility) attained by obtaining information prior to the decision. This Letter follows21 by adapting classical approaches and mathematically substituting expected net flow for utility. This adapted VoI sets an upper bound for the confidence a stakeholder can expect to gain from additional research on a topic.
In the base case, with no new information, optimal expected net flow is calculated as ϕ_No = maxk(Σn(ϕi(k))/n), effectively asking, ‘What is the average net flow of the one technology that most often ranked first?’ Here, a stakeholder's ‘confidence’ is expressed as the difference in average net flow between the highest and second-highest ranking technologies, over n simulated possible realities from the uncertainty distributions.
With perfect information on all criteria, we instead ask, ‘What is the average net flow of all the different technologies that ranked first in each trial?’ By shifting the decision point with respect to n samplings, as described by ref. 22, we calculate a new expected net flow ϕ_Perfect = Σn(maxk(ϕi(k)))/n. With perfect information available only on a subset C of the criteria (sometimes called partial-perfect information; for example, C could contain all of the manufacturing criteria but not the health criterion), we calculate the expected net flow Fij(k) = Σm(ϕ′ij(k))/m over m trials for each of the criteria j not in C (where m = n), and Fij(k) = ϕ′ij(k) otherwise. Finally, we calculate Fi(k) = Σjwij Fij(k) for each iteration of n and average these to obtain the expected net flow with information on C, ϕ_C = Σn(maxk (Fi(k)))/n. We then calculate the value of information as the increase in expected net flow from perfect information or information on C.
This work was supported by the Environmental Quality Research Program of the US Army Engineer Research and Development Center. The authors thank E. Ferguson, the manager of this programme. J. Steevens and M. Chappell of the US Army Corps of Engineers are thanked for their editorial comments and suggestions. Permission was granted by the Chief of Engineers to publish this information.
About this article
Balancing research and funding using value of information and portfolio tools for nanomaterial risk classification
Nature Nanotechnology (2016)