Sacramento–San Joaquin River Basin Case Study

Case Study Contents

Photo by akrassel / Getty Images

In this case study, we describe how Decisionmaking Under Deep Uncertainty (DMDU) methods were applied to Reclamation’s Sacramento and San Joaquin Rivers Basin Study (SSJRBS) [PDF] and present additional Robust Decision Making (RDM) analysis to show the potential of DMDU to inform long-term water resources management (Figure 1). Although the SSJRBS team did not formally use RDM, RAND Corporation researchers worked with Mike Tansey of the Reclamation Mid-Pacific Regional Office, a member of the SSJRBS team, to describe how elements of the original SSJRBS analysis aligned with key components of a DMDU approach. We then performed additional analysis informed by RDM to illustrate how RDM could help researchers interpret the results of evaluating water resources management strategies across a variety of different objectives identified by stakeholders. This retroactive approach illustrates how water management agencies that have completed studies could use RDM to elaborate on their results to guide more-robust decisionmaking in light of future uncertainty.

Map of the Sacramento and San Joaquin Rivers Basin, California. — **Figure 1.** Sacramento–San Joaquin Rivers Basin Study Area

Background

The Sacramento and San Joaquin Rivers Basin makes up California’s largest watershed and is a key source of water throughout the Central Valley, the San Francisco Bay Area, and Southern California. Total surface water supplies from the basin average 25 million acre-feet (MAF) per year, and groundwater contributes an additional 2–5 MAF per year. These sources supply agriculture (18–27 MAF per year), Central Valley urban use (2.2 MAF per year), dedicated environmental flows (1.2 MAF per year), and demands out of the basin (1–2 MAF per year).

California hydrology is highly variable and is undergoing significant changes as temperatures rise and the variability of supplies increases from what has been observed over the past century. Both drought and precipitation records have been broken in recent years. Specifically, the most recent drought, from 2012 to 2016, was significantly worse than the previous drought of record, from 1987 to 1992, which was used for most statewide planning. On the other hand, the water years (October 1 through September 30) 2016–2017 and 2018–2019 broke precipitation records throughout the state.

The SSJRBS, which is part of Reclamation’s Water Smart program, was a collaboration among Reclamation, California’s Department of Water Resources, California Partnership for the San Joaquin Valley, El Dorado County Water Agency, Madera County Resources Agency, and the Stockton East Water District. The SSJRBS team sought to evaluate future potential supply and demand imbalances and compare some potential adaptation responses. The SSJRBS addresses a key question: “How well will one or more water management actions alleviate anticipated negative impacts of climate change on water supplies, demand, infrastructure, and Endangered Species Act (ESA) species in the Central Valley?” The SSJRBS, which was completed in 2016, used DMDU analytic components to explore the potential impacts of climate change and assess mitigation strategies to the system, although in a different way than was done for the Colorado River Basin Water Supply and Demand Study (CRBS). Specifically, instead of evaluating hundreds of plausible climate projections, the SSJRBS team identified five climate projections to represent the range of projected temperature and precipitation trends from global climate models (GCMs) and three scenarios of demographics and land use, although these latter scenarios were not fully available for inclusion in our case study. The SSJRBS analysis used a suite of models to evaluate the existing water management plan along many metrics, including water deliveries, water quality, hydropower generation, and flood control. The SSJRBS team then developed seven adaptation portfolios and evaluated changes in performance that are attributable to these interventions.

This case study maps the analysis undertaken for the SSJRBS to the steps of RDM and adds some other analysis to demonstrate how RDM techniques could augment the original study. Specifically, this case study highlights

how the SSJRBS team framed the analytical problem (which we align to the decision framing step of RDM) and generated a database of results that can be explored interactively (which we align to the evaluate strategies across futures step of RDM)
how one can use RDM techniques to present the trade-offs among the many metrics of concern (the trade-off analysis step of RDM)
how additional analysis of a broader set of conditions would generate the data needed to conduct vulnerability analyses and identify robust portfolios.

This case study concludes with a discussion of possible next steps for Reclamation and its partners to expand on the SSJRBS.

A summary of the RDM elements in the SSJRBS is shown below Figure 2.

Figure 2. Robust Decision Making Steps Used in the Sacramento–San Joaquin River Basin Case Study

Decision Framing
Evaluate Strategies Across Futures
Vulnerability Analysis
Tradeoff Analysis
New Futures and Strategies

RDM Steps in This Case Study

Decision framing: The uncertainties, portfolios of management strategies, models, and performance metrics developed for the SSJRBS are shown in Table 1. These inputs were developed by the Reclamation project team.
Evaluate strategies across futures: Using the five climate and three socioeconomic (demographic and land use) projections developed in the decision framing step of RDM to represent plausible future conditions, the SSJRBS team used a detailed simulation model of the system to evaluate how well the current “No Action” management portfolio would perform through 2099 in each of these futures according to 13 different metrics. The SSJRBS team evaluated an additional seven water management portfolios (i.e., strategies).
Vulnerability analysis: Because of the relatively few futures in this study, a vulnerability analysis was not done for the SSJRBS. However, including more climate conditions and other uncertainties would be a clear next step in a follow-up study, particularly so that data-mining tools could be used to characterize vulnerabilities across the many futures and across the many metrics.
Trade-off analysis: The SSJRBS team evaluated the performance of the different portfolios across the scenarios, showing that no portfolio performs well on all of the performance metrics. This case study expands on this analysis and describes the key trade-offs between metrics that must be made.
New futures and strategies: The SSJRBS team developed seven additional adaptation portfolios during the decision framing step and evaluated those portfolios in the evaluate strategies across futures step, prior to any trade-off analysis. Using the expanded vulnerability analysis presented in this case study, another iteration of RDM could suggest modifications to the portfolios that would address vulnerabilities and result in more-robust strategies.

Next, we provide a detailed discussion of how the different elements of RDM were used in the SSJRBS. In addition, you will be able to explore interactive tools that were developed for this case study and that showcase how we applied RDM techniques to develop a more robust evaluation of strategies.

RDM Step: Decision Framing

The key elements relevant to the SSJRBS analysis can be summarized using an XLRM matrix (see Table 1). See the RDM overview for more information about the XLRM matrix. We elaborate on each element in the following sections.

Table 1. XLRM Matrix for Sacramento–San Joaquin River Basin Case Study

(X) Uncertainties	(L) Management Options and Strategies
Climate drivers Demographics and land use	Current management Seven portfolios Delta conveyance and restoration Expanded water storage Flexible systems operations Healthy headwaters and tributaries Least cost Regional self-reliance Water Action Plan
(R) Relationships or Systems Model	(M) Performance Metrics
Integrated WEAP-PGM-CV and CalLite-CV model	Water delivery: Total unmet demands End-of-September Sacramento Valley storage CVP/SWP Delta exports Water quality: Delta salinity End-of-May Shasta storage Hydropower: CVP Net Hydropower Generation Flood control: Folsom Reservoir flood control pool Recreation: Lake Oroville surface area Fish and wildlife habitats: pelagic species Delta habitat ESA species: adult salmon migration in Delta Flow-dependent eco-resiliency: Sacramento River floodplain flow processes

NOTE: The study also assessed water temperature as a facet of water quality, but this metric was not included in the main set of metrics ultimately presented to stakeholders. CalLite-CV = Central Valley Water Management Screening Model–coefficient of variation. CVP = Central Valley Project. SWP = State Water Project. WEAP-PGM-CV = Water Evaluation and Planning System–Plant Growth Model–coefficient of variation.

Uncertainties (X)

As population increases, municipal, commercial, and industrial water demands typically increase. These demands are dynamic and depend on a variety of factors, such as urban development and land-use density. Agricultural demand is also influenced by socioeconomic trends, but to a lesser degree. As shown in the upper-left quadrant of Table 1, the SSJRBS team was concerned with climate drivers (precipitation, temperature, solar radiation, atmospheric humidity, windspeed, and carbon dioxide) and changes in water use driven by shifting demographics and land use. The SSJRBS team developed five future scenarios for the period from October 2014 to September 2099 using a transient approach in which the climate and socioeconomic factors change as the simulation moves through time.

The SSJRBS team developed climate projections from the most recently available global climate change simulations of the Intergovernmental Panel on Climate Change (IPCC 2014) [PDF] and the Coupled Model Intercomparison Project Phase 5 (CMIP5) to characterize a wide range of future hydroclimate uncertainties. By using five representative future climates, the SSJRBS team sought to efficiently assess the impacts of a range of potential climate futures without having to perform a large number of simulations. For each of these future climates, projections were developed for temperature, precipitation, and other climate characteristics from 2014 through 2099. This approach differs from that used in the CRBS, which evaluated the specific projected time series of precipitation and temperature for the full GCM ensemble.

The legends in tabs 2 and 3 of Visualization 1, which were developed for this case study, show how the SSJRBS team generated the following five climate scenarios:

hotter, drier (Hd)
hotter, wetter (Hw)
warmer, drier (Wd)
warmer, wetter (Ww)
central tendency (Ct).

Tab 1: All Climate Projections presents the ensemble of 112 precipitation (horizontal axis) and temperature (vertical axis) trends derived from GCMs used as a starting point by the SSJRBS team. In Visualization 1, each blue circle represents the projection from a single GCM for the year 2050.

From these trends, the SSJRBS team developed five climate scenarios that reflect the range of changes in climate characteristics described by the ensemble of 112 GCM simulations. As shown in tab 2: Select Representatives, the SSJRBS team found the 10th- and 90th-percentile changes in both temperature and precipitation and developed four representative scenarios at the joint temperature-precipitation changes at these percentiles. The fifth scenario represents the 50th percentile of both temperature and precipitation and lies in the center (central tendency). These scenarios are shown as colored squares in tab 2. For instance, the green square in the lower-right corner reflects a wetter future (the 90th-percentile change in precipitation) with less warming (the 10th-percentile change in temperature) (warmer, wetter).

As illustrated in tab 3: Define Climate Changes, the SSJRBS team then used the GCM simulations nearest to each of the scenarios to inform the development of annual temperature and precipitation estimates throughout the basin for that scenario. For the four scenarios built on 10th and 90th percentiles, the study used the ten nearest GCM simulations. For the scenario at the 50th percentile of temperature and precipitation, the SSJRBS team used all of the GCM simulations between the 25th and 75th percentiles.

Visualization 1. Deriving Representative Climate Scenarios for the Sacramento–San Joaquin River Basin Case Study

The SSJRBS team also developed the following three demographic and land-use scenarios:

Current trends (CT)—This scenario was used as a baseline for comparison and projects the trend of current population growth and land-use changes. The California Department of Finance population projections, which go from the present day to 2050, were extended to the end of the century by the SSJRBS study team.
Expanded growth (EG)—This scenario assumes a high population growth rate and a low urban density, which leads to expanding urban development and reduced agricultural land use.
Slow growth (SG)—This scenario assumes a low population growth rate and a high urban density, which leads to a slower rate of urban expansion.

These scenarios were based on information developed for the California Water Plan (CWP) Update 2013, which created nine conceptual growth scenarios from a combination of three population growth and development density assumptions. The SSJRBS team used three of the CWP conceptual growth scenarios to develop projections combining different assumptions about the rate of population and land-use changes. However, the analyses for the EG and SG scenarios were not available for this case study.

Performance Metrics (M)

The SSJRBS team evaluated the performance of the water management system using 13 metrics, which they grouped into eight resource categories. These metrics are described in Table 2 in alphabetical order by resource category and then by metric name.

Table 2. Metrics Used in the SSJRBS, Organized into Eight Resource Categories

Resource Category	Metric Name	Description
ESA species	Adult salmon migration in delta	Reverse (negative) flows in the Old and Middle rivers indicates that higher-salinity water from the Bay is being drawn into the interior Delta, which is correlated with increased entrainment of adult salmonids migrating to spawning habitat.
ESA species	Cold water pool	Shasta storage at the end of September and April indicates the availability of cold water to support populations of listed salmonids and other fish species.
Fish and wildlife habitats	Food web productivity	Reverse (negative) flows in the Old and Middle rivers indicates that higher-salinity water from the Bay is being drawn into the interior Delta, which disrupts food web productivity.
Fish and wildlife habitats	Pelagic species Delta habitat	Pelagic species are fish that live and spawn in open water in the estuaries of the Bay-Delta. Spring Delta outflows above target levels have been shown to benefit these fish populations.
Flood control	Folsom River flood control pool	As reservoir levels increase, there is a decrease in the availability of storage to control floods. Reclamation is required to maintain reservoir storage levels below the flood conservation pool.
Flow-dependent eco-resiliency	Sacramento River floodplain flow processes	Floodplain processes are important to create and maintain the riparian habitats that support numerous aquatic, terrestrial, and avian species in the Central Valley.
Hydropower	CVP net hydropower generation	This is the difference between hydropower production and use. Generation increases in wet years, while use declines in drier years, because less power is needed to make water deliveries.
Recreation	Lake Oroville surface area	Reservoir surface area reflects the potential for recreational use.
Water delivery	CVP/SWP Delta exports	This is the amount of water exported from the CVP and the SWP from the Delta to regions in the south.
	End-of-September Sacramento Valley storage	A target minimum volume of water at the end of September helps provide resilience to offset reduced precipitation in future years.
	Unmet demand	Unmet demand is the difference between total agricultural and urban water needs and the supply available from surface water, groundwater pumping, and water recycling.
Water quality	Delta salinity	Salinity measured at key points in the Delta is an important indicator of water quality.
Water quality	End-of-May Shasta storage	This reflects the “cold water pool” available to support aquatic habitats below major reservoirs during the hot summer and fall months.

Because each of these metrics has significantly different units (e.g., some are percentages, some are in thousand acre-feet [TAF]), and because an increase in value is better for some metrics (e.g., net hydropower generation) while a decrease is better for others (e.g., total unmet demand), the SSJRBS team and we in this case study both normalize the units to a common frame. The SSJRBS team examined the performance of each portfolio in two ways. First, they calculated the performance of each portfolio relative to current conditions (i.e., reference climate). This was represented by the performance of the No Action portfolio in the reference climate. Second, they calculated the performance of each portfolio relative to the No Action portfolio in each of the five climate scenarios.¹

Our analysis normalizes the metrics in a manner that is consistent with the first approach taken in the SSJRBS; it allows for a single benchmark (current conditions) against which to compare each portfolio. First, we use as a baseline the performance of the No Action portfolio in the current reference climate. Second, we calculate the performance of each portfolio in each of the five other climate futures as a percentage change from this baseline (i.e., [performance – reference] / reference). Third, we adjust the sign as needed so that increasing values are always better.

Thus, if the No Action portfolio has 7,353 TAF of unmet demand in the reference climate and the Least Cost portfolio has 4,049 TAF of unmet demand in the central tendency scenario, then the relative performance is (4,049 – 7,353) / 7,353 = 45 percent. This means that, in terms of unmet demands, the No Action portfolio in the central tendency scenario is 45 percent better than the baseline.

Management Options and Strategies (L)

As with the CRBS, the SSJRBS team began by analyzing a baseline management strategy that involved no additional actions or changes to the existing management strategy. They also developed the following seven alternative adaptation-focused portfolios, which combine a variety of different water management approaches related to demand reduction, supply increase, operational capabilities, resource stewardship, and institutional flexibility:

Delta Conveyance and Restoration is designed to improve Delta export reliability by developing a new Delta Conveyance Project in combination with improved environmental actions in the Delta. These actions include both alternative Delta conveyance and water management actions needed for Delta restoration objectives.
Expanded Water Storage and Groundwater seeks to improve water supply reliability through new surface water storage and groundwater management actions. These include increased surface storage in higher elevations of watersheds, expanded reservoir storage in the Sacramento and San Joaquin Rivers Basin, and conjunctive use with increased groundwater recharge.
Flexible System Operations and Management includes actions designed to improve system performance without constructing new facilities or expanding the size of existing facilities. These actions include conjunctive use management with increased groundwater recharge.
Healthy Headwaters and Tributaries includes adaptation actions that improve environmental and water quality in the Central Valley and upper watershed areas. These actions include additional spring releases that resemble unimpaired runoff and additional Bay-Delta outflows² in the fall to reduce salinity.
Least Cost includes water management actions that either improve system operations at minimal cost per acre-foot of yield or provide additional yield efficiently. These actions include improvements in both urban and agricultural water-use efficiency, increased surface and groundwater storage, and Delta conveyance.
Regional Self-Reliance is intended to include regional actions that either reduce demand or increase supply at a regional level without affecting CVP and SWP project operations. These actions include improvements in urban and agricultural water-use efficiency and conjunctive use with increased groundwater recharge.
Water Action Plan includes all water management actions in the California Water Action Plan. Essentially, this portfolio includes all of the actions in the other portfolios.

Table 3 from the SSJRBS shows specific actions included in each portfolio.

Table 3. Actions Included in Each SSJRBS Portfolio

SOURCE: Adapted from Reclamation, 2016, p. TR-276, Table 6-10.

Relationships or Systems Model (R)

The modeling approach and analysis tools for the SSJRBS were developed as part of the CVP Integrated Resources Plan (IRP) [PDF] and the Sacramento and San Joaquin Rivers Basin Climate Impact Assessment Report [PDF] and were further improved for the SSJRBS. The Water Evaluation and Planning model of the Central Valley’s (WEAP-CV’s) hydrology model was used to simulate water supply and demand in each portfolio (L) under each of the uncertain scenarios (X). These results were used as inputs to the CalLite-CV model to simulate how the CVP, SWP, and other water management systems operate to meet urban, agricultural, and environmental needs. The SSJRBS team used the results from the CalLite-CV model as the basis for the supply and demand imbalance analysis and as inputs to other performance assessment tools. ³

RDM Step: Evaluate Strategies Across Futures

This step of RDM evaluates strategies across futures. In the case of the SSJRBS, the study team first assessed the performance of the current management strategy across the 15 climate and land use and demographics futures using 13 metrics. The SSJRBS used static graphics, such as the tornado graph shown in Figure 3, to show how the system would perform with respect to different metrics across the different climate scenarios.

Figure 3. Evaluation of the Current Management Strategy Across Climate Scenarios

SOURCE: This figure was provided to the authors by Michael Tansey of Reclamation.

Visualization 2, which was developed for this case study, shows each value for the normalized performance of a particular portfolio on a particular metric. That is, each value in the table is a percentage change in performance relative to the No Action portfolio in the reference climate. For all metrics, positive values indicate an improvement in performance.

Tab 1: Exploring Data shows performance in the central tendency scenario. Tab 2: Comparing Across Scenarios expands the viewable data to all five climate scenarios considered by the SSJRBS team. (Although the SSJRBS team considered three demographic and land use scenarios in addition to five climate scenarios, we were provided data only on the current trends scenario for this case study.)

Visualization 2. Projected Outcomes for Eight Water Management Portfolios

Because of the significant ways in which outcomes can be viewed across many metrics, strategies, and futures, RDM studies typically use interactive visualizations to help explore the results of large ensembles of data. In the next step, we provide additional interactive visualizations that allow the user to explore and interpret the evaluation data produced by the SSJRBS team and normalized by our team.

RDM Step: Identifying Vulnerabilities

An RDM study typically uses a vulnerability analysis to help understand the conditions under which a strategy performs poorly and when an alternative strategy might be warranted. The SSJRBS team described key findings based on the data shown in Visualization 2 but did not perform a formal RDM vulnerability analysis. In this section, we conduct a limited post hoc vulnerability analysis to demonstrate how one might approach it even with limited data.

The first step of a vulnerability analysis is to consider whether a portfolio meets or fails to meet decisionmakers’ goals with respect to a metric. That is, does the portfolio satisfy a performance threshold? For this case study, we have set as the threshold the baseline performance: the performance of the No Action portfolio in the reference climate. If a portfolio meets or exceeds the baseline performance, we say that it satisfies the criteria.

A second key question is whether a particular portfolio performs the best (or nearly the best) relative to others for a given metric in a given climate future. We can measure the regret of a particular portfolio as the difference between its performance and that of the best-performing portfolio.

For this case study, we implement a simple vulnerability analysis using the existing SSJRBS results. We first recast the raw numerical output shown in our normalized data table (Visualization 2) in terms of meeting or not meeting thresholds defined for each metric. This analysis helps convey metrics for which different portfolios lead to acceptable results. Next, we consider how performance varies across the different climate scenarios in terms of acceptable results and the best-performing portfolio. Finally, we identify vulnerabilities—i.e., conditions under which a portfolio fails to meet decisionmakers’ goals in terms of the climate uncertainties.

Performance Across Metrics

Visualization 3, which was developed for this case study, shows which portfolios (columns) are satisficing, which are the best-performing/lowest regret, and which are both for each metric (row) in the central tendency climate scenario.

For example, when looking at satisficing portfolios, we can see that no portfolio meets the performance threshold for pelagic species habitat or Delta salinity, while every portfolio except No Action meets the performance threshold for CVP net generation, adult salmonid migration, and unmet demands. The results in other metrics vary across the portfolios, and no portfolio meets the performance threshold for all metrics.

Because multiple portfolios meet the threshold for each metric, the user can highlight the best-performing portfolio for each metric, which is shown in the visualization as large squares and is equivalent to the portfolio with least regret. For example, the Water Action Plan is the best portfolio for reducing unmet demand, whereas Healthy Headwaters is best for flood control. You might choose a different portfolio depending on which metric is most important to you.

You can also move the Performance Tolerance parameter to the right to highlight portfolios that are almost as good as the best portfolio (i.e., have low but not zero regret). For instance, a tolerance of 10 percent means that all portfolios that perform within 10 percentage points of the best portfolio will appear as large squares.

The table can also show both types of information: which portfolios meet performance thresholds (red/green) or are the best or near best (large squares) for each metric. You can use the checkboxes on the right of the visualization to show only those metrics that are important to you.

Visualization 3. Satisficing and Best-Performing Portfolios in the Central Tendency Climate Scenario

Performance Across Metrics Under Uncertainty

We next introduce the performance of portfolios across metrics in different climate futures. In Visualization 4, which we developed for this case study, we have selected five metrics that represent a cross-section of resource concerns. The performance of each portfolio is shown both for the central tendency scenario and for the four other climate scenarios.

We can observe a few things. First, every portfolio meets the adult salmonid migration metric threshold, while no portfolio meets the Delta salinity metric threshold, except in the most forgiving warmer and wetter climate. As a result, we might remove these two metrics as decisionmaking criteria. You can do this by deselecting these metrics from the list of five at right.

With only three remaining metrics, we can see trade-offs more easily. For instance, the Regional Self-Reliance portfolio is robust for the recreation metric, and the Least Cost and Water Action Plan portfolios are both robust to climate for the water deliveries metric. We might choose different portfolios depending on which metrics are our priorities.

The second tab allows users to explore any combination of portfolio, climate, and metrics to identify trade-offs.

Visualization 4. Portfolio Trade-Offs for Different Climate Futures

Identifying Vulnerabilities

We now answer the question, “Under what uncertain future conditions does a portfolio fail to meet decisionmakers’ goals (i.e., meet or exceed the baseline performance)?” These conditions represent vulnerabilities of the current management strategy. With precipitation and temperature change as the uncertainties being explored in our analysis, we might look for specific thresholds for precipitation and temperature change that lead to acceptable versus unacceptable outcomes.

We can start by exploring the performance of portfolios under different precipitation and temperature changes, as shown in the next visualization, which was developed for this case study. Visualization 5 allows you to explore and compare the performance of up to four portfolios on a particular metric, plotted against the five climate scenarios. Each portfolio is shown as a subplot with five marks, which represent the temperature and precipitation change in a particular climate scenario. The color of the mark indicates whether the portfolio was successful according to the selected metric under those conditions. For this visualization, we say that a portfolio is successful in a climate scenario (shown with a green mark) if it meets the performance threshold for the metric or, if it does not, it is the best or nearly the best portfolio (i.e., within 5-percent performance tolerance) for that metric.

With five futures and reservoir storage as our metric, the Flexible System Operations and Management portfolio appears to perform the best, with successful performance in four of the five climates. Delta Conveyance and Restoration appears to be the next best, with successful performance in three of the five climates. The Least Cost and Expanded Water Storage portfolios appear to perform similarly, with successful performance in the same two climates.

Visualization 5. Comparison of the Performance of Four Portfolios on a Single Metric in Each of the Five Climate Scenarios

However, these plots are sparse. It is difficult to identify the full breadth of conditions under which a portfolio succeeds or fails, because there is much unexplored uncertainty space. A vulnerability analysis of the kind done in the CRBS is not yet possible without additional model simulations. If such an effort could explore the performance of the portfolios over a fuller range of GCMs, it would add more density to these plots and enable a more robust vulnerability analysis.

In Figure 4, we have added artificial marks to illustrate what a denser analysis of the portfolios’ performance across more climate conditions might reveal. For instance, with these added “data,” Expanded Water Storage appears to be the most robust portfolio, performing well across the range of climate conditions, with a few exceptions in some specific futures. In contrast to the sparser plots in Visualization 5, in Figure 4, the Flexible System Operations and Management and Least Cost portfolios appear to perform similarly, succeeding as long as precipitation does not increase. Finally, the Delta Conveyance portfolio appears to perform the worst, performing successfully only when precipitation declines significantly. Richer data, such as these, could contradict inferences made with sparser data. Additional insights could be gained if the futures also included demographic and land use changes, which would paint a still richer picture of how different portfolios can manage future risk.

Figure 4. Climate Uncertainty: Illustrative Example

Illustrative Climate Uncertainty

NOTE: This figure includes hypothetical data.

RDM Step: Trade-Off Analysis

The SSJRBS team described some key trade-offs based on a review of tornado graphs like the one shown in Figure 4. For this case study, we explore how RDM techniques can expand on these findings. Although a complete RDM-style vulnerability analysis is not possible, we can cautiously consider key trade-offs by further summarizing performance. Specifically, the next interactive table that we developed for this case study shows the number of climate scenarios in which a portfolio is successful for each metric (see Visualization 6).

In tab 1: Simple trade-offs, you can see that no portfolio is robust across all five climates for all three remaining metrics, so decisionmakers must make trade-offs among their priorities. For example, the Least Cost portfolio is successful in meeting CVP/SWP exports in all five climate scenarios—making it robust to climate uncertainty as explored in this analysis for that metric. However, it succeeds in flood control for only two climate scenarios, making it potentially more vulnerable to climate uncertainty for that metric.

In general, there appears to be a trade-off between flood control and water deliveries: Portfolios that perform particularly well on flood control (e.g., Flexible System Operations and Management and Healthy Headwaters and Tributaries) do not perform well on water deliveries, and vice versa (e.g., the Least Cost and Water Action Plan portfolios perform well on water deliveries but not flood control). This might be because, to make water deliveries, the reservoirs need to be filled in the winter and spring and then emptied in the spring and summer. In contrast, flood control benefits from inverse conditions: low reservoir storage in the winter and spring, which occurs in the Healthy Headwaters and Tributaries portfolio. In tab 2: Trade-offs, you can explore performance trade-offs for any set of metrics and climate scenarios.

As it is used here, this approach of counting futures is meant to illustrate trade-offs between strategies that are otherwise hard to see. However, this approach of futures counting should generally be avoided because it obscures information about the conditions in which a strategy works well or poorly. In other words, it is not a substitute for the vulnerability analysis step of RDM.

Visualization 6. Portfolio Trade-Offs Across Metrics Based on Counting Futures

RDM Step: New Futures and Strategies

The SSJRBS team developed seven portfolios to compare with the No Action strategy in a small set of climate futures. Another iteration of RDM could consider more climate futures and other uncertainties, including the demographic and land-use scenarios.

Additional management strategies could be developed that seek to combine aspects of the best-performing portfolios. For example, this case study highlighted that the Least Cost and Regional Self-Reliance portfolios perform well in roughly the same set of metrics. Healthy Headwaters performs poorly on those metrics but well on the remaining ones. Analysts might consider whether actions taken under these individual portfolios could be combined into a new portfolio that performs well across all climates and all metrics. This might not be possible if actions are mutually exclusive, but such comparisons as these can produce new strategies for achieving policymakers’ goals.

Conclusion

This case study describes how the SSJRBS team used DMDU methods to explore how uncertainty about future climate and demographics would affect the river basin under current management and several select portfolios. The analysis showed that the future has the potential to seriously stress the basin and its users.

Instead of evaluating a large set of futures to reflect future climate uncertainties, however, the SSJRBS team defined five different futures. This case study shows that this approach was successful in identifying how current management might perform in widely varying futures, but it also showed that the five futures left gaps in understanding about the strengths and weaknesses of strategies in the full range of conditions that might come to pass. The development and evaluation of more futures could reveal key vulnerabilities. The case study also suggests how the challenge of many diverse metrics can be handled. That is, many methodologies struggle to accommodate diverse goals and criteria in an analysis. Many methods try to compress multiple goals into a single score by using a utility function with weights across multiple objectives. This has the advantage of producing a single performance measure, but at the cost of making it difficult to understand how different underlying goals were traded off.

The case study suggests a different way in which the analyst and stakeholder can evaluate the performance of different portfolios across many diverse performance metrics. For example, this case study shows that no single strategy can meet all of the goals for the basin, and, therefore, decisionmakers might need to make difficult trade-offs. It also shows (1) how analysts can select a handful of metrics that might be representative of and correlate with a larger set of metrics and (2) how some metrics can be omitted if all strategies perform similarly, because such metrics offer little in distinguishing between strategies and thus have lower relevance to a decision. Finally, this case study shows how performance on a particular metric can be viewed in the uncertainty space—in this case, in a plot of precipitation and temperature—to identify future conditions in which strategies are vulnerable for particular metrics. These kinds of metric-selection and -consolidation techniques, coupled with interactive visualizations, can help decisionmakers home in on the most-salient information for comparing and choosing among their options.

Case study authors: Nidhi Kalra and David Groves

Acknowledgments: The authors would like to thank Michael Tansey of Reclamation for providing background information on and data from the SSJRBS and for his review of this case study and helpful suggestions.

Footnotes

Federal planning requires a comparison of a future with no action to a future with actions and a characterization of uncertainties. For the case study, this included future climate and socioeconomic conditions. (Return to text)
Bay-Delta outflows refers to the amount of fresh water from the Sacramento and San Joaquin Rivers that exits the Bay-Delta through the Golden Gate Bridge. Higher outflows generally reduce the salinity of the Bay-Delta. (Return to text)
Although it is not the focus of this case study, a unique aspect of the SSJRBS was the simulation of agricultural water demands using a biologically based Plant Growth Model (PGM) to calculate crop water use, including the effects of increasing temperature and changes in solar radiation, atmospheric humidity, and carbon dioxide. Another benefit of the PGM model was the estimation of yield changes under climate change. (Return to text)