Software Review
Equity 3
Software provides equipment procurement direction for United Kingdom's Ministry of Defence.
By Ian Mitchell
The United Kingdom's Ministry of Defence (MOD) procures equipment for the three armed services via the Equipment Plan (EP). A key stage is to decide the relative priorities of the equipment being procured in the light of prevailing assumptions about the security environment and financial conditions.
The MOD Equipment Capability Customer (ECC) has a Joint Capabilities Board (JCB) in charge of some 11 Directorates of Equipment Capability (DsEC, but more commonly DECs). The DECs each have their part of the EP comprising equipment "lines" for their projects, such as respirators, armored fighting vehicles, ships, aircraft or weapon systems. They are required to offer up a proportion of their programs as potential savings, and they are invited to propose enhancements.
There is never enough money to procure every piece of equipment that every DEC could aspire to, so there is always a need to prioritize which equipment should be acquired and which should be reduced or abandoned. Equity 3 is a key part of the process, which determines these priorities within the DECs and across the ECC as a whole from the point of view of the JCB. The DECs consider their programs in detail with the stakeholders in their respective areas of capability. The JCB has to decide which capabilities it should advance furthest. A means to develop a joint perspective with an audit trail is essential to obtain a rational approach for this.
The Solution
Decision conferencing seeks to develop a shared understanding of a problem, generate a sense of common purpose amongst the stakeholders and instill a commitment to the way forward. It achieves these aims through a socio-technical process. The social aspect is a professionally facilitated working meeting, attended by key stakeholders. The technical part relies on Equity 3 to generate a model of the decision problem comprising options scored within their areas against weighted criteria. This socio-technical approach makes the facilitation as important as the modeling.
Following experience with the use of Equity-based decision conferences to inform user requirements for future surface vessels and the balance of investment in capability to control and deny the "above water battlespace," its potential to assist in the design of the EP was recognized. Twelve staffers from the ECC including military officers and civilians were trained over 12 days in spring 2001. The first DEC and cross-JCB conferences took place that fall. The approach was a success and is now embedded in the cycle of EP management with DEC conferences in September and the JCB-level conference in November. At both DEC and JCB levels, Equity is run via laptop computer and projector so that all participants can see the descriptions and the data as they are discussed and debated.
A two-person team of an analyst from the DEC staff and a facilitator from outside the DEC conduct the decision conference. Both are trained in the use of Equity.
The analyst's first task is to create a model in Equity 3. A generic example for a notional detection capability area is shown in Figure 1, the cityscape, with three areas comprising two sensor capabilities X and Y and a repair kit area.
Figure 1: Cityscape-area detection.
Equity structures the equipment as options or levels within areas. Cumulative or incremental models, where any up to all options in an area may be selected, lend themselves to the EP's many equipment lines grouped within areas. Equity's other area structure is the mutually exclusive options where one and only one will be selected out of those available in area. There are two views for data input as these differ between the cumulative and mutually exclusive. There is also a mixed option available.
The C icon below the area names indicates that these are cumulative (or incremental) options. A P indicates that the option is a part of the current plan with identified funding, and a + that it is an additional project. The double-headed arrow indicates that the areas have been sorted into benefit: cost (bang for buck) order with the highest at the bottom of the tower.
In the ECC, criteria are mandated centrally to allow consistency sufficient for valid transfer between the DEC and JCB level. The criteria settled on represent the value of the equipment towards operational success in three types of military endeavor. Equity allows multiple criteria to be defined for both costs and benefits. The model criteria are defined by dialogue boxes. Areas and levels are added via menu commands and dialogue boxes
To populate the model with data, matrices are used as shown by Figure 2 for the left area. Two types of cost data are shown — Operating Cost (OC) and Equipment Plan Cost Total (EPCT) — with three benefit criteria for the value of the options in three types of terrain: town, country and mountain.
Figure 2: The input data for the area Sensor X.
Scoring Value
Much discussion at the DEC decision conferences concerns the relative values for the options in each of the benefit criteria. Scoring value, which incorporates subjective viewpoints, can be challenging to many participants more accustomed to hard metrics of effectiveness and performance. To assist, Equity 3 has graphical displays as shown in Figure 3 for the Sensor X area under the town criterion.
Figure 3: Histogram display for Sensor X.
Option 3, Training Sets, is valued twice as much as an extra 1,000 sensors (Option 4). The numeric display allows simple arithmetic to be used by the Balance Beam technique of contrasting scores to check for consistency. The data shows that Option 3, the Training Sets, is valued at 200. This is slightly more than the combined value of Option 2, the extra 1,000 sensors under the planned Option 4 and Option 5's enhanced features put together (75+100+15 = 190). The facilitator has to constantly probe for consistent scoring. The re-scaling feature of the graph to show the highest score allows the relative values of options to be made apparent to the group. The histogram displays help by showing relative value of options. With the earlier versions of Equity the facilitator often would indicate relative value with their hands or simple line displays on a board. Equity 3 does this by directly re-scaling other options so that the full effect is more visible.
The ability to call up the definitions of criteria and options focuses the arguments. The social aspect of decision conferencing is closely linked to focused argument, which the analyst records on-line. Seeing their comments transcribed on screen can help those with strong views understand that these have been noted, allowing discussion to move on.
Figure 4: Weights dialogue.
The weights place the scores on a common scale, allowing fair comparison in terms of cost-benefit for all the options. Benefits in this example are more complex than costs. Weights are set within criteria as shown in the columns above and then across in the bottom row. These can be set automatically based on the length of the ranges of scores using the thermometer icons, but this is usable only where it is known that the scales are common across areas. It is more common for the weights to be set manually via "swing-weighting." This is based on the swing between the lowest and highest points in each area and then across the criteria. With the costs, scores and weights input, the model is ready to be sorted.
Sorting the model marks a turning point in the decision conference. This is the main processing conducted by Equity 3, and it is extremely fast running. This is essential to allow rapid what-if analysis, which underpins the ability to explore implications of contentious scores and weights.
Each area can then be reviewed to see the weighted preference values set against cost as shown in Figure 5. This is a review point where the facilitator will look to identify the furrowed brows suggesting that the result is not accepted by some participants. The key is that the model produced represents the views of these. It does not imply that these views are correct.
Figure 5: Area diagram.
The area diagram in Figure 5 illustrates the characteristic convex slope of options sorted into descending benefit-cost ratio order. The gradient of the triangle formed by the benefit and cost vectors for each option shows the benefit-to-cost ratio. Equity 3 shows the triangles formed by costs and benefits, which assists the participants to understand the results. This is a good point to establish that they have provided scores, which reflected their true views. In the example, option + Type A — although with a lower benefit than the planned training sets options — has lower costs and so a high benefit-to-cost ratio. Some Defence equipment has materially longer service life than other equipment, so annualized costs are used to correct for these variations.
Having reviewed the areas to ensure that they represent the views of the conference, the potential portfolios can be viewed. The green area shows all possible combinations of the options. The top edge of this area comprises the most benefit-cost advantageous options in descending order. This is the frontier and shows the order in which to buy the options in order to maximize the benefit for the cost incurred.
Figure 6: Envelope diagram.
Equity 3 shows the triangles of benefit (Y-axis) against cost (X-axis) of options on the frontier that assists the participants to understand the display. When combined with the cityscape, Equity 3 allows a "walk" up the frontier so that participants can see each option's place in turn.
Seeing what lies "within" the line and what is beyond a given level of funding generates further discussion and debate. Sensitivity analysis allows the group to see the implications on the overall result of individual changes to scores and weights argued for earlier. This interaction is the key to buy-in, especially by those whose favorite options have fared badly in terms of benefit-cost.
The resulting priorities are available as an order of priority listing showing all options sorted into benefit-to-cost ratio is the tangible output.
This list provides a useful guide to the DEC on which options are most preferred by the stakeholders. This informs the DEC's decisions. A much longer list results from the JCB conference but it serves a similar purpose.
The Software
Catalyze's Equity 3 runs on most current and recent Windows operating systems, although older versions of NT should be avoided.
Figure 7: Envelope with Frontier diagram and corresponding cityscape.
Building the model is straightforward, provided an approach exists to structure the issues in terms of areas in which to invest and data for the options in them. Many situations can be represented equally well as incremental or mutually exclusive options according to the view of the analyst.
The key views available from Equity 3 are those used to discuss the inputs and outputs. These are accessed by menu. Inputs are scores and weights but also the comments raised in discussion of the options.
Weights are often allocated via whiteboard techniques complementing the use of Equity views. Again, teasing out the reasons why priorities are different offers material value beyond the production of the order of priority.
Equity can produce all its outputs as a report in HTML, which can then be rendered in Word.
Figure 8: Order of Priority.
The transmission of data between DECs and JCB relied on considerable manual effort. Equity 3 allowed the building of a merged model with the weighted preference scores brought forward to act as inputs to the higher level. It only allowed one cost criterion to be brought forward that constrained the sensitivity analyses available.
Criticisms of Equity
The most common complaint is that cheapest options do too well and the big, expensive options such as aircraft and aircraft carriers end up at the end of the curve. This problem arises from discomfort at scoring the benefits as radically as merited. People are not good at providing consistent relevant values without assistance. The graphics of Equity 3 assist in this, but a willingness to revisit scores and weights is essential if the model is to accurately represent the views of the participants.
The second most common complaint is that the model is subjective. In response, the use of multi-criteria decision analysis recognizes that decision-making ultimately has to be subjective. Equity offers an excellent means to structure debate, inform the decision-making and provide audit trails for the decisions reached.
Advantages of Equity 3
Equity 3 is the lynchpin of the ECC Decision conferencing process. The dialogue boxes allow initial definitions to be input before the conference starts, ensuring a common baseline definition. During the conference, the displays are key in eliciting opinions from a range of people, capturing the views for use in audit trails. The outputs are immediately available to inform sensitivity analysis
The ECC is unusual in the broad scope of its work with Equity. The options with the greatest potential for savings or enhancements in each DEC area are then transferred to a cross-JCB model in their cost-benefit order. This allows the JCB to identify what their priorities are in cost-benefit terms before the constraints of finance are applied in the definition of the proposals to be made to ministers.
At both levels, Equity 3 models illustrate situations to develop a shared understanding of the problems, generate a sense of common purpose amongst the stakeholders and instill a commitment to the way forward. It has done this for three years and is expected to continue to do so.
|
Along with his role as group leader with a research agency of the U.K. Ministry of Defence, Ian Mitchell is vice president of the OR Society.
OR/MS Today copyright © 2004 by the Institute for Operations Research and the Management Sciences. All rights reserved.
Lionheart Publishing, Inc.
506 Roswell Rd., Suite 220, Marietta, GA 30060 USA
Phone: 770-431-0867 | Fax: 770-432-6969
E-mail: lpi@lionhrtpub.com
URL: http://www.lionhrtpub.com
Web Site © Copyright 2004 by Lionheart Publishing, Inc. All rights reserved.