In the book, "Dangerous Company," authors James O'Shea and Charles M. Madigan discuss the consulting powerhouses and their effects on their clients. On the last two pages of this interesting book some guidelines are given for successfully using consultants. We have modified and extended those guidelines to help those considering the use of simulation consultants.

Simulation is increasingly being used to provide insight and analysis of complex systems and problems. The authors have managed simulation consultants for many years and don't know many consultants that would be "dangerous company." But we do believe that consumers of simulation services will be more successful if they consider the following.

• Do you understand everything about the problem? Is further research required before the problem can be adequately defined? It probably doesn't make sense to build a model of a system where important pieces of the system are unknown or ill defined. It is possible to get started and fill in missing information later, but this almost invariably results in additional project costs.
  
  
• What do you want the consultant to accomplish? The first rule of consulting is to know the desired goal. Carefully determine and document exactly what you expect of the consultant. What is the scope (breadth) of the system to be modeled? What level of detail (depth) is expected of the model logic and capabilities? The scope and detail of a model are specific and different for every project, and is perhaps the most important decision made early in the process. Scope and detail decisions are made primarily by the questions that the model is expected to answer. Careful attention must be paid to these decisions and their implications affect the model's robustness.
  
  
• What are the bounds of the consultant's responsibility? Do they first need to develop a document defining the system (if you haven't already)? Be clear on stating what tasks they will complete vs. what tasks you expect to complete or support internally.
  
  
• How should the model be tested? How much analysis is expected? Who can provide the system expertise needed for the tests? In some projects, the consultant will complete the simulation experiments and submit them along with a report to you. Directing the consultant to guide you through the process of producing and interpreting the results will yield much greater benefit from the process. Completing a written test plan at the beginning of the project will avoid disagreements over what constitutes sufficient testing in later project phases.
  
  
• Is simulation the correct technology to solve your problem? What are the alternatives? Look closely at the problem definition and determine if there are other, simpler ways to solve the problem (see "For Further Reading"). A simulation consultant might be unfamiliar with alternatives, so you should resolve this question before calling upon them.

Frequently, inexperienced consumers of simulation ask for or expect the model to develop optimal solutions to their problems. A simulation model can test a proposed solution or compare two or more solutions, but by itself it cannot provide the optimal solution except in few, limited cases. (Heuristic optimization capability has been added to several simulation software, but their capabilities are limited currently to answering well-defined questions about the use of resources, not about the appropriateness of the control algorithms. We anticipate increased capability of these types of products and improvements in the time required to obtain a solution.)

• What are the characteristics of a successful solution? Knowing what you are really trying to accomplish is essential to a successful consulting project. One of the frustrations that consultants have is working with companies where different levels of management or different groups in the company have differing objectives. This almost guarantees that the results will not be accepted, and the project may be regarded as not successful.

But for many problems you don't have time to develop the internal skills. You might consider having an internal engineer work closely with the consultant as a training experience. This could extend the project schedule, but ought to be considered if you intend on developing internal simulation capability in the future.

The consultant will never know as much about your operations as you (collectively) do. In fact, no one person knows everything about the process; many people need to be interviewed to generate a complete understanding of a system. Good simulation models of complex problems must include all of the interactions of the individual elements of the problem or system. If you use a consultant you will need to document and teach the consultant about these details. If you don't have the time to collect, document and review the accuracy and completeness of this information in advance (and few projects do), then plan on having the consultant do so (and be prepared to pay for it) as a first step in their engagement.


What kind of simulation consultant do you want?

Consultants come in many varieties and levels of competency. What is important to you ought to determine with whom you will work. Do you need more than model building (i.e., engineering or design assistance, project management, or data collection)? Do you need someone already familiar with your industry and problems? Important considerations might be schedule availability, total cost, related experience, other expertise and referrals from previous related projects. Academics (professors or graduate students) are generally less expensive, but usually work according to their own schedule rather than yours (consulting is not their primary business). Sole practitioners may also be less expensive, but they may or may not have access to the necessary resources to handle a large project.

Another possibility is to use the consulting staff associated with one of the simulation software companies, particularly if you might be purchasing their product. They'll be extremely knowledgeable about the product, and you might work a lower cost combining consulting and software purchase price. Software-specific consultants use their software of choice for all problems, but know that software very well.

There are also independent firms providing simulation consulting, either as a primary business or related to their other activities. These firms are usually independent, and can offer help and guidance if you haven't yet decided or don't know which simulation package is best suited to your needs.

Considerations in selecting a simulation consultant might include the following:

• Is this their primary business?
  
  
• How many years experience in simulation do they have?
  
  
• Do they have the necessary technical resources (people) and software?
  
  
• With which software do they have expertise?
  
  
• Are these appropriate to your needs?
  
  
• Have they completed similar projects?
  
  
• Can they provide references that you can contact?
  
  
• Who will actually do the work? (Make sure that you meet the project analysts, not just the spokesperson.)
  
  
• Do they have a well-defined project methodology?
  
  
• What do they propose to do?
  
  
• How do they price a project?

The answers should make sense for your requirements and situation. Related experience usually results in a higher quality project in less time. It may actually cost less overall (their daily rate may be higher, but they may be able to complete the project faster). And, it offers the advantage of the consultant having the experience to offer suggestions based upon other similar projects.


What does the consultant propose to do?

Every consultant should provide a written proposal, which should include a clear project methodology and responsibilities. Experienced consultants should be able to provide a budget price (if not a fixed price) and schedule from a few hours on the phone with you, and possibly some system drawings you supply. Many consultants will work only on a per hour or per day rate; others do most of their work fixed price. Some consultants provide a powerful guarantee: "Pay us only if you are satisfied."

After cost, schedule and experience, the following should also be considered:

• Is the cost greater than possible savings?
  
  
• Is the size of the consultant's organization appropriate to your project needs?
  
  
• If someone is ill is there capacity to fill in?
  
  
• What will it cost you to support the consultant (your people and other resources)?

Most projects evolve as insight is gained from the simulation process. How will this evolution be handled in project scope, responsibility and pricing? Address this with your consultant before beginning the project.

You might also consider soliciting bids from several consultants. In order to conduct a fair comparison, you will need to provide them with a detailed written project requirements document. However, many companies need the assistance of a consultant just to develop such a document!

Most modelers are not system designers. There is a significant difference between consultants who are merely "model builders," and those who are true consultants with experience in the design and implementation of systems. A true consultant who is already familiar with your industry will quickly learn your system and understand your problems, and offer informed suggestions or guidance in setting the project objectives and model expectations. They will be of great assistance in guiding the analysis and interpreting the simulation results. A "model builder" generally will only provide expertise with the mechanics of building the model � the client must supply all the rest. Determine ahead of time what your needs are and find a consultant who matches them.


Is the proposed solution unique to us?

Many companies experience similar problems. If a consultant has experience with your type of problem, he or she may have developed special tools to solve your problem quickly. This should result in a shorter schedule, and may result in a lower overall project cost. However, you are not just paying for some number of hours of consulting, you are also paying for the benefit of the consultant's expertise. Such expertise usually commands a higher daily charge rate, but the efficiencies due to it may actually result in a lower project cost.

Often, an experienced consultant will have completed previous models for problems similar to yours. These can be used as a starting point for your model and can help shorten the project schedule. (Note: In some highly competitive industries, the consultant's previous agreements may prohibit them from using previous models.)

However, make sure that the final model is specific for your problem, and includes all the necessary details that are specific to your situation. Rarely is a previous "off-the-shelf" model suitable for a new project with no changes.


How do we manage the consultant?

Some companies are of the opinion that a good consultant should be self-managing and given free rein in a project. Unless you have had a lot of experience with a consultant and gained a high level of trust with them, this is probably not a good approach.

Simulation projects require careful planning, review and execution. Begin with a kickoff meeting with all parties that are involved or who have a stake in the results. Insist on a project or model specification document that clearly indicates the design requirements of the model, inputs and outputs, project schedule and responsibilities. The plan should include progress reports or � better yet � meetings to review progress and discuss issues. The nature of simulation projects is that they evolve, and often grow in scope. New information must be jointly reviewed to determine how it affects the model design, assumptions and costs. If there is a disagreement over what needs to be done, first listen to the consultant's advice, then remember that you're paying the bills, so you're calling the shots.

Consulting agreements need to be very specific to avoid disagreements over scope, costs or responsibilities. Questions that must be answered include the following:

• Who is to complete the data collection?
  
  
• Is the analysis limited to a specific number of days?
  
  
• Who runs the model experiments?
  
  
• Who conducts the final presentation?
  
  
• Is there ongoing support required if you elect to continue to use the model after the project is completed?
  
  
• Who owns the model?
  
  
• Will the source code be delivered?

There is no right answer to these questions, but they should be determined and agreed upon as part of the written engagement.


How involved in the project do you want to be?

Do you buy a model and do the analysis or do you let the consultants do all of the work, giving you a report and recommendations at the end? There are situations where all that you are looking for is a single answer, and you have confidence in the consultant from previous work, where you might not need to be involved with the project until the end. We would still urge you to begin with a well-defined and documented problem statement, and set of objectives.

However, over many years and hundreds of projects, we have observed that the more involved the client is with the project, the better it turns out and the more the client gets out of the process. You will not be able to predict at the beginning of a project where the model results will lead. Design refinements, new scenarios to analyze, unexpected side effects � all can lead to an expanded role for the model. It often becomes a tool that you will use in the future. Would you choose to be uninvolved with the architect who you have hired to build your home? Probably not. You would work closely with the architect, the interaction resulting in a far better and more satisfying end result. Simulation modeling projects should be executed in the same fashion.


What do we do if things aren't going as we expected?

The best way to determine if things are going as planned is to have a project road map to follow. This should be the purpose of the project requirements document, which is completed during the first phase of the project and generally follows the formal methodology the consultant uses. The project plan should include several intermediate review dates and what tasks are expected to be completed at each milestone. You probably can't expect to understand every activity that a consultant is doing, but if you define and monitor appropriate milestones, you will avoid many potential problems.

The first milestone is always the production of a very detailed design document describing the simulated system flow, logic, explicit definition of all inputs to and outputs from the model (with sample screen shots), and assumptions as to what level of detail the model will include. Milestones should also include intermediate meetings for demonstration of portions of the model (where they can be separated or completed independently) using test data for a contrived scenario that should produce known results.

Generally, critical problems result from inappropriate project expectations or objectives (poor communication between the customer and the consultant), falling behind schedule, and the inability to test and validate the model. Properly constructed project requirements and design documents should avoid the first two problem types. Selecting an experienced and highly recommended consultant (rather than selecting only on the basis of availability or cost) is your best insurance against the last problem type.


How do we make sure that the promises of the consultant have been achieved?

We encourage you to follow these simple but very important rules that should be completed before proceeding to the analysis phase at the end of the project:

• Complete a written project specification.
  
  
• Stay involved during the project.
  
  
• Determine how the model will be validated (demonstrated to reflect the real system).
  
  
• Complete a model test and verification plan (as part of the project specification). This plan should spell out how many scenarios are to be simulated, and who is to collect and validate the data for each scenario.

Do not proceed with development or data collection until a written specification has been developed, reviewed and approved, no matter how great the time pressure of the schedule might be � it will only lead to problems later.

Lastly, allow "contingency" time in your project schedule for unexpected or additional tasks at the end. This may be the result of flaws uncovered in the model testing phase that require time to correct and retest. It might be necessary to collect or reconfirm key data elements that were overlooked. Or it could be from problems discovered or additional questions raised with the system design as a result of the simulation analysis.


How much involvement with the consultant, if any, is required after completion of the project?

In most successful simulation projects, when the project is over it is not really over. Typically, additional analysis questions are brought up at the end of the project by the client when the useful the model is seen. In the consulting agreement, you might wish to indicate how this will be supported. Identify exactly what is included in the scope of the agreement, at what point the end is reached, and what the costs will be to continue with further analysis.

Will training in the utility of the model be included? Who will write the final project report? Will the consultant be needed to assist with management presentations? If you purchase a license for the simulation software and intend to continue further analysis or model refinements internally, you will want to indicate these responsibilities in the agreement at the beginning of the project. You will want to include a support period after completion of the base project (at least 90 days), during which the consultant will be "on call" to answer questions. If logical errors are found in the model during this period, the consultant should be responsible to correct them and retest the model at no charge.

Simulation modeling technology can provide great benefit to a wide variety of otherwise difficult problems. Simulation consultants can provide an invaluable service to help you apply this technology, and are most effective when the project is properly defined and the user stays involved.

O'Shea and Madigan conclude "Dangerous Company" with a quote from Machaivelli about seeking advice: "Here is an infallible rule: A prince who is not himself wise cannot be wisely advised. . . .Good advice depends on the shrewdness of the prince who seeks it, and not the shrewdness of the prince on good advice." We think this is excellent advice for simulation consulting projects as well.

Jerry Banks is a professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. He is a frequent author and speaker, and a member of INFORMS and other professional organizations. He was previously a principal in Carson/Banks and Associates, Inc., a simulation consulting firm.

Van Norman is the chairman and founder of AutoSimulations Inc., the developer of the AutoMod simulation system and the AutoSched simulation-based scheduling software product family.

Web Site © Copyright 2020 by Lionheart Publishing, Inc.
All rights reserved.


Lionheart Publishing, Inc.
2555 Cumberland Parkway, Suite 299, Atlanta, GA 30339 USA
Phone: +44 23 8110 3411 |
E-mail:
Web: www.lionheartpub.com


Web Design by Premier Web Designs
E-mail: [email protected]