Bonus Technical Article – Source https://source.aacei.org Source Wed, 03 Aug 2022 11:49:10 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 https://source.aacei.org/wp-content/uploads/2020/12/AACE-Site-Icon.gif Bonus Technical Article – Source https://source.aacei.org 32 32 Getting Back to Basics: Update Schedule Review by the Numbers https://source.aacei.org/2022/08/03/getting-back-to-basics-update-schedule-review-by-the-numbers/ https://source.aacei.org/2022/08/03/getting-back-to-basics-update-schedule-review-by-the-numbers/#comments Wed, 03 Aug 2022 04:05:00 +0000 https://source.aacei.org/?p=5869

Getting Back to Basics: Update Schedule Review by the Numbers (Recommended Practice Numbers, that is)

Marina G. Sominsky, PSP
Ronald M. Winter, PSP FAACE
Katherine Baron

Abstract–Performing an update schedule review, submitted by a 3rd party, requires a vastly different skill sets than that of creating a schedule. The scheduler who creates a schedule works with a large number of construction professionals who will provide feedback and guidance. Typically, the scheduler who reviews a 3rd party’s schedule submittal works alone. This process can be quite daunting for the first-time reviewer. The reviewer does not need to work alone as long as there is access to AACE’s incredible library of Recommended Practices and technical papers. This article focuses on the key documents that aid in determining the main topics of the review memorandum, assist in identifying the culprits that make schedule updates non-compliant, as well as provide solid guidelines on how to gauge schedule’s overall quality. The authors hope to inspire and motivate project controls professionals to advance their skill set from being a proficient scheduler to becoming a trusted scheduling consultant. This article was first presented as PS.3648 at the 2021 AACE International Conference & Expo.

Introduction

The owner of a planned project is either a public entity, a private organization such as a corporation, or even a well-financed individual. This entity typically realizes that they do not have the time, knowledge, and experience to construct the project in an optimal manner. For this, they often employ a team of professionals who do have this ability to turn the plan into reality. This article refers to the team hired to perform the work as contractors. The owner creates documents that describe the work to be performed and contracts that work out to a contractor.

If the contractors are responsible for the timeliness, cost, and quality of the work, they must be allowed to make their own decisions on the means and methods of the work. The owner should keep the project at ‘arms reach’ and only monitor the work from afar unless the contract says otherwise. The project schedule is normally the method of communication and reference for decisions on the project.

The contract typically requires the contractor to show their work plan by submitting a project schedule to the owner. Reviewing this schedule usually requires the expertise of a professional scheduler, who is hired by the owner to represent owner’s interests. The schedule reviewer evaluates this schedule for compliance to contractual requirements and professional scheduling norms and best practices. The schedule reviewer may negotiate with the contractor for changes to the schedule that would allow for better monitoring and could be used as a reference for dispute resolution. The goal of this process is for the contractor to submit a qualified project schedule and for the schedule reviewer to accept it as the basis for completing the work. This ‘baseline schedule’ will be used to monitor and record progress throughout the project.

This article does not cover the process of reviewing a baseline schedule. Reviewing a schedule update is very different from reviewing a baseline schedule. Instead, it is presumed that the baseline schedule review process has already occurred, and the next step is covered: monitor and update the project schedule to reflect progress and changes to the work plan.

Instead of presenting the author’s opinions, this work is based upon professionally recognized AACE International publications, namely recommended practices (RPs), and technical papers. To assist the reader in finding more on each subject, references are liberally placed throughout, with more detail available in the reference section at the end. This article straightforwardly presents performing a schedule review by the numbers.

Inputs of Performing an Update Schedule Review

Schedule management/control is the third significant section of the traditional planning and scheduling phases (the first two defined as project planning and scheduled development). During this phase, actual project progress is reviewed, activities are monitored, especially critical and near-critical activities that impact the project, variances and trends are analyzed, and reports are prepared to deliver the project’s status to the owner and stakeholders.[1]

Performing an update schedule review is the process of analyzing the schedule update submittal for the accuracy and rationality of changes that reflect project status and progress.[2] This process can be broken down into the three stages of review: inputs, process, and outputs. Figure 1 depicts the data flow diagram of the process.

Figure 1–Schedule Update Review: Inputs, Process, and Outputs

In the first instance, the reviewer must have the appropriate knowledge to describe, communicate, and perform the schedule control and review tasks. Recommended Practice 14R-90, Responsibility and Required Skills for a Project Planning and Scheduling Professional, states that the person should be able to explain and understand concepts and methods involved in the schedule performance assessment: such as schedule variance and trends, and performance reporting and variance methods. In addition, the reviewer should evaluate the following information during the schedule review process:[2]

  • Contract specification requirements
  • Contract plans specifications and instructions
  • Availability of the resources
  • Professional scheduling practices and guidelines

Consequently, when the reviewer receives the schedule, and before starting the review process, he or she needs to have a good understanding of the project and the type of work that has been performed. Therefore, the reviewer must read the documents related to the schedule’s development to have a factual basis to conduct a fair and objective review.

Another consideration is to review as-built facts recorded in coexistent project documents. For instance, available sources to confirm the project’s status correctness can include daily reports, meeting minutes, progress photographs, submittal, and production logs, request for information, change order logs, productivity reports, and weekly or monthly progress reports.[3] Therefore, the reviewer should start by reading and understanding the following project documents:

Contract

A contract is the “legal agreement between two or more parties.” [4, p. 29] The contract is considered the “rule book” for the project. It is one of the most essential and primary project documents that need to be considered while performing an update schedule review because it establishes the schedule and review report’s requirements.[5]

The contract includes vital information, such as general conditions, methods, project specifications, drawings, start and completion dates, the baseline schedule, completion milestone dates, modifications to the contract or scope of work, delay clauses, types of delays, and the plans to execute and complete the project. These will help the reviewer to compare the updated schedule to the specified information on the contract and determine if it is in accordance with the terms of the agreement and other contract documents.

Examining the contract helps the reviewer to:

  • Have a proper understanding of the contract
  • Check if all scope areas, milestones, and contract completion dates are represented in the schedule correctly
  • Identify if all submittals required in the contract are included in the update deliverables, such as the schedule narrative and the native electronic schedule file

As a result, the reviewer should perform the schedule update review based on the requirements included in the contract specifications and other contract documents. While other factors, such as industry norms and recommended practices, may also guide the reviewer, the wording of the contract still takes precedent.

As-Planned Schedule

Another project document that needs to be examined by the reviewer is the as-planned or baseline schedule. This schedule is one of the key project documents that the contractor develops to execute the work. It must be approved and includes relevant information related to the original scope of work, such as activity duration dates, milestones, constraints, resources, logic relationships, cost, and project deliverables. The as-planned schedule is the primary document used to compare update schedules and track project progress (unless supplanted by an updated, approved baseline typically reflecting the requirements of a change order). [5] [2]

Schedule Update Submittal

Schedule update submittals should be developed based on the contract specifications. The owner either accepts them for meeting requirements and specifications or rejects them for not satisfying those requirements.[2] There are many possible components of the schedule update submittal that the contract might require, such as:

  • A written narrative
  • A native electronic schedule file
  • Printed reports like critical path method (CPM) reports and table of schedule changes
  • Graphics like progress and budget S-curves
  • Other required documents on the contract, such as recovery schedules or time impact analysis

Of the schedule update submittal components, the reviewer should at least have and review the schedule narrative and the native electronic schedule file.

Schedule Narrative

Schedule narratives are essential documents that help the reviewer understand the changes to the schedule, issues, and project status. Schedule narratives should record the reasons for the significant changes that the program has had during the reporting period. [5] This document should address the following information:

  • Identification of the longest path/critical path
  • Changes that have altered the critical path include changes to logic relationships and activity durations or added or deleted activities
  • Project’s status and progress
  • Explanations of any project issues and opportunities
  • Work completed during the reporting period
  • A list of the activities that are scheduled to progress during the next reporting period (forecasted work)
  • Any other topics required by the contract

It is in the best interest of the submitting contractor to include a schedule narrative with the schedule. This narrative can state what the contractor intended to submit, even if the actual schedule fails to deliver the intended message.

Native Electronic Schedule File

Once the reviewer receives the native electronic schedule file, for instance, a Primavera or Microsoft Project file, it is recommended that he or she imports the file to the software as soon as possible to review the integrity of the data and identify possibly corrupt data. The reviewer shall request a new file in case of damage.

If obtaining an uncorrupted file is not feasible, the reviewer could employ procedures or use third-party software to repair and access the schedule. The following listed resources explain methods available to the reviewer if Primavera P6 scheduling software is used:

  • A Practical Application of Identifying and Correcting P6 Corrupt Data,” this AACE technical paper focuses on the different ways to avoid or mitigate corrupted data issues. Offered solutions involve SQL queries, third-party software, or revision and edition of the XER or XML files.[6]
  • Another method is presented in the “P6 File Corruption, Part 2” AACE technical paper, which entails the application of Oracle Primavera P6 technical solution to purge XER file from corrupted records, such as the public operational breakdown structure (POBS).[7]

It is important to note that schedules in portable document format (PDF) do not qualify as electronic files because they show only a view of the current status instead of all the information that is included in an electronic schedule file, which can be imported into scheduling software.[2]

Meeting Minutes

Project meeting minutes are records produced to accurately represent the project’s status and the plan for the forecasted work. Meeting minutes become a reliable document because it is assumed that the information mentioned in it is agreed upon by all the stakeholders that were present in the meeting. The reviewer should read the meeting minutes because of their valuable information. Usually, these minutes contains the following topics:

  • Safety issues
  • Review of open issues from previous meetings
  • Project schedule: identification of delays and time extensions, review of permit approvals, critical work, and contractual milestones
  • Review of drawings and RFIs
  • Change order status
  • Payment logs
  • Work inspections

When the reviewer cannot find the cause of a specific change in the schedule narrative, he or she should check the meeting minutes, which are likely to include an accurate and fact-based accounting of possible causes.

Plans and Change Orders

The schedule revision is a fundamental process in schedule management and control that should be clarified in the contract scheduling specification plans. [5] It is beneficial for the reviewer to identify these plans as they will help him/her to have a better understanding of the process and what should be included in the schedule updates. Schedules that are not updated are not valuable to the project due to their inaccuracy; they do not reflect actual and ongoing work. Likewise, if the owners fail to approve change orders on time, this can affect schedules’ reliability and accuracy since they will not include changes and will not reflect what is actually occurring with the project. [8] These are some key elements in schedule management and control that the reviewer should identify: “frequency of progress status updates, method of reporting impacts and/or delays, incorporation of change orders, and level of detail to be reported.” [5, p. 8]

Progress Photographs

Project completion photographs are significant additional project documents to be obtained and reviewed for accuracy of the schedule. Construction photography should be done professionally and by a person who is aware and understands the schedule, issues, and construction methods.

Photographs are important for the following reasons:

  • They can support the issues that are being referenced in the request for information (RFI).
  • They can be used to confirm and show that progress has been made.
  • Using photos on the status reports can avoid confusion and misunderstanding of the progress or issues.
  • Construction photographs can also be used to support or defend claim disputes.

Photographs clearly communicate and present “more information than any other medium.” [9. p. 2]

Process of Performing an Update Schedule Review

The task of performing a substantial review of the schedule update can be overwhelming not only for a novice reviewer, but also for a knowledgeable scheduler who is accustomed to providing expert judgments. This could be attributed to the fact that the auditing/evaluating process is inherently stressful. To help alleviate some of this stress and successfully navigate the task of performing a schedule update review, a process roadmap is laid out in this section. It includes a list of most relevant RPs, checklists of items to be considered, data analysis techniques, and practical recommendations.

Gather Data

We will now describe two methods of collecting data to perform an update schedule review. The first one is using information from the relevant AACE International recommended practices and published technical papers. The second method is employing checklists of the points that must be considered during an update schedule review.

AACE International Recommended Practices and Technical Papers

Gathering relevant information from the published RPs and technical papers aids a reviewer in applying the best practices vetted and widely accepted by fellow professionals. This approach also frames inputs of performing an update schedule review into the appropriate context. Table 1 lists inputs specified earlier in this article in relation to referenced RPs and technical papers.

Table 1–Schedule Update Review: Inputs by Recommended Practices and Publications

Checklists

The method of creating a checklist is a simple but very useful tool that can assist a reviewer of schedule update in identifying and focusing on the most relevant indicators of a project status and potential schedule issues. It is also an effective way of capturing and tracking the status of items that have been identified during previous reviews. The authors recommend that reviewers build and maintain such schedule update checklists and capitalize on lessons learned and consistent approach to the task at hand.

In addition to providing a structured and focused schedule update review, checklists also aid in maintaining consistency from one review to the next. This benefits both – the contractor’s scheduler, as well as the reviewer – setting the stage for the understanding and expectations.

Analyze Data

The following data analysis techniques can be used for the process of performing an update schedule review. These techniques include, but are not limited to:

  • Compilation of schedule statistics
  • Pinpoint of data manipulation
  • Critical path analysis
  • Near critical path evaluation
  • Variance analysis

Compilation of Schedule Statistics

Close attention must be paid to all changed items in the schedule, because even changes that presently appear insignificant have a potential of becoming crucial during the next review. [2] Most scheduling software can provide information about deleted, added, or modified scheduling components, such as activities and work breakdown structure (WBS). Third party applications can also aid in the process of gathering the changes in one packaged report.

Recommended practice No. 53R-06 lists changes that should be noted during the schedule update review. Note that the following list is a high-level outline of notable changes, and the reviewer must refer to the above RP for details. For example, ‘Activity data’ category unfolds into granular specifics, including not only information about deleted, added, or modified data, but also particulars such as milestones, constraints, and codes:

  • Scheduling software system checks
  • Work breakdown structure (WBS)
  • Activity data
  • Relationships
  • Activity logs, memos, or notebooks

Data Manipulation

A schedule update reviewer must be able to identify techniques which are used to manipulate the schedule to the contractor’s advantage. The technical paper authored by Christopher W. Carson, and presented at the 2012 AACE International Annual Meeting in San Antonio, lists the following ‘scheduling tricks’ [10]:

  • Logic and activity duration manipulation
  • Float suppression or sequestering
  • Misuse of relationship types or lags
  • Forced or predefined critical paths
  • Misuse of calendars
  • Out of sequence work

Critical Path Analysis

The reviewer must have a solid understanding of how to determine the critical path, as well as how to analyze it. The term critical path denotes only activity paths identified by float calculation using early and late dates and represents the longest continuous chain of activities’ which establishes the minimum overall project duration’. [4]

The frequently used and industry accepted methods of determining the critical path are: [11]

  1. Lowest Total Float
  2. Negative Total Float
  3. Longest Path
  4. Longest Path Value Method

Each of these methods has its own list of factors that must be considered when reviewing schedule update data. Referenced RP 49R-06 is an instrumental resource for the reviewer because it details out these factors and describes how it affects the critical path activities.

Near Critical Path Evaluation

The necessity of including the review of near-critical path is dictated by the understanding that activity duration estimates can change during project execution, thus making near critical activity potentially becoming critical if this activity is delayed significantly.[11]

There are deterministic and probabilistic methods that can be applied to identify near-critical paths. Deterministic methods include:

  • Near-critical float (with established ranges of a total float)
  • Near-longest path
  • Professional judgment

Probabilistic methods consist of techniques like program evaluation and review technique (PERT) or the graphical evaluation and review technique (GERT).

Variance Analysis

An update schedule review is focused on changes made to the schedule after the most recent accepted update. Therefore, it is recommended that the reviewer should establish a routine that aids in making the change identification process more efficient. [2] Most scheduling software packages provide the ability to automatically compare the current update to the previous schedule.

Some of the practical recommendations include, but are not limited to the following: [13]

  • Technical checkup via schedule recalculation
  • Confirmation of data integrity using custom graphical reports
  • Comparison of current schedule to the last version via custom layouts or reports
  • Identification of variances and trends by calculating the schedule performance index (SPI) and cost performance index (CPI)
  • Updates validation with the project team

Formulate Recommendations

The reviewer should formulate recommendations to the owner based on the findings from the process outlined in the “analyze data” section of this article. The recommendation can be to either accept or reject submitted schedule update, and the justification and explanation must be provided. Existing compliance metrics should provide the reviewer with the minimum standards applicable to CPM schedule. [14]

Outputs of Performing an Update Schedule Review

Once the schedule update review process is complete, the schedule reviewer has another, all-important process to perform; respond to the owner and contractor as to their findings. The schedule reviewer’s response can run a spectrum of no response, minimal response, or one of an optimum response. Finally, this section covers the question of what a schedule reviewer should do when they are unable to negotiate a schedule that can be approved.

No Response

Some reviewers may be tempted to not comment on their review findings, thinking that gives them the least exposure to later schedule failure. This is usually the worst plan possible. A failure to either accept or reject a schedule submission generally has been interpreted to consider the submission as accepted. [8]

In fact, it is the owner’s responsibility (and thus, the schedule reviewer’s job) to review and accept schedules. [5] If the owner contractually requires the contractor to submit a schedule, then it is their responsibility to review and respond. The response should be made in writing and directed to the contractor. [2]

Minimum Response

It is in everyone’s best interest to achieve a workable, approved project schedule depicting all of the significant work to be performed. An approved schedule allows for agreement as to project status, variances from planned, schedule analysis, trend assessment, a record of scope changes, and projected project completion. [1] [2]

The project’s contract sets the requirements of what should be in the reviewer’s report. Items typically covered include the categories and responsibility for delays, the methods for managing and controlling change, and methods for delay analysis. [3] Part of the review output also should be a listing of those activities that the schedule reviewer believes to be critical and near critical. [11] [12]

If the contract calls for a schedule submittal, then the schedule reviewer must state their reasons for rejection in writing. For the sake of the project and in the interests of helping the scheduling process progress with as little friction as possible, there is nothing wrong with approving the schedule submittal with the proviso that certain, small schedule issues be corrected in the next schedule submittal. [2]

Should the owner not object, it is also permissible for the schedule reviewer to wave scheduling specifications that the contractor finds objectionable or too costly to observe in the interest of project success. To maintain scheduling contractual enforceability, the explicit contractual issue being waved should be stated in writing. [2]

Obtaining a good schedule that contains all the work to be performed and properly depicts reasonable durations, and correct relationships between them can be a difficult and arduous process. This process may require cooperative work from both the contractor and owner to achieve.  [6] Properly explaining the reasons for rejecting a schedule submittal can be just a step in that process and not the end. The suggestions for schedule improvement should be general in nature, so as to not be considered a directive. [15]

If a revision to the schedule submittal will be required, the schedule reviewer should clearly state the requirements as quickly as possible. This will allow the contractor to respond with a schedule revision before the next schedule update cycle occurs. Having the contractor update an unapproved schedule should be avoided, if possible. [3] Additionally, the schedule reviewer must be aware of the difference between requesting a schedule update that recovers lost time and a recovery schedule that requires acceleration. [5] [16]The later may contractually require additional costs to be borne by the owner.

Before the owner can accept a project schedule submission, the contractor and owner must first reach agreement on the current project status. If the owner disagrees with the contractor about the project’s status, they must clearly state their position in an expeditious manner in writing. [2]

The owner should make every effort to provide time, resources, and motivation for the contractor to adequately plan the project before proceeding to construction. The owner might postpone the notice to proceed to allow for the necessary time. Financial penalties or rewards may be included in the contract to encourage timely schedule submittals. Progress payments should be set up to encourage preparation of long-lead material items (a common early critical work process). [10]

Before formally rejecting a schedule submission, the schedule reviewer is encouraged to hold a joint meeting between the owner’s and the contractor’s project controls personnel. There they can discuss the objections and possibly prevent misunderstandings. Negations and considerations for compromise may be worked-out, increasing the possibility of a quick and successful turn-around. Delays in resolving a particular schedule submission risk impacting the next schedule update submission, complicating the resolution of both updates. [10]

Optimum Response

A written narrative review is only one way that the schedule reviewer can communicate their objections or findings of the schedule. Good communication will enhance the understanding and perhaps the increase the likelihood of both parties being successful. A joint schedule review meeting should be considered so that the schedule reviewer can explain the intent and meaning behind the observations. [15] Other means and analysis should also be considered.

A written variance analysis report (VAR) can be created to enumerate significant variances from the plan and begin the documentation of the reasons for these variances. While the contractor often creates this document, there is nothing to prevent the schedule reviewer from taking the initiative. This VAR should be freely shared with all interested parties to encourage feedback and to reach consensus. [17]

An optimum VAR should contain information on root causes, corrective actions, changes to the budget, and estimate for project completion. It is an excellent place to also document needed management actions. [17] The VAR can also prove to be a trigger to create or update the project risk register to add a new risk strategy. [12]

Like a VAR, a schedule log may be created to enhance communication between parties. Such a log helps explain the project history, explain the current status of the project, provide for review continuity, as well as a verified source for developing time impact analyses (TIA) to adjust contractual requirements. Just as with a VAR, schedule logs are often created by the contractor but may be created and maintained by the schedule reviewer as well. All interested parties should have free access to this document. The contractor should be encouraged to raise objections to the contents of the log so as to perhaps be able to negotiate a common, agreed consensus. [18]

The schedule reviewer’s report can be vastly enhanced using graphics. It is best if the schedule reviewer makes their own graphics for analysis so that the parameters involved in their creation are known and accepted. Time scaled diagrams make for wonderful platforms for discussing the timing and the flow of upcoming work. Most CPM software already has built-in graphics capability, so this process usually does not require additional resources in order to accomplish.

A good review technique entails adding crew resources to a schedule if it is submitted without resources. A graphical display of daily or weekly crew loading can be a great help in communicating the owner’s concerns as to the reasonableness of the timing of the work. They can be optimized to display complicated reactions in the processes that make it easier for non-technical observers to understand the issues being discussed. [19]

Negotiations can be as straightforward as making a simple logical compromise. A common source of disagreements as to project status sometimes involves out-of-sequence progress. Out-of-sequence progress is normally an inevitable occurrence on projects. Some CPM scheduling software has various settings to handle the resolution of out-of-sequence activity float calculations. A retained logic setting may be thought to be too restrictive, and a progress override setting too liberal. A compromise change of replacing the finish-to-start logic with a finish-to-finish relationship and then recomputing the schedule using retained logic often gives the most realistic answer in these circumstances. [20]

Rejection of Substandard Schedule

It is always in the project’s best interest for the schedule reviewer and contractor to be able to agree on the use of a single schedule to represent the project’s progress. Such an approved schedule allows for a single interpretation of project status, progress trending, projected project completion, as well as the basis for dispute resolution. It also allows other involved parties to coordinate their efforts. This is especially helpful in coordinating submittal review. [10]

But what should the schedule reviewer do when they cannot approve the schedule update? Perhaps no schedule is submitted for review. Perhaps significant portions of the work are omitted, and the contractor refuses to correct this. Perhaps changes to the schedule are not in accordance to contract requirements. In such cases, it is likely that the schedule reviewer’s acceptance of that schedule would compromise the contract and lead to increased time and costs. In this case, is the owner to do nothing?

In situations where the owner is unable to accept a schedule update, the owner has a lot more work to perform than under ‘normal’ conditions. If the contractor is going to be shirking scheduling duties, it will fall upon the owner to step-up and perform more. They should ensure that a very competent full-time scheduler is in place to oversee the owner’s interest. The scheduler should create an independent CPM schedule to monitor the work, even if they have to guess at the contractor’s means and methods. This schedule should be modified as needed to keep-up with changes. [10]

Without an approved schedule to use as a framework for monitoring progress, the owner will need to heavily rely on good, contemporaneous documentation. Processes and procedures should be formalized and communicated to the owner’s staff. As-built progress should be recorded daily by the owner’s project control staff in a method that facilitates its retrieval later. One such process is called a daily specific as-built (DSAB) record. In addition, a force account procedure should be established and published so that the contractor is aware of the informational requirements beforehand. [10]

Conclusion

AACE International offers a wide variety of recommended practices and technical papers that aim to present valuable information and technical foundation to enhance skills and knowledge in the schedule update review process. These resources provide the schedule reviewer with the technical guidance and the information necessary to execute every stage in the process of performing an update schedule review: inputs, process, and outputs.

The schedule reviewer has a wide range of data sources that they must draw from in order to accomplish a thorough review. Besides the update submittal, the reviewer must be intimately familiar with the project contract. Seeing as this update schedule is a continuation of the previous ones, earlier schedules must be available to reference for changes. A written narrative should accompany the electronic CPM submittal. Plans, meeting minutes, and change orders must be properly reflected in the schedule basis. Whenever possible, the reviewer should physically observe the work being documented. Professionally created photographs documenting the critical progress and work disruptions or quality failures are needed to ensure that the reviewers can back-up their findings.

The reviewers are not alone in performing their job; numerous AACE recommended practices exist to assist the process. Hundreds of selfless scheduling professionals have collaborated in writing guides to the review process. Checklists should be created to help orchestrate the review process. Schedule statistics should be amassed to help spot trends and benchmark the quality of the data. These processes will help spot any possible data manipulation. Critical path analysis (along with near-critical) and variance analysis is a must for understanding the importance of the schedule changes.

Finally, a schedule review is of very little value if the reviewer fails to formulate and communicate the opinion as to the quality of the schedule submittal. The optimum response can be multi-faceted including a narrative, VAR schedule log, as well as graphics depicting technical analyses of various schedule quality indicators. The reviewer can negotiate with the contractor for changes in the schedule to improve schedule’s ability in depicting the accurate project status, should shortcoming be detected. This allows the project schedule to become the common source of project measurement and control.

The authors encourage the first-time reviewers or even experienced schedulers to read every reference listed. This paper merely gathers all relevant – in author’s opinion – materials, summarizing the schedule update review. It is a roadmap for what it would take to conduct a successful schedule update review. The task might seem daunting for both knowledgeable scheduler and inexperienced reviewer, if for no other reason that auditing process is inherently stressful. The reviewer now has an opportunity to not only provide identification of deficiencies, but also recommend improvements to the schedule for the update to be accepted.

REFERENCES

  1. AACE International, “Recommended Practice No. 14R-90, Responsibility and Required Skills for a Project Planning and Scheduling Professional,” Morgantown, WV, Morgantown: AACE International, Latest revision.
  2. AACE International, “Recommended Practice No. 53R-06, Schedule Update Review – As Applied in Engineering, Procurement, and Construction,” AACE International, Morgantown, Latest revision.
  3. AACE International, “Recommended Practice No. 90R-17, Statusing the CPM Schedule – As Applied in Construction, Morgantown, WV: AACE International, Latest revision.
  4. AACE International, “Recommended Practice No. 10S-90, Cost Engineering Terminology,” Morgantown, WV: AACE International, Latest revision.
  5. AACE International, “Recommended Practice No. 45R-08, Scheduling Claims Protection Methods,” Morgantown: AACE International, Latest revision.
  6. Winter, R.M. and M.G. Sominsky, A Practical Application of Identifying and Correcting P6 Corrupt Data, Morgantown: AACE International, 2020.
  7. Winter, R.M. and M.G. Sominsky, P6 File Corruption, Part 2, Morgantown: AACE International, 2017.
  8. Livengood, J. and K. Patrick, The Law of Schedules, Morgantown: AACE International, 2012.
  9. AACE International, Construction Photography to Document Project Status, Morgantown: AACE International, Latest revision.
  10. Carson, C., Dealing with Contractors Schedules That Cannot Be Approved, Morgantown: AACE International, 2012.
  11. AACE International, “Recommended Practice No. 49R-06, Identifying the Critical Path,” Morgantown: AACE International, Latest revision.
  12. AACE International, “Recommended Practice No. 92R-17, Analyzing Near-Critical Paths,” Morgantown: AACE International, Latest revision.
  13. Olsen, C., Schedule Review Basics, Morgantown: AACE International , 2000.
  14. Boyle, G., A. Podolny and W. Menesi, Evaluating CPM Schedules for Best Practices: A First Principles Approach, Morgantown: AACE International, 2016.
  15. Winter, R., Reviewing Resource Leveled Schedules Using P6™, Morgantown: AACE International, 2015.
  16. AACE International, Recommended Practice No. 50R-16, Trending and Forecasting of CPM Schedules,” Morgantown: AACE International, Latest revision.
  17. AACE International, Recommended Practice No. 86R-14, Variance Analysis and Reporting,” Morgantown: AACE International, Latest revision.
  18. AACE International, Recommended Practice No. 93R-17, Schedule Logs,” Morgantown: AACE International, Latest revision.
  19. Kuhn, A.J., Artificial Resource Loading for Schedule Review, Morgantown: AACE International, 2007.
  20. Plotnick, F., Reviewing Means of Calculating Schedule Updates With Out-of-Sequence Progress, Morgantown: AACE International, 2014.

ABOUT THE AUTHORS

Marina G. Sominsky, PSP, is with MGS Consulting Inc. and can be contacted by sending email to: msominsk@asu.edu

Ronald M. Winter, PSP FAACE is with Schedule Analyzer Software and can be contacted by sending email to: ron@ronwinterconsulting.com

Katherine Baron is with HKA and can be contacted by sending email to: katherinebaron@hka.com

]]>
https://source.aacei.org/2022/08/03/getting-back-to-basics-update-schedule-review-by-the-numbers/feed/ 1
Practical Planning and Scheduling for Capital Project Delivery in the Pharmaceutical Industry https://source.aacei.org/2022/06/01/practical-planning-and-scheduling-for-capital-project-delivery-in-the-pharmaceutical-industry/ https://source.aacei.org/2022/06/01/practical-planning-and-scheduling-for-capital-project-delivery-in-the-pharmaceutical-industry/#comments Wed, 01 Jun 2022 05:30:00 +0000 https://source.aacei.org/?p=5592

Practical Planning and Scheduling for Capital Project Delivery in the Pharmaceutical Industry

Francisco Cruz Moreno, PE

Abstract–Initiating a pharmaceutical capital project requires a deep understanding of the unique processes related to current good manufacturing practices, along with experience in the design, procurement, construction, commissioning, qualification, validation of facilities, utilities, and equipment. Additionally, planning should account for the regulatory affairs that approve bringing a product to market in this highly regulated industry. While most of the pharmaceutical capital projects are schedule-driven in a first-to-market business model, improper planning and delays constantly jeopardize the release of new products. This article describes practical techniques to plan and schedule new construction and renovations of existing buildings; typical sequences and durations used for commissioning, qualification, and validation activities; timing for engagement with different stakeholders; common pharmaceutical terminology employed during all phases of a project; and recommended approaches to leverage previously performed work to potentially gain time in the project schedule. Novice and seasoned practitioners alike will benefit from following these procedures. The author has successfully used these practices in manufacturing facilities for oral, parenteral, and topical administration products located in the Americas, Europe, and Asia. This article was first presented as PS-3597 at the 2021 AACE International Conference & Expo.

Introduction and Background

The global pharmaceutical industry is projected to grow during the upcoming years due, in part, to an aging and growing population, rising income levels, new diseases, and emerging medical conditions [1, p. 1]. While the pharmaceutical industry is one of the most regulated and capital-intensive on the planet, it is also more profitable than any other large public company [2, p. 841]. Since these companies operate in an increasingly competitive market, they have adopted creative business strategies to expedite product delivery. The current SARS-CoV-2 pandemic is just one example of how the pharmaceutical industry can quickly scale research, development, and manufacturing processes to be first to market with a product. The pandemic environment also supports the assertions as to why most pharmaceutical capital projects are schedule driven.

The term pharmaceutical as used in this article refers to the pharmaceutical, biotechnology, and nutraceutical sub-industries as they relate to the use of chemical and biological compounds, dietary supplements, and food additives. Furthermore, the term includes capital projects associated with oral, parenteral, dermal, and injection applications and research projects, including vivarium and process development.

Facilities and engineering teams work on a variety of capital projects to address the needs related to products and processes specific to this industry. These projects may include new production or support facilities, central utility plant upgrades, new cleanrooms or renovations, operational expansions, interior fit-outs, technology transfers, automation, and building and environmental management systems. Projects can be executed on one site, within one building or across multiple buildings; or on multiple sites, located in the same country or across the globe; and can be completed concurrently or sequentially.

These teams face common challenges as they begin to plan and execute projects. Common denominators include project complexity; massive coordination with multiple workstreams; stringent room environmental conditions; qualified workforce resources; changing equipment and software technologies; rigorous documentation, validation, and regulatory requirements; operational consolidation projects; and multiple company reorganizations. The fact that these capital projects may be executed in different countries adds to the complexity since the process requires qualifying designers, contractors, vendors, and suppliers to ensure that they are familiar with the industry’s quality, environment, and documentation requirements.

This article presents practical planning and scheduling techniques that address these challenges by focusing on the timing for engagement with different workstream leaders; typical sequences and durations used for commissioning, qualification, and validation (CQV) activities; and recommended approaches to leverage previously performed work to potentially gain time in the project schedule. The author uses pharmaceutical terminology familiar to all phases of a project. This document will not cover the business process, sanction of candidate products, procurement of architectural and engineering services, or supply chain management after products are approved by regulatory agencies as these are company-specific and tend to vary widely.

Phases of a Pharmaceutical Capital Project

Pharmaceutical projects require additional phases that are not common in other industries since they need to maintain the integrity, quality, and safety of their processes and products by ensuring that they meet current good manufacturing practice (CGMP) and government regulatory requirements. In addition to design, procurement, construction, and commissioning, these projects must go through qualification, validation, and regulatory phases. Project teams ensure that the asset meets not only the design intent and requirements but, most importantly, the manufacturing processes and the product.

The project scope definition and planning stages follow a phase-gate process where front-end loading (FEL) and a project definition rating index are the most widely used rating systems. Steering committee members evaluate projects at every gate and decide whether to approve them, place them on hold, ask for a reevaluation, or cancel them. Projects with weak scope definition fail to secure approval and funding to move to the next gate. To secure approval, deliverables must be organized, and the first two gates should be conducted with discipline and additional rigor.

To be successful, the project team should start a coordination effort and an integrated documentation process as soon as practically possible after project kick-off. Identifying the departments and leads that will be involved on the project should start as soon as the project is sanctioned; the goal is to create a core team that reports to a project sponsor and to a steering committee. This team will identify, collect, confirm, and document the project requirements. A typical team composition is shown in Table 1.

Table 1–Typical Project Team Composition

Owing to the regulatory requirements, the project manager, along with the CQV team, will start reviewing and updating the validation master plan (VMP). This is the document that defines and documents the elements of the site’s validation and qualification program. The contents of the VMP, or an equivalent document, are often neglected during the early planning stages and are not included in the project schedule. To mitigate this issue, the project team should review and amend the VMP as soon as there is an approval to any significant change to equipment, a section of the facility, processes, or testing methods that could affect the quality of the product.

Before the design phase begins, the planning and scheduling professional should review and understand the following items within the VMP as part of the project schedule planning and development:

  • Validation philosophy
  • Requirements for establishing validation (e.g., direct and indirect product impact, and critical and noncritical components)
  • Site process and product description
  • Approval levels required for documentation
  • Inventory of products
  • Manufacturing processes
  • Systems to be validated (e.g., environmental management system)
  • Planning and scheduling of the validation activities

Design

The design of a pharmaceutical facility addresses key features, including adequate space to facilitate the functions and efficiency of manufacturing, cleaning, and maintenance operations; sustain specific environmental conditions related to temperature, humidity, and differential pressure; prevent cross contamination between adjacent areas, corridors, and rooms; minimize exposure to hazardous material; and enable the proper flow of personnel, materials, and waste. During this phase, the project team starts collecting information, such as user requirement specifications (URS); functional requirement specifications (FRS); software requirement specifications (SRS); budget, product volumes, schedules, and key milestones; utility and safety requirements; and standard operating procedures (SOP).

A typical work breakdown structure includes disciplines related to architectural, civil, and structural; process and manufacturing equipment. This list includes utilities, piping, and instrumentation; automation, building automation systems, and environmental management systems (EMS); heating, ventilation, and air conditioning (HVAC), including air handling units (AHU) and fan coil units (FCU); mechanical services; fire alarm and fire protection systems; electrical, environmental, and health and safety systems; and information technology, cameras, and access control [3, p. Subpart C].

The permit strategy should be thoroughly assessed during the design phase since the site may require that additional environmental permits related to biological and process byproduct waste, discharge (or even treatment) be secured, potentially increasing the permitting duration. Meeting with the local building department to discuss the details of the project, understand the permitting requirements, and anticipate any additional demands can facilitate and expedite the permitting process. Additionally, some building departments have checklists to identify specific items that the department will review in the project drawings. The team can use these checklists during 90% design to provide feedback to the designer to anticipate potential design omissions. The author has used this approach to reduce the permitting schedule by one to two months.

The qualification team will perform a risk analysis and impact assessment (RA/IA) on product and equipment quality and process fitness. This will allow them to identify the systems that will be deemed direct or indirect impact, as well as critical and noncritical components. System boundaries are also identified as part of this assessment to assist the construction and CQV teams in organizing the documentation and in identifying responsibility assignments.

The outcome of the RA/IA typically classifies the documents into three categories: baseline documents, controlled documents, and validation documents. This key step should occur early during the design phase because documentation requirements may be different depending on the critical classification of the systems (e.g., good engineering practice guidelines may be sufficient to document noncritical components or indirect impact systems). The team should capture the development, execution, and approval of this documentation in the project schedule.

Besides the RA/IA, the team should perform at least three project risk analyses, specifically for Class 5, Class 3, and Class 1 schedules. Class 5 schedules contain limited information, and the team will most likely need to rely on limited historical data [4, p. 10]. While the project risk analysis focuses primarily on the facilities and equipment installation, the team should incorporate the risks identified for the product and processes, which are usually captured in a failure mode and effects analysis register.

The design can be executed in two stages (i.e., a combined schematic design/design development and construction documents) or in three stages (i.e., schematic design, design development, and construction documents), depending on the organization maturity, project size, procurement practices, and chosen project delivery method. Small projects that do not include process equipment usually fall under the two-stage design approach, whereas medium to mega projects fall under the three-stage design approach. The project team will hold design qualification (DQ) review meetings to ensure that the design captures all quality features, facility finishes, and flows; instrumentation performance; and functional criteria to move from one stage to the next. DQ also helps the engineering team make a decision regarding the equipment vendor selection.

The project team should explore the option of modularization and prefabrication, especially for sites where operations will run concurrent with new construction or renovations. During the mid-to-later stages of the design phase, the project team must decide what process and manufacturing equipment should undergo formal factory acceptance testing (FAT) and site acceptance testing (SAT). The team will also develop and approve FAT and SAT test protocols documentation; these activities should be included in the schedule and are considered pre-commissioning tasks.

The team will start planning the allocation of staff, resources, implementation strategies, and raw materials needed for the commissioning and qualification phases. The development of an integrated master schedule (IMS) is key to support the business plan and target the commercial date with a higher level of certainty. Figure 1 illustrates an IMS incorporating all major phases found in a typical capital project.

Figure 1–Common Schedule Sequence

Procurement

Besides procuring the services of a general contractor or a construction manager, the team must identify whether they will use in-house resources to perform and execute the commissioning and qualification (C&Q) protocols. If the project team decides to outsource the C&Q services, these consultants should be brought in early during the design process. For each equipment identified in the approved commissioning documentation, the commissioning team will perform an FAT at the manufacturer’s facility to ensure that the equipment operates as intended and that it complies with the URS, SOP, and design specifications (DS). A sample equipment and utilities list is shown in Table 2.

Table 2–Sample Equipment and Utilities List

Some of the functional testing will include drawing and wiring verification; instruments and calibration; utilities verification; safety, alarm, and security verification; and software quality assessment. Since process and manufacturing equipment are often deemed long-lead items, the team should include shipment and clearing of customs durations to reach the site. The project team will identify the commissioning team that will support FAT and SAT activities; this team could be composed of in-house personnel, consultants, or a combination of the two.

Construction

The project team will oversee the construction of each room and the installation of systems and devices, keeping quality as a paramount criterion for the completed area. Because there is minimal interaction with existing systems, production, and operations personnel, new construction usually has less coordination and constraints than renovation, additions, or remodeling.

For construction in existing manufacturing areas, the project team will coordinate with several departments, including facilities, manufacturing, quality, planning, and quality assurance to prevent and monitor the collection of airborne particles and dust; maintain the environmental conditions of adjacent rooms (i.e., temperature, humidity, differential pressure); and prevent vibrations to avoid recalibrating devices and equipment. The project team should install high-grade, opaque temporary partitions, seal space penetrations, and install differential pressure devices, such as magnehelics and photohelic gauges to monitor room pressurization. They should also install air scrubbers with high efficiency particulate air (HEPA) filters to maintain negative pressure and to avoid cross contamination between areas.

Additionally, construction personnel will receive training on avoiding triggering fire alarms and impairing the fire protection system following the site’s insurance carrier instructions. It is recommended that the project team perform a test and balance (TAB) verification of the existing HVAC system, with a goal of ensuring that environmental conditions required for manufacturing a product remain the same as before construction. A TAB must also be performed after construction completion. Implementing these preventive actions will ensure swift execution of the construction phase within the constraints of existing controlled areas.

The project team should also take advantage of site shutdowns that occur once or twice per year; these usually take place during the summer and winter seasons and can last between one to two weeks. There may be a need to request a shutdown for a longer period to install, commission, qualify, and validate a manufacturing area located in the core of a building, or, due to the nature of the process, the area may require extensive testing. In this case, in addition to getting approval from senior management, the project team should coordinate with the manufacturing team to build inventory and allocate space in the warehouse for additional finished goods.

The engineering team will coordinate with manufacturers and vendors to begin the SAT activities, which start after the confirmation of both mechanical completion and available site utilities. Note that some equipment manufacturers and vendors may have to travel from other countries to the construction site, so planning needs to occur well in advance. The commissioning team will start the equipment installation verification and functional testing to verify that the equipment operates as it did during the FAT.

It is recommended that the commissioning team witness the physical testing, perform visual examination, and document construction activities (e.g., loop testing, pressure testing, test and balance of HVAC, special welding) to expedite the commissioning process and receive certification. Additionally, the team could potentially leverage some of these testing results and certifications during the follow-on installation qualification (IQ) execution in the qualification phase. 

Commissioning

Depending on the maturity of the organization, commissioning may be the last phase in which contractors and vendors provide support services. Refer to Figure 2. Mature companies will perform both qualification and validation tasks using in-house personnel. There should be milestones denoting handover activities as well as a period of knowledge transfer captured in the schedule.

Figure 2–Flowchart of the Commissioning and Qualification Process

A successful commissioning phase provides documented evidence that the installed or modified equipment complies with the URS, FRS, approved design, and manufacturer recommendations. The commissioning activities include equipment and components testing, drawings and wiring verification, instrumentation and calibration, utilities verification, software assessment, security access, and personnel training. These may take from several days to two months depending on the complexity of the equipment and systems.

Although it may require more rigorous testing and inspection protocols, enhanced commissioning should be implemented whenever possible (i.e., usually for indirect impact systems) to avoid repeating activities during the qualification phase. Some commissioning activities may overlap with qualification activities (e.g., SAT and IQ).

Qualification

Qualification is the action of verifying, proving, and documenting, with a high degree of assurance, that the product, process, equipment, and ancillary systems are properly installed, work as specified, and lead to the anticipated results. Refer to Figure 2. The development of the qualification master plan (QMP) should start during the design phase and include a listing of protocols related to IQ, operational qualification (OQ), performance qualification (PQ), and resources that support the execution and documentation of the protocols. These protocols ensure that the equipment installation consistently meets the quality requirements of the finished product.

While qualification teams usually execute these protocols sequentially, the IQ and OQ protocols could be performed concurrently for systems already in operation in the case of equipment replacement, but the process remains the same; personnel may refer to it as the IOQ protocol. The deterministic duration of IQ, OQ, and PQ protocols depend on the complexity of the equipment, the qualification plan, and the experience of the team executing the protocols. The duration could take between twenty days to several months for each protocol.

The project team needs to account for material supply since the PQ could potentially use production material or a qualified substitute surrogate media for qualification testing. As the final step in the qualification phase, the qualification team will execute a PQ to test all equipment and components as a partial or completed manufacturing process; this is different from IQ and OQ where personnel test and verify each equipment and component one by one. The qualification team must complete the appropriate qualification of critical equipment and ancillary systems before starting validation activities.

Qualification of the HVAC system is required owing to potential impacts to product quality. Depending on the cleanroom classification, the qualification team will test several design features that include temperature and humidity thresholds and controls (T and RH); airborne particle control, room airflow direction, air changes (ACH), and differential pressure (DP); and HEPA filter integrity testing.

Table 3 shows key criteria that the qualification team needs to identify, plan for, and verify for each process equipment identified in the project. Generated reports provide the documented evidence and will be included as part of the regulatory submission package; these reports will be kept at the project site for potential regulatory audits.

Table 3–Purpose and Key Criteria for Qualification Protocols

Since there may be a need to hire personnel to support a new area or manufacturing process, planning for the training of personnel in the processes, equipment, safety and handling of material and waste is key during this phase. These trained personnel will then execute the validation tasks during the next phase and will be responsible for the manufacturing process once the product receives approval from the regulatory authority.

Validation

Process validation refers to the collection and assessment of data to establish scientific evidence that a process can consistently deliver quality products. It is a regulatory requirement for any company aiming to obtain licensure of a pharmaceutical product. Process validation occurs from process design through commercial manufacturing and aims to detect and control variability in the manufacturing process. This is the final phase of a pharmaceutical project before submitting all documentation to the regulatory agency that will either approve or reject a product.

For new facilities, the team will use prospective process validation prior to manufacturing and selling a commercial product. For existing manufacturing operations and facility renovations, the team will use revalidation to maintain the current process validation while making changes to equipment, process, or product. There may be instances where the team needs to perform a retrospective process validation to assess the consistency of a process for products already in the market, but this approach may require several levels of approvals from senior management due to the increased level of effort required to execute it.

There is great emphasis on implementing well-understood control strategies and on demonstrating that the delivery team used a risk-based approach to identify critical process parameters. Some projects may include technology transfer efforts that require early planning and additional rigorous coordination between two or more sites, sometimes located in different countries. Transferring processes and products between countries also triggers additional regulatory requirements. Note: Due to the complexity of this subject, this article will not discuss technology transfer projects.

A main schedule risk driver that may impact the execution of validation tasks is the availability of skilled resources. Personnel assigned to carry out validation activities need to have the qualifications, training, and process experience related to cGMP and be familiar with the company’s SOP.

The team should also assess the cleaning validation procedure during the validation process. This validation is usually performed during the part of the process when risk of contamination or carryover of material that could compromise the product quality is the greatest. Cleaning validation is performed concurrently with product validation; however, the team may perform it prior to shorten the validation timeline. Existing validated cleaning procedures may not require additional validation if a new product belongs to the same classification and family type or if the product is a like-for-like replacement of existing equipment, pipes, and vessels (e.g., CIP systems).

The validation team is typically a cross-functional unit composed of key personnel from the quality control, quality assurance, manufacturing, product development, and regulatory departments. The validation team must plan to develop validation protocols using the site’s SOP, and the schedule should reflect at least the following activities: protocol development, review, and approval; protocol execution; protocol report generation, review, and approval; and validation documentation change control. Validation procedures and protocols must support the validation activities related to computer hardware and software, critical systems and equipment, process utilities, and facilities validation. The deterministic duration of the execution, report generation, and review and approval of a protocol typically ranges from five to 20 days.

There are additional key activities that occur as part of validation to confirm the manufacturing process. These activities are sequential in nature and there are few opportunities to perform them concurrently. They include engineering runs, process performance qualification (PPQ) runs, and stability periods.

Engineering runs are the initial runs, at the expected production scale, that simulate the formal CGMP manufacturing run using the actual qualified process equipment. The goals of the engineering runs are to confirm the successful scaling up of the new manufacturing process, complete the batch records [5, p. 210.3], vet the process control strategy, and train production personnel. Note that the validation team needs to plan the acquisition and disposition of material associated with these runs. The material could be either the same production material or what is referred to as water batch, which is material that may be composed of media or buffer. The quality control team oversees the engineering runs and performs full testing; the quality assurance team performs assessments and investigations on deviations that could affect product manufacturing. Depending on the risk appetite of the organization, engineering runs, along with their stability testing results, may be used as part of the submission package. This approach is considered mainly for manufacturing processes that have been operating for more than several years.

One of the most debated subjects within the validation phase is related to the appropriate number of PPQ batches. A decade ago, the gold standard was to run at least three PPQ batches to satisfy regulatory requirements. Now there are many different approaches in determining the number of PPQ batches after the FDA issued its new guidance in 2011 [6, p. 4] . The 2011 guidance is based on process experience and knowledge; statistics (e.g., expected coverage, process confidence, and capability); risk management; or a combination of these.

The validation team must identify the strategy and approach to run the PPQ batches before the end of the commissioning phase. Ideally, the number of PPQ batches may range between two and five depending on the product and robustness of the process. Note that the PPQ runs are considered commercial runs.

The last validation activities are related to collecting product stability testing data and developing reports to submit to regulatory agencies as shown in Figure 3. The validation and quality teams will evaluate the data from the PPQ runs to identify any deviations and degradation of the final product due to storage conditions (e.g., temperature, humidity, light); number and size of the batches; and closure systems [3]. The stability testing program must be defined in conjunction with the validation strategy and plan. The validation team may decide to collect stability testing data for the engineering runs to submit as part of the regulatory submission, potentially saving several months during the validation phase. The deterministic duration of the stability testing activities will vary depending on the product and process and will typically range between one, three, and six months.

Figure 3–Schedule Showing the Validation and Regulatory Process Phases

Regulatory Process

One of the most important steps of a pharmaceutical project is to assess and ensure compliance of all laws and regulations pertaining to chemical and biological manufacturing products. A regulatory affairs (RA) department will define a regulatory strategy and prepare a regulatory submission package to request authorization to manufacture, promote, and sell the finished product. The regulatory strategy identifies whether a product will be registered with a domestic regulatory agency (e.g., United States Food and Drug Administration), or if it also will be registered internationally (e.g., European Medicines Agency in the European Union); the latter will increase the effort to comply with different regulatory requirements. These tasks may be outsourced to RA consultants; the core team must track them in the schedule.

The regulatory submission package not only includes the results from clinical studies and the stability testing data but also the commitments for post-registration and market surveillance to ensure patient protection. New products will most likely have a regulatory strategy defined and completed during Phase 1 or Phase 2 clinical trials. Existing products going through an increasing market demand, and that are approved for a facility expansion, have their regulatory strategy approved during the last stages of design of the facility. Regulatory approvals typically range from one month for existing products to twenty-four months for new products.

Change Control

A critical process within the pharmaceutical industry is related to the control of changes impacting the facilities, engineering, manufacturing, quality, research and development, and information technology departments. Additionally, this process needs to address whether changes are related to new or existing products, modification or decommissioning of critical equipment, new or revised drawings, or regulatory changes. The project team needs to initiate, document, approve, implement, track, and control these changes before starting the project. This is especially critical since systems and equipment can be impacted during the planning and execution of a capital project and these change control tasks should also be included in the project schedule. Once created, a change request may stay open for several months since it requires verification of satisfactory completion of tasks and subtasks to close it.

The qualification team, along with personnel from the quality assurance department, will use an RA/IA (see Figure 2) to initiate a change request and to determine which documentation, tests, verification levels, and approvals will be included. The project may need only one change with several subtasks, or it may need several change requests.

Conclusion

This article presented the unique phases found in a pharmaceutical capital project, identified the key team members along with their interaction throughout planning and execution, and displayed a flowchart to aid practitioners in comprehending the intricacies of the commissioning and qualification processes. A specific set of criteria were listed to verify and document the installation, operational, and performance qualification protocols based on the practical applications of a risk-based validation approach. Coordination and implementation of the change control process early in the project is key to meeting interim and finish milestones and avoid unintended delays.

Understanding the complexities of the processes, requirements, and regulations to which capital projects in the pharmaceutical industry need to comply increases the likelihood of achieving regulatory compliance and obtaining market approval. Early planning, extensive coordination, and solid documented evidence are essential to moving through the various project phases. Whereas there are traditional sequential approaches to plan and execute protocols and testing, this paper presents methodologies, including enhanced commissioning and concurrent IOQ, that could potentially shorten the project schedule while maintaining the robustness of the results.

Each design, procurement, and construction method, as well as discussed recommendations, is vital to accomplish a successful commissioning, qualification, and validation campaign of a full-scale CGMP run, especially on existing manufacturing sites. Scheduling activities using the sequence and durations presented during the CQV phases will give the project team a good starting point when planning, with the added value that they could anticipate and eliminate potential sources of risk, failure, and deviations.

References

  1. NAVADHI Market Research, “Global Pharmaceuticals Industry Analysis and Trends 2023,” 17 May 2019. [Online]. Available: https://www.navadhi.com/publications/global-pharmaceuticals-industry-analysis-and-trends-2023. [Accessed 19 December 2020].
  2. S. V. G. C. E. Ledley FD, “Profitability of Large Pharmaceutical Companies Compared With Other Large Public Companies,” JAMA, vol. 323, no. 9, pp. 834-843, 2020.
  3. FDA, “Electronic Code of Federal Regulations, Title 21, Part 211,” 19 January 2021. [Online]. Available: https://tinyurl.com/y3bwkm66. [Accessed 20 January 2021].
  4. AACE International, Recommended Practice No. 27R-03, Schedule Classification Systems, Morgantown, WV: AACE International, Latest revision.
  5. FDA, “Electronic Code of Federal Regulations, Title 21, Part 210,” 19 January 2021. [Online]. Available: https://tinyurl.com/y6rjhxdz. [Accessed 20 January 2021].
  6. United States Food and Drug Administration, “Guidance for Industry – Process Validation: General Principles and Practices, Revision 1,” FDA, 2011.

ABOUT THE AUTHOR

Francisco Cruz Moreno, PE, is with PMA Consultants. He can be contacted by email at: fcruz@pmaconsultants.com

]]>
https://source.aacei.org/2022/06/01/practical-planning-and-scheduling-for-capital-project-delivery-in-the-pharmaceutical-industry/feed/ 1
Improving Project Gatekeeping Through Stakeholder Alignment and Behavioral Decision Making https://source.aacei.org/2022/04/06/improving-project-gatekeeping-through-stakeholder-alignment-and-behavioral-decision-making/ https://source.aacei.org/2022/04/06/improving-project-gatekeeping-through-stakeholder-alignment-and-behavioral-decision-making/#respond Wed, 06 Apr 2022 17:09:02 +0000 https://source.aacei.org/?p=5083

Improving Project Gatekeeping Through Stakeholder Alignment and Behavioral Decision Making

David C. Wolfson

Abstract–When looking at why many major projects fail to meet their cost and schedule expectations, the root cause is often a misalignment between the engineered solution and the owner’s business case. This article will explore this issue by briefly describing the gatekeeping process. At its core, gatekeeping is a decision-making process. The article looks at decision making in the context of projects. Stakeholder management, salience and interaction with gatekeepers are also explored. The interrelationships between gatekeeping, decision making, and stakeholder management is illustrated by looking at both a simple and a slightly more complex project. Even in these simple examples it is apparent how misalignment can occur. From an owner’s perspective, misalignment at the project approval stage can have very significant financial repercussions. This article offers three strategies to mitigate potential misalignment, based on the author’s examination of decision making and stakeholder management. This article was first presented at the 2021 AACE International Conference and Expo as OWN.3666.

Introduction
When looking at why many major projects fail to meet their cost and schedule expectations, the root cause is often a misalignment between the engineered solution and the owner’s business case. The owner’s gatekeeping function sets the parameters that various aspects of the business are evaluated against. This in turn defines the planning deliverables required at each gate, especially for the project execution team. When this effort breaks down, ill-defined and ill-prepared projects make their way through the process resulting in late changes and poor project results. This article shows that by understanding stakeholders, gatekeepers, and the decision-making processes, this misalignment can be mitigated.

The article will explore this issue by first describing the gatekeeping process. At its core, gatekeeping is a decision-making process. The article will look at decision-making in the context of projects. Key aspects of stakeholder management will be summarized. The interrelationships between gatekeeping, decision making, and stakeholder management will be illustrated by looking at two different projects as case studies. Finally, aspects of alignment and misalignment will be explored.

Gatekeeping
Best practice in capital project execution is based on a stage gate approach. For asset creating projects, most organizations use stages and gates as shown in Figure 1.

Typical Stage Gates for Asset Creating Projects

Figure 1–Typical Stage Gates for Asset Creating Projects
 
Fundamentally, gatekeeping is a quality process. Its purpose is to ensure the project team is doing the right work at the right time. And it ensures that the work in the previous phase is done with the correct quality. For example, during the definitive estimate phase, most organizations have clear criteria to judge the estimate quality, including piping and instrumentation diagrams (P&ID) quality, level of quotations received, amount of work done exploring local labor rates and technology and risk assessments. Gatekeeping is essential to project governance.
 
The purpose of the gatekeeping of course is to make a decision. Decisions can be thought of as either strategic, tactical, or operational (or even transactional). The characterization is a matter of perspective. For example, the decision to release a certain drawing for construction might be viewed as an operational decision by the owner, but strategic to the engineering contractor on a fixed price contract. Which bucket a decision falls into is less important than understanding that the perspective changes depending on if the decision maker is an owner, a contractor, or a subcontractor.
 
From an owner’s perspective, strategic decisions can be thought of as stage gate decisions. For example, project approval and final acceptance are strategic decisions. Figures 2 – 4 show the various perspectives of the EPC company, the owner’s project team, and the owner. Figures 5 – 7 add in the shareholders and their notional motivations. These figures represent some examples and are not meant to be hard and fast dogma. The purpose is to show that different perspectives have different decisions, different gatekeepers, and different stakeholders. Figure 7 shows the owner’s company perspective. External stakeholders include banks that are looking for sound financial returns, the local community that is looking for a good neighbor (low noise, good employer, environmental steward) and regulators that are looking for compliance. How to manage these stakeholders will be discussed later.

Figure 2–The EPC Perspective of Strategic Decisions

Figure 3–The Owner’s Project Team Perspective

Figure 4–The Outside Perspective of a Project

Figure 5–The EPC Perspective with Stakeholders

Figure 6–The Owner’s Project Team Perspective with Stakeholders

Figure 7–The Outside Perspective of a Project with Stakeholders

Decision Making
Essentially projects are a series of decisions and gatekeeping is an example of a strategic decision from the owner’s point of view. From an owner’s project team’s perspective, the following decisions can be thought of as either strategic, tactical, or operational:

  • Strategic: Decision to fund development (initiate), decision to fund definitive estimate (gate 2), project authorization, technology decisions, decision to commit funds (long lead procurement items, easements for pipelines, environmental permitting modeling) when project or money is at risk, high level execution strategy/risk assignment (partnering, EPC role, self-execution, degree of oversight), permit strategy decisions, decision to take over operation. 
  • Tactical: Execution strategy fundamentals (outsourcing decisions), schedule milestones, procurement packaging strategy, major equipment specification and T&C requirements, modular strategies, design basis/design margins to be used, foundation design strategies (large mats or many individual foundations, piling type to be used) controls philosophy, drawing detail approach (home run I&E package vs routing details), above ground vs below grade decisions, sub-contractor risks allocations, field staff strategies. These might be considered strategic by an EPC contractor.
  • Operational: Specification inclusions, drawing package decisions (ex: structural steel details to be shown, drawing approvals and comments), decisions to add field staff as necessary, decisions to source inspect and/or expedite equipment, decisions to accept vendor deliveries in the field and/or hold suppliers accountable for changes, decisions to make up work with increased crews vs more hours (4 x 10’s vs added Saturday make up days). These might be considered tactical or strategic by an engineering or construction contractor.

The AACE International Total Cost Management Framework lists an objective approach to decision policy and investment decision making including planning for decision making, determining deterministic decision models, and evaluating alternatives prior to an approval decision. [4]

Engineers are trained to think rationally. Much of the project management and cost engineering literature assumes both project teams and gatekeepers are thinking rationally. For example, if construction is not fully complete, the assumption is that the gatekeeper will deny the project from moving into the startup state gate. Things are never quite so black and white of course. Typical questions that need to be clarified include how many punch list items are acceptable to be considered complete, can the plant be [HQ1] turned over system by system or area by area (i.e., have multiple gates), are utilities available and should that be considered, what level of equipment spares, or temporary piping are needed on site before turnover is acceptable. 

At the project approval decision gate, a project approval might be assured if all requirements are completed per company requirements. For example, execution risks have been developed and quantified, financial analysis with sensitivities have been run, financing is secured and contracts or letters of intents on key operating assumptions have been legally reviewed. Yet the decision makers need to understand the basis and assumptions made for each of these inputs as each have built in biases. Probing questions might reveal, for example that the project contingency or scope completeness score was modified after some pressure was exerted on the engineering team. Gatekeepers want to believe the numbers brought before them as they provide a rationale for yes/no decisions.

Yet beyond definitional issues such as those listed above, is the realm of non-rational thinking. Gatekeepers and project team members have biases and motivations that are impossible to strip away. So, it’s important to understand some behavioral elements of decision making.

Broadly speaking there are three ways of understanding decision making in projects. This article will only focus on two. [5] The first, “the reductionist school” assumes that decisions should be rational and that deviations from a purely rational approach are due to biases or errors. The second approach, “the pluralist school” argues that decision makers are rational but strongly influenced by personal and political interests which can be in conflict with the interests of the project. Here deviations from a purely rational approach are due to intra-group conflicts resulting in opportunistic behavior, bargaining, and conflicts. [5] The author argues that most of the AACE audience falls in the reductionist school, but that a pluralist approach must be understood and considered to improve outcomes. Moreover, decision makers such as gatekeepers need to identify and appreciate biases and political interests much better. This includes understanding systemic biases.

The literature, summarized by Stingle, Geraldi [5], identify three key ways biases manifest themselves in project decision making.

  1. Escalation of commitment: Projects continue despite clear warnings of failure. Optimism bias (the over estimation of positive outcomes), illusion of control (perceived control over project risks lead to downplaying these risks), group think (or sunk cost bias) and anchoring. These all contribute to this impact.
  2. Overoptimistic initial plans and forecasts: Planning fallacy, optimism bias, as well as strategic misrepresentations where individuals push pet projects. These are each contributing factors. [6]
  3. Negotiations: “A failure to balance interests, or the adoption of a strongly self-interested strategy in negotiations can lead to sub-optimal negotiations” [5] and therefore decisions. This becomes even more important in the context of stakeholders. As project decisions are made, negotiations made in bad faith might result in a stakeholder’s ire later in the project and thus affect project success.

Behavioral decision making has been well documented. For example, Parth [7] lists 5 major flaws that cause errors in decision making:

  • Errors in logic
  • False assumptions
  • Unreliable memories
  • Mistaking the symptom for the problem
  • Biases 

Mitigation measures from Parth [7] include:

  • Self-feedback
  • Critically questioning the biases of those that provide information 
  • Document experiences 
  • Critically thinking about reality 

Mitigation involves awareness of biases and understanding of our own personal biases. 

Building on this knowledge, this article proposes three improvements to gate keeping decisions:

  1. Debias the gatekeepers as much as possible.
  2. Improve communication among gatekeepers and between the gatekeepers and the project team.
  3. Promote intellectual honesty across all decision makers and providers of information.

Each of these will be discussed in detail in the context of the case studies that follows. 

Stakeholders
AACE International defines stakeholders as “Decision makers, people or organizations that can affect or be affected by a decision” [8]. Dalcler [9] informally defines a stakeholder as “anyone affected, or potentially affected, by your work.” Bahadorestani et al [10] defines it as “individuals, groups or organization that have a stock or interest in project activities or outputs.” Recently, the Business Roundtable [25] revised its statement on the purpose of a corporation moving the emphasis from satisfying shareholders to that of satisfying stakeholders, which are described as customers, employees, suppliers, communities, and shareholders.

Embedded in these broad definitions is the notion that project success includes stakeholder criteria. Recent notions include the stakeholder as decision makers as opposed to creators or targets of value [11]. Thus, key project decisions such as project selection, allocation of resources, development of risk management strategy, managing contracts and the determination of the rights and responsibility of project participants all need to include key stakeholders. Nguyen [13] explicitly makes project success a function of stakeholder management and needs are always considered to be a critical success factor.

There are many ways to characterize stakeholders, but most focus on salience (the degree to which managers give priority to competing stakeholder claims [12]). Salience typically includes an assessment of stakeholder power, interests, attitudes [13] or more power, legitimacy, and urgency. [12] Aaltone promotes mapping stakeholders over time, across different project phases. Most project literature [22],[23] includes a stakeholder management section which typically includes stakeholder analysis, engagement, influence, and management strategies (such as inform, involve, proactively influence, collaborate, monitor, ignore, compromise, avoid or dismiss) [13]. Salhan [14] supports a power vs interest grid with strategies directly aligned with each 2 x 2 grid. Figure 8a and 8b shows some examples of stakeholder mapping to determine stakeholder salience and therefore which strategies to deploy. Figure 8a indicates that stakeholder salience is a function of stakeholder interest and power. Figure 8b indicates that once salience is determined, stakeholder’s can be further characterized by their level of project support. Further, stakeholder mapping should be done at various phases of the project since both salience and support can change over time. Smyth et al [15] explicitly states the natural conclusion of the above: that project managers need to consider more than just the iron triangle (cost, schedule, quality) and include stakeholder outcomes resulting from the downstream political, economic, and temporal risks of the asset created by the project.

Figure 8a–Determining Stakeholder Management via Power and Interest

Figure 8b–Stakeholder Mapping over Time Including Project Support

Stakeholders vary from project to project. Even in the simplistic case where a single company is executing a seemingly benign project to create a new asset, external stakeholders might include existing landowners, neighbors and the greater community, regulators (local, regional, state, federal – in the US) and utility suppliers (such as power companies, natural gas companies, water, and sewer authorities) as shown on Figure 7. Internal stakeholders will include the accountable person leading the project (ex: senior member of engineering organization), subcontractors, suppliers, treasury department, legal department, purchasing and real estate departments, operations department, and the business for which the asset is to be deployed. When projects get more complicated, the execution of the project starts to involve many other entities such as strategic partners, engineering contractors and their employees, construction subcontractors, FEED contractors, labor unions or groups, community activists, banks, owners, and public relations firms. And the most complicated projects have large roles for governmental entities (including ownership), multiple engineering and construction firms, project manager contractors, and many consultants (i.e., teams of lawyers, environmental and permitting consultants).

The key for the project manager is to understand which of these stakeholders is also a gatekeeper. This will change as the project progresses of course – each stage gate requires different interests. The focus of the rest of this article will be on the project authorization stage gate since this is typically the most important gate of project execution.

Some [16] have advocated that there are two approaches to stakeholder management:

  1. Management OF stakeholders. That is, getting the necessary resources from stakeholders.
  2. Managing FOR stakeholders. That is, stakeholders have a right and legitimate need for attention.

Depending on the project situation (small or large, simple, or complex, public or private) and specific stakeholders (salient or not, independent or interrelated stakeholders), each project needs to determine which of these two approaches to take.

Note figures 5 -7 reflect both internal and external stakeholders, regardless of an owner’s or contractor’s perspective. One question is which stakeholders will influence or should influence the decisions to be made by the gatekeepers. Another question is in which situations the project team should manage the stakeholders and when should the project be managed FOR the stakeholders. For example, in figure 7, the local community is shown as an exterior stakeholder. If the project is a public works project, the local community should influence the gatekeepers (and in fact they may very well be a gatekeeper) and the project needs to manage FOR the needs of the community. Cohesiveness, reasonableness, and supportiveness of the community will greatly determine the strategy to be deployed by the project team. But recognizing the local community as a stakeholder and managing them accordingly will greatly help the gatekeeping process and therefore the project.

Case Studies
To better understand the relationship between gatekeepers, stakeholders and decision making, two case study examples are provided – a simple project and a slightly more complex one. Truly complex projects, such as mega projects involving public entities will of course have much more complex issues. However, by focusing on two relatively simple projects, the essential issues become apparent and can be applied to more complex projects. The cases will illustrate the application of the three recommendations for improving decision making introduced earlier in this article.

Case Study 1: Simple Project

Project background 

This project will be developed, financed, constructed, owned, and operated by the same company, primarily using its own resources. The asset will be located in an industrially zoned section of a US city. The locality is excited by the prospect of new jobs and increased tax revenues. There’s a primary customer for the asset’s production and that customer is expected to sign a long-term contract for the primary products. The customer is a long-time employer of the locality.

Gatekeepers

The gatekeepers at time of project approval (sometimes called project sanction or authorization) are:

  • The business lead who will have profit and loss (P&L) responsibility for the asset.
  • The engineering lead who has accountability for the cost, schedule, and quality performance of the asset creating project.
  • Various corporate interests such as the lawyers (accountable for contracts and compliance), corporate real estate (responsible for land acquisition), finance (accountable for providing funding for the project), human resources (accountable for providing suitable resources), corporate purchasing groups (accountable for obtaining contracts for utilities and raw materials).

Note that these are all internal stakeholders. In this case the external stakeholders are determined to have low salience. Even though the project will affect neighbors (perhaps with noise) and require regulatory approval (say for emissions), the project approval stage holds the accountable internal stakeholders as the key decision makers or gatekeepers.

Decision-making

The project manager’s role in the project approval step varies from company to company but their signature is generally required on the approval documents. This signifies their accountability for cost, schedule, and quality requirements. They also need to clearly communicate project risks to the other gatekeepers. This is the most important job of the project manager during project approval – to provide the gatekeepers with a thorough and honest discussion of risks. John Hollman’s book [17] on risk quantification provides many excellent examples and techniques. But gatekeeping meetings must translate project risks for nontechnical stakeholders and must be done in the context they are familiar. Some methods toward that end are the front-end loading index (IPA) or project definition rating index score (CII). The discussions generated by these methods are typically more important than the absolute value of the score.

Note that in this simple case, the project stakeholders and gatekeepers are the same. That’s because in such a simple project, the external stakeholders are in a subservient role; even though they have a clear stake in the outcome of the decision (to approve the project) they have no voice in the decision. Thus, this is an example of management OF stakeholders.

Improvement of decision-making

Improvement of decision-making in this example relies on the following key elements:

  1. Debias: there’s a natural tendency toward optimism bias, biases in probability estimation, political bias, renumeration system bias and many others. [24] Other biases such as anchoring, cognitive inertia, confirmation bias, planning fallacy and other well-known biases [7] also come into play. Even in a simple example, human errors can easily slip into key decisions [20]. By simply being aware these biases exist, decision making can improve. In this simple project example, it is likely the gatekeepers are in their roles routinely and therefore have prior relationships. Depending on group dynamics, one effective method to debias is for a checks and balances approach where individuals can “call each other out” when exhibiting biases. Some examples:

    a.  The business lead might have optimism bias and challenge the schedule proposed by the project manager. The project manager should work to expose the bias based on facts, past history, what if analyses and logic.

    b.  The project manager might be biased by a previous project and be anchored in costs, risk or schedule assumptions that are not necessarily appropriate for the specific project. Reminding the project manager of other project performances and/or the specifics of the referenced project needs to be highlighted.

    c.  A stakeholder such as the end use customer might have a political bias and pressure the business sponsor that the proposed contract pricing should exclude needed environmental controls or actions. Exposing that bias by highlighting regulatory risks and realities is needed. 

  2. Communication skills: too many times, decision makers are not good listeners. If a gatekeeper is not listening properly to stakeholders, key points might be missed. Of course, the speaker needs to be clear, concise and have facts available to support their case. Examples:

    a.  A discussion on permit or land acquisition risks and contingency plans: Here the project might need an alternative site resulting in significant scope change later. But if this is poorly communicated at the project approval gate meeting (either poorly portrayed or not properly heard), future surprises might be in store.

    b.  Risks associated with the ability to obtain critical raw materials at a given price: This might result in changes to the product portfolio that the project needs to produce. And in turn, this could result in a scope change or a change to the sales forecasts.

    c.  The project manager’s risk discussion regarding construction productivity risks associated with a very busy environment: If gatekeepers don’t understand the manifestation of those risks might result in significant cost or schedule overruns, an incorrect decision to approve the project might be made. 

  3. Intellectual honesty: Too often, a gatekeeper has a preconceived notion and refuses to acknowledge key points being made. This might be because the gatekeeper is not interested in or does not respect information provided (or the individual providing the information). Earlier in example 2c, regarding construction productivity risk, the business leader may have been told by a colleague at another company not to worry about construction productivity because the other projects proposed (that is driving the forecasted very busy environment) will likely not happen. The project manager might provide sufficient proof to the contrary but if the gatekeeper is not intellectually honest about the input, they are putting the project at risk. Part of intellectual honesty is integrity; the author acknowledges that it’s very hard to counter individuals that don’t have integrity. But many people with integrity are not intellectually honest for other reasons such as: it’s hard work or takes too much time, aren’t intellectually curious, don’t want to look foolish (or admit that previous beliefs were foolish), biases and a lack of humility. Establishing norms which promote and incentivize intellectual honesty is important – both at an organizational level and a project level. [26] 

These three suggestions – debiasing, improved communication skills and ensuring intellectual honesty are typically intertwined. They are presented separately here only for illustration purposes. In practice, all three are necessary to improve the decision-making process.

Case Study 2: A Slightly More Complex Project

Project background 

While a mega public project would provide a starker contrast, a project that is not so complex makes a clearer illustration of the intended point. For this case, it is assumed that the project will be developed, constructed, owned, and operated by the same company using its own resources along with major engineering contractors but financed by others. The asset will be located in highly visible part of a US city on the land of the customer. While the locality is excited by the prospect of new jobs and increased tax revenues there’s likely local opposition from environmental groups. There is a primary customer for the asset’s production and that customer is expected to sign a long-term contract for the primary products. The customer is a long-time employer of the locality however has had environmental issues in the past. The project needs significant regulatory approvals to proceed, and that approval is not at all certain.

Gatekeepers

The gatekeepers at time of project approval are similar to the above simple case. However, a decision board (or project steering committee) of senior corporate executives might be appointed as a formal gatekeeping entity, especially if the capital project size is large or it is deemed a complex project. Plus, the project must now include regulators as a salient stakeholder. The project sponsor may decide to take the risk and approve the project prior to regulatory approval. But a denial of project permits would in effect shut down the project and therefore it is essentially a gatekeeper as well.

External gatekeepers now become important during project approval processes. The power and urgency of the regulator suggests that that the project needs to pay much more attention here. Thus, the political process (community and government relations and lobbying), public outreach efforts and closely working with the local company will be key. Each of these entities might be considered de-facto gatekeepers as well. The decision might be to treat manage the stakeholders rather than execute the project FOR the stakeholders. However, management of the stakeholders might be much more nuanced and balanced than in case study 1. 

Decision-making

In addition to all the factors discussed above, other issues start to come into play with complexity. Many companies have standard risk forms and processes which help put a specific project risk in perspective. Regardless, the project manager must ensure the documents properly communicate these risks in the context of their cost, schedule, and quality numbers. Flyvbjerg [6,18] has pointed out that through either omission, political sensitivity or outright lies (what he refers to as strategic misrepresentation), decision makers may or may not be thoroughly honest in these discussions. This is especially true with projects involving public entities and by extension perhaps public regulators. This applies not just in project risk discussion but in contract terms, sales forecasts and asset input pricing, quality, and availability. 

As previously discussed, there’s also a planning fallacy argument that states that organizations underestimate costs and overestimate benefits. This presumes either a nefarious element or systemic bias. While this has been debated in the academic community [19], it’s not difficult to see the planning fallacy at work. Even in private business, decision makers have biases that might manifest in lies or strategic representation. For example, a business sponsor might be promoted upon expansion of a business line based on the asset creation process. That individual might not be in position 3–4 years later when the asset comes on stream, or 6–10 years later when profitability can be determined. Therefore, they might feel less accountable for their actions and influenced to promote the project by down-playing risks or overstating sales projections. In this example, the decision to approve the project despite regulator concerns might be examples of over optimism.

Improvement of decision making relies on similar elements as discussed earlier:

  1. Debias: Reference class estimating has been suggested as one mechanism to debias decisions. Reference class estimating involves having independent eyes review the estimate to provide an outside view. The author’s experience in private industry is that in the estimating space, reconciliation to other data points (prior projects, previously approved estimates, or even benchmarked data) provides a much better methodology. Most projects are sufficiently complex that an outside view will simply miss nuances of the process, scope or estimate assumptions. Further in the reality of competitive bids, projects naturally need to be part of a continuous improvement process. Therefore, prior projects or an outside view might miss the fact that value engineering has been done to legitimately reduce costs or schedule relative to the referenced projects or estimates.
     
  2. Communication skills: On complex projects, where stakeholders might be inter- dependent, communication skills go beyond what was described in the simple project example. A member of the project team might be appointed with the sole task of stakeholder management, for example, and/or there will be a much more active role for corporate communications’ involvement, media relations and proactive messaging. Outreach sessions might be needed to proactively listen to the stakeholders prior to a gate keeping meeting. Further, higher level meetings with the regulatory agencies might be wise prior to an approval meeting to understand the level of support that is there. In the case of an air permit for example, gatekeepers should meet with the permitting agency, consulting firm, as well as existing customers to understand mitigation, technology and permitting strategies. This is an example of a pro-active communication on the part of the gatekeepers.
     
  3. Intellectual honesty: For more complicated projects it is more difficult since each stakeholder has their own interests at heart. Though the principles discussed under case study 1 still apply. Politics will be involved and therefore value judgements of different gatekeepers and stakeholders will be different. Of course, not all stakeholders have the same outcomes in mind. The ability of project team members to appeal to logic, consistency, precedent (other projects) as well as other decision makers can help draw out the intellectual honesty of the individual. Other influence techniques such as appealing to the individual’s mentor or other respected person or even their superior may also be needed. This requires relationship building between and among the gatekeepers and salient stakeholders.
     

Alignment
This all speaks to the issue of alignment. The Construction Industry Institute (CII) publishes best practices and one of the most important is alignment. CII defines alignment as “the condition where appropriate project participants are working within acceptable tolerance to develop and meet a uniformly defined and understood set of project objectives.” While CII work is focused within the project, many of their recommendations apply to stakeholders: Team building and team alignment, on boarding members and stakeholders, consideration of cultural differences and constraints of stakeholders, ensuring continuity of key team members and development a mechanism to check on stakeholder alignment throughout the project life cycle. [21] 

Project managers typically consider their teams consisting of engineering, procurement, construction, and commissioning team members. But by broadening the view team to include not only the key project team members, but salient stakeholders and gatekeepers, alignment starts looking differently. Ensuring alignment of both internal and external stakeholders is a key to success. When stakeholders are aligned, gatekeeping will be a much easier venture. The three improvements discussed above: debiasing, improved communication skills and intellectual honesty builds alignment as do the CII recommended practices listed above.

Impact of Misalignment
When the project authorization gatekeepers are misaligned with stakeholders, ill-defined and ill-prepared projects make their way through the process resulting in late changes and poor project results. This can happen when looking at even the simple project presented above.

The business lead is a key stakeholder but has goals which are typically not aligned with the project team responsible for execution. For example, the costs of going through the first three stages of project development are typically born by that business unit’s P&L if the project is ultimately not approved. If approved, the costs will be capitalized as an asset on the corporation’s books and therefore will not appear on the P&L. Thus, the business lead has significant incentive to limit project development and definition costs. The result of course is a potential for projects to lack proper definition. Of course, the business lead has a strong incentive for the project to succeed since both their customer and the long-term business P&L depend on it. 

This contrasts from the execution team’s perspective which is to deliver on cost, schedule, and quality requirements. Their incentive is therefore to spend more time and money on project development so as to improve project definition, per project best practices. This is a natural tension that plays out all the time and typically results in a compromise of project development costs, scope, and schedule. It will be up to the execution team to properly communicate risks associated with any shortfalls of project development.

Regulators such as an air permit agency have very different perspectives. They will be interested in compliance, public input, and the project’s response to their questions. At project approval stage, it is unlikely for a large project to have their permits approved, especially since downstream engineering and design are required to even make permit application. However, gatekeepers need to be aware of the risk associated with regulatory approval and therefore need to consider the regulatory agency’s input. They can’t simply “manage” this stakeholder away. Gatekeepers may rely on an outside environmental consulting agency to provide an independent assessment of risk.

On more complex projects, the project steering team may decide, for example, to enlist a community relations and lobbying effort. Their feedback may suggest that the project is not ready to be approved because of significant resistance from either political agencies or the public at large. This may put them at odds with the business lead.

Table 1 summarizes the various gatekeepers for these simplistic cases:

Gatekeeper/Stakeholder

Motivations

Business Lead

  • Minimize project development costs and schedule
  • Identify lowest reasonable project cost and shortest schedule
  • Satisfy customer

Engineering Lead

  • Develop full class 3 estimate
  • Uncover all risks
  • Develop budget and contingencies so performance is positive

Regulatory agency

  • Compliance, ensure best available control technologies used (as example)
  • Minimize time spent on reviews / negotiations
  • Avoid escalation to public OR Be as inclusive as possible (depends)

Table 1 — Summary of Various Gatekeepers for Simplistic Cases

Each gatekeeper has competing motivations and thus it’s easy to see the potential for misalignment. The business case might be based on getting to market quickly and at a certain budget. However, the engineered solution might take much longer, and cost more than originally anticipated due to either regulatory compliance or other unforeseen risk.

The issue of misalignment between stakeholders, gatekeepers and the project team can get more complicated very quickly. Figures 5-7 depicts even more stakeholders with even more varied motivations, each of which are potentials for misalignment. Furthermore, Figure 7 depicts the interaction between the external environment, the owner, the project, the EPC contractor, and the EPC’s sub-contractors. Between each of these layers, there exists stakeholders with various objectives and biases, communication challenges and opportunities for intellectual dishonesty. Both the external environment (with many stakeholders) and internal project environment (with many layers of responsibility) present opportunities for misalignment within the context of gatekeeping. And although beyond the scope of this paper, there are even further opportunities for misalignment in looking at more tactical and operational decision points as well.

Conclusion
This article reviewed gatekeeping as a series of strategic decisions from an owner’s viewpoint, though the perspectives of various contractors were also considered. Behavioral aspects of decision making were reviewed with emphasis on understanding decision making biases. Stakeholders and stakeholder management was discussed, noting the difference between internal and external stakeholders, as well as stakeholder salience. Projects are a series of decisions (strategic, tactical, and operational) and, from the owner’s perspective, stage gates are the most important strategic decisions. By adding stakeholders, these most important of decisions quickly become complex and subject to misalignment.

The article presented two case studies, both relatively simple projects relative to many mega projects. These simple cases demonstrate how stakeholders can be easily misaligned with gatekeepers. For the owner, no other decision is as important as project approval. The case studies suggest that even in simplistic projects, these decisions can be very complex. The same of course, holds for all the other decisions the owner, the project team, the EPC contractor, and individuals make. While it is easy to see the impact of poor decisions around project authorization, the same applies to any number of project level decisions and gates.

Project outcomes will be better aligned with the business case by improving our understanding of the decision-making process at key stage gates. Although the paper looked at rather simple projects, the potential for misalignment is clearly evident. More complex projects will have significantly more tension between stakeholders and gatekeepers, but the same principles will apply.

The article presented ways to improve project decisions by addressing bias, communication skills and intellectual honesty. When applied to gatekeepers and especially at the project authorization stage, these techniques can greatly mitigate impacts of misalignment. A savvy project team should facilitate these techniques to improve project outcomes.

Since the root cause of project failure often is a misalignment between the engineered solution and the owner’s business case, it is important for project teams to understand sources of misalignment. The focus of this paper is to explore one such source which is that stakeholders and gatekeepers have different objectives. By understanding and applying methods of stakeholder management and decision making, gate keeping can avoid being a source of misalignment.

REFERENCES

  1. Morris, P.W.G., 2013. Reconstructing Project Management. John Wiley and Sons, Sussex, UK
  2. Walkup and Ligon, “The Good, the Bad, and the Ugly of the Stage-Gate Project Management Process in the Oil and Gas Industry.”
  3. Cab Cabano, Stephen L., Paul G. Williams, and Robert C. Mathias. “The Owner’s Project Control Function,” 2011 AACE International Transactions OWN.503.
  4. L. Stephenson, Ed., Total Cost Management Framework: An Integrated Approach to Portfolio, Program and Project Management, 2nd ed., Morgantown, WV: AACE International, Latest revision, section 3.3.2.
  5. Stingl, Verena, and Joana Geraldi. “Errors, Lies and Misunderstandings: Systematic Review on Behavioural Decision Making in Projects.” International Journal of Project Management, 35, no. 2 (February 2017): 121–35.
  6. Flyvbjerg, Bent. “From Nobel Prize to Project Management: Getting Risks Right.” Project Management Journal, 37, no. 3 (August 2006): 5–15.
  7. Parth, Frank R. “Critical Decision Making for Project Planners and Controllers,” AACE International Transactions,2014, RISK.1504
  8. AACE International, Recommended Practice No. 10S-90, Cost Engineering Terminology, Morgantown, WV: AACE International, Latest revision.
  9. Dalcher, Darren. “In Whose Interest? Repositioning the Stakeholder Paradox,” PM World Journal (Issn:2330-4480) Vol IX, Issue IX, September 2020.
  10. Bahadorestani, Amir, Jan Terje Karlsen, and Nasser Motahari Farimani. “A Comprehensive Stakeholder-Typology Model Based on Salience Attributes in Construction Projects.” Journal of Construction Engineering and Management, 145, no. 9 (September 2019): 04019048.
  11. Derakhshan, Roya, Rodney Turner, and Mauro Mancini. “Project Governance and Stakeholders: A Literature Review.” International Journal of Project Management, 37, no. 1 (January 2019): 98–116.
  12. Aaltonen, Kirsi, Jaakko Kujala, Laura Havela, and Grant Savage. “Stakeholder Dynamics during the Project Front-End: The Case of Nuclear Waste Repository Projects.” Project Management Journal, 46, no. 6 (December 2015): 15–41.
  13. Nguyen, Tuan Son, Sherif Mohamed, and Kriengsak Panuwatwanich. “Stakeholder Management in Complex Project: Review of Contemporary Literature.” Journal of Engineering, Project, and Production Management, 8 (June 5, 2018).
  14. Salhan, Pranav. “A Study on Stakeholder Management of an Engineering Project.” International Journal of Engineering Research and V9, (June 5, 2020).
  15. Smyth, Hedley, Laurence Lecoeuvre, and Philippe Vaesken. “Co-Creation of Value and the Project Context: Towards Application on the Case of Hinkley Point C Nuclear Power Station.” International Journal of Project Management, 36, no. 1 (January 2018): 170–83.
  16. Eskerod, Pernille, Martina Huemann, and Grant Savage. “Project Stakeholder Management—Past and Present.” Project Management Journal, 46, no. 6 (December 2015): 6–14.
  17. Hollmann, John K. Project Risk Quantification. Probablistic publishing, Gainsesville, FL 2016
  18. Flyvbjerg, Bent, Massimo Garbuio, and Dan Lovallo. “Delusion and Deception in Large Infrastructure Projects: Two Models for Explaining and Preventing Executive Disaster.” California Management Review, 51, no. 2 (January 2009): 170–94.
  19. Ika, Lavagnon A. “Beneficial or Detrimental Ignorance: The Straw Man Fallacy of Flyvbjerg’s Test of Hirschman’s Hiding Hand.” World Development, 103 (March 2018): 369–82.
  20. Pinto, Jeffrey K. “Lies, Damned Lies, and Project Plans: Recurring Human Errors That Can Ruin the Project Planning Process.” Business Horizons, 56, no. 5 (September 2013): 643–53.
  21. https://www.construction-institute.org/resources/knowledgebase/best-practices/alignment
  22. Project Management Institute, ed. A Guide to the Project Management Body of Knowledge (PMBOK Guide). 2000 ed. Newtown Square, Penn., USA: Project Management Institute, 2000.
  23. Turner, J. Rodney, ed. Gower Handbook of Project Management, 4th ed. Aldershot, England; Burlington, VT: Gower, 2007.
  24. Williams, Terry, Hang Vo, Knut Samset, and Andrew Edkins. “The Front-End of Projects: A Systematic Literature Review and Structuring.” Production Planning & Control, 30, no. 14 (October 26, 2019): 1137–69.
  25. Business Roundtable. “Statement on the Purpose of a Corporation.” https://Opportunity.Businessroundtable.Org/Ourcommitment/, 2019.
  26. Samson, A. (Ed.)(2020). The Behavioral Economics Guide 2020 (with an Introduction by Colin Camerer). Retrieved from https://www.behavioraleconomics.com.

ABOUT THE AUTHOR
David C. Wolfson is with Pathfinder, LLC, and can be contacted by sending email to:
dswolfson@yahoo.com

]]>
https://source.aacei.org/2022/04/06/improving-project-gatekeeping-through-stakeholder-alignment-and-behavioral-decision-making/feed/ 0
Earned Value Management Systems for Operations Activities https://source.aacei.org/2022/03/02/earned-value-management-systems-for-operations-activities/ https://source.aacei.org/2022/03/02/earned-value-management-systems-for-operations-activities/#comments Wed, 02 Mar 2022 09:46:00 +0000 https://source.aacei.org/?p=4851

Earned Value Management Systems for Operations Activities

Dan Melamed, CCP EVP FAACE; Crystal Williams;
Lisa Ramdas; and Rodney Lehman

Abstract–An earned value management system (EVMS), which monitors contractor performance, is a requirement for program and project management for all major acquisitions by the United States Federal Government with development effort: (i.e., an asset requiring management attention because of its importance to an agency’s mission; high development, operating, or maintenance costs; high risk and/or high return).

As an area less explored in earned value management (EVM) practices, this paper will survey the application of an EVMS for operations activities defined as:

  • Non-capital asset activities that are projects (or project – like) with definable start and end dates, with discrete scopes of work, and measurable accomplishments; as well as
  • Routine or recurring facility or environmental operations.

This paper will examine the use of an EVMS to evaluate performance of operations and maintenance activities required once construction of a capital asset is complete and being used as intended. Such activities include upgrades and maintenance in order for capital assets to meet their mission function over a life-cycle (through repair, replacement, etc.).

This paper will provide background on this topic from the perspective of the Department of Energy (DOE) Environmental Management (EM) Program. In addition, it will provide material from a panel discussion provided by a group of experts from the October 2019 Office of Environmental Management Project Management Workshop, as well as material from subsequent research on this topic.

Introduction

This paper discusses the applicability for using an earned value management system (EVMS) for measuring performance of operations activities for government programs, specifically with a focus on the Environmental Management (EM) Program of the United States Department of Energy (DOE). EM was established by Congress in 1989 to manage the clean-up of areas where federal activities have contaminated the environment – such as nuclear weapons production facilities and military installations.

This paper will supply the necessary background to provide a better understanding for the role of operations activities in the management of project and program baselines as well as their budgets for the Environmental Management Program. The general requirements of an EVMS within the United States Government for capital asset projects and operations activities will be discussed. To support this effort, this paper provides definitions for both capital asset projects as well as operations and maintenance activities which very often coexist in environmental cleanup efforts and combine to form an EM program baseline.

The Government Accountability Office (GAO) – whose mission is to serve the Congress and the American people by collecting, analyzing, and reporting on information about federal programs and services was directed to review the EM program, with a focus on its management of operations activities. A summary of the results of their investigation, published in February 2019, stated “Until EM revises its cleanup policy to follow leading practices, EM’s operations activities are at risk of uncontrolled changes to scope, exceeding initial budget and schedule, and failing to meet their original goals.” [1, Highlights] As part of the response to this report, in October 2019, the EM Office of Project Management held a panel discussion by a group of experts on EVMS for operations activities during the Office of Environmental Management Project Management Workshop. The panel, listed in Appendix 1, consisted of one employee of the EM Program and four contractors, each with extensive experience with EVM and its application at a number of agencies, including previous experience at: National Aeronautics and Space Agency (NASA) and Department of Homeland Security (DHS). Their current work experience covers the following federal agencies:

  • The DOE EM Program (described above);
  • The National Nuclear Security Administration (NNSA) a semi-autonomous agency within the U.S. DOE, responsible for enhancing national security through the military application of nuclear science; [2, page 6] and
  • The Assembled Chemical Weapons Alternatives (ACWA) – a United States Army program dedicated to the disposal of chemical weapons in a safe, environmentally sound, cost-effective manner. [3 pages 12, 22, and 87]

The panel members’ experiences with these technically challenging programs, would provide good insight for best practices for the use of EVM in operations activities that can be readily applied to the EM Program. The next sections discusses and expands upon some of the major points addressed during the panel discussion, along with extra information not discussed during the workshop. This includes a discussion of an industrial sector where EVM for operations has already been implemented: the shipbuilding industry.

The Advantages of Earned Value Management

EVM provides a measurement of the work accomplished on a given project and/or program scope within a given period and compares it with the planned work scheduled for that same period along with the actual cost of work accomplished during that period of time. This provides improved understanding for project and program performance status, including an estimated cost and time to complete. In addition, EVM provides tools to manage risks, alerting management to potential problems and informing their forecasts to project completion. A general summary of what EVM enables is shown in Figure 1 (below).

Figure 1–A General Description of What Earned Value Management Enables.
[4, page 18]

Requirements for an Earned Value Management System in the Federal Government

An EVMS, is defined within the Recommended Practice (RP) 10S-90 Cost Engineering Terminology as a project progress control system that integrates work scope, schedule, and resources to enable objective comparison of the earned value to the actual cost and the planned schedule of the project. [5, page 46] This is required for many projects funded by the United States government to assure compliance with the Electronics Industries Alliance (EIA)-748 EVMS standard. The Federal Acquisition Regulation (FAR), which regulates purchasing of goods and services by the U.S. government, codified in the Code of Federal Regulations (CFR), provides the following requirements for an EVMS:

  • 48 CFR   34.2031 Solicitation provisions and contract clause:
    (c) The contracting officer shall insert a clause that is substantially the same as the clause at FAR 52.234-4, Earned Value Management System, in solicitations and contracts that require a contractor to use an EVMS.
  • 48 CFR   52.234-4 -Earned Value Management System:
    As prescribed in 34.203(c), insert the following clause:
    (a) The Contractor shall use an earned value management system (EVMS) that has been determined by the Cognizant Federal Agency (CFA) to be compliant with the guidelines in Electronic Industries Alliance Standard 748 (EIA-748) (current version at the time of award) to manage this contract.
  • OMB Circular No. A–11 (2019) Capital Programming Guide. [6, page 18]
    I.5.5.4) Planning for Acquisition Management

    The risk associated with the asset selected for consideration will determine the type of performance-based management system that should be used to monitor contractor performance in achieving the cost, schedule, and performance goals during the contract period. All major acquisitions with development effort will include the requirement for the contractor to use an Earned Value Management System (EVMS) that meets the guidelines in EIA Standard—748 to monitor contract performance.

Therefore, it is clear that the Federal government’s projects entailing construction of capital assets constitute major investments and thereby require an EVMS. Herein, a rigorous definition of a capital asset would be useful. This is discussed in detail in the OMB Circular No. A – 11 (2019) Capital Programming Guide (which uses the same language since June 2006), [6, page 2] that includes the following definition of capital assets,

Capital assets are land (including park lands), structures, equipment (including motor and aircraft fleets), and intellectual property (including software), which are used by the Federal Government and that have an estimated useful life of two years or more.

This definition continues [6, page 54] to include language relevant to government activities involving environmental cleanup:

Capital assets include the environmental remediation of land to make it useful, leasehold improvements and land rights; assets owned by the Federal Government but located in a foreign country or held by others (such as Federal contractors, State and local governments, or colleges and universities); and assets whose ownership is shared by the Federal Government with other entities.

OMB Continues [6, page 54] with some details, as well as specific examples:

Capital assets include not only the assets as initially acquired but also additions, improvements, modifications, replacements, rearrangements and reinstallations, and major improvements (but not ordinary repairs and maintenance). Examples of capital assets include the following, but are not limited to them:

  • Office buildings, hospitals, laboratories, schools, and prisons;
  • Dams, power plants, and water resources projects;
  • Motor vehicles, airplanes, and ships;
  • Satellites and space exploration equipment;
  • Information technology hardware, software and modifications;
  • Department of Defense (DOD) weapons systems; and
  • Environmental restoration (decontamination and decommissioning efforts).

This paper will focus on operations activities relating to environmental remediation as well as operations at facilities after their construction is complete. Discussion of weapons platforms, motor vehicles, satellites, space exploration equipment, software development and other information technology are beyond the scope of this paper.

Additional Requirements for an Earned Value Management System in the United States Department of Energy

DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, [7, page C-10] also documents the requirement for the use of EVM on capital asset projects.

An EVMS is required for all projects with a Total Project Cost (TPC) greater than $50M. In accordance with FAR Subpart 52.234-4, a contractor’s EVMS will be reviewed for compliance with EIA-748C, or as required by the contract.

This is supported by NNSA Policy (NAP) NAP 413.2, Program Management Policy, that requires NNSA programs covered by DOE Order 413.3B to follow its requirements (including an EVM System). In addition, NAP 413.2 also references NNSA Business Operating Procedure (BOP) 06.07, Program Management Policy for Weapons and Strategic Materials Programs2 for guidance on life extension programs, [8, page AP1-2] which directs federal program managers to:

Implement project controls (such as a requirements management system, an integrated master schedule, and earned value management protocols) to track costs and synchronize execution of program scope across the nuclear security enterprise and with DOD.

A Capital Asset Project Versus an Operations Activity

Differentiating a Capital Asset from Operations Activities

It is important to distinguish a capital asset from an operations activity.

As described in OMB Circular No. A – 11 (2019) Capital Programming Guide, capital assets are distinguished from operations [6, page 54] as follows:

Capital assets exclude items acquired for resale in the ordinary course of operations or held for the purpose of physical consumption such as operating materials and supplies.

Also, it is important to understand that the life-cycle cost of a capital asset, as defined by OMB Circular No. A – 11 [6, page 54] requires inclusion of operating costs:

The cost of a capital asset is its full life-cycle costs, including all direct and indirect costs for planning, procurement (purchase price and all other costs incurred to bring it to a form and location suitable for its intended use), operations and maintenance (including service contracts), and disposal.

It is worth noting that projects for the construction of capital assets also inherently contain some operations activities. In general, these operations activities are managed in a project’s EVMS as routine undertakings without discretely measurable products: as level of effort. Examples of EM Program operations activities within a capital asset include: a portion of the overall site’s landlord activities and site services such as infrastructure maintenance, emergency management, safeguards and security; health and safety; and land management.

Defining an Operations Activity

As a general definition the definition from RP 10S-90 Cost Engineering Terminology for “operations” and an “operating cost” [5, page 82] are as follows:

Operation – Ongoing endeavor, or activities that utilize strategic assets for a defined function or purpose.

Operating Cost: The expenses incurred during the normal operation of a facility, or component, including labor, materials, utilities, and other related costs. Includes all fuel, lubricants, and normally scheduled part changes in order to keep a subsystem, system, particular item, or entire project functioning. Operating costs may also include general building maintenance, cleaning services, taxes, and similar items.

The first definition provides a general description of the term while the definition of operating cost provides good examples. More specific examples can be found when focusing on a specific industry. The AACE Recommended Practice, RP 107R-19, Cost Estimate Classification System – As Applied in Engineering, Procurement, and Construction for the Environmental Remediation Industries, describes the general stepwise environmental cleanup process within compliance of the major Federal laws such as the Resource Conservation and Recovery Act (RCRA) and the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA). These projects often have an operations and maintenance phase [10, page 11]. In this phase, the cleanup project completes activities associated with operations and maintenance for the selected treatment or remediation alternative(s) as part of an environmental cleanup project. Operations activities in environmental projects, such as groundwater pumping and treatment, as well as radioactive waste, treatment, characterization packaging and disposition that end when clean up and/or waste treatment goals are met.

Looking within the EM program, a general listing of mission-related, stand-alone cleanup operating activities [9, page 4] is as follows:

Operating Activities: This category includes the retrieval, stabilization, packaging, storage, transportation and disposition of waste and nuclear materials; the operation of environmental remediation systems such as groundwater treatment systems; post-construction and post-closure care of remediated land burial sites; long-term environmental stewardship including environmental monitoring and institutional controls (e.g., maintaining a secure perimeter); as well as facility shutdown and deactivation activities designed to place the inactive structures, systems and components in a safe and stable configuration pending final decommissioning. Other EM activities in this category include: landlord activities and site services such as infrastructure maintenance.

It is important to note that the aforementioned is a partial list and that there are other EM activities that should be included in this category that are omitted to allow a focus on the most common operations. In order to provide a more comprehensive listing, EM Program mission-related, stand-alone cleanup operations activities also include: safeguards and security; emergency management; fleet (vehicle transportation) management; technology research, development and deployment; community-support, public outreach, regulatory support; and program management.

Operations Activities and Estimating Capital Asset Life-Cycle Costs

Operations activities such as those discussed above, are a key component for estimating the life-cycle costs for a capital asset. This can be seen in Figure 2 (below) which shows includes the costs for:

  • Constructing a capital asset,
  • Operations, as well as
  • Decommissioning, deactivation, decontamination, demolition, dismantlement, and disposition (commonly referred to as D&D) [11, page 3] at the end of the it’s life cycle.

Figure 2– A General Description of the components of the life cycle cost of a capital asset which contains the sum of the construction of a capital asset, operations and D&D.

Maintaining a rigorous EVMS for operations activities is a good means for both measuring these costs as well as a tool to help archive historical records of these costs to be archived in a cost database, a useful tool for estimating future operating costs. Accurate life-cycle cost information is used to determine future year budget needs, project and program costs, manage changes in project scope and technical approach, as well as prioritize work activities. As an example, life-cycle cost analysis is required for making a business case for programmatic budget decisions. This is discussed in OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs, which provides guidance for the methods to perform cost-benefits analyses for U.S. government programs. In addition, life-cycle cost estimate is critical for the evaluation of capital asset management. These life-cycle cost analyses are vital for a variety of budget processes, managing changes in project and program scope and technical approach, as well as to prioritize work activities. [9, pages 1-2]

Another example found in environmental projects operating within the CERCLA regulatory framework the selection of a specific remedy (or remedies) for the site clean-up is always done on a life-cycle cost basis. [10, page 8]

Operations Activity as Production

DOE Order 413.3B states that once construction of a facility is completed (also known as Critical Decision 4, CD-4), there is a transition to operations. This includes activities such as the development of operations and maintenance manuals, identification of the necessary budget, and the procurement of any materials required for initial operations. However, it is important to note that in certain cases (depending on the facility built) operations will also include initiation of production activities. Some examples of operations activities from the DOE EM Program as well as the DoD that involve production activities from completed facilities are provided below:

  • Salt and sludge processing for dispositioning the waste in the Savannah River Site (SRS) High Level Waste (HLW) tank farms in Aiken, South Carolina. [12, pages 4-6.]
  • Uranium Mill Tailings Disposal – the Moab Uranium Mill Tailings Remedial Action Project. [13, page 14];
  • Transuranic (Radioactive) Waste Disposition Advanced Mixed Waste Treatment Project (AMWTP) located at the Idaho National Laboratory (INL), Idaho Falls, ID. [14, pages 3-4.]; and
  • Destruction of Chemical Weapons (Demilitarization) – Various facilities [e.g., Pueblo Chemical Agent Disposal Pilot Plant (PCAPP), Blue Grass Chemical Agent Disposal Pilot Plant (BGCAPP) as well as the Umatilla Chemical Agent Demilitarization Facility (UMCDF) and four other completed projects] [3, pages 13-15.].

A significant point is that the aforementioned production/operations activities align to discrete EVM techniques facilitating their successful management. This has been discussed in a white paper produced by NDIA, “Earned Value Management in a Production Environment,” [16, pages 4-7]

The Government Accountability Office (GAO) Review of the EM Program’s Operation Activities

The GAO was directed by Congress to review the EM program, particularly its management of operations activities. This is significant because operations activities constitute the vast majority of the program’s work. As an example, in the fiscal year 2019 budget, operations activities accounted for over 75 percent of EM’s budget while capital asset projects accounted for under 20 percent of EM’s budget. Using the definition of operations discussed in this paper, a significant fraction of these activities may be classified as production, particularly where facilities within the EM Program process radioactive wastes for disposition.

In February 2019, GAO issued the results of their investigation as a report to Congress. “DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices,” (GAO-19-223) [1, page 44] that stated:

EM’s EVM systems for operations activities are not comprehensive, do not provide reliable data, and are not used by EM leadership to measure overall performance of the EM program.

Information from this report was also highlighted in the update to the GAO Cost Estimating and Assessment Guide (GAO-20-195G) [15, page 15]

As part of a response to this challenge in the GAO Report to Congress, the EM program organized a panel discussion Earned Value Management Systems for Operations to further address these challenge areas. The results of that panel discussion, as well as a review of the existing literature is provided in the remaining points that follow.

Topics to Consider for Creating an EVMS that Includes Operations Activities

Using Production Metrics to Measure Earned Value

As mentioned earlier, a significant portion of the operations activities in the DOE EM Program, as well as the DoD, involve production activities from completed facilities (e.g., the Moab Uranium Mill Tailings Remedial Action Project, the AMWTP and the destruction of chemical weapons (Demilitarization)). Production metrics should be an important component in the use of EVM for these types of activities.

The EM Program divides its cleanup work into six work areas that sometimes include both operations activities and capital asset projects. A significant portion of the EM Program’s cleanup work is managed through operations activities that involve production in facilities at several sites dispositioning wastes. Some examples of these facilities, their connection to examples of the EM cleanup work area (specific cleanup activity), as well as examples of specific metric associated with those facilities are shown in table 1 (below):

Table 1–A listing of the EM cleanup activity with examples of sites with operating facilities dedicated to the disposition of both solid and liquid wastes. Also listed are the metrics associated with the production activities at these facilities.

A key requirement for EVM is to measure the percentage of work completed, which is vital for determining the earned value at any point in time. The accuracy of the calculated earned value hinges on a reliable reporting system. In this case, a percentage completion rule that is based upon production metrics is recommended. As an example (shown in Table 2 below), predetermined annual completion milestones for each month of a specific year may be: 10, 25, 33, 40, 50, 60, 65, 75, 80, 85, 95 and 100 percent. This could reflect the metric targets for the first year in a five-year contract period of performance. This (notional) site has an estimated 4,500 cubic meters of waste, the contractor has developed a baseline where it will remove 250 cubic meters per year over a five-year period. The table below shows the first year listing the monthly targets as:

  • Individual targets
  • A cumulative target (a running total for the current year); and
  • As a percent complete of the annual target,

Table 2. A listing of the targeted quantities of cubic meters of waste to be disposed at a (notional) site for (1) year, (2) a five-year contract period, and (3) its life-cycle. Also provided is a listing of monthly metrics for a calendar year both as cubic meters of waste as (1) a monthly target (2) as a cumulative total and (3) as well as percent complete.

The status of each active task completed in the effort reflected above is reviewed for a specific time period and updated. The extent of the work completed is measured for a given time period which is then compared to the planned value of the work for the period as well as the actual cost of the work. With these metrics, the project’s EVM is measured and updated.

As was discussed earlier in this paper, once construction of a capital asset is completed, the facility transitions to operations that (depending on the facility) may initiate production activities; which may continue years after that facility starts operations. An example would be the Moab Uranium Mill Tailings Remedial Action Project in Utah which started as a capital asset project (to establish physical infrastructure) with an approved baseline and a construction start in August 2008. The Moab Project then transitioned to an operations activity program in 2010, initiating the transport and disposal of uranium mill tailings and other contaminated materials, collectively known as residual radioactive material currently scheduled to end in 2025. For this program, the production metric: the disposal of uranium mill tailings is distributed over a 15-year period. This metric is subject to a wide variety of systemic variables (e.g., funding variations) as well as site-specific risks (e.g., a rockslide near the railroad area that occurred in in November 2014). [13, pages 14-15] Therefore, operations activities in the EM Program, in particular those involving production, as described in the aforementioned GAO report (GAO-19-223), can be readily quantified, measured, tracked, reported, and managed (including addressing risks) for a given performance period (spanning a number of reporting periods over many years).

The Challenge of Minimizing Level of Effort (LOE) Activities

A significant finding in the GAO Report on Operations Activities at DOE EM Sites [1, page 39] involves how activities are categorized within an EVMS:

  • A large portion of the work is categorized as level of effort which has limited utility for EVM.

This requires a clear discussion of how activities involving production (discussed above) are categorized in an EVMS [16, pages 20-21]: production support labor is defined as all the other functional labor activities that assist, support or inspect the manufacturing labor efforts (e.g., tooling, quality, production control, industrial engineering, design engineering, manufacturing engineering, manufacturing supervision and other related disciplines). These activities are treated as level of effort (LOE) as it is not known when support will be needed and therefore it cannot be discretely planned. LOE work:

  • Has the same budgeting requirements as discrete work;
  • At a given time, Earned Value = Planned Value (regardless of the amount of work performed);
  • Has to be scheduled with a start and finish date; and
  • Lacks any milestones that could be used to measure performance.

This description of LOE, is also consistent to that given in RP 79R-13, Level of Effort Planning and Execution on Earned Value Projects – Within the Framework of EIA-748. [17, pages 2-3.] In addition, although these LOE tasks may have discrete deliverables (e.g., monthly reports), they do not represent physical progress of readily measured (i.e., discrete) efforts. Budget is scheduled and distributed over the period of performance, however, when an LOE task is performed during a given period, the value budgeted for that task is always claimed.

Also listed in RP 79R-13, [17, pages 2-4] is a possible solution to this challenge: thoroughly examine all work scope classified as LOE within a project or program with a goal and see if it can be reclassified into one of the two following categories:

    • Discrete Effort – defined as “Tasks that have a specific measurable end product or end result” Discrete work has defined products. Discrete work is measurable and can be broken into steps or quantifiable tasks, each of which can be budgeted based on resources required to complete the work.
    • Apportioned Effort –defined as: “An earned value technique in which status is assessed consistent with a discretely measurable base task(s). The earned value percent complete of the base effort is used to status the apportioned effort work package. Apportioned effort is technically related and time-phased proportionally to the based unit designated”

Lastly, RP 79R-13 recommends that LOE activities should be cordoned off as separate within its own control account (CA). Otherwise, LOE activities can distort the true performance of the CA’s EV data. Therefore, if LOE is embedded with discrete work activities in a CA, it should be reorganized to remove the LOE. [17, page 4]

An example of shifting the application of LOE from an environmental activity to apply apportioned effort instead (as the Earned Value Technique-EVT), is the use of radiation safety technicians-a necessary part of an environmental cleanup project. Often such activity is designated as LOE because it is not known precisely when the technician’s support will be needed and therefore it cannot be discretely planned (16, page 21). However, a portion of the radiation technician’s work can be apportioned when associated with the characterization, packaging and dispositioning radioactive waste as shown in Table 2 (see above). Therefore, the radiation safety technician’s portion of the work relating to the disposition of radioactive waste can be more discretely measured.

Challenges in Calculating an Accurate Estimate at Completion (EAC)

The estimate at completion (EAC) is the estimated total cost for the authorized work scope, defined as the sum of the actual cost of work performed (AC) to date plus an objective estimated cost to complete (ETC) of the remaining work scope. [18 page 3] For operations activities within a program baseline establishing an accurate EAC is challenging. As demonstrated by a large portion of the EVM literature, EVM works well when there is an end point and deliverable completed at the end of a discrete project life-cycle (e.g., a facility or weapons platform). However, many operations activities can continue for an extended period far exceeding the duration required to construct an actual facility. For such operations activities, their associated “end points” are not defined by the completion of a specific deliverable (e.g., construction of a facility). Instead, the operations activities are defined by the duration of their associated work scope defined within a contract period (or some other discrete period of performance). As an example, in the case of the Moab Project, the necessary capital project has already been completed several years ago. Currently, there is a specific quantity of uranium mill tailings required to be retrieved and dispositioned over a specific contract period of performance (an estimated 15-year period over the life-cycle of operations starting in 2010 and projected to end in 2025). [13, page 13-14] Estimating an accurate full EAC to disposition the mill tailings over a contract period (as distinct from the remaining life-cycle) can be a challenge to accurately and effectively communicate as part of the standard reporting requirements for an EIA 748 compliant system. In addition to conventional challenges that environmental cleanup activities are prone to unique uncertainties, [19, page 4] including:

  • Uncertainties concerning the extent of the quantities and types of contaminants;
  • Delayed site cleanup (often the only option considering limitations to available funding) resulting with increased contamination spread;
  • The requirement for multiple regulatory approvals with potential changing requirements; and
  • Unforeseen environmental science and technology challenges

These uncertainties have implications to establishing and maintaining accurate EACs and executable performance baselines for environmental cleanup in general.

EVM Tailoring

In general, an EVMS has found effective application in capital asset projects. This can be seen in the following quote from Office of Management and Budget (OMB) whose mission is to assist the Executive Branch in carrying out their budgetary duties. OMB Circular A – 11 provides guidance for federal agencies to prepare their budget requests from Congress. A portion of OMB Circular No. A–11 (2019), Capital Programming Guide [6, page 15] discusses tailoring an EVM:

Implemented properly, an EVM system will measure progress against a baseline and provide an early warning of cost overruns and schedule delays. Most likely, a practical application of EVM will involve tailoring the principles to a project’s unique circumstances.

The GAO Cost Estimating and Assessment Guide (GAO-20-195G) discusses the possibilities for tailoring [15, page 217] an EVM (defining scope with a Work Breakdown Structure (WBS)) for short term and small dollar projects:

EVM can be tailored and ingrained into the culture to ensure that program cost and schedule goals are met for smaller or shorter projects; smaller projects will benefit from having the work scope defined by a WBS and having a detailed plan and schedule for accomplishing the work. Small-dollar projects still need to have a baseline in place to manage changes and variances and risk management plans to address issues.

In addition, the NDIA also produced White Paper EVMS Guideline Cross Reference Addendum [20. Page 2] which discusses tailoring a WBS: since typical manufacturing operations control accounts may not be created at the lowest level of the WBS. Therefore, the depth of a WBS may be tailored in a production environment.

Lastly, the Integrated Program Management Data and Analysis Report (IPMDAR) Data Item Description (DID) Tailoring and Implementation Guide provides general guidance for tailoring an EVMS. Some examples [20, pages 53, 75-80] of what can be changed with an EVM Tailoring Approach:

  • Rigor of Variance Analysis – Variance Thresholds.
  • Formality of Change Control process – Work Authorization.
  • Deployment of Charge Numbers – Cost Collection Level.
  • Level of Reporting (e.g., Level of Reporting (Work package, Control Account, WBS, etc.).
  • Contract Performance Report (CPR) Formats.
  • Reporting Frequency (Monthly, Weekly, Quarterly, Annually, Incremental, Upon Request).

The Use of EVM for Operations in Shipbuilding

A review of the literature shows that there is substantial use of EVM applied to operations in the shipbuilding industry.

A RAND corporation study done for the United Kingdom’s Defense Procurement Agency (DPA), discusses how shipbuilders track project and program progress [22 page 31] “Nearly all shipbuilders use earned value management to monitor the progress of production and design.” This point was elaborated upon in an article presented at an AACE meeting in 2018 [23, page 5]: “It is common practice in shipyards to follow the ANSI/EIA earned value management (EVM) standard in order to comply with federal contract requirements, especially for military ships. The budgeting process, accounting considerations, and governance features are well-defined in this standard.” Lastly, the NDIA White Paper EVMS Guideline Cross Reference Addendum which discusses specific differences between a development and production effort or a shipbuilding environment [21, page 1] details the relevant differences between the environments required for tailoring for the shipbuilding industry. These three publications provide an overview for the effective use of EVM within the shipbuilding industry.

A specific example of EVM tailoring can be found discussing managing a WBS. Canadian surface ship and submarine owners provide a common framework for the overall planning and control of an entire ship life-cycle; this framework, called a ship work breakdown structure (SWBS) has to be mapped properly to the WBS, and the organizational breakdown structure (OBS) [23, page 5]. The NDIA discussed very similar processes for an Expanded Ship WBS (ESWBS) that must be mapped to the externally reported WBS [21, pages 2-3]. In addition, the RAND study there is a strong requirement to limit level of effort activities [22 page 18]. A good example of limiting LOE in a shipbuilding EVMS had also been provided: when production support tasks that cannot be readily measured on their own, or which cannot be broken down into discrete work packages but which are related to the planning and performance of other measured effort, are measured using the apportioned effort technique [22, pages 20-21].

The uses of EVMS for operations that is mature within the shipbuilding industry can serve as a resource for lessons learned and examples for implementation EVMS for Operations for the Department of Energy EM Program. This is especially interesting in areas where the shipbuilding industry deals with environmental issues such as when the United States Navy has to dispose of its obsolete National Defense Reserve Fleet vessels in a manner that maximizes financial returns to the US. However, the ship dismantling must be done in an environmentally safe manner due to concerns about the environment and worker safety and health [24, page 1].

Conclusion

This paper describes the advantages and requirements for an earned value management system as well as a description of the requirements of an EVMS for government programs.

Along with the requirements for an EVMS for government projects, this paper also provides a detailed definition of capital asset projects, as distinct from operations activities, including specific examples within the EM Program. Also included is a discussion of some of the challenges that impact the effective use of EVMS for operations activities (especially production) including a discussion of the uncertainties involved with measuring operations in an environmental cleanup program such as:

  • Uncertainties with the extent of the quantities and types of contaminants;
  • Delayed site cleanup (often the only option considering limitations to available funding) resulting in greater contamination spread as well as increased scope; and
  • A complex regulatory framework;

An effective EVMS can provide effective performance controls, an understanding of operations activities, as well as a means to accurately record their historical costs which are required for estimating accurate life-cycle costs. Life-cycle costs are vital for determining budget priorities, managing changes in project scope and technical approach, as well as to prioritize work activities.

In addition, there is discussion of some of the practices that would best enable the use of EVM to measure operations activities, including:

  • EVM Tailoring;
  • Using Production Metrics; and
  • Minimizing the use of Level of Effort Activities.

Lastly, this paper discusses some of applications of an EVMS for operations that is mature within the shipbuilding industry.

References

  1. US General Accountability Office. “DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices,” GAO-19-223 (February 2019).
  2. Planning the Future: Methodologies for Estimating U.S. Nuclear Stockpile Cost, Frank Douglas Lewis; Cash Fitzpatrick; Matt Proveaux; Cole Lillard; William P. Todd, Cost Engineering, Vol. 58, No. 05, AACE International, Morgantown, WV, 2016.
  3. Remediation of Buried Chemical Warfare Materiel, National Research Council 2012. Washington, DC: The National Academies Press; (Taken from the internet January 1, 2021: https://doi.org/10.17226/13419)
  4. Earned Value Management Systems for Operations – A Panel Discussion a presentation given at, Virtual Workshop – sponsored by U.S. Department of Energy, Office of Environmental Management, Office of Project Management (EM-5.22), Washington, DC, October 21, 2020.
  5. AACE International, Recommended Practice No. 10S-90, Cost Engineering Terminology, Morgantown, WV: AACE International, Latest revision.
  6. The United States Government Capital Programming Guide (a supplement to the United Government Office of Management and Budget, Circular A–11: Planning, Budgeting, and Acquisition of Capital Assets).
  7. DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, Change 6, January 12, 2021. ([Taken from the internet January 1, 2021: http://www.emcbc.doe.gov/Office/ProjectManagement]
  8. BOP 413.5, Program Management Policy for Weapons and Strategic Materials Programs
    [Taken from the internet February 6, 2021: https://directives.nnsa.doe.gov/bop/bop-0413-005.]
  9. U. S. Department of Energy (DOE) Office of Environmental Management (EM) Cost Estimate Development Handbook, EMCBC-OOCE G 002 Revision 1, August 22, 2019 [Taken from the internet January 1, 2021: http://www.emcbc.doe.gov/Office/ProjectManagement]
  10. A Discussion of the Cost Estimate Classification System: As applied in the Engineering, Procurement, Construction and Operations for the Environmental Remediation Industries, Dan Melamed, CCP EVP; Bryan A. Skokan, PE CCP; Gregory Mah-Hing, PE; Rodney Lehman; Jake Lefman, 2020 AACE International Transactions, EST-3542, AACE International, Morgantown, WV, 2020.
  11. Interstate Technology & Regulatory Council (ITRC) Technical and Regulatory Guidance Document: Decontamination and Decommissioning of Radiologically Contaminated Facilities (RAD-5) 2008; [Taken from the internet March 20, 2021]: https://itrcweb.org/GuidanceDocuments/RAD5.pdf
  12. U. S. Department of Energy (DOE) Savannah River Site Liquid Waste Plan SRR-LWP-2009-00001 Revision 21, January 2019 [Taken from the internet January 24, 2021: https://www.energy.gov/sites/prod/files/2019/05/f62/SRS-Liquid-Waste-System-Plan-January-2019-0.pdf]
  13. Managing a Large Mill Tailings Cleanup Project, Donald R. Metzler, 2015 AACE International Transactions, OWN.2057, AACE International, Morgantown, WV, 2015
  14. Progress and Lessons Learned in Transuranic Waste Disposition at the Department of Energy’s Advanced Mixed Waste Treatment Project, J. D. Mousseau, S. C. Raish, F. M. Russo, WM’06 Conference, February 26 – March 2, 2006, Tucson, AZ
  15. US General Accountability Office. ” Cost Estimating and Assesment Guide, Best Practices for Developing and Managing Capital Program Costs,” GAO-20-195G (March 2020).
  16. Earned Value Management in a Production Environment, Production Earned Value Management Working Group, Adams et al, National Defense Industrial Association, October 10, 2011.
  17. AACE International, Recommended Practice No. 79R-13, Level of Effort Planning and Execution on Earned Value Projects – Within the Framework of EIA-748, Morgantown, WV: AACE International, Latest revision.
  18. AACE International, Recommended Practice No. 80R-13, Estimate at Completion (EAC), Morgantown, WV: AACE International, Latest revision.
  19. Tailored Risk Management Assessment Tool, Bryan Skokan, Dan Melamed, Rodney Lehman, R., and Jake Lefman, RISK.1284, , 2013 AACE International Transactions, AACE International, Morgantown, WV. 2013.
  20. Integrated Program Management Data and Analysis Report (IPMDAR) Data Item Description (DID) Tailoring and Implementation Guide (DI-MGMT-81861B) issued 12 March 2020.
  21. White Paper EVMS Guideline Cross Reference Addendum, Production Earned Value Management Working Group, Adams et al, National Defense Industrial Association, January, 2015.
  22. Monitoring the progress of shipbuilding programmes: how can the Defence Procurement Agency more accurately monitor progress? Mark V. Arena, John Birkler, John F. Schank, Jessie Riposo, and Clifford A. Grammich. 2005, RAND Corporation (Taken from the internet April 8, 2021: https://www.rand.org/pubs/monographs/MG235.html).
  23. Integrated Cost and Schedule Model for Shipbuilding Projects, Ahmad Emam, 2018 AACE International Transactions, CSC-2855, AACE International, Morgantown, WV, 2018.
  24. Ship Dismantling Cost Review, Robert C. Creese; Pooja Sibal, 2001 AACE International Transactions, EST.03, AACE International, Morgantown, WV, 2001.
  25. RP 107R-19: Cost Estimate Classification System – As Applied in Engineering, Procurement, and Construction for the Environmental Remediation Industries, Morgantown, WV: AACE International, Latest revision

Appendix 1
A listing of the Members of the Panel for EM Program’s Earned Value Management Systems for Operations Activities

Table 3–Members of the panel Earned Value Management Systems for Operations along with a summary of their experience.

]]>
https://source.aacei.org/2022/03/02/earned-value-management-systems-for-operations-activities/feed/ 4
Successful Design Scheduling https://source.aacei.org/2021/10/05/successful-design-scheduling/ https://source.aacei.org/2021/10/05/successful-design-scheduling/#comments Tue, 05 Oct 2021 16:38:26 +0000 https://source.aacei.org/?p=4076

Successful Design Scheduling

Christopher W. Carson, CEP DRMP PSP FAACE;
Aaron Fletcher, PSP; Noah A. Jones, PSP;
and Leo Carson-Penalosa

Abstract–Delays often originate within the architectural and engineering (A/E) design effort. The schedules developed to plan, organize, and monitor design tend to be high-level and not very effective to accurately model the work at a level of detail that allows for on time, on budget, and quality completion. When the schedule does not provide the right level of detail or complexity, its value for monitoring is limited. Sometimes there is even a failure to recognize the difference between consumed hours and progress. Failure to use the right schedule can lead to performance issues resulting in late design delivery, over budget delivery, or poor-quality design delivery, or any combination thereof. A well-designed and managed A/E design schedule promotes quick and accurate updates, supports proactive analysis to minimize delays and claims, and aligns with other project controls functions to enable integrated cost-schedule-risk design scheduling. The authors, working for firms that provide engineering design services, have experience in working with the designers to develop the right level of detail for the design portion of a project, to establish a stage-gate approach to design milestones so they can align with cost, schedule, and risk monitoring, and so performance can be accurately measured. The authors bring a wide range of perspectives, from process engineering design scheduling to design-build A/E scheduling, to construction manager (CM) agency A/E monitoring, to CM at risk A/E support scheduling. This article will offer a proven approach that demonstrates guidelines for schedule design, development, monitoring, analysis, updating, and reporting, as well as set the benchmark to facilitate mitigation when necessary. This article was first presented as PS.3427 at the 2020 AACE International Conference & Expo.

Introduction

Design scheduling is handled by several different parties, or parties working in several different roles and contracts. A/Es might work directly for the stakeholder or employer on a project, a contractor in the form of a design-builder, or a contractor working as a concessionaire for a public-private-partnership (PPP or P3).

Background

To discuss this issue, it is important to define design scheduling and establish the value of providing professional critical path method (CPM) scheduling to plan and manage the design process. Many projects fall behind during the design process, in design-bid-build (DBB), design-build (DB), engineering-procurement-construction (EPC), and construction manager at risk (CMAR). Each of these project delivery processes has its own challenges, but design schedule management is a primary one. The complexity of architectural-engineering (A/E) design is mostly because of the variety of disciplines necessary to complete a project, the scope development maturity, and the ability to assess progress in design. At a given stage of design, say 30% design development, the structural design might be at 60% complete, while commissioning might be at 0% complete. Developing a schedule of this sort can be difficult and monitoring can be even more difficult.

What is Design Scheduling?

Design scheduling is the act of planning the design activities, developing a CPM schedule for those design activities, and maintaining that schedule through a regular updating process that captures progress information, assesses performance, predicts trending and delay issues, and provides corrective actions to enable the design to complete all milestones on time.

How is it Commonly Handled?

Typically, A/Es develop and maintain their own progress schedules using different software and methods. Since the schedules are most often not shared with other stakeholders, there is a reduced need for a high level of technical scheduling effort as would more often be seen in the construction process. However, all designs evolve through phases, a series of increasing levels of scope definition, and assign milestones representing the completion percentage at the end of each phase. In many projects, especially the more successful, the milestones act as the gates at which cost and schedule reviews are performed to ensure that the design is progressing and maintaining the original budget and duration established in the conceptual stage. As cost and schedule are verified at each stage, the gate opens and allows design to move into the next phase.

This is often called a stage-gate approach, and studies have shown that when done appropriately, the project is more likely to achieve the design-to-budget goal, defined as ensuring that the design stays within the planned budget as it evolves to full maturity. Meeting this goal is enabled by a formalized process of detailed review at the different levels of scope definition, ensuring that the budget, schedule, and risk issues have been updated and validated. Combined with a good contingency process, this evolving process of projects controls support and verification, helps identify any design issues that are at risk of driving up budgets and extending schedules.

Value of Design Scheduling

Depending on contract type, many projects cannot proceed into procurement and then construction without design progressing as planned. Many projects under DB and EPC delivery methods progress to procurement and or construction without a completed design. EPC or DB project durations tend to be both quite long duration and technically complex projects that often require multiple phases of design. For these projects, the design effort is often scheduled in the CPM schedule. In both cases, design costs are related directly and only to labor, so productivity and production are both important. Issues in design delivery are caused by taking more time than anticipated or quality concerns. Using the stage-gate approach limits both potential sources of over run.

Risks of a Weak Design Schedule

If the A/E does not have a plan for completion of the design within the planned duration, it is unlikely that they will engage the right individuals, or the number of individuals needed, and the design effort will fall behind, ultimately delaying the start of the project.

There are also risks associated with the designers working without project controls monitoring, so the scope of work can grow, or features can be added, since there is no feedback related to the costs of the design, or the feedback comes after the design is complete. This is called scope creep when additional scope is added, often unintentionally, or gold-plating, when the A/E designs more expensive features than originally budgeted, which will increase the costs and duration often unbeknownst to the stakeholders.

Benefits of a Good Design Schedule

A well-designed and managed design schedule will allow the project to meet the stakeholders’ goals in progressing through the project phases into procurement and then construction as scheduled and within budget. Conceptual project schedules will show time allowed for phases of design, producing increasing levels of scope definition or design maturity, until final construction documents are produced.

As design progresses, according to the original schedule, staff work is assigned consistent with the design budget. Senior, more expensive, designers are assigned for the innovative and technical requirements. More junior, less expensive, computer design operators produce the details and production drawings. Each stage-gate milestone receives reviews to ensure compliance with budget, schedule, constructability, value planning, and risk. When done properly, the design matures in a controllable fashion, the stakeholders are aware of the adherence to the goals, and the phase-gate process allows corrective actions when there are deviations from the original benchmarks.

Studies Related to Design Scheduling

The value of this effort is reinforced in a 2014 study, by McGraw Hill, which notes that, “The ability to develop a set of documents that meet the stakeholder’s program requirements and are constructible within budget is unanimously cited as the most-valued metric for design team performance . . .” [page 6, of reference 8]

In the same study, the second highest cause of uncertainty with greatest impact on cost is stakeholder program or design changes and the third and fourth are design errors and design omissions, with the program design changes ranking significantly greater than the design errors/omissions. [page 14, of reference 8] Progressing through the design process with a weak phase-gate process can allow design development that is focused only on the design and not the budget or schedule, making it subject to scope creep. Many design changes can cause delays to other disciplines and hours expended on designing features or even options that may or may not be adopted. Without a phase-gate approach delays to time and budget, due to accumulated changes, can and do go underestimated until the end of the design process causing overruns in budget and/or time. The impact on the schedule can be surprisingly significant since budget overruns that are discovered late in the design process will often create a need for cost reductions. These cost reductions, commonly poorly named as value engineering efforts, can become increasingly critical as the team is trying to get construction work started. Value engineering should take place much earlier in the design process, in fact, it can be done as soon as there are drawings to review and the next step after value planning. However, when the budget is overrun and there is pressure to release construction drawings to start work, what is called value engineering is usually just quality reductions, materials changes, or even scope deletion. A detailed state-gate project controls effort integrated with the design process can help ensure the project is designed to the original budget, or if there are legitimate changes that require budget increases, the stakeholder is notified early enough to minimize impact on the schedule.

Project Delivery Methods Associated with Phases of Design

The type of project delivery method contributes to some of the uncertainties in A/E design since in traditional delivery the A/E has ultimate quality control that is tested by the bid market. In many other delivery methods, the contractor either takes over the design or is intimately involved in monitoring design as it advances through the stages from conceptual to bid or tender documents.

Design-Bid-Build – Complete Design = Stakeholder to 100%

The traditional, private sector project delivery method typically involves three sequential project phases: The design phase, which requires the services of a designer who will be the designer of record for the project; the bid phase, when a contractor is selected; and a build or construction phase, when the project is built by the selected (typically low bid) contractor. This sequence usually leads to a fixed-price contract. [page 3, paragraph 2 of reference 7] In a design-bid-build type delivery method, the stakeholder will be responsible for most if not all aspects of the design.

A risk to a stakeholder with a design-bid-build delivery method is that they warrant the sufficiency of the plans and specifications to which the selected contractor will ultimately build the project. This means the stakeholder/designer will be responsible for any issues that may arise during construction related to errors or omissions in the plans. There is little or no opportunity for the designer and contractor to collaborate during the design phase, which can ultimately contribute to time and cost overruns due to undiscovered design errors. Further, without input from experienced construction expert staff, the design will likely be less constructible and efficient.

These design errors or omissions can often stem from lack of planning during the design. A well-developed stage-gate approach with clearly defined design deliverables helps the teamwork towards distinct achievable interim goals as opposed to three or four major deliverable dates. Also, early involvement of the contractor or construction manager can help to recognize many of these potential issues from a construction point of view. For this reason, part of the stage-gate approach should include constructability review. Constructability reviews are part of the evolution from value planning in pre-design to value engineering and constructability reviews during design as drawings are produced. These quick reviews by construction experienced individuals ensures that designs can be built as they bring eyes of physical, real-world constraints to the design. Questions generated during this process always add value and support a more robust design.

EPC – Stakeholder to ~15%, Contractor to 100%

In the engineering, procurement, and construction (EPC) type of project delivery method used commonly in industrial and process industries, the stakeholder performs studies or research, decides to go forward with a project, determines the results needed from the project, develops the appropriate scope definition to ensure understanding of the stakeholder’s needs, then hands the scope definition over to the selected contractor who proceeds with advancing the conceptual phase design, moving on to feasibility, to front-end engineering and design (FEED), to detail design, then finally to implementation.

Generally, these are lump sum/fixed price contracts. This places the risk of final scope definition and performance on the contractor. In addition, it makes it easier for the stakeholder since managing one contractor is simpler than managing several contractors, and much simpler than interceding in the historical conflicts between contractors and designers. Using an EPC firm will allow for clean lines of communication from the design to construction phases and less mistakes because of the team being unified under one company or contract. The stakeholder must be clear with their goals in the early program and communication throughout the project. Without a clear vision there will be scope ambiguities and potential scope creep and it will be difficult for the EPC firm to plan, monitor, and control the project. The main disadvantage for the stakeholder will be less design control.

The success of the EPC firm will depend on the robustness of their planning and scheduling, which will reduce confusion and conflicts in the field where the costs for rework are much higher than costs of design changes earlier in the process.

Design-Build – Stakeholder to ~30%, Contractor ~30% to 100%

Design-build delivery method in commercial projects is similar to the EPC delivery method, however, this concept is commonly used in many industries outside of the EPC process world, particularly as a means to shorten the construction duration and limit risks to the stakeholder for performance. The stakeholder details the drawings to the point at which it defines the scope of work without eliminating opportunities for innovative design by the contractor. The documents that the stakeholder prepares for use in bidding and procuring the contractors are often called “bridging documents” or the “program definition” documents. The contractor includes the detailed design firm on the DB contractor team, so there is a more engineering-construction integrated approach to design. The contractor can detail the design in specific scopes of work to the point at which it allows construction to proceed, and it accommodates early release of some design packages, advancing the construction. An industry term for this staged release of early trade contract work packages is often called “fast-track” construction, and there are implementations relying on fast-track delivery such as “advanced work packaging” that formalize the effort and helps improve the quality of the transition to construction.

CM at Risk (CMAR) Project Delivery – Stakeholder to 100% with CMAR Providing Project Controls

With CMAR delivery, the CM acts in the interests of the stakeholder, providing project controls expertise and construction experience to help monitor design by VE, constructability reviews, estimating, and scheduling. This allows the CMAR to coordinate with the designers to plan and schedule early trade contractor work in packages that allow construction to start earlier than would be possible with full design required before bidding. This method is commonly contracted on a cost-plus-a-fee in the early stages to enable the CMAR to provide these services, but most often, converts to a fixed fee for a fixed duration contract once the scope definition has advanced sufficiently for an accurate bid. Risks to the stakeholder lie in the reputation and competence of the CMAR, as well as the full engagement of the CMAR in validating the budget and schedule of the design as it matures. However, with an experienced and competent CMAR, the project can sequence trade work and shorten the duration while minimizing stakeholder risk. Once the CMAR is willing to convert to the fixed price and duration, his role is more akin to a traditional general contractor, taking on all the risk of performance.

Alignment of Design Stages and Project Controls with Project Delivery Method

Designs mature through the design phases and so project controls services must track with estimates and schedules aligned by the level of scope definition and the usage of the budget and schedule. The characteristics of the estimates and schedules and the alignment with scope definition are best explained in the AACE International recommended practices for cost and schedule classes. [References 2 and 3, (page 3 in both publications)].

Pre-Design Phase
In the pre-design phase, there are no drawings to define the scope of work. The project is defined by a statement of work or other narrative description of the project. Estimates and schedules are very high-level, providing analogous or parametric level products with a wide range of accuracy to the lack of scope definition. AACE Class 5 estimates are typically provided during this phase, as well as AACE Class 5 schedules. However, these early budgets and schedules are the beginning of the benchmarks against which the project will be monitored and analyzed, so it is vital that pre-design phase estimating and scheduling is appropriate and reasonable.

At this stage, value planning is useful as it helps provide guidelines as the project moves into early design using appropriate techniques and products for the location and industry.

The delivery method at this phase has generally not been determined so the project controls services are typically provided by the stakeholder, whether in-house or by consultant. Engaging the project controls professionals during pre-design provides guidance in selecting an appropriate project delivery method and is a strong indicator of a successful project.

Concept Phase of Scope Definition
In this phase, project controls services continue to be comprised primarily of developing conceptual budgets in the form of capacity factored or parametric modeled AACE Class 5 concept screening estimates, preliminary schedules in the form of AACE Class 5 concept screening milestone schedules, and value planning to help determine the construction means and methods. In some industries, when the professionals providing estimating and project durations have extensive experience and benchmarked databases, the range of accuracy can be narrowed for both budget and schedule, which reduces risks during later phases.

Design Phase
In the preliminary design phase, there are likely to be sketches, plan layouts, line drawings, and site concepts. However, as design progresses through this phase, further development will result in much more detailed drawings. The design phase is divided into sub-phases based on design development progress stages. Good practices suggest that completion of each phase should result in a thorough cost, schedule, risk, and scope review to ensure that the design is progressing satisfactorily and can be built using efficient techniques, at the cost specified in the budget, and within the planned project duration. With the stage-gate process, the design engineering halts until the cost, schedule, risk, and scope review verifies that the design still fits to the budget and project duration.

Concept Development – to 15%
In this phase, project controls services are comprised primarily of developing study budgets in the form of equipment factored or parametric modeled AACE Class 4 estimates [1] for study or feasibility purposes, planning milestone and project events AACE Class 4 schedules [2] for feasibility study purposes. During this phase, scope definition is primarily still the responsibility of the owner.

Design work is an iterative process. As a result of the nature of this work there is a tendency to only want to track hand-offs between disciplines in the schedule for all the phases, especially the conceptual phase. However, the more detail in the inter-discipline relationships, the better. The best practice is to plan for the known scope then adjust as the details become clearer. Tracking only hand-offs never yields an accurate schedule or a schedule that can be tracked and reported on in a valuable way. In schedules that only track hand-offs the logic ends up needing to constantly change to match what is really going on, which is unnecessary work. Planning for the known scope, then making adjustments, also helps in successive phases because the teams may change (and probably will), but if the new team can see what happened in the previous phases in detail, then they can avoid unnecessary rework.

Schematic/Scoping – 15% to 30% (FEED 2 Package)
In the 30% maturity of scope definition, the design phase is likely to be the start of detailed design documents, providing more scope definition. Project controls in this phase are comprised primarily of developing sanction budgets once again in the form of budget authorization AACE Class 4-unit cost and line-item estimates, and more detailed planning AACE Class 3 authorization or sanction schedules [2]. At this point in the scope definition maturity, the responsibility for design depends on the project delivery method, as noted above. However, the discussion here about the project controls services supporting design monitoring may be provided by the owner, the A/E, a consultant to the owner, or the contractor when the delivery is design-build or EPC.

During the approximate 15% to 30% design, commonly referred to the schematic and preliminary design stages of the design, the key opportunity is to ensure the design is beginning to align design deliverables with the construction schedule needs. The CPM schedule can now start to support further refinement of the deliverables and specify how interim design stages will be monitored and tracked. One of the goals and focuses during this stage of scheduling for design should be establishing design definable features of work for the major deliverables.

Establishing these interim design features of work deliverables can help to:

  • Define task flow through each major deliverable
  • Establish critical path dependencies necessary to drive a design deliverable to completion
  • Help alert the project team to specific needs required by one another.

As discussed earlier in this article, key design deliverables are typically tracked to certain percentages throughout the life of the design, but the problem is assigning progress to these percentages. To successfully do this, decisions need to be made as to what percentages apply to disciplines at each stage gate. It is during this stage of design that the designer should include additional design players, i.e., sub-consultants, out-side agencies, or the technical team to collaboratively discuss roles and responsibilities as it relates to the design and schedule. It is important during this early stage to collaboratively agree on interim design delivery milestones to help ensure clear deadlines providing scope definition for each related discipline. These discipline-specific scope definition completion percentages should be used in scheduling.

During the conceptual design stage, a milestone schedule was developed to establish the major deliverables. This stage of design is where the schedule should start to move from conceptual/milestone driven to evolve into a more detailed logically driven schedule that contains these discipline-specific scope definition completion percentages.

See the following list for some possible expected design submittals goals during the 15% and 30% stages that can influence the schedule planning:

15% (Schematic Design)
Develop design draft schematics and concept layouts
Initial opinions of probable construction costs
Determine that the concept is consistent with the design budget

30% (Preliminary Design)
Finalize basis of design and advance spatial design
Design is suitable for an initial constructability review
Begin drafting the project manual

The 30% preliminary design phase is also the stage where not only the design schedule criteria is developed, but the construction implementation of the project should also start to be included in the project schedule. This should help the design team better understand how the project will likely be constructed with insight provided by experienced construction managers (CM). This will of course require involvement of CMs to assist the project team in bringing their construction background to the table. Their input early in the design process will beneficially impact design, ultimately yielding a better, more efficient, and more complete design, as well as promoting early resolution of design problems before impacting the schedule.

Planning to support the design submittal goals occurs during this stage; with products such as contract time determination, schedules used to establish contract delivery dates, and pre-bid contractor’s schedules from a select group of approved contractors to help align the construction with early design. These schedules will need to be reviewed by the stakeholder and are used for confirmation of procurement decisions and to start monitoring progress at the various stages to ensure they are on track either through durations or interim milestones.

Another often overlooked value to the design schedule at this stage is to start to establish critical paths through the design criteria so the CPM schedule can ensure contributing disciplines provide their submittals on time to allow successor design to proceed. This effort will increase in later stages as more detailed designs are developed.

Collaboratively establishing design deliverables will significantly improve the quality and efficiency during this design stage. While this stage of design is still relatively early, construction and schedule input now could lead to significant value engineering savings that would not affect bidding documents.

Design Development – 30% to 50% (FEED 3 Package)
In the 50% maturity of scope definition, the design will be expanded to more detailed drawings in multiple disciplines, requiring considerably more coordination between the disciplines. Project controls in this phase are comprised primarily of developing control budgets in the form of control AACE Class 2 estimates [1] providing detailed unit cost and line-item take-off estimates and bottom-up planning AACE Class 2 schedules [2] for control or bid/tender use. These estimates and schedules are provided by different entities based on the project delivery method. For design-bid-build, the owner still has the responsibility and provides the integrated project controls support, either in-house or with a consultant, for CMAR, the CMAR provides the support sometimes with the owner providing a program or construction manager to validate the results, and with design-build or EPC, the contractor starts to pick up the integrated project controls support to their own design effort. These responsibilities for the integrated project controls services are consistent through the rest of the stages of design maturity.

Construction Documents – 50% to 80%
As the design matures from 50 to approximately 80% complete scope definition, the design is being finalized, disciplines are coordinated in a feedback loop, quality control is implemented with check sets and/or building information modeling with 3D drawings. Project controls in this phase are comprised primarily of developing check estimate budgets in the form of bid/tender Class 1 detailed unit cost with detailed take-off estimates at the same level of detail as contractor bid documents, producing detailed planning Class 2 or possibly Class 1 bid/tender control schedules [2].

Bid/Tender Documents – 80% to 100%
The goal of this design stage is typically to produce a complete set of drawings and specifications that have passed internal quality control and coordination and are ready for construction. The design is suitable for final client review, permit application and allows an opinion of probable construction cost and project contractual duration that will serve for review and selection of the contractor. This is also the stage for a final bid document review to ensure everything aligns with stakeholder and third-party requirements. Lastly, the final goal is to publish the bid documents and supplemental information for the market.

Depending on the project delivery method, the responsibility for completing the design can vary. The scheduling effort during this final stage will help ensure each of the design package criteria is met with all submittals and is complete. Accurately modeling the final stages can ensure all reviews or meetings are scheduled in advance to help ensure a collaborative communication between all design team members or stakeholders.

Recommended Practices for Design Schedule Development

Phase-Gate Approach

The approach that is commonly called a phase-gate approach to project controls support during design has proven success when followed appropriately (Phase Gate is addressed by the Construction Industry Institute in discussions of Front-End Planning effort). This approach uses the design development phases or stages noted above and recommends the use of an approval gate at each stage. When the design has developed to match the stage of scope definition, design is not allowed to proceed until a thorough review of the cost, schedule, risk, constructability, and value engineering shows that the project is still tracking within the target benchmark, opening the gate to allow design to proceed into the next phase up to the next gate.

Figure 1–Phases and Gates from CII Front End Planning Process

Clear Program Definition
Clear understanding of the design statement of work or concept write-up is vital to establishing the design definable features of work. This starts with the design package criteria that clearly states each design submittal requirement; “The project will require 500 feet of pipe” is an example of a poorly defined feature. A better-defined example would be that the project requires 500 linear feet of RCP and two drainage structures for adequate site drainage across a specific area.

The scheduling effort will be less effective if the contractual requirements for the schedule are vague and non-specific. Often, the schedule specifications clearly state the requirements for construction, therefore a construction schedule is typically detailed to build the project. So, the question needs to be asked: “Why can’t the schedule specifications require similar level of detail to monitor and track design?”

The schedule specification is the stakeholder’s opportunity to ensure the designer monitors progress appropriately for the full design effort. It is crucial that the design and construction documents are well developed and clearly depict the stakeholder’s needs. This is especially true for design-build, and CM at-risk type deliveries since most of the project risk falls to the contractor, which can limit the reporting to a stakeholder. Generally, the schedule is and will be one of the best communication tools between the designer and stakeholder for all delivery methods. Therefore, a vague or non-specific schedule specification can reduce the accuracy of information provided to the stakeholder. This tends to lead to unreliable schedule predictions as well as over or underreported progress that may lead to late design and design cost overruns. An appropriate schedule specification should address the following items.

Establish Completion Milestones (Phase-Gates)
There is no single standard for the percentage of design completion at each milestone, so it is vital that these are established to accommodate the needs for quality control review, value planning and engineering, constructability review, budget and schedule review, and risk assessment. Sometimes these milestones align with payments or are designed to enable certain package releases. Establishing these milestones is important and should be carefully reviewed by the team to ensure that the scope of design work required by each milestone is defined and understood. With the appropriate definition and team understanding, progress monitoring is improved to allow successful design at the appropriate time for the expected project duration.

Define Detailed Scope Required before Each Milestone (Phases)
Once the phase-gate completion milestones are established, the detailed scope required to be completed in each phase must be identified. There should be milestones for each phase, they could be in the form of deliverables. It may be an approval package or a submittal depending on the phase. The requirements for each milestone must be specifically identified and the level of quality outlined. Note that depending on the project delivery method, the stakeholder responsible for accepting the final design package might be the stakeholder or the contractor.

Document Specific Completion Definitions to Support Project Controls Review
For this approach to work well, it is important that the milestones are well defined so monitoring the completion of each phase-gate milestone is straightforward and simple. This is a tougher problem than it should be because the cumulative percentage of completion for any given milestone includes a wide variety of different levels of completion for many disciplines. While a 30% design completion might show structural design as 60% complete, commissioning may not have even started. No matter how progress is tracked, the definition allows a good estimate and evaluation of completion. The authors have found that resource loading of the schedule using design hours and costs is the best approach when the designer team participates. However, when the designer is not cooperative in establishing discipline hours and costs, there are other simple implementations of progress monitoring, such as earned value based on the number of drawing sheets, which can still be useful. No matter the progress monitoring, the stakeholder must specify completion goals and the designer still must identify which sheets must be complete to achieve the gate milestone terminating each phase.

The scheduler should meet with the designers, get a list of all the disciplines that must be completed to meet the milestones, and then determine with the designers how mature each discipline must be at each milestone. Developing a clear definition of each milestone with detailed understanding of the maturity of each discipline is vital to implementing this approach.

Key Risks in Design Schedule Development
Poor or little validation of the key design milestone completion percentages can often be the primary contributing factor to a design submittal failure. Without the use of a stage-gate approach to scheduling, a design firm’s progress monitoring of design deliverables can be inconsistent and inaccurate. Often one design firm’s definition of 30% drawings is very different from another firm’s idea of the 30% design submittal. Or, after submission of the deliverable, the submitted package is only approvable to 20% complete based on design criteria. These discrepancies can stem from a design firm’s inability to accurately model and track to design submittal deliverables, which can affect cost, time and quality. The following list of questions should be considered when updating and progressing the planning design schedule:

  • What is the overall estimated completion percentage for the design package?
  • How will the deliverable milestones be measured, by money, or hours expended?
  • How are the design definable features of work developed?
  • Do the costs and hours in the activities in each submittal requirement equal the total submittal budgeted cost/hour?
  • Do the activities clearly define the measurable deliverables?
  • What is the level of effort?
  • Is the schedule able to monitor the design milestones, i.e., 15%, 30%, 60%, 90%?

The importance of accurately modeling and updating the schedule can significantly increase the reliability for the design submittal deliveries. This is achieved primarily through the appropriate modeling of the activities and managing the stage-gate project controls reviews to reach the overall submittal requirement. By appropriately validating each design submittal milestone completion, potential issues should be identified and an acceptable status for the submittal goal established.

Develop Activity Detail per Milestone
This will help define the specific activities included in the schedule that must be completed before opening the gate to the next phase. This is the area where the A/E input is needed because most A/Es have internal planning documents that suggest the disciplines and the discipline completion that should be associated with each milestone. The scheduler will need to collaboratively acquire this information and then validate it with the A/E team to ensure an appropriate detailed design schedule that models the expected maturity of the individual discipline documents.

Establish Resource Availability
Establishing design resource availability by discipline should be done after activities are identified and sequenced, and then activity durations can be determined. A key part is knowing what is needed (i.e., civil structural architectural (CSA), electrical & instrumentation (E&I), process, plant design, mechanical), as well as knowing what level of resource is needed (i.e., junior/senior level), to provide the expected design development.

Knowing the full design scope is paramount to acquiring the team needed. The next step after establishing the scope is understanding the complexity of the scope to help the designers determine the mix of senior and junior staff needed for the disciplines. Of course, the designers will consider the risks of using junior personnel (i.e. increased durations, quality of work) and evaluate those risks with the costs of more senior staff providing the work.

Once the resources required are known, then the process of modeling staff availability can begin. Once the resources are developed in the scheduling software, the next step is to establish the discipline resource calendars. Later the team’s schedules can be incorporated into the master schedule to ensure that all other required work is planned and monitored, and performance in design work will be sufficient to maintain the program goals.

During the establishing of resource availability, consideration needs to be given to construction equipment availability since the delivery will be a function of the approved design releases for procurement. The design schedule should show any of these or other long-lead procurement items that may have to be ordered or reserved during the design phases to have it in time for the construction. With an accurate design schedule, the stakeholder can determine if the schedule will allow for the selected contractor to order these items or if the tight schedule might require the stakeholder to consider direct purchase. These are decisions that should be evaluated as soon as possible after the design schedule is approved.

Identify Time and Cost Budget (Load Hours and Costs)
Once milestones are defined and resources established, the resource loading is pretty straightforward, except that it has to meet the milestone completion. This requires resource loading the schedule, which is the individual discipline costs and hours broken down into the work that is associated with each milestone definition.

Suggested Practices for Design Schedule Monitoring and Analysis

Enforce the Stage-Gate Approach
Monitoring of design is straightforward once the definitions are set up, so this step simply requires adhering to the project controls services identified for each of disciplines. The stage-gate approach to integrated project controls monitoring of design deliverables is relevant to any of the project delivery methods. No matter whether the delivery is design-bid-build where the owner is at risk for design, or CMAR, where the owner and CMAR collaborate to minimize the risk of design, or design-build and EPC where the contractor takes over and assumes full responsibility for design, the stage-gate approach improves design durations and quality.

Monitor Design Progress at Activity Level
The main goals of accurately modeling and updating a construction CPM schedule are to monitor progress, identify delays early enough to mitigate, provide corrective actions to maintain planned progress, provide change management support, and avoid disputes in construction. With design scheduling, the goals are reduced to monitor progress, provide corrective actions to maintain planned progress, identify changes from the project controls reviews, and provide change management support.

Much of this can be achieved through a combination of critical and near-critical path analysis and CPM schedule trending analysis with design scheduling because design combines a productivity effort with a sequencing effort; multiple teams take on the design concurrently, but each team has cross-discipline relationships with the other teams that drive delays across the disciplines. The trending analysis is most useful in activity performance trending, such as duration overruns and plan adherence monitoring, rather than just sheet count or earned value metric trending. The updated project schedule is the primary monitoring tool for the project. It provides the benchmark for analysis of progress and delays. Review and approval is the stakeholder’s opportunity to ensure a quality update schedule that will serve as the model for analysis of change, trending and completion predictions.

The first step in the update process, prior to any data input, is to gather the data. The scheduler should provide the update report to the team and responsible parties at regularly scheduled intervals based on the frequency of the update process. Suggested information to provide in the update report or layout is: activity identification code, activity name, original duration, remaining duration, percent complete, float, actual start, actual finish, with a blank column for remarks.

One aspect of planning includes assigning specific roles and responsibilities for measuring the progress and performance. During the schedule development process, the scheduler should assign responsibility to each activity in the schedule. In Primavera P6™, this is easily accomplished using an activity code (i.e., responsibility).

Using a responsibility activity code allows the scheduler to break down activity progress by the discipline or individual performing or responsible for the work. Each responsible party can, by organized use of filters and layouts, review a report of work progress and effectively plan for near term activities. The scheduler can then also more easily identify potential issues and mitigate risks based on performance factors.

Use Earned Value Management
Since the design scheduling effort is heavily oriented to production and productivity along with sequencing progress, monitoring is well served with a focus on the productivity and production. A critical and near-critical path analysis ensures that delays are identified when they occur, such as a failure of the architect to provide initial floor plan layout to the structural engineer. Non-critical path delays can also occur, so it is important to monitor the progress of all activities, including non-critical path activities.

The typical failures in design scheduling occur when cross-discipline hand-off dates are missed, progress falls behind, or budgets start to overrun. With performance issues, too often the designer simply redefines a poorly defined milestone to claim completion at the planned level, say 30% completion. But when that milestone has been defined in detail, that redefined 30% completion milestone is identified as more likely 20% or 25% and that has serious ramifications for the range of accuracy of budgets and schedules associated with that phase.

Monitor Hourly Earned Value for Progress
Architectural and engineering design is budgeted and monitored in hours, so loading the schedule with hourly resources is relatively straightforward. There still is a decision to make with respect to budgeting; if the firm has targeted specific designers, then there might be a need to load individual resources, designers by name. This would allow for monitoring to ensure that each person has the appropriate workload and will not be over allocated by the schedule as performance is updated.

More commonly, hourly resources are loaded by roles; grouping individuals by levels or types of roles such as Engineer, Senior Engineer, or perhaps Engineer Level II. The premise is that anyone who carries the title, for the project, of one of the roles is interchangeable with anyone else, so if four Engineers are Senior Engineers, the loading would allow monitoring of over allocation of the group of Senior Engineers.

This role-based approach allows for a certain amount of flexibility and ensures that tasks assigned to Engineers Level I will only be scheduled for the Engineers in that category, so it is easier to maintain the budget. The loading and monitoring is also simplified since specific person performance does not need to be reported.

Monitor Cost Earned Value for Efficiency/Productivity
Although monitoring hourly earned value works well, consumption of hours does not necessarily relate to progress. While it is important to monitor hours, that does not indicate the efficiency of the work, and with the likelihood that there is a range of salaries within each role, the risk is that too many hours are consumed at the high end of the salary range. The design firm needs the ability to recognize when they are at risk of exceeding budget even when operating within the hourly totals for each phase.

This risk can be managed by a loading of costs in addition to the resources into the schedule. The cost-based earned value system can be used to monitor the productivity while the hourly-based system monitors production. Running metrics from both loading will help the design firm stay on schedule and budget.

Provide Critical and Near-Critical Path Analysis
Providing critical and near-critical path analysis is vital as it will ensure that the discipline work that must be sequenced is managed and monitored. For example, the mechanical engineer designing the plumbing system requires multiple input to proceed with 30% preliminary design piping sizing, which might include:

  • Plumbing fixture requirements from the architect
  • Drainage requirements from the process engineer
  • Fuel-fired boiler information from the HVAC mechanical engineer

If the plumbing design cannot proceed without the necessary input, delays from the one discipline can be magnified as other disciplines are unable to complete significant portions needed for the team to meet the 30% preliminary design milestone.

Provide Trending Analysis
Trending analysis is the use of historical records to identify a pattern of production or productivity results that will eventually cause cost and time overruns. These patterns are often insidious and some areas are not easily identified without a detailed schedule used to monitor performance.

An example of schedule trending is the duration overrun risk; if the structural steel engineering has performed the first 25% of their scope in durations that are twice as long as planned, analysis of that trend might show that continued performance at that same rate will cause them to miss a stage-gate by 30 days. Those types of delays will be magnified as they are driven into other disciplines waiting on the structural steel designs, and the trending analysis requires the CPM schedule to determine the expected impact from the delays.

Other trends are more productivity-based only, and could be managed without the schedule, for example, if a project task required 400 sheets of engineering drawings and committed to 40 days to complete engineering, that means the team must complete 10 sheets per day.

If they get 20 days into the task and have completed 100 sheets, they have completed 5 sheets/day. This trends to:

20 remaining days x 5 sheets/day = 100 sheets by day 40
100 sheets (complete) + 100 sheets (trend) = 200 total sheets by deadline

This performance would miss the target by 50%, AND, with proper tracking, scheduling and stage gate milestones the overrun could have been prevented.

Provide Mitigation Needs and Opportunities
One of the benefits of a good scheduling effort is the ability to provide predictive analysis to identify slippage in performance and budget, and the resulting corrective actions that can be recommended to mitigate the delays. Done properly, this can allow the A/E to re-assign resources to supplement or replace weak performers, ensure the right level of experience in each task, and improve the achievement of the assigned level of completion stage.

The same is true of gains; analysis of schedule trending when there are improvements in performance will show potential opportunities to improve delivery dates.

Conclusion

One of the common failures in construction programs is the timely production of architectural and engineering drawings. This shows up as missed partial completion milestones, under-designed milestones, late delivery of completed packages, and the risk of incomplete packages that are released just to meet deadlines.

The further symptoms and risks from this failure revolves around the scheduling and budgeting that is performed at the wrong range of accuracy because the documents are not really completed to the appropriate level of detail in all disciplines. The range of accuracy is dependent upon the maturity level of the documents, so inaccuracy in the level of completion will limit the ability to produce a more accurate project duration (and budget) as the design matures.

A commitment to thorough technical scheduling at a level of detail that allows appropriate monitoring of production and productivity would improve design completion results, with the accompanying improved ability to design-to-budget and schedule. Owners would benefit from requiring and funding an integrated project controls design effort by reduced project durations and lower costs, no matter the project delivery method.

The integrated project controls phase-gate effort has been proven to improve the completeness of the deliverables and the confidence in the measured level of maturity of those deliverables. [5]

While studies related to design performance rarely isolate scheduling of the design, CII has a number of studies of assessments for measuring project scope definition and alignment between project participants that have a strong scheduling component for design stages. One of those studies notes that the index “allows its users to measure the level of scope definition and to compare scope definition to anticipated project success.” This study looked at a Project Definition Rating Index (PDRI), comprised of a number of contributing factors which included the scope definition and appropriate maturity of the scope supported by design scheduling, and found an average “Schedule reduction by 13 percent” in the projects scoring higher in pre-project planning (all the design leading to detailed engineering). [4]

This commitment to an accurate and detailed technical design schedule requires time and costs to develop and monitor the schedule, but the authors have seen huge losses to A/E firms due to designs that take considerably longer than planned and lead to unhappy stakeholders. The costs to provide an integrated stage-gate project controls effort are much less than many of these losses. All the stakeholders, whether owners, contractors, or designers, would see benefits from spending the additional costs up front to provide this effort.

REFERENCES

  1. AACE International Recommended Practice No. 18R-97, “Cost Estimate Classification System – As Applied in Engineering, Procurement, and Construction for the Process Industries”, Revised March 1, 2016.
  2. AACE International Recommended Practice No. 27R-03, “Schedule Classification System,” Revised November 12, 2010.
  3. AACE International, Recommended Practice 56R-08, “Cost Estimate Classification System —As Applied in Engineering, Procurement, and Construction for the Building and General Construction Industries,” Revised August 7, 2020.
  4. Construction Industry Institute, “Pre-Project Planning Tool: PDRI for Buildings,” 1999.
  5. Construction Industry Institute, CII Research Team 331, “Assessing the Maturity and Accuracy of Front End Engineering Design (FEED) to Support Phase-Gate Approvals,” 2017.
  6. C.R.T. 331, “Assessing the Maturity and Accuracy of Front End Engineering Design (FEED) to Support Phase-gate Approvals,” Construction Industry Institute (CII), Austin, 2017.
  7. DBIA, “Choosing A Project Delivery Method, A Design-Build Done Right Primer,” Design-Build Institute of America, 2015.
  8. S.M. Report, “Managing Uncertainty and Expectations in Building Design and Construction,” McGraw Hill Construction, 2014.

ABOUT THE AUTHORS

Christopher W. Carson, CEP DRMP PSP FAACE, is with Arcadis U.S., Inc. He can be contacted by email at: chris.carson@arcadis.com

Aaron Fletcher, PSP, is with Arcadis U.S., Inc. He can be contacted by email at: aaron.fletcher@arcadis.com

Noah A. Jones, PSP, is with Arcadis U.S., Inc. He can be contacted by email at: njones@schiavone.net

Leo Carson-Penalosa is with Burns & McDonnell. He can be contacted by email at: carson.penalosa@gmail.com

]]>
https://source.aacei.org/2021/10/05/successful-design-scheduling/feed/ 2
Ramifications of Owner’s Baseline Schedule Approved Decisions https://source.aacei.org/2021/08/17/ramifications-of-owners-baseline-schedule-approved-decisions/ https://source.aacei.org/2021/08/17/ramifications-of-owners-baseline-schedule-approved-decisions/#respond Tue, 17 Aug 2021 17:50:42 +0000 https://source.aacei.org/?p=3889

Ramifications of Owner’s Baseline Schedule Approval Decisions

by Christopher W. Carson, CEP DRMP PSP FAACE and Gino Napuri, EVP

Abstract–Just when a team thinks that a project is smooth sailing in its early stages, a critical deliverable is missed… now what? While many owners and scheduling specifications have requirements for owner approval of contractors’ baseline and update schedules, too often there is concern about possible risks associated with approval, or approvals are simply ignored. Sometimes specifications require ‘approval,’ sometimes ‘review,’ sometimes they mention ‘acceptance,’ occasionally they simply address a ‘record submission.’ Often the concerns are related to the worry that review might ‘direct’ the contractor to a specific means and methods that could be problematic for the owner later if the project runs into delays related to the means and methods employed. Studies show that one of the traits of project success is the involvement of the management team (owner, project controls group, construction manager and contractor management staff) in schedule review. [4] This article discusses the different review and approval options for owners and makes recommendations for the appropriate level of approval, including discussion of the benefits and risks of any approach. The reasons for schedule rejection are discussed, as well as recommendations for requirements that will improve the chances of receiving an approved schedule. Working for a top-50 program management firm, the authors have dealt with the issues of drafting scheduling specifications and schedule approval. They bring an extensive depth of experience in scheduling and schedule review, through working with contractors, consultants, and owners. This article was first presented as PS-3428 at the 2020 AACE International Conference & Expo.

Introduction
Creating and approving a workable baseline schedule in the first precious months of a project is of critical importance. Project baseline schedules (“baseline”) are meant to be used by all stakeholders (owner, owner representatives, contractors, etc.) to compare project progress against the plan. Unfortunately, most “plans” are not agreed upon by all the stakeholders within a reasonable time after the project starts. This causes misalignment between stakeholders and weakens the ability to provide proper analysis of disputes and delays occurring in the early stages of the project. These misalignments (or disconnects or confusions or frustrations) could be solved by agreeing on a baseline early in the project. This article explains the important value of an approved baseline and details options for the stakeholders can use to achieve this key deliverable early in the project or program.

Background
Stakeholders discuss and decide early in the project on key milestone dates including interim and final completion dates. What seldom is discussed by the stakeholders is when the project will be baselined. The “how” to get to these milestones is usually ignored or at least not prioritized. Early in the ‘game,’ stakeholders such as subcontractors, suppliers, inspectors, and management staff rarely have all the information needed to make fully informed contributions. Their contributions typically are expressed in commitment dates so the ‘how’ to get to these milestones is often ignored, or at least not prioritized as preferred.

In this article, the terms ‘owner’ refers to the client and their representative; ‘contractor’ refers to general contractor and sub-contractors within a project.

Unless stipulated in the contract, there is always the question of how long after notice to proceed (NTP) is granted should the contractor submit a baseline schedule for approval. The contractor is usually challenged to achieve submission within the contractually allowed time, and this often leads to a baseline schedule that is submitted late. This might be because the contractor recognizes that once the baseline is completed and approved, their work progress will be subject to monitoring and possibly judged more harshly or intensely by the owner and the owner representatives. This sense of judgment can freeze a contractor’s project controls team and cause shortcomings in the schedule if not acknowledged and dealt with early.

Alternatively, a late submittal could be due to the limited amount of time available to develop the schedule, the lack of professional scheduling staff, or the inability to facilitate a schedule development session with the full construction management team.

The goals of this article include making the case that the owner should require formal submission, provide formal approval, and require the contractor to accommodate any review comments. Additionally, it hopes to challenge the assumption that the baseline is a threat and risk to the contractor and their team. It is important that the baseline is recognized as an integral deliverable which should be championed to be achieved by every stakeholder. A successful baseline process leading to approval early in the project will improve project success.

Submitting a baseline just because it is a contract requirement without much effort is counterproductive. The submission should be scrutinized by the team to make sure it conforms with standard practices and contract specifications.

Only after the baseline is vetted and improved will the project reduce the risk of receiving change orders incrementally during progress due to missing logic, insufficient durations, and/or constructability issues in the baseline.

Review and Response is Important
The baseline schedule is the benchmark against which progress is measured, performance is determined, and delays are analyzed. Review of a project baseline and response is important because it improves the quality of the schedule which increases the likelihood of completing the project on time and on budget. Depending on the size, complexity, and duration of the project, the baseline should be submitted as early as possible so it can be used to manage the early stages of the project when there are likely to be conflicting delays. These early delays tend to fall into one of two categories; owner delays due to quality of the plans, and contractor delays due to the inability to negotiate and award all the early trade contracts on time.

It is common in claims to discover inaccuracies or missing information in the drawings during shop drawing preparation by trade contractors. These are clarified or answered through formal requests for information (RFI) that tend to take time to resolve and thus delay the project. If these were the only delays and the schedule was approved in sufficient detail to analyze and prove the delay quantum and responsibility, the entitlement to an extension of time (EOT) would be clear and simple. But without a baseline in place, the delays are often not identified, and analysis is very difficult without an agreed-upon baseline.

At the same time, the general contractor may run into a delay in bringing on early trade contractors like underground plumbing, causing a delay to construction. With no baseline, these delays just accumulate and corrode the collaborative culture necessary for a successful project. On top of the problems with attempting to analyze without a baseline, the delays tend to be concurrent which are more difficult to analyze and resolve than most isolated delays.

One of the solutions to this quandary is to allow a two-step baseline schedule submission such as owner representatives like the US Army Corps of Engineers (USACE) specifies. This approach requires an initial project schedule (IPS) due within a fairly short time frame, such as 45 calendar days (cd), and a detailed project schedule (DPS) due within a longer time frame, such as 120 cd. The IPS is required to include all the paperwork and the early construction activities in full detail with the rest of the project scheduled at a summary level, and the DPS is required to be built on the IPS and provide the full project detailed schedule. The IPS can be updated routinely to capture the actual dates and monitor performance until the full DPS is approved, and since the DPS is built on the IPS, if there are multiple updates of the IPS, the as-built data in those updates can be easily imported into and match activities in the full DPS.

Regardless of the solution chosen for schedule submission timing, one good approach is to require a reasonable period of time between the award of the contract and the issuance of the notice to proceed (NTP), using that time to develop the schedule. The issuance of the NTP could be dependent on schedule approval, as long as care is taken to protect the project duration and owner’s responsibility for delays due to any prolonged failure to get the schedule approved.

Besides the issue of dispute and claims prevention, timely and practical baseline schedules have enormous value to any company. The main issues can be elaborated based on the value in professional, psychological, collaborative, and contractual matters.

Professional Value
Building relationships and a professional network in the industry is complicated. Achieving early deliverables on time for a project – especially the project baseline – increases the chances to build stronger and lasting relations with colleagues, consultants, contractors, and owners. The stakeholders will recognize the value of the project controls team with early submission of a reasonable baseline that helps prevent conflict. This makes them more likely to reach out to the project controls team for support because of their professionalism in getting this key early deliverable successfully completed.

Psychological Value
The added value from working on a project schedule with the team early in the project improves the quality of the schedule, based on a study that shows good correlation between better project performance and “detailed review of schedule by the project team.” [1] This also increases the likelihood of everyone getting along, strengthens team confidence and improves owner satisfaction. The likelihood of the project controls team being involved and interconnected with the other stakeholders of a project increases if the team works together early to achieve a comprehensive project baseline.

Collaborative Value
Interaction between the project controls teams (owners, contractor, etc.) to achieve an approved baseline increases collaboration early in a project. It is one of the first key milestones which indicates that the stakeholders are in agreement for contractor and owner actions necessary to meet the scheduled means and methods. A well-designed baseline also helps define the scope and the implementation plan so that all stakeholders understand how the project will be built. This opens the door for better cooperation among stakeholders as the project matures, and also helps prevent misunderstandings over the nature of project deliverables.

Legal/Contractual Value
Most project contracts require analysis of delays based on the critical path method (CPM) schedule and stipulate a sanction if the baseline is not approved within a reasonable time. In addition, not achieving a project baseline approval will increase the likelihood of disputes later in the project. All time disputes will be analyzed based on the baseline and subsequent updates and if those do not reflect the implementation plan and contemporaneous field operation, those disputes will be harder to resolve. The baseline benchmarks this plan and provides the basis for monitoring and analyzing with each update. This risk and the subsequent additional cost are generally reduced if the stakeholders agree on a project baseline within a reasonable time from NTP.

Review and Response Options
Project baselines should be reviewed by the owner’s project control team, who will analyze the received schedule and provide recommendations on how to respond to the submission. In most projects the owner defers to their project controls team’s conclusion for their response to the submission. The following are the most common options used by owners for dealing with the baseline schedule and subsequent updates.

Require Approval and Reconciliation of Comments
Most contracts require that the baseline schedule be reviewed and approved by the owner before it becomes a project document. The achievement of receiving “approval and reconciliation of comments,” also known as “approved with comments” or “approved as noted,” should be the most desired response for a contractor. This is not just because, in a dispute resolution forum, the contractor can rely on the schedule for analysis when the baseline was approved by the owner, but also because they know the owner is supportive of their plan. An approved schedule starts and helps guide the project team on a clear path toward project completion. The achievement of receiving approval of a baseline schedule should not be taken lightly and all comments by the reviewing team should be required to be answered within a reasonable time. This approach has significant benefits to the contractor, as noted in language from a recent online presentation by one of the authors, [2]

  • “Schedule approval is the owner’s opportunity to ensure reasonableness, not missing vital scope, positioning for claims, or unrealistic in expectations for owner’s work.
  • Approval does not mean that the owner is directing the contractor to follow the schedule, merely that it meets the specification requirements.
  • Approval would imply that the owner has agreed to perform his work within the constraints of the schedule, it appears reasonable, and meets owner needs.
  • Approval also means that the owner agrees to cooperate with the schedule for any owner responsibilities or coordination with third parties.”

Require Acceptance Only
Unlike the previous option, merely accepting the submitted project baseline is more of an acknowledgement that is sometimes used when the owner believes they might take on some liability if they approve the baseline. Unfortunately, this option removes the owner’s ability to help improve the quality of the schedule and ensure owner tasks are modeled appropriately in the schedule. In some contracts acceptance might be a term indicating only a step in the approval (or rejection) of a baseline submittal process to notify the stakeholders that the submission has been received and is under review.

Require Submission Only
Some project contracts (usually small) might just require that the contractor shares with the owner their schedule and leave it entirely to the contractor to implement their means and methods to complete the project without any owner interaction or feedback. This might be reasonable for small design projects where the owner does not have a design engineer or the ability to support the review of a contractor’s schedule. However, it still creates risks to the owner because of the contractor’s expectations for the owner to perform some tasks in alignment with contractor work. Regardless, it is recommended that the owner reviews the contractor schedule and provides feedback to help improve quality of the schedule network.

Some textbooks caution owners to not require acceptance of a schedule if they do not have the facilities to properly review it, address it as an internal product [5], or treat it as a product that is “sold” [7] when accepted. The act of only accepting a schedule can involve the owner by not identifying planning risks that could have been addressed.

No Required Owner Involvement
The lack of owner involvement in the baseline schedule review is not recommended, and often contributes to the project being managed without a schedule. Being completely indifferent to a project schedule and providing no feedback to the contractor for any reason whatsoever leaves the owner exposed to the contractor’s expectations memorialized in this primary communication and contractual tool. The risk of completing a project on time without owner involvement or a schedule is very high, as recognized by inference in some studies, such as a 2011 study showing that the “most serious risks to project success” are led by “changes in schedule” by 49% of respondents surveyed. [6] Other studies, while not directly addressing approval of schedule, speak to the value of the detailed review of the schedule by the project team, noting in one study, “demonstrated less cost growth.” [4]

In addition, the owner takes on some measure of liability from failing to engage with the baseline schedule, as determined in the “Stanley Miller Construction Co. v. State” case, where the trial court held that an owner breached a construction contract because the baseline schedule was “both logically flawed and hopelessly incomplete” when it included inaccurate float forcing the contractor to spend more labor and equipment than planned to meet project deadlines.

Rejection Options
When the owner specifications require review and approval and the submittal does not meet the minimum requirements, the schedule submittal should be rejected. Rejection options vary according to how closely the submission meets the requirements. The owner or program/construction manager reviewing team evaluates the submission and determines if it considers the contractor’s schedule submission inadmissible, which may be due to a number of issues. Here are the most common rejection options.

Contractual Noncompliance
Contractual noncompliance occurs when the contractor’s schedule does not comply with contract requirements. It could be as simple as an oversight by the contractor’s scheduling team in not reviewing the contract prior to submitting the baseline schedule or it could be as complicated (and political) as the contractor using different requirements and refusing to deviate from them. It is not uncommon for the contract to require a schedule to be loaded with activity-based costs, and this is one of the sources for noncompliance when the contractor fails to load costs.

Occasionally, the noncompliance is due to a failure to meet the phasing specified, incorrect notice to proceed or milestone dates, or missing owner or third-party activities. All parties need to ensure alignment of the schedule and the contract and negotiate a reasonable plan forward as quickly as possible. The goal should be to move away from rejecting a baseline schedule (with a risk of contributing to a project completion delay) and toward a common ground approach of collaborating with the contractor to gain approval. If practical, the owner might decide it makes sense to modify the contract to accommodate reasonable needs by the contractor. More commonly, the contractor will adjust and try to align the developing schedule with the contract requirements in order to gain early approval of the baseline from the owner and team.

There are commonly three categories of reasons for rejecting a schedule submission; technical noncompliance, outdated information, and inappropriateness.

Technical Noncompliance
Most construction contracts spell out the technical details required for the baseline schedule to be accepted for review. Since the baseline is one of the first schedule submissions by the contractor and often required within a short period of time, there is an elevated risk that the contractor will fail to comply with a technical requirement. It could be a simple misunderstanding or lack of knowledge by the contractor. One such misunderstanding might be incorrect software or version of a software used as the scheduling tool by the contractor to which the owner does not have access. Another reason for technical noncompliance might be a failure to involve the appropriate trade contractors in developing the schedule, leading to problems in sequencing or alignment with deliveries. Other significant reasons include failure to meet contract requirements to load costs or resources into the schedule. Understanding the technical specifications and making sure that all the stakeholders are involved in the development and review of the submitted documents is key for a successful baseline project schedule.

Inclusion of Information Not Available at Contract Award
Sometimes it happens that the baseline project schedule is submitted right after a major change or delay has occurred in the project. Anything that affects scope, budget and time would affect the schedule, but the baseline schedule is a model of the contract scope of work and cannot be used to include work not in scope. It may happen that some changes might be on the critical path and therefore affect time of completion for the overall project, but the baseline schedule should still not include non-contract work. However, this is certainly support for the intention to get an approved schedule in place as soon as possible after NTP.

Occasionally, a contractor will include information as part of the baseline schedule which more properly should be handled in a change management effort, or the owner may encourage the same approach. But when the schedule contains changed conditions in the baseline submission, it tends to confuse the definition of original scope of work. Any changes necessary due to new or changed field conditions or requirements should be clearly identified and discussed as changes and not included in the baseline schedule. These changes should, however, be resolved as an updated schedule or time impact analysis as quickly as possible after baseline schedule is approved.

Schedule Not Appropriate
If the submitted baseline schedule does not fit the standards based on industry best practices –even though the contract might not spell them out – a knowledgeable scheduling professional can identify schedules that should not even be considered for review. This could be for missing scope of work, lack of detail, missing or inappropriate logic, schedule development problems such as activity names not clear or duplicated, or even something like the schedule is in a foreign language. Other reasons for rejection include under- or over-development of trade work, high proportion of owner-responsible work that appears to be “forced” onto the critical or near-critical path, just the failure of the schedule to appropriately model the contractor’s means and methods to build the project. This type of failure in the schedule will require the owner reject a baseline submission, and the rejection, just as with any submittal, should be returned as soon as possible to minimize the time for resubmission.

A reasonable approach to recognizing this type of problem is maintaining the purpose of the schedule, to model the contractor’s plan to achieve the contract goals. A good model, built with industry recognized good CPM scheduling practices, should ensure an appropriate baseline schedule.

Benefits to Owner for Approval and Reconciliation of Comments
There are numerous benefits to the owner in achieving early approval of the baseline schedule. With the approved baseline in place and available for updating and analysis, the project is managed by the schedule from the earliest date. This dramatically reduces the risk of early delays that cannot be analyzed without a schedule, and it provides a monitoring tool that helps keep the project on track while providing a basis against which to analyze any contractor caused delays.

The schedule review is the owner’s opportunity to improve the quality of the schedule for performance monitoring and analysis, as well as protecting the owner from project failure and any associated owner risks. It is the best approach for providing comments necessary for the contractor to address deficiencies by revising the schedule. The quicker the comments are provided to the contractor after schedule submission, the sooner the contractor can address the comments. Since the schedule is most commonly on a monthly cycle, if the comments are not received by the contractor early enough to revise prior to the subsequent month’s schedule update process starts, the next update will still contain the same problems and risks.

Risks to Owner for Other than Approval
Rejection is certainly a risk to the project and owner (as shown in the legal citation in the Review and Response Options section above), and repeated rejections tend to erode the relationship between the contractor and owner. As noted in a paper related to baseline approval, “These rejections cause production to be severely affected, inflicts stress on the contractor’s project management team, and places the contractor in an untenable situation.” [8]

Approved as noted is a reasonable and preferred compromise if the contractor is required to make corrections according to the notes or comments, and prevents the lag in schedule approval that happens with rejection. The “as noted” stipulation refers to the review comments supplied, and best practice is for the owner to carefully consider the list of problems with the schedule and split them into two categories. These categories are minor best practice improvement suggestions and secondly deficiencies that would render the schedule less valuable or even risky for the owner.

The deficiencies are listed in the comments that carry the understanding that, if the contractor corrects them satisfactorily, the owner will approve the submission. The other items on the list that amount to best practice suggestions are issues that, even if not fixed, will not cause the schedule to calculate incorrectly or position the schedule so it can easily develop into claims. If the contractor properly corrects the deficiencies, the schedule can be approved.

Approval and Project Delivery Methods
The project delivery method has an effect on the ability to recommend approval since some methods allow work to start without a full schedule in place. With (design-bid-build), it is common to require full schedule approval before construction starts since the contractor has the approved drawings showing full scope. With DB (design-build), PPP or P3 (public-private-partnership), and EPC (engineering, procurement, and construction) project delivery, the contractor takes over the design at some fairly low level of scope definition, so the project is not fully designed. In these delivery methods, there must be consideration from both parties to get an approved schedule in place as soon as possible, and a multi-part submission is often the answer. As noted in a paper by Colbert and Forbes, “the CPM Schedule goes through multiple stages – similar to a rolling wave approach.” [3] This provides a detailed schedule for early work with the balance of the work summarized, and then another submission where that summarized work is detailed.

Multi-prime contracts will often have a variety of project delivery methods employed, and CMAR (construction manager at risk or CM@Risk) project delivery requires the CMAR to engage with the development and monitoring of scope definition during design. With CMAR delivery, the submitted schedule should already be developed and coordinated with design to meet the requirements and provide high value. By the time the schedule is ready for baseline schedule approval, it should include the full scope of work with a good explanation of the contractor’s means and methods. Approval should be straightforward unless the CMAR has not fulfilled their role in the stage-gate process of updating and expanding the schedule as the design matures and scope is further developed.

Combining Time and Cost in a Schedule
When the schedule is loaded with costs, it can be an additional complication to submit and approve these schedules, but it is in the stakeholders’ best interests to cooperate and resolve it quickly. The additional complexity of using cost-loaded schedules is due to the differences in the level of granularity needed in the schedules between cost and time. Cost, which is normally invoiced monthly, can often be carried at a higher level than the daily/weekly work activities necessary to monitor performance. This can lead to reduced detail in the schedule if the cost input drives development or perhaps a need to add cost-only activities in the schedule.

The authors have found that if the contractor informs the subcontractors and suppliers that the schedule will be cost-loaded when developing the schedule; then the sequencing and depth of detail from the subcontractors and suppliers will be better and more appropriate. This is due to their need to ensure the ability to bill against their sequencing delivery and installation. For example, without recognition of cost loading, a structure steel fabricator/erector on a passenger cruise terminal provided a very shallow set of activities to cover erection of columns, setting of beams, plumbing/locking in, setting bar joists, and welding deck. However, once the requirement for cost loading was communicated, the same subcontractor supplied activities for six sequences, each with their own columns, beams, bar joists, and deck, in order to ensure that they could bill for each sequence. This addition made the project schedule much better for analysis and monitoring and helped ensure on-time completion.

Developing a common work breakdown structure (WBS) between scheduling and estimating helps with the alignment of the cost and schedule activities. The cost loading can be performed at the work package level of the WBS rather than down at the more detailed activity level which is needed for schedule monitoring.

Following good guidelines for cost loading should simplify the complexity of this requirement and reduce the conflicts common when schedules need to be loaded with costs.

Recommendation
The best choice of the options for schedule review is the requirement of schedules to be reviewed and approved by the owner, supplying deficiencies in comments, and for the comments to be addressed by the contractor and resubmitted for final approval, with both processes performed timely. These steps will ensure deficiency corrections are completed for the next update, so there is a contractual agreement to promote the discussion of the quality of the schedule and provide high quality solutions. The preference is to approve, either in full, or as noted, working with the contractor to address any as-noted comments rather than to formally reject the submission. This ensures that the comments are addressed quickly and builds on the collaborative culture desired. Outright rejection of a baseline schedule should be avoided unless the schedule has so many problems that it cannot be approved without additional risk to the owner. One of the authors developed a paper on how to deal with a schedule submission that cannot be approved for use in this type of situation. The paper discusses an approach that is recommended for use to protect the project and owner when the schedule has too many deficiencies that the contractor refuses to correct, and notes, “It is in the best interests of any project to get an approved as-planned schedule in place as soon as possible to ensure that appropriate planning is done.” [1] That is still a vital point and worth effort to facilitate.

Conclusion
With the right approach and thorough but timely review of the contractor’s baseline schedule submission, it is possible and reasonable to agree to an approved schedule early in the project. Achieving this goal starts with a contract requirement to provide a timely baseline schedule, whether in a single or a two-step submission as necessary. The two-step submission is a useful requirement and the authors recommend the practice as a way to put in place a detailed and appropriate schedule early in the project so accurate performance monitoring and analysis can be provided.

The schedule review should be performed in a detailed, thorough manner, separately identifying deficiencies that must be corrected as well as good industry practices that can help the contractor improve their scheduling. However, these are two different categories and the reviewer must be careful to emphasize the need to correct deficiencies while not using non-deficient practices to delay schedule approval. It is important to require the contractor to address the schedule review comments and correct the deficiencies so the schedule will serve as the basis for completion predictions, performance monitoring and improvement, and analysis of delays. This process of the owner’s team identifying and commenting on deficiencies and the contractor reviewing and correcting them should be done as quickly as possible in order to avoid interrupting the schedule update cycle.

In addition, in keeping with the preferred collaborative culture on the project, both parties should operate in a transparent and open manner, and enter into negotiations when differences arise, allowing for a reasonable resolution in a timely approach. When the schedule just cannot be approved without imposing additional risk on the owner or project, the reviewer should embrace a process to protect the project from the risks, using industry publications that offer good processes. [8] With these efforts, an appropriate baseline schedule can be achieved as early as possible. Each of these elements are important in support of the goal for on-time completion.

REFERENCES

  1. Carson, C. “Dealing with Contractors Schedules that Cannot be Approved,” Cost Engineering, Vol. September/October, pp. 4-16, 2013.
  2. Carson, C. “Can a Contractor Use Baseline Durations in a Request for EOT?,” Project Control Academy Masterclass, Houston, 2019.
  3. Colbert, J. and K. Forbes, “Understanding Risks in Design-Build Schedules,” in AACE International Technical Paper, Morgantown, 2019.
  4. Griffith, D. “Scheduling Practices and Project Success,” in AACE International Transactions, Morgantown, 2005.
  5. Lewis, J. Project Planning, Scheduling & Control, Probus Publishing Company, 1991.
  6. McGraw Hill Construction, Mitigation of Risk in Construction, Smart Market Report, McGraw Hill Construction, 2011.
  7. Uyttewaal, E. Forecast Scheduling with Microsoft Project 2010, Best Practices for Real-World Projects, ProjectPro Corporation, 2010, p. 693.
  8. Zafar, Z. and D. Rasmussen, “Baseline Schedule Approval,” Cost Engineering, Vol. 43, No. 8, 2001.

ABOUT THE AUTHORS
Christopher W. Carson, CEP DRMP PSP FAACE, is with Arcadis U.S., Inc. He can be contacted by sending email to: Chris.Carson@Arcadis.com

Gino Napuri, EVP, is with Arcadis U.S., Inc. He can be contacted by sending email to: Gino.Napuri@Arcadis.com

]]>
https://source.aacei.org/2021/08/17/ramifications-of-owners-baseline-schedule-approved-decisions/feed/ 0
Analysis of the Earned Schedule Forecasting Accuracy https://source.aacei.org/2021/06/01/analysis-of-the-earned-schedule-forecasting-accuracy/ https://source.aacei.org/2021/06/01/analysis-of-the-earned-schedule-forecasting-accuracy/#comments Tue, 01 Jun 2021 05:00:00 +0000 https://source.aacei.org/?p=3340

Analysis of the Earned Schedule Forecasting Accuracy

by Keight Charles Navarro Hurtado, PE PSP

Abstract–The independent estimate at completion (IEAC) is a time-based earned schedule (ES) method that allows the analyst to estimate the duration of a project according to the information available for a period of time. The fact that there are two approved equations that may give different results hinders the acceptance of this important analysis.

The purpose of this paper is to describe the two equations and determine which is more accurate to estimate the final duration of a project. In this analysis, the average absolute percentage error, completed percentage, seriality of the network diagram and productivity index (PI) have been used to evaluate the accuracy of the two equations.

A study of eight real projects with early and late completion will be used to determine the accuracy of each method. The conclusion is that when the schedule performance index based on the time is less than 1 or the project has a delay, the equation of the IEAC with the PF=1 begins to lose accuracy and the equation of the IEAC with the SPI begins to gain accuracy or has fewer errors.

Introduction

The time-based independent estimate at completion is an indicator that allows estimating the final duration of a project for an early or late completion. The IEAC has two methods to predict the duration. This work analyzes and determines which equation of the IEAC is more accurate throughout the life cycle of the project. In addition, a contrast will be made with the analysis of the writer Lipke [5], who dealt with the earned schedule forecasting method selection, its two forecasting methods and an optimum selection.

Case Study

This case study is the analysis of eight real construction projects of an oil refinery with an evaluation and accuracy of the two IEAC forecasts through the simple formula and the more general formula IEAC. Both forecasts allow one to estimate the total duration of a project. This paper will verify which of the two forecasts have a greater precision in the estimation of the total duration of a project. Better precision will help create a better estimate of the date of completion of the project. The two IEAC forecasts to be evaluated are the schedule performance index (SPI) and the equation with PF=1. The term PF=1 means the performance factor or future execution efficiency, which is used in the more general equation of the IEAC.

Methodology

The study begins with the calculation of the percentage error in each period of the project. This indicator allows evaluation of the degree of error in the forecasting of the IEAC with SPI and PF=1. Therefore, its purpose is to calculate the percentage error and observe the variations in the accuracy of the forecasts in the projects analyzed, once the percentage errors have been obtained.

The analysis continues with the calculation of the mean absolute percentage error (MAPE). Its formula is furnished in Appendix A, fig. 01 where the “RD” value means the actual duration of the project and the “Ft” value is the estimated forecasting of the IEAC in each period calculated for the analysis. In the study, the lowest MAPE calculated in both equations of the IEAC will be used as the more accurate method. Another parameter analyzed is the percentage completed. This factor impacts the evolution of the IEAC through the schedule performance index and, in addition, the percentage completed will allow the reader to observe its influence on the IEAC forecasts in each phase of the project.

In the study of the percentage completed with the MAPE, three phases are considered. The early phase is in the range of 0% ≤PC <30%, the intermediate phase in the range of 30% ≤PC ≤70% and the late phase in the range of 70% <PC≤100%

Another factor to consider in the impact of the IEAC forecast is the seriality. Seriality refers to the topology of the project network diagram. The study of the seriality with the MAPE has the following intervals to categorize the topology of the network diagram. If the network diagram is in parallel, the range is 0% ≤SP<40%, in series-parallel the interval is 40%≤SP≤60% and in series, its interval is 60% <SP≤100%. In the study, seriality is observed in two phases for analysis at the beginning of the project and at the end. This will help in understanding how the topology of the network diagram of the project changes towards its completion and how this change can impact the IEAC forecasts.

Another factor to consider is the index productivity. It will be used to evaluate the performance and its relationship with accurate IEAC forecasting through the observations in the IEAC analysis. The IEAC forecasting changes when low index productivity is high or low.

To understand how these change in the accurate forecasting occur, the SPI and ES of the earned schedule, as obtained, will be plotted through their natural logarithms and the standard deviation of the Ln (ES (t)). This will allow obtaining a range of intervals when the SPI is more accurate that the PF=1. The same happens with a range of intervals obtained when the PF=1 is more accurate than the SPI. These intervals will be used to calculate the forecasts and make a comparison with the real forecasts (the periods in which the SPI and the PF=1 are used depends on the MAPE).

Result

The result section summarizes the data of the methodology used. It contains the calculation of the MAPE, percent complete, seriality and productivity index.

MAPE

The results of the absolute average percentage errors made to the eight sample projects are shown in Table 01, along with %Error SPI and PF=1 in the early, intermediate and late phases of the eight projects.

Table 1

Table 2

Table 3

In Tables 01, 02 and 03, the reader can see the summary of all the average absolute percentage errors calculated in the eight projects. As can be observed, PF=1 is the most accurate at the beginning of the project in all the eight projects. It can also be observed that the error changes as the project continues towards its completion. This change is the result of two important parameters. When the delay and productivity have been calculated and analyzed, there are considerations, as mentioned below:

  1. When there is a delay and low productivity in terms of the performance index SPI, the forecast of the IEAC with SPI is more accurate than PF=1.
  2. When there is a delay and high productivity in terms of the performance index SPI, the forecast of the IEAC with PF=1 is more accurate than SPI.

In Table 04, MAPE Summary of PF=1 Calculations, the reader can see the difference of error between the SPI and the PF=1 in 15 periods of analysis and also check the two considerations previously indicated.

Table 4

Table 05 shows the MAPE summary of the eight projects analyzed. It is also observed that the forecast with the PF=1 is more accurate than that with SPI in six of the eight projects analyzed, while two reflect greater accuracy of the SPI forecast. It is also important to specify that only three projects out of the eight were completed early, viz., projects 01, 02 and 04.

Table 5

Percent Complete (PC)

Table 06 shows the calculated values of the SPI and the PF=1 in the early, intermediate and late phases of the eight projects.

Table 6

Readers can see in the Table #06 the forecast of the IEAC with SPI and PF=1. The forecast with PF=1 is more accurate in the early phase of the eight projects. However, in the intermediate phase, six of the eight projects were more accurate with PF=1, but this change of accuracy in the PF=1 occurs in the intermediate phase of the projects #5 and #7. In the revised data of the projects #5 and #7, the forecast with SPI is more accurate than PF=1 and it occurs in the range of 0.1 and -0.6 of the natural logarithmic of the SPI. Reader can confirm the accuracy of the SPI from the following table with the data of the projects 05 and 07. All these values relate to the intermediate phase of both the projects.

Table 7

In the late phase, some projects changed their accuracy with respect to the intermediate phase. This can be seen from Table 08 of the projects #3 and #6 and it occurs for the schedule performance index (SPI) or, in other words, for the productivity. For that reason, the forecast of the SPI is more accurate than that of the PF=1. The reader can observe the trend of the SPI in Table 06.

Table 8

Project Seriality (SP)

Seriality is an important concept to consider. It is the description of the structure of the project network, reflecting how close it is to a structure parallel, serial-parallel or serial. This topology structure of the network diagram varies from the beginning of a project until its completion. This change is due to the delay in the project. Table 07 shows the seriality at the start and end of each project and the MAPE of the forecasts with SPI and PF=1.

Table 9

In Table 09, the reader can see that project #1 started with a seriality in serial and ended with a seriality in parallel-serial. In all the life cycle of the same, the forecast with the PF=1 was more accurate than the SPI, and this project was completed early. Project #2 started with a seriality in series and ended in parallel, and in all the life cycle of the same, the forecast PF=1 was more accurate than the forecast SPI, and it was completed early. Project 03 started with a seriality series-parallel and ended in parallel, and in all the life cycle of the same, the forecast PF=1 was more accurate than the forecast SPI, and it was completed late. Project 04 started with a seriality series-parallel and finished with a seriality in series and was completed late. In this project, the forecast with PF=1 was more accurate than the SPI. Projects 05, 06, 07 and 08 had a delayed completion, and the accuracy of their forecasts were SPI, PF=1, SPI and PF=1, respectively. Readers may question why and when this change happens. As indicated earlier, the replies to these questions depend on the performance index of the SPI, which will be explained in this paper.

Productivity Index (PI)

In the analysis of the forecast of the earned schedule, the PI (index productivity) is also analyzed. Readers can see in Table 10 the index productivity of the eight projects with the average absolute percentage errors (MAPE) of the forecasts through SPI and PF=1. Project #1 has a PI of 0.7987 and its forecast with PF=1 is more accurate than the forecast with SPI. Project #3 has a PI of 0.7042 and its forecast with PF=1 is more accurate than the forecast with SPI. Project 05 has a PI of 0.6069 and its forecast with SPI is more accurate than the forecast with PF=1. The analysis of all the data indicates that there is no relationship between the PI and the performance index of the SPI. Hence, there is no link between the forecasts of the IEAC and the productivity index.

Table 10

Ln(SPI) and σLn(ES(t)) are analyzed in this section of the paper. This is done to have a compression above the accuracy of the forecasts as explained in the MAPE, PC, Seriality and PI and about the relationship with them of the performance index SPI.

In Figure 1, the reader can see the values of the six projects where the PF=1 was more accurate than the forecast SPI. These values were obtained from the calculation of the MAPE. In the observations, the values of the ln (SPI) are between 0.1 and 0.6, and if the values of the ln (SPI) are greater than 0.6, these values will be out of control. The reader can see in Fig.01 the values of the σLn (ES (t)) are in the range of 0.0 and 0.5.

Figure 1–Projections using PF=1

Figure 2 shows the values of two projects where the forecast with the SPI is more accurate than PF=1. These values were obtained from the MAPE. The range of accuracy of the forecast SPI is between -0.1 and -0.6, but the analysis of the data of all the values of the Ln(SPI) shows a transition period, where the accuracy of forecast of the IEAC with the PF=1 changes from more to less, and the forecast with the SPI changes from less to more. This transition period occurs in projects with late completion. The interval of the transition period is between 0.1>Ln(SPI) and Ln(SPI)<-0.1. Hence, from the intervals described earlier in this section, it can be concluded that the interval of the SPI is between 0.1 and -0.6. If in the analysis of the data, values of the ln (SPI) are greater than -0.6, it will be considered out of control. The reader can see in Fig.02 the Ln (ES) is in the range of 0.0 and 0.5. The answers to the questions that arose previously when the seriality was analyzed are as follows. The change of accuracy of one forecast vis-à-vis the other happens when there is delay in the project, and it has a low productivity or high productivity in terms of the performance index SPI. This will happen in the intervals analyzed to the case of the PF=1, its interval being 0.1≤Ln(SPI) ≤0.6 and 0.5≤σLn (SPI), and in the case of the SPI, 0.1<Ln( SPI≤-0.6 and 0.5≤σLn (SPI).

Figure 2–Projections of SPI

In addition, it is important to indicate that there are five projects with late completion, of which two reflect accuracy of the forecast SPI and three with the PF=1. Why does this happen? The answer is in the observations derived from the data: it happens when the project has a tendency in which the Ln(SPI)<-0.4 and a low productivity. Hence the forecast with SPI is more accurate than the forecast with PF=1, as can be observed in Figure 3.

Figure 3

Figure 3 shows all the values of the six projects with the PF=1. There are values where the natural logarithmic exceeds -0.4 in some periods, so in this case, the PF=1 is more accurate. Because its productivity in terms of the SPI has been high, this raises the accuracy of the PF=1. It also consolidates the lower error in the MAPE, compared to the forecast of the SPI.

In Figure 4, all the values where the SPI is more accurate than the forecast of the PF=1 are furnished. These values have a tendency where the Ln (SPI)<-0.4. In this case, these projects had a low productivity and they have been delayed. Hence these projects had a low productivity in terms of the SPI plus delay, resulting in a lower error MAPE with the forecast through SPI with respect to PF = 1.

Figure 4

Walt Lipke made this analysis in his paper Earned Schedule Forecasting Method Selection and he lists below his selection rules for the forecast of the IEAC with the SPI and PF=1.

Use PF=1 when -0.1≤lnSPI≤0.1 and lnESpσ≤0.8
Use SPI when lnSPI>0.1 or < -0.1 and lnESpσ≤0.8
Use SPI when lnSPI≤0.6 or ≥-0.6 and lnESpσ≤0.8
Use PF=1 when -0.6>lnSPI>0.6 or lnESpσ>0.8 = Out of Control

The reader can see in Table 11 a comparative listing of the SPI, PF=1, Real, Calculating and Lipke.

 

Table 11

The values of the selection rule of Lipke have been compared with the other values in the same table, to calculate the accuracy grade of the intervals. MAPE of the observations in the SPI and PF=1 are given in the table. The range of intervals to the forecast of the IEAC with the PF=1 is 0.1≤ln (SPI) ≤1.0 and 0.5≤σln (ES). However, in the case of the forecast of the IEAC with the SPI, its range is between the intervals 0.1> ln(SPI)>-0.6 and 0.5≤σln (ES). In the same table, the reader can see that Real MAPE is lowest MAPE of all the values and that the calculated MAPE and Lipke are closer to the real values. The reader can see in Table 12 the difference between calculated and Lipke MAPE.

Table 12

The calculated MAPE and Lipke’s are closer to the real figure. However, of the two the calculated MAPE are closer to the real and Lipke has one project near the real MAPE.

Conclusion

The analysis performed with the MAPE, Percent Complete, Seriality and Index to the forecast of the IEAC resulted in PF=1 emerging with the highest accuracy. The productivity index as a factor of analysis of the accuracy about the forecasts of the earned schedule did not show a relationship with the forecast of the IEAC.

The forecasts of the IEAC have a logarithmic tendency.
From an objective point of view, the accuracy of the forecasts has been estimated in intervals to know when each forecast is more accurate.

The evaluation of the forecast of the IEAC with PF=1 and the SPI has been made with the natural logarithm.

The interval of the forecast of the IEAC with PF=1 is 0.1≤ln(SPI)≤1.0 and 0.5≤σln (ES).

The interval of the forecast of the IEAC with SPI (t) is 0.1> ln(SPI)>-0.6 and 0.5≤σln (ES).

There are periods in the projects where the values have a tendency towards the accuracy of the SPI due to the delay. However, when calculating the MAPE of that project, its accuracy is better with PF=1. This occurs due to the high productivity in terms of the performance index SPI. The same scenario occurs with the PF=1 too. There are periods in the projects when the values have a tendency towards the accuracy of the PF=1. While calculating the MAPE of the project, SPI displays more accuracy, which is due to low productivity in terms of the performance index SPI.

Only six projects have been covered here, so the analysis is based on limited data.

References

  1. Dr. Scott J. Amos, PE., Skill and Knowledge of Cost Engineering, 5th edition, AACE International, Morgantown, WV, 2007.
  2. James A. Bent and Kenneth K. Humphreys, Effective Project Management Through Applied Cost and Schedule Control, Marcel Dekker, New York 1996.
  3. Forrest D. Clark and A. B. Lorenzoni, Applied Cost Engineering, 3rd edition, Marcel Dekker, New York 1997.
  4. Dr. Frederic C. Jelen, Dr. James H. Black and Dr. Aaron Rose, Jelen’s Cost and Optimization Engineering third edition, AACE International, Morgantown, WV, 2013.
  5. Lipke, Walt. “Earned Schedule Forecasting Method Selection”, PM World Journal, January 2019, Vol. VIII, Issue I.
  6. Lipke, Walt. “Forecasting Schedule Variance Using Earned Schedule”, PM World Journal, February 2017, Vol. VI, Issue II.
  7. Jordy Batselier and Mario Vankhoucke, “Empirical Evaluation of Earned Value Management Forecasting Accuracy for Time and Cost”, ASCE 2015.
  8. Mario Vankhoucke, Measuring Time Improving Project Performance Using Earned Value Management, Springer, New York 2009.
  9. Quentin W. Fleming and Joel M. Koppelman, AACE International’s Professional Practice Guide to Earned Value Project Management.
  10. Dr. Joseph J. Orczyk, PE., Productivity, Analyzing Construction Productivity, AACE International, Morgantown, WV.
  11. Anghel Patrascu, Construction Cost Engineering Handbook, Marcel Dekker, New York 1988.

Appendix A

Earned Schedule

Earned schedule (ES) is a method that uses the earned value elements available based on duration variance, to calculate the performance and schedule variations. The results can be used to predict the total duration of the project in time units. The theory of the ES focuses on the time in which the earned value must have occurred.

ES is calculated from the formula:

ES = C + I

Equation 1

Where,
C = Earned Value but satisfying the condition EV ≥ PVn
I = Linear interpolation

“C” is determined by comparing the EV with the periodic values for PV. For example, PVn. “C” is the highest value of “n” that satisfies the condition, EV ≥ PVn. “I” is an interpolation using the equation:

I = (EV – PVC) / (PVC+1 – PVC)

Equation 2

Where,
C = Value progress time
I = Linear interpolation

Indicators of the Earned Schedule (ES)

The indicators of the earned schedule are the variation of the schedule (SV (t)) and the schedule performance index (SPI(t)). The equations are determined by the following:

SV(t) = ES – AT

Equation 3

SPI(t) = ES / AT

Equation 4

Where “AT” is the actual time, for example from the beginning until the EV is measured

Independent Estimate at Completion (time)

This indicator estimate the completion of the project and there are two methods of calculating it. The simple form is:

IEAC = PD/SPI

Equation 5

And the more general form is:

IEAC = AT + (PD-ES)/PF

Equation 6

Mean Absolute Error

The mean absolute percentage error (MAPE) is a statistical measure of how accurate a forecast system is. It measures this accuracy as a percentage and can be calculated as the average absolute percent error for each time period minus actual values divided by actual values.

BTA-6-2021_Equation7

Equation 7

Where,
At= The actual value
Ft= The forecast value
n= Period numbers

Percent Complete

The completed percentage enables measurement and analysis of productivity and measurement of the completed percentage of work activities. There are six methods in this regard:

1. Units Completed
2. Incremental Milestone
3. Start/Finish Percentages
4. Ratio
5. Supervisor Opinion
6. Weighted or Equivalent Units

Seriality

The seriality is the topological network diagram and is directly linked to the number of critical activities in a network.

Index Productivity

The productivity index consists of the credit workhours and the actual hours. The PI is calculated through the following formula.

PI = Credit Workhours/Actual hours

  Equation 8


Table and Figure Image Details.
Click on any image to launch widget.

]]>
https://source.aacei.org/2021/06/01/analysis-of-the-earned-schedule-forecasting-accuracy/feed/ 3
Estimating As It Pertains to Risk Management https://source.aacei.org/2021/04/28/estimating-as-it-pertains-to-risk-management/ https://source.aacei.org/2021/04/28/estimating-as-it-pertains-to-risk-management/#respond Wed, 28 Apr 2021 17:40:45 +0000 https://source.aacei.org/?p=3072

Estimating As It Pertains to Risk Management

by Shoshanna Fraizinger, CCP

AbstractThe outputs of estimating are typically a primary input for business planning, cost and risk analysis, management decisions, and project cost and schedule control processes. All these aspects of corporate strategy and project planning are bounded, or guided by, an organization’s risk appetite. 

Estimating is fundamental to the ‘Assess’ & ‘Treat’ steps of risk management, as defined in AACE’s Total Cost Management Framework (TCM) and Skills and Knowledge of Cost Engineering 6th Edition (S&K6).  Estimator skill is required to determine the cost impact of the risk (assess), and the cost to implement the plan to address the risk (treat), respectively. The cost impact of the risk contributes to the amount of contingency required. However, there are several facets of strategic planning wherein the cost estimating process can introduce, assess, or mitigate risk.

This document addresses the topic of estimating as it pertains to risk and the various facets of project risk, which can be affected by the cost estimate and the methods by which the estimates are developed. This paper is aimed at the junior to intermediate cost engineering professional and provides a single source for AACE references on this subject. This article was first presented as EST.3429 at the 2020 AACE International virtual Conference and Expo.

Introduction

Corporate investment, which results in the continued successful operation that meets or exceeds, defined strategic goals and objectives, drives the need for portfolios, programs and projects.

Regardless of which industry a practitioner may consider, there is inherent uncertainty concerning the selection of the right mix of business endeavors to make up the projects, programs and portfolios which are expected to achieve these investment objectives. The level and type of uncertainty are both driven and affected by an organization’s data, processes and decisions.
 
At the core of the AACE® Total Cost Management Framework® (TCM), is the fundamental principle that the success of any corporate strategic mission, or objective, relies on the effective integration of the various cost engineering disciplines to develop the most accurate and applicable data, and use the most refined and exacting processes to make the most well-informed decisions.
 
The processes of cost estimating and risk management provide primary inputs and outputs which directly influence and inform the investment planning, decision making and implementation process. This process is illustrated at a high level by the process map shown in Figure 1.
 
Understanding how the inputs to the cost estimating process, as well as the outputs of this process, interact with the risk management process inputs and outputs significantly improves the estimator’s ability to control and limit the amount of uncertainty around the investment decision-making process.

Figure 1–Investment Planning, Decision, and Implementation Process Flow

This paper aims to provide a general overview and discussion of the cost estimating process, the risk management process, the various facets of the cost estimate process and outputs which pertain to risk at each interface point within the investment planning, decision and implementation process and provides general recommendations to the cost estimating practitioner concerning the minimization, or elimination, of those aspects of risk which are within their control to manage.

Discussion on the Cost Estimating Process

The Cost Estimating Process

As a concept, the “estimation’” of anything (be it a final project cost amount, a total project schedule duration, the expectation of project success against objectives, or the amount of risk in the endeavor) is a calculation.
 

This calculation is ideally performed using:

  • Applicable, quality (i.e. best available) input data as a basis for scope and factoring,
  • Industry best practices and methodologies for development,
  • Consistent processes for documentation, review and assessment; and,
  • Reasonable judgement in ascertaining if the outputs are useable

The outputs of this calculation result in some range of probable values, from which a single number is often be selected for use. The users of this value have expectations of it being credible, and reliable, for the purposes of appraising the scope, and whether the thing being evaluated is worth pursuing (or continuing).

In AACE’s Total Cost Management Framework [1], cost estimating is defined as “the predictive process used to quantify, cost, and price the resources required by the scope of an investment option, activity, or project.” [2, p. 90]. The output of this process is the cost estimate itself, which is often described and documented in a basis of estimate document.

It is apparent when referring to figure 1 above that output of the estimating process, may be used for many purposes, including:

a)     The determination and selection of the best investment option from several alternatives,

b)    The economic feasibility of the selected option for decision making,

c)     The establishment of the project budget once the decision to proceed has been made,

d)    The proper application of the risk management process for the determination of contingency and reserves,

e)    The management, reporting and control of the project against that budget once implementation is underway; and,

f)      The provision of additional inputs to a corporate historical database, which provides the basis and benchmarking for future endeavors of similar requirements.

The Function of the Cost Estimator

AACE’s Recommended Practice 10S-90, Cost Engineering Terminology [3] defines cost estimator’s function as “COST ESTIMATOR (PROJECT) – Project cost estimators predict the cost of a project for a defined scope, to be completed at a defined location and point of time in the future. Cost estimators assist in the economic evaluation of potential projects by supporting the development of project budgets, project resource requirements, and value engineering. They also support project control by providing input to the cost control baseline. Estimators collect and analyze data on all of the factors that can affect project costs such as: materials, equipment, labor, location, duration of the project, and other project requirements. (November 2012)” [3, p. 35]. This is further elaborated in AACE’s Recommended Practice 101R-19, Roles and Responsibilities of a Project Cost Estimator. [4]

The Value of Cost Estimating

The value of the cost estimating process lies in its capacity to predict the final cost of a planned scope of work, with some measure of reasonable certainty; however, by its very nature, the cost estimate is a prediction and, therefore, uncertain. It is therefore vitally important that:

  1. The cost estimate is accurate, concerning each of the stakeholder’s end uses to limit the amount of uncertainty within the cost estimate and,
  2. That the estimator understands the concept of risk and the risk management process as well as the fundamentals of developing and documenting a cost estimate correctly.

AACE has several recommended practices (see table 1), as well as professional practice guides, which describe in detail the industry best practices for cost estimating, including the requirements for the development, documentation, and classification of cost estimates to ensure a high degree of accuracy. The cost engineering practitioner is advised to consider the guidance provided in these documents when performing cost estimating process activities. 

Table 1–List of AACE Recommended Practices (Cost Estimating Topics)

Discussion on the Risk Management Process

What is Risk?

AACE’s Recommended Practice 10s-90, Cost Engineering Terminology [3], defines risk as:

(1) An ambiguous term that can mean any of the following: a) All uncertainty (threats + opportunities); or b) Undesirable outcomes (uncertainty = risks + opportunities); or c) The net impact or effect of uncertainty (threats – opportunities). The convention used should be clearly stated to avoid misunderstanding.

(2) Probability of an undesirable outcome.

(3) In total cost management, an uncertain event or condition that could affect a project objective or business goal.

See also: CONDITION (RISK CONDITION); EVENT; OPPORTUNITY; THREAT; UNCERTAINTY. (December 2011) “ [3, p. 102]

In reviewing this definition, it can be extrapolated that a cost estimate, due to its probable nature, can be a risk in and of itself to the project. If the cost prediction is wrong, there is certainly a higher probability of an undesirable outcome for the project.

What is Risk Management?

Risk management is the process by which an organization plans for, identifies, quantifies, analyzes, responds to, monitors, controls and reports risk within a strategic investment option (be it a project, program, portfolio).

AACE defines risk management  in Skills and Knowledge 6th Edition (S&K6) [2] as:the process of identifying risk factors (risk assessment), analyzing and quantifying the properties of those factors (risk analysis), treating the impact of the factors on planned asset or project performance and developing a risk management plan (risk treatment), and implementing the risk management plan (risk control)” which is a direct citation from the TCM Section 7.6.1. [2, p. 407].

The S&K 6 further elaborates that “the goal of risk management is to increase the probability that a planned asset or project outcome will occur without decreasing the value of the asset or project. Risk management presumes that deviations from plans may result in unintended results (positive or negative) that should be identified and managed” [2, p. 407]

From this definition, it can be seen that there is an expectation that the risk management process results in a variety of response alternatives being identified and which must then be assessed to evaluate whether they address the risk sufficiently such that established thresholds for the project (be they cost, schedule, quality, safety or technical performance) are not exceeded.

In aide of these risk management process activities, the cost estimator  is expected to contribute to the process by providing the following inputs directly to crucial risk management process steps:

  • The characterization and quantification of the cost impact of the risk element(s) in the project, which are usually captured in the project risk register.
  • The characterization and quantification of the “cost to implement” for the various response alternatives so the best solution for addressing the risk element can be selected and incorporated into the base estimate for the project
  • Evaluation and quantification of residual contingency requirements needed to achieve the necessary estimate classification level.

The Nature of Estimates and their Relationship to Risk

Project cost and schedule estimates are predictions based on empirical data. Because they are predictions or put another way “very-well educated guesses,” it means these estimates are inherently uncertain, and there are a variety of factors which can contribute to that uncertainty, which include the following:

  • The level of maturity and definition may be very low because the project is still in the earliest stages of planning and many elements of scope, contracting and execution strategy may as yet, be undefined (or poorly defined)
  • The organization’s estimating processes, procedures and tools may be rudimentary, poorly documented or exercised in an ad hoc manner leading to inconsistencies in both historical information for estimating input reference as well as a loose definition of estimating development practices.
  • The strategic planning assumptions forming the basis of the estimate were not necessarily validated with, or accepted by, all estimate stakeholders
  • The organizational objectives for the project were not well communicated to the estimator and, therefore, not documented appropriately in the estimate development files.
  • Factors external to the estimating process, which might have influence over the cost in the estimate were not readily identified and addressed in the estimate. These could include such things as the cost of raw materials, labor  rate changes, unique contract terms and conditions, etc.
  • The organization operates in a siloed environment and does not necessarily integrate the estimator adequately in the risk identification and quantification processes leading to potential inaccuracies in the ranging information and risk modelling.

The Value of Risk Management

The value of applying the risk management process to the activities defined in the investment decision-making process (including the cost estimating process), lies in the fact that that it is a structured process to anticipating and assessing the various ways an investment endeavor can be affected by uncertainty to inform effective decision making that can reduce that uncertainty and thereby improve the success of the endeavor.

AACE has several recommended practices (see table 2), as well as a professional practice guide, which describe in detail the industry best practices for risk management, including the requirements for the planning, identification, assessment, treatment of risk as well as development and use of risk models for analysis. The cost engineering practitioner is advised to consider the guidance provided in these documents when supporting the risk management process and identifying required contingency within the estimate.

Table 2–List of AACE Recommended Practices (Cost Estimating Topics)

The Many Facets of Risk an Estimator Needs to Consider within the Estimating Process

As discussed above, there are many aspects of cost estimating, which pertain to the subject of risk, and the process of risk management on a project. There are also several ways in which the cost estimating process can affect or even introduce risk to a project. The cost estimator should be aware of these considerations when preforming their tasks to ensure they effectively capture and document the technical risks associated with executing scope in the project, as well as trying to minimize the potential for introducing new risks to the project through their actions.

Recall from AACE RP 17R-97, Cost Estimate Classification System [5], that “the Cost Estimate Classification System maps the phases and stages of project cost estimating together with a generic project scope definition maturity and quality matrix” [5, p. 1] and helps guide the estimator for the correlation of required scope maturity, to the potential estimate categorization and end-use, as well the required methodologies, accuracies and effort necessary to achieve that categorization. This correlative mapping guidance is summarized in figure 2.

Figure 2–Generic Cost Estimate Classification Matrix Provided as Table 1 in RP 17R-97 [5, p. 5]

Many of the characteristics identified in figure 2 can be subjective, depending on the standards and processes employed by the organization developing the estimate, which can introduce the potential for uncertainty that an estimate as presented truly represents the characterizations expected by the RP. It becomes the responsibility of the estimator to then employ the guidance provided in RP 31-03, Reviewing, Validating, and Documenting the Estimate [6] to ensure the cost estimating process has been followed judiciously and with prudence.

Further, if the assessment of the estimate characteristics are not appropriately aligned (vis a vis the applied definition of the characteristic) to the expected, or required, characterization of the estimate class designation as defined by the guidance in RP 17R-97, this also creates the potential for uncertainty as the applied ranging values may be incorrect and inadequate for proper contingency evaluation. An excellent AACE technical paper by J. Hollman, R. Bali, C Germain, M. Guevremont and K. Kai-man entitled “Variability in Accuracy Ranges: A Case Study in the Canadian Hydropower Industry” [7], which discusses this issue and can be found in the AACE virtual library for further reference. While the case study centers on a particular industry, the lessons learned from the example can be applied elsewhere.

Recall from RP 17R-97, Cost Estimate Classification [5], that “the estimate accuracy range is an indication of the degree to which the final cost outcome for a given project will vary from the estimated cost. Accuracy is traditionally expressed as a +/- percentage range around the point estimate after the application of contingency, with a stated level of confidence that the actual cost outcome would fall withing this range” [5, p. 5]. From this it can be understood that, within AACE, the industry best practice with respect to communicating about the base estimate is more properly conducted using a “range of ranges” rather than one particular “point value”. AACE Recommended Practice 18R-97, Cost Estimate Classification System -As Applied in Engineering, Procurement, and Construction for the Process Industries [8] (recently updated in March of 2019), provides a graphical representation of the variability in accuracy ranges as shown in figure 3 below.

Figure 3–Illustration of the Variability in Accuracy Ranges for Process Industry Estimates as provided in RP 18R-97 [8, p. 5]

Risk Management Considerations within the Estimating Process

Some of the risk management considerations which must be evaluated or addressed in the cost estimating process by the estimator are noted below. The items identified by no means represent a comprehensive list, as there are industry-specific nuances which may also need to be considered, however, these are presented as a few general guidelines for review:

    1. The Risk appetite of the Organization – As per Chapter 9, Cost Estimating of the S&K6 [2], “ The capital cost of a proposed project is one of the key determinants in evaluating the financial viability and business case of the project” [2, p. 90]. This financial viability evaluation is always made in relation to the risk appetite of the organization. An organization willing to “take a risk” on a project may dictate the development of an aggressive (low-cost value) estimate, and conversely, the organization which does not have much free capital to lose in pursuit of an endeavor may choose to move forward with a conservative (higher cost value) estimate. If the estimator does not understand the risk appetite of the organization, i.e. how aggressive or conservative they wish to be in pursuit of the project, they may run the risk of developing or documenting the estimate inappropriately. As an example, an organization that leans toward an aggressive estimate may want very few allowances added to their estimates; they may want to adjust the skew applied in the risk modelling.

    2. The Risks “Internal to the Estimating Process” – The process and diligence by which the estimator has quantified base costs and pricing, the logic and planning and duration basis and assumptions, applied allowances etc. can all introduce uncertainty and affect risk in the project. If an estimator is not aware, for example, that; the organization has changed elements of the estimating standards to be applied, a template has changed concerning resources rates or applicable overhead percentages, a norms reference has been used but it hasn’t appropriately aligned into the estimating database with the correct location basis for the project – All of these can create error in the estimate, and that creates uncertainty that the estimate presents a correct base cost for the scope of the project.

    3. The Risks “External to the Estimating Process” – Projects, by their nature, can be moving targets as they are iterated through their lifecycle from initiation to closeout. The definition of scope and execution may completely shift from what was initially understood by the project team and communicated to the estimator. When this happens, it introduces uncertainty that the estimate as developed to that point remains valid going forward for the project. Examples can include situations where contractual constraints are applied such that project cost and schedule have dictated maximum allowable thresholds that must be maintained, or a primary stakeholder does not like the price in the estimate, leading to a complete revamp of scope through a value engineering study late in the project lifecycle. The skilled estimator should understand that these situations can happen on a project and seeks to maintain the highest levels of communication with stakeholders so they can be in the best position to address these constraints and changes.

    4. Self-Created Risks – The quality and level of maturity of the inputs, the bias of reviewers, and other factors influencing the cost estimating process all contribute to the relative inaccuracy of the base estimate, regardless of how diligently the estimator has applied standards and processes. An estimator must be cognizant of these aspects of the inputs to the cost estimating deliverable and ensure they do not “over-present” the characterization of estimate class; this would result in what is being termed as a “self-created risk.” If the project has provided estimating inputs that can at best characterizes the project as 5% complete, they should not accept a request to develop an estimate required for budget authorization or control; however it is all too often the case that estimators are provided with inputs with a much lower level of definition than the requested end-use for the estimate.

    5. The Risk Quantification Step of the Risk Management Process Itself – In this context, the reference is to the action of providing input to the risk register line item costing and ranging of these items. The cost estimator is in the best position to understand which elements of uncertainty were addressed in the base estimate costs and ensure that the risk register accounts for the elements of uncertainty that were carried over from the base (i.e. not included as base cost). The base estimate, in conjunction with the risk register (line item and response plan) costs will form the complete foundation from which the project contingency range of values will be formulated through risk modelling and analysis.

    6. The Cost and Schedule Risk Modelling and Analysis Used to Develop Contingency Levels for the Project – Just as a skilled risk analyst will have some understanding of the cost estimating process, so too should the skilled cost estimator have an understanding of the risk modelling process and analysis methodologies. If the model and analysis do not account; for example, for all of the assumptions and exclusions applied in the development of the base estimate, or the overall strategy for the project, and contractual constraints, then the familiar adage “garbage in, garbage out” applies and this results in a new introduction of uncertainty concerning how appropriate the contingency values will be to maintain confidence in the project budget.

    7. The Budget Control and Performance Forecasting Used During the Management and Control Aspects of the Project – There are two aspects of the management and control process for a project that the estimator should be aware of when considering the estimate and associated risk management because the process of risk management does not stop once the budget has been authorized against the base estimate and work will be allowed to proceed.

      The first aspect is the level of rigour applied by the project team to the management and control against the baseline. There are significant case studies that show projects having overrun their cost and schedule estimates, and very often, the general commentary as to why this occurs runs along the lines of “the estimator didn’t get it right.” Poor adherence to control and management processes will introduce uncertainty in the project’s ability to adhere to the original baseline. Ill-defined rules of credit, loose change management practices, and misaligned reporting intervals can all contribute to the scenario whereby the project actuals do not match the project estimate. The estimator should ensure they are engaged throughout the lifecycle of the management and reporting process to ensure that actual execution is progressing within the assumptions and scope limitations that define the base estimate.

      The second aspect comes from the application of a well-defined change management process, which involves the assessment of the change impact (be it cost or schedule) which must then be calculated by the estimator and the baseline, and associated review, validation and documentation appropriately updated to reflect the new state of play for the project.

The quality, quantity and applicability, of the inputs that define our knowledge of some “thing” (be it a project, program or portfolio) all affect the accuracy and certainty of the outcomes of any process which relies on these inputs. This is most true for cost estimating and risk management processes. If the project scope, execution and estimating strategies, risks, etc. (inputs) for a project (the thing) are not defined or are poorly understood, the estimate (as the output of the cost estimating process) will also be poor. This creates uncertainty that the risk management process which follows, will produce reasonable results in evaluating and defining the required contingencies to enable project success against objectives.

There is also the matter of personal “bias” to consider. The bias of the user of any cost estimating and risk management process inputs will affect the way the inputs are used to create outputs, and the bias of the receiver of these process outputs will also affect how they interpret the results of the process. A skilled estimator will seek to understand and remove bias from their work and communicate about their estimates in such a way as to remove the bias of those who are estimate stakeholders.

The estimator should remember that the quantification of risk is relative to whatever has been developed as a base estimate (this is an observation that applies to both cost and schedule base estimates). It is also influenced by the base. If the base value is aggressive (very lean and without many allowances), the associated contingency required to achieve funding approval will be correspondingly higher to achieve the same level of confidence in the number provided. (This is assuming these values and confidence ranges are decided based on estimate classifications as per AACE RP 17R-97 and associated industry-specific variants). Therefore, the base should be as complete and well documented as possible so it can be understood and evaluated appropriately.

Conclusions

Estimating of risk items following industry best practices should be standard procedure for an organization’s project controls and project management functions.

Whether these estimates are developed using range estimating, conceptual or definitive estimating techniques (as dictated by the level of project definition available to the estimator), the resultant quantified risk dollar value will be as accurate as possible to either over-fund, or under-fund, the required project risk value (i.e. amount of contingency).

Ensuring a project has a valid, credible, and accurate amount of contingency available to address risk throughout the project lifecycle is a key element to ensuring a successful project outcome.

The collective AACE recommended practice [HQ8] guidance on cost estimating and risk management provides the fundamental direction to the estimator that summarizes to the following observations:

  1. To quantify risk as part of the risk management process, one needs a well-developed base estimate from which to assess the remaining risk in the project. Further, cost estimating input is needed for the expected costs associated with risk exposure, expected monetary values before and after mitigation, and the costs associated with any response plans.
  2. For the estimator to do this well, they first need to understand the risk appetite of the organization performing the project and the attendant estimating strategy.

 In essence, the concept of cost estimating (as defined and used in AACE), includes the elements of risk quantification, and base estimates should always be considered the starting point to the iterative process of identifying, quantifying, assessing and managing risk.

When developing the base, an estimator must ensure the following to support the successful output of the cost estimating process for input to the risk management process:

  1. Reference data is relevant and correct
  2. Source has been scrubbed before use
  3. Project-specific factors have been applied consistently
  4. The outputs have been reviewed for quality and alignment against governance procedures
  5. The basis, assumptions, plan, strategy and development outputs have been well documented (i.e. in an accurate and fulsome manner).

When communicating about the base estimate, the estimator should be aware that because of the uncertainty within; the estimating inputs, the estimating process and the other elements as discussed earlier, the base estimate is best described as a range of contextualized values which should incorporate the risk elements and aspects of the projects.

References

1.     H. L. Stephenson, Ed., Total Cost Management Framework: An Integrated Approach to Portfolio, Program and Project Management, 2nd ed., Morgantown, WV: AACE International, Latest Revision.

2.     D. M. Hastak, Ed., Skills and Knowledge of Cost Engineering, 6th ed., Morgantown, WV: AACE International, 2015.

3.     AACE International, Recommended Practice No. 10S-90, Cost Engineering Terminology, Morgantown, WV: AACE International, Latest revision.

4.     AACE International, Recommended Practice 101R-19, Roles and Responsibilities of a Project Cost Estimator, Morgantown, WV: AACE International, Latest Revision.

5.     AACE International, Recommended Practice No. 17R-97, Cost Estimate Classification System, Morgantown, WV: AACE International, Latest revision.

6.     AACE International, Recommended Practice No. 31R-03, Reviewing, Validating, and Documenting the Estimate, Morgantown, WV: AACE International, Latest revision.

7.     J. K. e. Hollman, RISK.1721 – Variability in Accuracy Ranges: A Case Study in the Canadian Hydropower Industry, Morgantown, WV: AACE International, 2014.

8.     AACE International, Recommended Practice No. 18R-97, Cost Estimate Classification System – As Applied in Engineering, Procurement, and Construction for the Process Industries, Morgantown, WV: AACE International, Latest revision.

]]>
https://source.aacei.org/2021/04/28/estimating-as-it-pertains-to-risk-management/feed/ 0
Top Ten Successful Approaches to On-Time Completion https://source.aacei.org/2021/01/31/bonus-technical-article/ https://source.aacei.org/2021/01/31/bonus-technical-article/#comments Sun, 31 Jan 2021 09:23:00 +0000 https://source.aacei.org/?p=1001

Top Ten Successful Approaches to On-Time Completion

by Glen R. Palmer, CFCC PSP FAACE and
Christopher W. Carson, CEP DRMP PSP FAACE

Abstract

Many projects today fail when it comes to completing on the planned final completion date. This failure is generally because of several issues related to the quality of the analysis of delays, owner commitments, and contractor performance problems.  A deep-dive technical analysis, supported by lessons-learned and deep experience in problem resolution and mitigation of delays, greatly improves the opportunities for achieving on-time completion.  Unfortunately, this deep dive technical analysis is often not performed, whether it is because of inexperience, lack of competence, limited time for analysis, or weak analysis.  The use of a system to provide completion prediction and analysis streamlines the effort and ensures that these ten approaches are consistently followed for a successful completion.  In this article, the authors continue their series of “Top Ten” issues and will give you their top ten approaches for enhancing a project’s chances of meeting this planned date. The authors of this article are widely experienced in planning and scheduling complex projects, dispute resolution analyses, project controls, project management and have testified as experts in forensic and project schedule analysis. This article was first presented at the 2020 AACE International Conference and Expo as CSC.3429.

Introduction

In the last decade, many projects have failed when it comes to finishing on the planned final completion date. There are several project management approaches that greatly improve the chances of on-time completion and reduce the risks of performance issues.  In keeping with the authors’ top-ranked series of papers and presentations on the ten best or worst approaches to various aspects of project controls; this article will provide the authors’ top ten favorite and most effective approaches for enhancing a project’s chances of meeting its planned completion date.  These approaches are the aggregation of two lifetimes of successful projects, support a collaborative approach to project controls which is promoted by the authors, and have worked well on past projects. 

The authors of this article are widely experienced in planning and scheduling complex projects, dispute resolution analyses, project controls, project management and have testified as experts. With a combined experience of more than 75 years in the engineering and construction industries. They have seen the execution of many successful and many unsuccessful projects. This combined experience has shown both authors that many unsuccessful project management teams fail as a result of not taking one or more of these important approaches to achieve the project completion date. 

It is important to understand that these steps will only help a project succeed when that project has both experienced and committed project personnel, preparing and analyzing the project’s data. There are no adequate substitutes for project experience and a committed project team.

These ten approaches will be discussed in ascending order of importance, but the authors believe that all ten approaches are very important to the success of any complex project.

APPROACH NO. 10 – “Manage the Project with an Approved Project Schedule”

A well-developed project schedule, designed to achieve approval, requires a number of steps in order to result in a schedule to which all of the stakeholders agree and are willing to support its use as a management tool. The following important points will be discussed to support the development of an approved schedule.

Important Discussion Points

  1. Scheduling Specification
  2. Detailed Planning Session
  3. Constructability Review
  4. Weekly Scheduling Meeting
  5. Accountability
  6. Schedule Updating
  7. Float

Detailed Discussion Points

Scheduling Specification – Many of the complex projects executed today are subcontracted on a discipline basis. This means that the integrated project schedule will be developed and updated by multiple schedulers, most of whom may use a different approach to creation of a Work Breakdown Structure (WBS), activity detail, constraints, activity relationship requirements, and so on. The only way to ensure that the input is done consistently is, in the very beginning, to define the method for baseline schedule development and periodic updating. Projects that require multiple inputs without a scheduling specification struggle to have this required consistency and often must pay a premium by not spelling out these requirements prior to the bidding process.  The owner can only require from the contractor those conditions that are specified in the contract, so the scheduling specification is vitally important.

Detailed Planning Session – Every complex project should be planned in a joint multi-day session which includes all the stakeholders, including the owner. This is the only way to get a consensus on the project plan. This process is best accomplished by using an experienced facilitator who is neutral, can moderate the discussion, and keep the process moving. It is also best to hold this session in a neutral location to avoid interruptions from a busy office environment. These sessions are best accomplished by including the experienced personnel who will actually execute the project, especially the construction personnel (if the project includes construction), because they have the knowledge from previous projects to understand what it takes to accomplish the work needed to support the schedule requirements. Fixed price contracts and subcontracts can complicate these planning sessions due to the inability to dictate the means and methods associated with fixed price work, but these sessions still need to be held. This may require an individual meeting with those contractors prior to the session that includes all stakeholders, but it is necessary in order to get a plan for the project on which everyone agrees and will support.

Constructability Review – Project schedules that include construction must be driven by construction. The planned construction sequence will determine how engineering and procurement need to be sequenced. The only way to ensure this happens is to have construction personnel involved in the detailed planning session and have periodic engineering and construction reviews of the 3D model (if applicable), and/or the drawings and specifications. Construction also must be part of all-important purchase order approvals. The industry has long recognized that construction involvement during the engineering process improves engineering documents.  Constructability reviews are also shown to reduce the likelihood of resource-draining litigation.

Weekly Scheduling Meeting – Once a schedule has been approved and becomes a project management tool, there needs to be a weekly scheduling meeting, including all parties currently performing project work. The only way to ensure the project team will use and perform to the schedule is to make the schedule the focus/agenda of a weekly meeting. The schedule needs to be issued as a look ahead schedule, showing only 2-4 weeks in the future, depending on the size of the project. It is very important to limit the number of pages for these schedules so that it is easy for the team to review and update for the meeting. This is where problems are discussed, and action items are developed, ensuring the schedule can be supported. 

Accountability – The project must develop an atmosphere where all team members are held accountable to support the plan and schedule that they helped develop for the project completion date to be achieved. In most cases this can be accomplished using a non-adversarial approach, but there may be instances where team members will need to be replaced throughout the project to achieve the schedule requirements.  

Schedule Updating – Updating a project schedule must be completed accurately and quickly. The main purpose of a schedule is to accurately forecast a project’s status with respect to important milestones. An updated schedule needs to provide management a timely and accurate projection of the schedule status to allow them to have the necessary information to manage effectively. When a schedule update takes too long to issue, it is essentially only tracking history which may not be actionable. Requiring the schedule to do too much, such as detailed cost loading, can impede the ability to get the schedule updated, reviewed and issued on a timely basis.

Float – Recent history and industry studies indicate that a very small percentage of complex projects finish on the planned completion date. Too many two to three-year complex projects finish on time without any float or time contingency available.  It is almost impossible to predict every possible schedule risk that can derail a project. Having float and/or time contingency should be no different than having dollar contingency on a fixed price estimate, but it is. Most owners want their project completed by a certain date to create revenue. Some projects produce hundreds of thousands of dollars per day in revenue and this makes a two-month delay in production a very painful issue. Some owners need to complete by a certain date in order to beat their competition and survive. There is no easy answer to this issue but in many cases, it requires a contractor to build an effectively risk-adjusted schedule by an expensive acceleration program in order to achieve the schedule completion date.

Important Advice

If the team wants to make a completion date, it is important to develop a schedule with a good plan, ensure the entire team has bought into it, approve the plan and schedule, and then go make it happen every day. If the schedule is simply thrown over the fence to the stakeholders and then not used as a management tool, THE PROJECT WILL FAIL.

APPROACH NO. 9 – “Update the As-Planned and As-Built Separately, No Less than Monthly”

Each schedule update consists of two parts, the as-built portion and the as-planned portion, and accurate data is important for both.  The as-built data, to the left of the update data date, documents the TRUE performance and progress or lack of progress.  The as-planned data, to the right of the update data date, shows the INTENDED plan, often a mitigation plan, designed to recover the lack of progress from the as-built performance.  Schedules are most commonly updated monthly, but contractors may fail to submit each monthly update, and this can create numerous problems.

Important Discussion Points

  1. Update and validate the as-built data separately
  2. Update and review the as-planned predictions separately
  3. Ensure that the schedules are updated no less than monthly

Detailed Discussion Points

The important performance part of the as-built update portion of the monthly schedule update is the accuracy of the actual dates and the assignment of the remaining duration for activities that have started but are not completed.  These are actual indicators of performance, similar to the actual invoices for time and material work that is completed.  The as-built side of the schedule update is not subject to change except for accuracy mistakes, so it should be validated to ensure the dates are correct.  

Whatever performance was achieved will be documented in the as-built side of the schedule, so this is very useful in discovering if the field team achieved the planned performance.  The as-built data can be reviewed either as a stand-alone update, or can be developed in a temporary, created update, that is often called the end-of-period (EOP) update to indicate that it is the end of the actual performance period just before the contractor makes logic and other changes to the as-built schedule.  If the contractor updates the as-built data, and it predicts completing on time, it is likely that there will be no logic changes made and the schedule will be submitted with only the completed changes in the as-built data.  However, when performance has slipped, the contractor will commonly “adjust” the schedule to attempt to mitigate the lack of performance.

The as-planned side of the schedule update is a PLAN that has not yet happened..  When the contractor makes logic, or more accurately, non-progress, changes (including changes to calendars, future durations, relationships, lags, and constraints) those create a model of actions that they plan to undertake but have not started.  There is no guarantee that that revised model will be followed, that it is reasonable, or that they will be able to achieve the planned mitigation that is modeled.  

In Figure 1, the top bar chart is the EOP update, the as-built progress-only side of the update.  This schedule shows clearly that the critical path driver was the installation of Pier Cap #3, perhaps caused by lack of performance (analysis would need to be made to determine if so and the cause of any lack of performance).  This resulted in an actual performance delay in July.

In the bottom bar chart of Figure 1, for the full Update #3, that same activity’s original duration was reduced in the as-planned portion, so it showed less impact on the critical path.  This removed the delay in July and showed the update as if it was still on target.  If an owner-caused delay occurred in August, it would have been measured from the zero critical path on-time schedule instead of having 5 days of float due to the pier cap delay.  Therefore it is vital to separate the progress from the mitigation efforts in the as-planned portion.

Figure 1 – End of Period vs. Full Update Schedules (Mark Nagata and Chris Carson)

Many projects complete late due to a failure to work the plan as shown in the baseline and each subsequent update, and this split analysis approach is an important method to identify this failure and take corrective actions.  In addition, the reason for delays that are mitigated by the contractor’s logic changes are often concealed so it appears that the wrong party is responsible for delays from update to update.  However, the real delays exist in the as-built schedule, while the concealment can be disguised by the as-planned mitigation effort.  This is not to say that there is necessarily any intent to conceal the performance failure, more often than not, it is just weak scheduling practices.

In addition, often contractors fail to provide regular monthly updates and there are numerous reasons, from the loss of their scheduler, leaving no one who can use the software, to performance issues that they believe will be corrected by the next update, so they simply do not provide the poor performance update.  However, going without updates generally creates larger problems as the project continues to slip, and the slip is too often performed without analysis into the cause.  

Analysis is easier and more accurate when the update periods or “windows”  are narrow, so monthly is a good update frequency for most projects, and it tends to align with invoices thus providing some correlation between cost and progress.  The longer the project runs without updates, the more problems exist that are concealed by not performing the monthly analysis, demonstrated by many claims showing long periods of no updates with large amounts of delay during those periods.  This makes for a difficult analysis to determine what happened and who was responsible, and removes the ability for the owner to participate in any mitigation effort if it should prove to be required by owner-caused delays.

Important Advice

Always review the performance side of the update, the as-built data, separately to monitor failures to meet the plan (“work the plan”), identify causes of performance delays, review the as-planned side of the update to determine the reasonableness of the plan and identify any mitigation plans that may or may not be achieved.  Also ensure there are frequent updates, no less than monthly, and if the contractor fails to provide these, perform an update even if temporary, to analyze the performance, assign responsibility for any delays, and engage the owner in mitigation if there are owner-caused delays.

APPROACH NO. 8 – “Manage Resources by Using Off-site Prefabrication (Modularization) on Large Projects”

The authors have both seen instances where pre-fabrication or modularization have hugely benefited the project by accomplishing parallel work without adding to site resource requirements. This should be planned early in the project’s life cycle. This approach can accelerate the burn rate of project manhours without adding workers to the site.

Important Discussion Points

  1. Planned Resources
  2. Specialized Skills/Controlled Environment
  3. Acceleration
  4. Cost Benefit Analysis
  5. Engineering Sequence

Detailed Discussion Points

Planned Resources – Reducing the planned resources by reducing the craft manhours, that would normally be required to be expended on a construction project, can be very beneficial to on-site performance. Any time the number of workers on a project is lowered without a loss of work being accomplished, the potential for trade stacking becomes less and this can have a tremendously positive impact on project performance.

Specialized Skill/Controlled Environment – Many construction projects require specialized skills, such as alloy welding. Having some of this work performed in a vendor’s shop can reduce the number of these qualified welders required on the job site, as well as the number of welds that must be tested on site. This can cause the number of weld inspectors required on site to also be reduced, thus saving on staffing costs. In many cases, the welding may be performed in a more controlled environment with better welders because the vendors have weeded out less capable workers, producing a higher quality product. 

Acceleration – Taking worker hours out of the field and putting them in a vendor’s shop and performing that work in parallel with other work can have a hugely positive acceleration impact on the project. An example is discussed under the pipe rack module in Figure 2.

Cost Benefit Analysis – It is always important to do a cost-benefit analysis when considering this step to ensure that the schedule advancement benefit is worth any additional associated costs. However, avoid assuming that this possible acceleration benefit is going to cost more, since, on subcontracted projects, this can be just another way of subcontracting and can even be less expensive. 

Engineering Sequence – Most of the time, prefabrication will alter and usually accelerate a portion of the project’s engineering. This may require engineering sketches to give a structural steel vendor a pipe rack size and configuration to get on the vendor’s fabrication schedule for a prefabricated pipe rack.  At the same time, it could result in an acceleration of the pipe and support design to get a vendor to fabricate a pipe rack module (See Figure 2).  This also applies to sequences for structural metal and steel fabrication and erection and other opportunities for prefabrication.

Figure 2 – Example of a Pipe Rack Module

The piece of pipe rack in Figure 2 (pipe rack module) is built and all the pipe is welded, supported, insulated and lagged off-site while the on-location or field project team is completing the rack foundation design and construction. That could not happen on a “stick built” project. Once two modules are set next to each other, the joining ends of the pipe are welded together, the weld is touched up with paint, insulated and lagged. All this work is completed with just a fraction of the worker hours being expended on site.

Important Advice

Prefabrication is always worth looking at early in a project’s lifecycle, as a possible schedule execution benefit, but perform a cost-benefit analysis to fully understand the cost ramifications.

APPROACH NO. 7 – “Develop and Maintain a Schedule-Oriented Culture”

A schedule-oriented culture is one where everything is viewed through the lens of the schedule, which places the focus on the regardless of related float.  Without this culture, meetings tend to discuss all submittals, requests for information (RFI), designer directives, non-compliance reports, issues logs, change request logs, change order logs, correspondence, and other items, with the schedule treated as a separate discussion.  When the culture is schedule-oriented, the schedule is the opening discussion, and all these discussion items are raised only when they are critical or near-critical.  This removes high float items from the discussion, allowing a strong focus on the high priority items.  This approach does require a good CPM schedule, so the float values are reasonable and accurate, but the schedule orientation improves project performance and validates the value of the time investment in developing the better schedule.

Important Discussion Points

  1. Senior management support
  2. High-end technical scheduling effort
  3. Continuous change management effort
  4. Documented processes and procedures
  5. Continuous review of performance
  6. Schedule prominent in all meetings

Detailed Discussion Points

This culture can only be established if senior management has embraced the culture and the construction management (CM) team has bought into this approach.  Without this support, scheduling will not be seen as providing the value that it can demonstrate within the right culture.  Senior management must demand succinct and clear summary reports that keep them informed as to the performance, lack of performance, change management, analysis of delay and disruption, action plans to mitigate poor performance, and continuous efforts to improve even already reasonable performance.

There must be support for a deep-dive detailed technical scheduling so there is time and budget allowed, and the scheduling staff are competent and a vital part of the CM team.  Part of the reason why the team will appreciate the scheduling effort is if the analysis process is insightful and provides value in identifying delays and trends that will result in delays, as well as continuous recommendations for action to mitigate and to improve performance.   The scheduler must have a close relationship with the construction manager so they can discuss issues that affect schedule, and the scheduler must feel comfortable in making suggestions related to performance and progress.   Achieving value from tasks like walking the project to review the update data is simplified and improved using standardized checklists and forms, as shown in Figure 3.

Figure 3 - Example of a Field Update Form

Changes occur in all projects, and part of a good schedule-orientation culture is a solid change management effort that includes not just costs, but, just as importantly, the time associated with change.  Changes occur and create delay situations, but the schedule has an important role in identifying delays that have already been absorbed into the schedule, as well as future delays that may result from changes in project conditions or performance issues.

As noted in the 2016 Carson and Palmer paper, “The project schedule is the primary coordination tool, and the communication tool, so without a schedule in place early, the project is being managed without that primary tool.  So, construction management is left to flounder and maybe, worse, as the project is started, unforeseen conditions can be discovered but not easily analyzed.  It takes a good, detailed CPM schedule to enable credible and detailed analysis of delays.  Without the ability to support delay analysis, additional scope and unforeseen conditions would be absorbed into the contractor’s schedule and ultimately, scope of work.  Without the detailed early schedule, the project is at a higher risk of failure.”

The scheduling program should be written down as standard or best practice documents and explained in a step-by-step procedure to ensure consistent and standardized implementation.  Every project should have a detailed schedule, developed with the CM team so everyone has input into development and ownership of the schedule.  Of course, industry best practices should be incorporated into any standard practice documents, like the AACE Recommended Practices. [1], [2], [3], [4], [5]

Figure 4 – Examples of Processes & Procedures for Scheduling Program

The value of good scheduling is most often recognized through continuous review of performance and a good scheduler may have more insight into the schedule than the CM or contractor, because the scheduler is digging deeply into the logic and plan that is explained by the schedule details.  With continuous review of performance, the scheduler provides action items that promote correction of performance slippage, and uses trending analysis to find negative trends that can be corrected before they cause too large of a problem.  In addition, the analysis supports recommendations for improvement suggestions that could not just mitigate known delays but will help avoid potential delays.  Organizing the schedules with a good schedule log is valuable, provides insight from a summary level, and can ensure practical use of the schedule data.

Figure 5 – AACE “Schedule Logs” Recommended Practice

The last important point is that the schedule must be prominent in all meetings in order to reinforce the importance, and to help identify issues and actions that can be taken to improve performance.  Running the project by the schedule is the best approach and helps to both establish, as well as to reinforce, the schedule-orientation of the project.   One good way to ensure visibility and use of the schedule is through good reporting in the form of dashboards that summarize the schedule information into customized reports for each of the stakeholders.  An example of a project dashboard is shown in Figure 6.

Figure 6 – Example of a Program Dashboard (Carson, Divvela, and Napuri)

Important Advice

Embrace the value of a good schedule-oriented program, ensure the CM team is supportive and works with the scheduler to get the utmost value out of the technical scheduling effort, ensure that the project is run from the schedule plan, and provide insightful, succinct reporting in customized dashboards for each stakeholder.  This will yield valuable returns and enable the ability for the project to complete on time or early.

APPROACH NO. 6 – “Manage Project Delay Claims Timely to Ensure Best Efforts to Maintain the Schedule”

The authors have both seen instances where slow responses to delay claims have ruined relationships on a project and caused progress to slow down. Completing a complex project by its completion date requires the steady commitment of all its team members.

Important Discussion Points

  1. Timely Communications
  2. Improperly Developed Claims
  3. Monthly Delay Analysis
  4. Constructive Acceleration

Detailed Discussion Points

Timely Communications – Timely response to all project time extension requests and claims is key to project schedule performance. This is the only way to protect relationships with project team members and ensure the proper focus is on getting the work completed and supporting the ability to manage by the schedule. When a contractor or subcontractor is waiting for an answer on a request for a time extension for delay or other type of claim, that party often has additional pressure put on them from their management to get the issue resolved.  This pressure often distracts from the other important issues and actions required to maintain schedule.

Improperly Developed Claims – Many delay claims are prepared improperly. An un-absorbed delay needs to be developed using a prospective time impact analysis (TIA) where an absorbed delay needs to be developed using a retrospective forensic analysis method. These methods are detailed in AACE’s Recommended Practices – RP52-R06 “Prospective Time Impact Analysis” and RP29-R03 “Forensic Schedule Analysis.”  Studies show that this is one of the top causes for disputes, in fact, “Poorly drafted or incomplete and unsubstantiated claims” ranked as the third cause for disputes in North America.

Figure 7 – North American Dispute Causes Survey

Monthly Delay Analysis – It is extremely beneficial for a complex project to perform a forensic schedule analysis on a monthly basis during the month following each schedule update. Doing so allows the analysis to be performed and discussed when the events are fresh in everyone’s mind, which makes causation analysis easier, usually more accurate, and always less expensive than hiring an expert at the end of the job when the project team has been disbanded. It also highlights the party or parties that may need to take on the responsibility to accelerate the schedule and make up for a prior schedule delay. Finally, it can help avoid an expensive dispute process by settling the delay issues monthly. It is always easier to negotiate and award 5 days of delay, 10 times during a project, than it is 50 days of delay at the end of the project.

Constructive Acceleration – Delaying the response on a delay claim can cause the other party to send a letter claiming constructive acceleration on top of the delay claim. Constructive acceleration is when an owner denies or is silent on a legitimate excusable delay claim forcing contractors to accelerate their work. Constructive acceleration can be much more expensive than a simple extension of time.  And failure to respond could turn an excusable but non-compensable delay that would not carry any cost to the owner, like unusually adverse weather, into an acceleration claim costing money.

Important Advice

Respond quickly to all claims on all projects to protect the schedule and seriously consider doing a monthly forensic delay analysis to deal with the project delays.

APPROACH NO. 5 – “Provide a Resource Plan to Ensure Resource Availability”

A common symptom of project failure is an inability to provide sufficient resources on the project, often because of a lack of available resources in the local culture.  Often, resource consumption is not considered during schedule planning, and resources are not benchmarked. Therefore, usage is not evaluated on a regular basis.  A schedule is not a legitimate schedule if it does not accommodate the available resources and there is no easy way to monitor resources without planning for the consumption of resources into the schedule as the project progresses.

Important Discussion Points

  1. Types of resources and best use in resource loading
  2. Challenges in loading resources in schedules that impede resource loading
  3. Risks of out-of-sequence progress in monitoring resources

Detailed Discussion Points

There are basic challenges in loading resources in schedules, starting with the type of resource to be loaded.  The industry has three common resource pools used for scheduling; individual named resources, role-based resources, and crew resources.  Individual named resources are most commonly used in engineering schedules where each person is tracked to ensure they are fully billable and efficient. They may be the only resource who can perform some scope of work.  Role-based resources are used when it is important to understand how many trade and role-specific staff are needed, such as the number of plumbers or plumber laborers planned to accomplish a task or activity.  Use of crew resources recognizes that the practical issue with respect to resources is that the crew composition is best left to the trade contractors to handle crew composition decisions, which might be at the foreman level.   

While the role-based resource loading is very useful, there are risks to the contractor with out-of-sequence progress in monitoring resources, the use of which frequently creates problems.   When schedules are developed to a great level of detail with almost all finish-to-start relationships, the role-based resource loading works well, but if there are lags and/or progress that causes numerous out-of-sequence work, the roles tend to stack up unrealistically.   The authors have seen projects where a major trade contractor was at risk of termination due only to the number of role resources scheduled by the out-of-sequence updates compared to the count of roles on the project.  On one mid-rise cast-in-place concrete structure, the concrete subcontractor was threatened with termination because the updated schedule called for over 600 role-based resources and they only had about 350 workers onsite each day.  Once the out-of-sequence progress was corrected, the schedule required dropped back down close to the actual count, and analysis determined that the out-of-sequence work was mostly caused by poor drawings, multiple requests for information, and slow response, which turned out to be an owner-responsible set of delays and disruptions.

It is crucial that resources are planning and monitored, but a good choice is to consider the trade crews for loading, keeping the crew composition as information in the basis of schedule or written narrative.  With crews, analysis can be done for similar crews, such as all the concrete crews or even subdivided into work scope resources such as concrete formwork crews, as well as for total crews that are planning to work in any given space or time frame.  Analyzing these types of crew resources is very insightful and can help determine when there are insufficient crews planned or too many crews planned for the spaces, time, or availability of local resources.  

For example, looking at the total plumbing or storm drainage crews promotes a discussion about how many crews the contractor expects to employ on the project, when, and where those crews are scheduled to work.  One of the benefits of this analysis is based on the recognition that one crew should be capable of completing the scope of work identified in each activity within the original duration of that activity.  That requires each crew to have some lead worker that can lay out, direct the crew, and ensure the work is completed, as well as any unique equipment for that activity.   If there are four paving crews that are scheduled to install pavement at the same time, the project will require four pavers to make that happen.  These analyses allow good insight with relatively simple analysis that does not take long to implement. They can be done even if the contractor’s schedule does not contain the crew loading; a few global changes in a software like Primavera P6 will yield the crew-loaded schedule.  This is shown in Figure 8, where the comparison between the planned early and late dates shows significant slippage of workers, pushing the crews to later in the project and stacking those crews higher than the original plan.

Figure 8 – Resource Plan – Based on Both Early and Late Dates

The analysis can include similar crews and any overallocation, similar spaces and any overloading, and limited time frames and overloading in those time frames.  If the as-planned schedule shows 25 drywall crews needed to hang and finish the third floor of a building over a one-week time frame, and the contractor only has 10 crews, or the space cannot accommodate any more than 15 crews without overloading the available spaces, the schedule is not reasonable and will ultimately be delayed just by insufficient crews.  At this level of analysis, the composition of the crews is not a vital part of the issues raised.

Important Advice

A common symptom of failure is inability to provide sufficient resources on the project, often as a result of a lack of available resources in the local culture.  Often resource consumption is not considered during schedule development, partly because it can be difficult to load role-based resources (carpenters) since there is often a lack of detail in many schedules.  And during routine updates, out-of-sequence work causes the resource need to double with each out-of-sequence pair of activities.  Loading crew resources is an excellent alternative that provides a solid basis of analysis without significantly increasing the workload on schedulers or the team to collect information.

APPROACH NO. 4 – “Assign a Specific Team Member for Each Schedule Activity”

Each activity within a complex project schedule must have an experienced team member in charge of that activity. This person oversees developing the total scope of that work by communicating with the other team members to ensure that the work scope is understood, not duplicated, and that the total scope of work for the project is covered. Assuming that the scope is understood is a common error made on projects and often leads to the project team even disputing the definition of an activity. This person is also the person in charge of getting the work accomplished in the planned duration and may very well handle a trade, scope of work, or perhaps a sub-specialty.  The important point is the assignment of responsibility. 

Important Discussion Points

  1. Accountability
  2. Schedule Sorting
  3. Risk Management
  4. Schedule Quality

Detailed Discussion Points

Accountability – Every activity in a schedule needs to have a single experienced person that is accountable for making sure that scope of work gets done as planned. This person is the main point of contact on the project for that work scope and the person who discusses (in the weekly scheduling meeting) the progress, the problems and risks, and the actions needed to make the work happen as scheduled. This approach usually shortens the length of the weekly scheduling meeting because everyone knows who is in charge of the item about to be discussed.

Schedule Sorting – The project schedule should have sort code for these people to avoid having to distribute 50 pages of the schedule to them and make them have to hunt for their activities. Often, when given a 50-page schedule, project people do not look at them because of the time required to navigate through the pages.

Risk Management – These people become the backbone of the project’s risk management program as they are the experts on this scope of work.

Schedule Quality – There is no better scheduling approach to improve the quality of a project schedule. As this experienced team member takes a deep dive into the scope of an activity, he/she will learn, in many cases, that either the schedule logic or the duration needs to be modified to be more accurate. This process always improves the quality of the schedule and the team understanding of the schedule.

Important Advice

Assigning a specific person to be responsible for each activity is effective, saves time, enhances a risk management process, and improves the schedule.

APPROACH NO. 3 – “Use Formal Trending Analysis to Recommend Mitigation Actions”

Critical path slippage is important to monitor because those are the activities that will immediately cause a delay, and near-critical path delay is also very important to monitor because those are the activities that might suffer a mid-period critical path delay, overtake the critical path, and cause a delay to the project; these near-critical path delays tend to be insidious because often no one is really monitoring that near-critical path.  But the third important factor in delays is even harder to identify and monitor and this is the non-critical path or all the activities.  When the work in general is not being installed at a rate commensurate with the schedule time remaining, there will be slippage of the work activities to later in the project, causing work to stack at the end of the project.  This will increase risk of cost overruns, quality problems, disruption, and delays to the project.  

Important Discussion Points

  1. Trending delays are typically not related to the critical path
  2. Delays that are not recognized can accumulate over multiple months into an unrecoverable delay
  3. There are a number of methods to analyze these types of delays and identify negative trending
  4. Metrics collected to help identify trends toward negative performance should be part of a standard schedule update analysis.

Detailed Discussion Points

A delay during one period is usually not earthshattering, but a delay that is caused by the same situation in each of a number of subsequent update periods will accumulate to a large, potentially unrecoverable delay.  These delays often are unrelated to the critical path, and may have significant float, so they tend to be overlooked during routine analysis.  These are the delays that tend to stack up and overwhelm the construction management team with concurrent workload.

Identifying a delay in one period is useful, but it is more valuable to analyze to determine if that delay will likely occur again and again, until the accumulation is much greater than the original delay.  A four-day delay in June may not affect the schedule, but if the condition causing the delay continues into November, the four-day delay can easily become a 24-day delay, which is much more difficult to mitigate.

Trending analysis is even more important in programs than in projects because often a delay in one project on a large program can drive a delay into another project or contract.  Trending analysis can be performed using a variety of tools and techniques:

Earned value management (EVM) is useful, particularly looking at the time metrics of schedule performance index (SPI) and schedule variance (SV) over subsequent updates and earned schedule. The importance of this increases if the project is running late and approaching the 2/3rd point of completion when typical EVM results start to become misleading.  These metrics are lagging indicators (providing historical data only) but trending of the changes in slope of SPI is significant in predictions of completion.

In addition to the historical indicators, it is valuable to run the predictive EVM calculations like to-complete performance index (TCPI) using the predicted budget at completion (BAC).  TCPI is calculated as (BAC – EV)/(BAC – AC), using earned value (EV) and actual cost (AC), recognizing that if TCPI is less than 1, there is improvement required to meet the goal.   TCPI can help determine the efficiency needed to bring the project back to an on-time completion.

Another useful metric that is often overlooked is the estimate at completion (EAC) which should be analyzed by one of two calculations:

  1. Total budgeted cost/cost performance index (CPI) is a somewhat optimistic prediction of total project cost at completion, useful when project is still predicting on-time completion. 
  2. Total budgeted cost/ (CPI * SPI) is a bit more pessimistic prediction of total project cost at completion because it takes into consideration schedule performance, so this is the better metric to monitor if the project is starting to predict late completion.

Actual duration overruns, commonly called “Tipper” or time performance ratio (TPR), from the name of the original software, are very useful metrics to monitor.  These values should ideally be 1.0 or less for activities on the critical or near-critical paths, and somewhere between 1.0 and 2.5 for all the activities.  In addition to the value for monitoring weak performance, Tipper is also useful for identifying opportunities for improvement when certain trade contractors are running very good historical Tipper values that could be used to reduce future durations.  As noted in the original Tipper manual, “Running a TPR chart on activities that were on the critical or near critical path can indicate manipulation of the schedule if the TPR is greater than 1.0, but the end date of the schedule never slips.”

Implementing Tipper for analysis requires using the historical data and modeling future durations at the same ratio, so if the TPR ratio for concrete work is 1.8, increasing all future concrete work original durations to that ratio will show the impact if that productivity continues.  The authors have found this to be a very useful analysis technique.

A common metric collected, missed starts and finishes, is valuable because it indicates when work is not started on the early dates, which allows slippage towards the critical path.  The most important impact of this metric is when the slippage forces resources, especially labor, to slip toward the end of the project, causing stacking of trade work.  This elevates the risk of overallocation of resources, perhaps beyond the capability of the local market, and forces more dissimilar trades to work concurrently in restricted spaces.  This problem can have a major impact on cost overruns, quality, and schedule overruns, and is most at risk in areas like equipment rooms and commercial kitchens.

Erosion of float has been a traditional method of monitoring this work, however, it relies on good critical path method (CPM) schedules that are not missing significant logic.  With good schedules, activities enter the float path on the high end, see their float reduced and eventually show up on the near-critical and finally critical paths.   Graphing and monitoring this can help identify patterns in where the trade work is focused and help avoid ignoring work in some trades or areas of the project.  For example, in the float dissipation chart in Figure 9, it is obvious where the program focus was on one facility and float evaporated in the paths to the other facilities’ completions.

Figure 9 – Trending Tool – Float Dissipation or Erosion of Float Graph

For programs, erosion of interdependency buffers is a significant and important metric to monitor.  This is critical in multi-prime programs or projects since any delay to one prime is likely to show up delaying work in another prime’s scope.  The driving relationships between contracts or primes must be identified and monitored to avoid two claims in the event of an owner-responsible delay.

Important Advice

Non-critical path delays, or delays to all the work activities unrelated to float values, cause work to slip later and later in the project, and eventually the work requires too much effort, resources, and space to complete within the remaining time.  Just as a 10-mile roadway that needs to be completed in 10 months must install work at some rate commensurate with getting done on time, maybe one mile per month, the non-critical work must be put away in a similar rate.  Trending analysis is vital in monitoring this aspect that is commonly a cause of late project delivery.

APPROACH NO. 2 – “Perform a Formal Critical Path Analysis on a Project”

Both authors have been involved in claims where a contractor did not perform a monthly critical path analysis and when their schedule was filtered by the “longest path” the only activity that appeared was the final completion milestone. Critical path analysis is the fundamental element of CPM scheduling. The chances of achieving a project’s planned end date without closely monitoring the critical path is slim at best.

 A study performed in 2015 showed that only a quarter of all the projects surveyed came within 10 percent of their original completion deadlines. It is probably not a stretch to assume that the number of projects that completed on time is a much lower number.

Important Discussion Points

  1. Longest Path
  2. Near Critical Activities
  3. Forward Looking

Detailed Discussion Points

Longest Path – Perform a critical path analysis (longest path) sort every time the schedule is updated and make sure the CPM network is functioning properly (a continuous string of activities, proper float values and so on). If the schedule end date or an important milestone is falling behind schedule, have an impromptu meeting with management at the end of the day to discuss the path forward. Avoid making logic changes or reducing future durations to hold the end date, but instead, put a formal recovery plan in place to recover slippage after the team knows what is impacting the schedule. Have a formal critical path discussion at least monthly in the weekly meeting.

Near Critical Activities – Also examine the near critical activities to avoid these pieces of work taking over the critical path.

Forward Looking – Do not just examine the current period activities during the analysis; look at all of the activities in the path and get commitments from the leaders that those activities are on track to finish as planned (see Figures 10 and 11).

Figure 10 – Commonly Used Approach To Critical Path Analysis

Figure 10 shows how most people analyze critical paths – looking only a few activities in the near future.

Figure 11 – A Better Approach to Critical Path Analysis

Figure 11 shows how critical paths should be analyzed – looking at all of the activities in the future.

Important Advice

This process is a lot of hard work, but there are no shortcuts in making this effort successful– this is the fundamental basis of CPM analysis and done correctly is one of the best validations of a schedule’s completion date.

APPROACH NO. 1 – “Maintain a Continuous No-Cost Performance ‘Acceleration’ Effort”

High value schedule review services include facilitating a collaborative culture and working with the contractor in an transparent schedule effort working as a team.  One of the most valuable services that can be offered to facilitate the team approach is to not only monitor performance failures, but also to look for opportunities to shorten the schedule.  This requires a continuous effort.

Important Discussion Points

  1. Collaboration with full CM and contractor team
  2. The recovery or mitigation effort
  3. Changing focus from one-time mitigation to no-cost acceleration opportunities in each update

Detailed Discussion Points

Projects always go better when they are run collaboratively, with the owner, contractors, designers, and construction manager all working together to plan, manage, and resolve problems.  This is one of the reasons why integrated project delivery has increased in popularity; it requires a contractual commitment to partnering or collaborating.   Moving the culture away from a confrontational approach to just protecting the opposing parties’ interests and toward a collaborative effort improves project successes.  

Recovery scheduling is a process, and the process can be used for both recovering lost time and for gaining time by a no-cost “acceleration” effort.  As noted in the relevant Recommended Practice, “While often recovery plan efforts are developed outside the project schedule, the recovery plan should be shown in the schedule, with clear and well thought steps for recovery.”  Acceleration often requires additional resources and cost, but the goal is to look at true acceleration as a last resort.  This process starts with identifying fast-track opportunities; those critical path activities that could be performed in parallel to save time without additional impacts to other stakeholders.  Then near-critical activities can be examined in the same manner.  

A process that works well for recognizing fast-track opportunities is to sort the schedule by the original durations, from highest to lowest, which provides the best opportunities in a prioritized list format.  The larger the duration, the more likely that there are successors that would not have to wait until that activity is complete before they can start.  Sometimes this is already demonstrated with start-to-start relationships and lags, but these large duration activities are the best place to start.  What the sort looks like is shown in Figure 12.

Figure 12 – Schedule Sorted by Original Duration

This is pretty straightforward, and working down this list, the Fabricate Production Piles in three activities seems to be the first opportunity.  These activities are scheduled for 25 workdays for each group of rows, overlapped with total fabrication taking 6 weeks.  These activities in the schedule, sorted now by early dates, looks like what is shown in Figure 13.

Figure 13 – Fast-Track Opportunities Sorted by Early Dates

The next step is to review the predecessors and successors to look for opportunities.  In this case, the fabrication could not be expedited, however the Drive Piles activity could be started earlier, saving time on the critical path.  The opportunity can be modeled in a what-if scenario, using “Reflection” views in Primavera P6 schedules, and if it shows gains, this could be a suggestion.

This effort can be performed quickly each month, and opportunities identified and modeled to ensure that there appear to be gains, and if so, they can be communicated to the contractor.  These are suggestions only, not intended to direct the contractor or to dictate means and methods but intended as potential ways for the contractor to gain time.  

Other analysis suggestions should include issues like out-of-sequence (OOS) work, identified by modeling corrections to the OOS activities, if the model shows gains.  A quick check, if using Primavera P6, as to whether correcting the OOS work would result in an opportunity to gain time is to change the P6 setting from “Retained Logic”, the preferred setting, to “Progress Override”.  “Retained Logic” provides a slightly conservative (and safe) approach making sure that no successor can complete before one of its predecessors completes, while “Progress Override” provides a more optimistic view by ignoring the logical relationships between the activities that are OOS.  If the re-calculated schedule under “Progress Override” shows significant gains to contractual milestones, that indicates that the OOS work is critical or near-critical and making the corrections will yield reasonable benefits without additional risks.  This is a temporary setting change only to serve as an indicator, and industry best practices do not suggest operating under “Progress Override.”

Figure 14 – Example of Out-of-Sequence Correction Suggestions

Another analysis that is very useful is to take the trending metrics such as better performance in original durations using the time performance ratio mentioned earlier and modeling savings from the trade historical data, such as the F/R/P (form/reinforce/pour) activities shown in the graphic in Figure 15, which completed with a TPR ratio of 0.50.  If the contractor performing this concrete work recognizes that they are performing their work at half the original durations, they will often agree to reduce future durations by 50%, saving what appears to be 12 calendar days to the project.  The benefit of this reduction is that all other work on the project will be rescheduled to allow this trade to complete early.

Figure 15 – Gains Suggested by Modeling Productivity from Historical TPR Reports

Important Advice

There is a huge opportunity in all projects to analyze for trending and fast-track gains by modeling the ideas, and then communicating the potential gains to the contractor.  Once the contractor buys in to reviewing suggestions to improve productivity, they can take advantage of the strong technical schedule review skills of the reviewer.  Teaming in this manner will provide opportunities for the contractor to regain lost time and reduce the risk of paying liquidated damages due to performance delays.

Conclusion

Completing projects on time is important to all the stakeholders on the project and there is no single approach or step that can be taken that will ensure on-time projects.  However, there are certainly lessons learned to be acquired from successful projects, as well as failed projects.  The authors have experienced a high level of success by implementing the approaches discussed in these ten suggestions.

For a project to succeed in achieving its planned completion date, these are the successful approaches most recommended by the authors:

 

  1. Manage the project with an approved project schedule.
  2. Update the as-planned and the as-built separately and no less than monthly.
  3. Manage resources by using off-site prefabrication (modularization) on large projects.
  4. Develop and maintain a schedule-oriented culture.
  5. Manage project delay claims timely to ensure best efforts to maintain the schedule.
  6. Provide and manage a resource plan to ensure availability of project resources.
  7. Assign a specific team member to be in change of each project schedule activity.
  8. Preform a formal trending analysis to recommend mitigation actions.
  9. Perform a formal critical path analysis on a project.
  10. Maintain a continuous no-cost performance “acceleration” effort.

REFERENCES

[1] AACE International, Recommended Practice 29R-03, “Forensic Schedule Analysis“, Morgantown, WV: AACE International, Latest revision. 

[2] AACE International, Recommended Practice 93R-17, “Schedule Logs“, Morgantown, WV: AACE International, Latest revision. 

[3] AACE International, “Recommended Practice 53R-06, “Schedule Update Review – As     Applied in Engineering, Procurement, and Construction,” AACE International, Morgantown, Latest revision.

[4] AACE International, “Recommended Practice 78R-13, “Original Baseline Schedule Review – Applied in Engineering, Procurement, and Construction“, AACE International, Morgantown, Latest revision.

[5] AACE International, “Recommended Practice 49R-06, “Identifying the Critical Path“,  AACE International, Morgantown, Latest revision.

ABOUT THE AUTHORS

Glen R. Palmer, CFCC PSP FAACE

Christopher W. Carson, CEP DRMP PSP FAACE

]]>
https://source.aacei.org/2021/01/31/bonus-technical-article/feed/ 3