1. EARNING VALUE FROM EARNED VALUE USING EV TO IMPROVE THE PROBABILITY OF PROJECT SUCCESS EVM WORLD 2009 PMI-CPM 25TH ANNUAL INTERNATIONAL CONFERENCE 1
2. Learning Outcomes Cost and Schedule Indices (SPI, CPI, TCPI, and IEAC) do not represent the underlying statistical nature of projects. Modeling this statistical behavior requires addition activities build around Monte Carlo simulation and stochastic modeling. Creating value from the Earned Value requires both bottom-up and a top-down work processes. 2
3. 3 Creating Value from EV is More Than Calculating SPI and CPI LAUNCH ABORT SYSTEM FOR ORION CAPSULE What are the technical requirements that fulfill the mission capabilities? How is the work partitioned for the producing the deliverables that fulfill these requirements? What are the Work Packages, Cost and Schedule elements and programmatic risk elements to assure on-time, on- budget performance? CONSTELLATION MISSION TO THE MOON What capabilities are needed to fulfill the mission? How are these capabilities connected to the technical and operational requirements? What are the programmatic trade space decisions between capabilities, requirements, cost and schedule? BRICK LAYING PROJECT What are the measures of scope, effort, and duration? What are the measures of physical percent complete? Can we forecast of final cost and completion date? What Does Done Look Like? How will we recognize done? What is the unit of measure of Done?
4. Questions Every Program Manager Needs to Ask and Answer How much money will I need to complete this program on time and on specification? What is my confidence that well complete on or before the planned finish date? Do I have enough management reserve to protect this finish date? What is the inherent risk built into my cost and schedule baseline? How can I tell if the program is making progress to plan? 4
5. GAINING COMPLIANCE WITH THE EARNED VALUE GUIDANCE THE BOTTOM UP APPROACH 5
6. The Logical Starting Point Of The Earned Value Journey 6
7. This progress is represented by physical, tangible, measurably quantifiable evidence. How much did we plan to get done on this date, measured in Dollars? How much did we actually get done on this date, measured in Dollars? In This Standard Paradigm Earned Value = Physical Percent Complete of the Planned Value 7
8. What About the Planned Technical Performance? Were On Schedule, On Budget But We Had A Slight Technical Problem With Our Flying Machine 8
9. 9 From the NDIA Earned Value Intent Guide (EVIG) Notice the inclusion of Technical along with Cost and Schedule Thats the next step is generating Value from Earned Value EV MUST include the Technical Performance Measures
10. 10 Relate time-phased budgets to specific contract tasks and/or statements of work (SOW) Objectively measure work progress Properly relate cost, schedule, and technical accomplishment Allow for informed decision making and corrective action Is valid, timely, and able to be audited Allow for statistical estimation of future costs Supply managers at all levels with status information at the appropriate level, and Is derived from the same EVM system used by the contractor to manage the contract. The First Set of Promises That Need Actionable Steps
11. The EV numbers are just numbers They can show trends They can be the basis of forecasts But they must be placed in a context of an underlying statistical model They must be connected with the relationship between time and money 11 Moving from reporting the past performance to forecasting the future is the first step in Earning Value from Earned Value Management
12. Does Money Equal Time? This is How EV Measures Schedule Variance But What If It Is Not Linear? Time Is Not Money Unless They Are Linearly Related 12
13. The REAL Problem Models Of Earned Value Are Inherently Probabilistic With Interdependent Couplings Between Network Elements 13
14. Using Static, Deterministic, and Decoupled Measures of Performance (e.g. The Standard EV Model) is at best 14
15. But Lets Visit A High Level View of the Earned Value Activities 15
16. The 32 Criteria for a ANSI/EIA-748B Compliant Earned Value Management System 16
17. MANAGEMENT PROCESSES EXTRACTED FROM 748B 17
18. Questions Every Program Manager Needs to Ask and Answer Part 2 How much money will I need to complete this program on time? What is my confidence that well complete on or before the planned finish date? Do I have enough management reserve? What is the inherent risk built into my cost and schedule baseline? How can I tell if the program is making progress to plan?
19. CAN WE MEASURE THE BENEFITS OF EARNED VALUE MANAGEMENT?
20. How Can We Get A Return On Our Investment In Earned Value? 20
21. Some Straight Forward Benefits Better Visibility into Program Performance Reduce Cycle Time to Deliver a Product Foster Accountability Reduce Risk From the DACS Gold Practices Track Earned Value Focus Area
22. The Missing Piece Is How To Obtain These Benefits How do we create better visibility into the programs performance? How do we reduce cycle time to deliver the product? How do we foster accountability? How do we reduce risk? Deploying Earned Value Alone Will Not Get Us There How Do We Get From Where We Are To Our Destination? 22
23. Claimed Benefit The Reason The Method to Achieve Better Visibility into Program Performance The combination of advance planning, baseline maintenance, and earned value analysis yields earlier and better visibility into program performance than is provided by non-integrated methods of planning and control. This requires a credible baseline. But this baseline must be more than just a list of work, it must represent the Programmatic Architecture of the system, matching the technical architecture. This programmatic architecture formulates the flow of work that produces products with increasing maturity. Step Action Details Define the Integrated Master Plan (IMP) The IMP is the Strategy for the successful completion of the program. It is formed from Program Events, the Significant Accomplishments (SA), and their Accomplishment Criteria (AC). Sequence the SAs The sequence of SAs for each single PE by the IPTs is a diagram sufficiently detailed to show how the increasing maturity of the deliverables can be achieved. Assign Budget Estimates (BOE) Collect the initial basis of estimate from the duration and sequence of these SAs 23
24. Claimed Benefit The Reason The Method to Achieve Reduces Cycle Time to Deliver a Product Earned Value Management is premised on careful detailed planning task decomposition, scheduling, and budgeting. This planning often addresses or prevents problems from surfacing later in the effort that result in rework. As rework is prevented cycle time may be reduced. Again the carefully detailed plan must adequately represent a credible schedule, cost allocation, and resource assignments. How can we know this is the case? Step Action Details Reduce cycle time by forecasting delay Using TCPI and IEAC the macro level forecast (scalar) can be generated. What is needed is to locate the specific tasks that are the source of this delay. This is a network analysis process. Identify alternative paths through the IMS With the network process flow indentified, alternative paths can be assessed. These path have associated cost and down stream schedule impacts that need further analysis. Probabilistic cost and schedule The result is a probabilistic cost and schedule analysis coupled with a Monte Carlo simulation tool to construct the probabilistic critical path? 24
25. Claimed Benefit The Reason The Method to Achieve Fosters Accountability When the engineer understands how the pieces fit the overall project effort, they tend to focus on delivery of a quality product. Over time they are better able to estimate the work required to complete a task, thereby improving the overall accuracy of the budget / estimating process for future efforts. Defining the logical sequence of product or service flow is the starting point. This systems engineering approach starts with a description of the increasing maturity of the product or service, and only then assigning the Earned Value elements Step Action Details Work Package budgets owned by producers of value Once the Value Stream as been defined this is the logical flow of the Significant Accomplishment the ownership for producing this value is identified in the Work Package owners. Dependencies between WPs is a Systems Engineering process The Value Stream of the collection of Work Package must remove an impediments to maximizing this value. This is the role of the planning and controls staff in conjunction with the subject matter experts accountable for delivering the technical solution. As a collective they are accountable for the resulting value stream. 25
26. Claimed Benefit The Reason The Method to Achieve Reduces Risk Because earned value measures enable realistic estimates of completion to be derived early in the project, it is possible to make adjustments and take corrective action to mitigate the risk of cost overruns and schedule slippage. Probabilistic risk analysis requires a credible baseline and an understanding of the underlying probabilistic process driving cost, schedule, and technical performance If the probabilistic process is stationary, a Monte Carlo simulation can be used to analyze the behaviors of the system only as a stationary model. Step Action Details Probabilistic modes expose risk Only probabilistic models enable realistic estimates. No point estimate in the absence of a variance is credible. Risk adjusted work in IMS All risk mitigation and retirement activities need to be in the IMS, along with funding profiles and probabilistic assessment of their effectiveness. EV alone will not address the issues. An active Risk Management process is needed. 26
27. STEPS NEEDED TO EARN THE VALUE FROM EARNED VALUE THE TOP DOWN APPROACH 27
28. 28 The Cost of These Benefits Is Already Built Into a Credible PMB The BCWS spreads define visibility into costs The sequence of Significant Accomplishments (SA) defines visibility into deliverables Risk mitigations, buy downs, and retirements are embedded in the PMB Small incremental measures of value are defined by Work Packages Visibility provided through measures of physical percent complete Reduced cycle is actually just maintaining the planned completion dates, on budget, that meet the planned technical performance measurements
29. Five Core Processes of Successful Program Planning and Controls 29 Program Success Process Areas Questions To Be Answered Identify Needed System Capabilities What capabilities are needed to fulfill the ConOps and System Requirements? Establish the Requirements Baseline What technical and operational requirements are needed to fulfill these capabilities? Establish the Performance Measurement Baseline What is the schedule that delivers product or services that meet the requirements? Execute the Performance Measurement Baseline What are the periodic measures of physical percent complete? Perform Continuous Risk Management What are the impediments to success and what are the mitigations ?
30. Partition system capabilities into classes of service within operational scenarios Connect capabilities to system requirements using sysML Define Measures of Effectiveness (MOE) and Measures of Performance (MOP) Define in the delivery schedule the achievement of each Technical Performance Measure Define Operational Concepts Define scenarios for each system capability Connect these scenarios to a Value Stream Map of the increasing maturity of the program Assess value flow through the map for each needed capability Identify capabilities mismatches and make corrections to improve overall value flow Define Capabilities Needed To Implement Concepts Assign costs to a system element using a value model process model Assure risk, probabilistic cost and benefit performance attributes are defined Use cost, schedule and technical performance probabilistic models to forecast potential risks to program performance Assess Needs, Costs and Risks Simultaneously Make tradeoffs that connect cost, schedule, and technical performance in a single trade space model Measures of Effectiveness and Measures of Performance are the raw materials for these tradeoffs Define Explicit, Balanced, and Feasible Alternatives Define the set of capabilities to be employed to achieve desired objectives or a particular end state for a specific scenario. Take the ConOps and define the details of who, where, and how it is to be accomplished, employed and executed. Identify Needed System Capabilities What capabilities are needed to fulfill the ConOps and System Requirements? 1.0 1.1 1.2 1.3 1.4 30
31. Produce an overall statement of the problem in an operational context. Develop the overall operational and technical objectives of the target system. Defined the boundaries and interfaces of the target system. Perform Fact Finding Gather required system capabilities, functional, nonfunctional and environmental requirements, and design constraints. Build a Top Down Capabilities and Functional decomposition of the requirements in a flow down tree using a Requirements Management System. Gather and Classify Requirements Answer the question why do I need this? in terms of operational benefits. Build a cost benefit / model using probabilistic assessment of all variables and dependencies. For technical requirements, perform a risk assessment to cost and schedule. Evaluate and Rationalize Requirements Determine criticality for the functions for the system mission. Determine trade off relationships for all requirements to be used when option decisions are made. For technical items prioritize on cost and dependency. Prioritize Requirements Address completeness of requirements by removing all TBD items. Validate the requirements agree and are traceable to system capabilities, goals, and mission. Resolve any requirements inconsistencies and conflicts. Integrate and Validate Requirements Define the technical and operational requirements that must be in place for the system capabilities to be fulfilled. Define these requirements in terms isolated from any implementation. Establish the Requirements Baseline What Technical and Operational Requirements are Needed to Fulfill the Capabilities? 2.0 2.1 2.2 2.3 2.4 2.5 31
32. Decompose the Project Scope into a product based Work Breakdown Structure (WBS), then further into Work Packages describing the production of all deliverables traceable to the requirements Decompose Scope into Work Packages Assign Responsibility to Work Packages (the groupings of deliverables) for the named owner accountable for the management of resource allocation and cost baseline and technical delivery Assign Responsibility for Deliverables Arrange the Work Packages in a well formed network with defined deliverables, milestones, internal and external dependencies, appropriate schedule and cost margin. Arrange Work Packages in Logical Order Develop a TimePhased Budgeted Cost for Work Scheduled (BCWS) from labor and material costs in each Work Package and the Project as a whole. Assure proper resource allocations can be met and budget profiles match expectations of the project sponsor Develop BCWS for Work Packages Assign object Measure of Performance (MOP) and Measures of Effectiveness (MOE) for each Work Package and summarize these for the Project as a whole Assign WP Measures of Performance Build a timephased network of schedule activities describing the work to be performed, the budgeted cost for this work, the organizational elements that produce the deliverables, and the performance measures showing this work is proceeding according to plan. Establish the Performance Measurement Baseline A Baselined Schedule that Creates the Services or Products to Meet The Requirements 3.0 3.1 3.2 3.3 3.4 3.5 Establish a Performance Measurement Baseline (PMB) used to forecast Work Package and Project ongoing and completion cost and schedule metrics Set Performance Measurement Baseline 3.6 32
33. Deliverables Based Planning Handbook for A&D, Copyright 2008, 2009, Lewis & Fowler Using the Work Package sequencing, release work to be performed as planned With the RACI based RAM, the Accountable delivery manager guides the development of the products or services for each Work Package Perform the Authorized Work Using Physical Percent Complete or Apportioned Milestones capture measures of progress to plan for each Work Package Report this Physical Percent Complete in a centralized database for each Work Package and the project as a whole Accumulate and Report Work Package Performance Compare the Physical Percent Complete against the Planned Percent Complete for each period of performance Construct cost and schedule performance indices from this information and the Physical Percent complete measures Analyze Work Package Performance With the Cost and Schedule performance indices, construct a forecast of future performance of cost, schedule, and technical performance compliance efforts Take management actions for any Work Packages not performing as planned Take Corrective Management Action Record past performance based on Work Package completion criteria Record past future forecast performance estimates in a historical database Forecast next future performance against the Performance Measurements Baseline Report this next future performance estimate to the project stakeholders Maintain the Performance Baseline Execute work packages, while assuring all performance assessment are 0%/100% complete before proceeding. No rework, no forward transfer of activities or features. Assure every requirement is traceable to work and all work is traceable to requirements. Execute the Performance Measurement Baseline Weekly, Bimonthly, or Monthly Measures of Physical Percent Complete 4.0 4.1 4.2 4.3 4.4 4.5 33
34. 34 5 Practice Areas of Successful Program Controls How Where When Who Why What Identify Business Needs Establish a Performance Measurement Baseline Execute the Performance Measurement Baseline Capabilities Based Plan Operational Needs Earned Value Performance 0% /100% Technical Performance Measures System Value Stream Technical Requirements Identify Requirements Baseline 1 2 3 4 Technical Performance Measures PMB Continuous Risk Management Process Changes to business strategy Changes to requirements Changes to project plan 1 2 3 4 5 6 7 8 9 10 10OrganizingPrinciplesofDeliverablesBasedPlanningsm
35. The 10 Organizing Principles That Drive The 5 Practice Areas 35 Capabilities Drive Requirements Requirements are Fulfilled by Work Packages Work Packages Define Deliverables Measure Physical Percent Complete of Each WP Perform Only Authorized Work Earned Value Defines Progress Adjust EV for Technical Performance Measure Use Past Performance to Forecast Future Performance 1 2 3 5 6 7 8 9 10 PMB Describes Work Sequence 4 9 Assess the capabilities being provided through the deliverables Fulfill the requirements through effort held in the Work Packages Produce deliverables from Work Packages Planned BCWS Physical % Complete WPs contain deliverables that fulfill requirements Capabilities topology defines requirements flow down WP flow must describe the increasing maturity of the product or service Producing the deliverables in the planned sequence maintains the value stream to the customer
36. Business Management Is the Purpose of 748B of the Criteria are NOT About Dollars The Units of Measure in Earned Value 36
37. Connecting Principles With Earned Value Processes 37 ANSI/EIA-748B Areas 10 Principles Driving Program Controls Success Organization Planning and Budgeting Accounting Analysis Revisions Capabilities Drive Requirements WBS Requirements Identify Deliverables WBS Work Packages Describe Production of Deliverables WBS CWBS IPTs BCWS Master Schedule Sequences Deliverables through WPs IMS IMS Progress Must Be Measured As Physical Percent Complete Perf Measure 0/100 or AP TPM Work Authorization Assures Proper Sequencing of Work Packages OBS WAD Late Starts Late Finishes CCB Earned Value Identifies Current Deliverables Performance Physical % Complete BCWP Monte Carlo Technical Performance Measures Adjust Earned Value TPM Measure TPM Monte Carlo Performance Feedback Adjusts Work Package Sequencing Upper Lower Bounds CAM Review CCB Future Performance Using TCPI, IEAC & Adjusted Work Sequence IEAC New BL
38. 38 What Did We Learned?
39. What Did We Learned? CPI/SPI are necessary but not sufficient for success program management. All EV variables have probabilistic foundations. The sunk cost of successful Earned Value is already absorbed in a credible PMB and the process used to build it 39
40. 40 The Light at the End of the Tunnel Start with the Top Down approach and define the needed capabilities to drive the requirements. Use these capabilities and requirements to establish the business value, and related Value Stream Map of the Master Plan. Partition the allocated work into Work Packages that deliver the business value in a credible sequence meaningful to the customer. Use the Bottom Up approach to construct the details. Consider all variables as random variables. Use the 32 Criteria as guidance for business processes, not just EV. Remember EVs Unit of Measure of Dollars not Time. Time Money on any real program.