A Strategic Approach to Program Metrics

CDM Smith program man­age­ment leaders, Tom Lutzen­berger (mod­er­a­tor), Gerry Benson and Brian McCarthy, held a webinar, “Com­plex­ity Under Control: Defining and Achiev­ing Program Ex­pec­ta­tions", which focused on the key com­po­nents and benefits to using a program man­age­ment plan to ef­fi­ciently manage major capital projects. Time ran out before they could answer the last question re­gard­ing key metrics that should be in­te­grated into a program plan. Here’s their response.

“When talking about key metrics, what strate­gies are typ­i­cally used for mon­i­tor­ing purposes?”

The purpose of es­tab­lish­ing metrics is to create a series of in­di­ca­tors that can be used to quickly assess program health through focused at­ten­tion on those pro­gram­matic ac­tiv­i­ties and tasks that are of interest or concern to owners.​ With most programs com­pris­ing numerous projects and tasks, project-related metrics are also gathered for in­di­vid­ual design and con­struc­tion con­tracts. Where the progress of an in­di­vid­ual project or task impacts the overall program or suggests a trend that needs to be mon­i­tored for all projects and/or tasks, the metric becomes pro­gram­matic.

A strate­gic approach to iden­ti­fy­ing these pro­gram­matic metrics involves the fol­low­ing steps:

1. Start with de­tail­ing the metrics for ac­tiv­i­ties on the critical path. These should include in­di­vid­ual project and task metrics that will best measure critical elements of per­for­mance and con­sis­tent progress with program ex­pec­ta­tions. For example, pro­cure­ment, per­mit­ting and right-of-way ac­tiv­i­ties or even owner-managed ac­tiv­i­ties such as O&M and warranty services are all critical elements that should be con­stantly checked. Metrics for these phases often address process duration com­pli­ance and con­flicts with com­pet­ing phase work.

2. Identify a con­sis­tent way to present and pri­or­i­tize critical metrics. With so many ac­tiv­i­ties hap­pen­ing si­mul­ta­ne­ously, all stake­hold­ers should be able to analyze each metric in the same way. A program man­age­ment in­for­ma­tion system or PMIS can be used to gather and maintain this in­for­ma­tion and provides owners an easy-to-follow visual for tracking their program's progress. With the PMIS, all stake­hold­ers involved in a program can clearly read metrics to assess the health of a task or project. Using methods such as a ‘stop-and-go’ concept, the program health of a task or a project is dis­played as green-good, yellow-changing and red-poor. This clear display of per­for­mance will help stake­hold­ers make more cohesive de­ci­sions as they can set thresh­olds based on the same un­der­stand­ing of the metric.

3. Dif­fer­en­ti­ate which thresh­olds deserve the most at­ten­tion. Mea­sure­ments (numbers, per­cent­ages, and du­ra­tions) are not nec­es­sar­ily in­di­ca­tors of issues worthy of mon­i­tor­ing. Thresh­olds should be iden­ti­fied and indicate when mon­i­tor­ing should proceed and when action should be ini­ti­ated. The thought process for and dis­cus­sion on these thresh­olds should be doc­u­mented and used in the staff training for any pro­ce­dure uti­liz­ing that metric. These thresh­olds should be pri­or­i­tized on a program to fa­cil­i­tate as­sign­ing the most staff time to tracking the most critical, metric-defined sit­u­a­tions.

4. Define which approach will be used to attain metric in­for­ma­tion. Based on the priority es­tab­lished, the approach to mon­i­tor­ing can be ac­com­plished in three ways. 100 Percent Sur­veil­lance is most ap­pro­pri­ate method for in­fre­quent tasks or tasks with strin­gent per­for­mance re­quire­ments. Random Sampling is used mostly for re­cur­ring tasks. Periodic In­spec­tion works best for tasks that occur in­fre­quently.

5. Figure out what to look for re­gard­ing vari­ances. In­ves­ti­gat­ing the causes for vari­a­tions from baseline ex­pec­ta­tions can become com­pli­cated and should be managed closely. Keep an eye out for these three factors: fluc­tu­a­tions from baseline program ex­pec­ta­tions for schedule, cost, scope and quality program ac­tiv­i­ties. Program trends, whether upward or downward, can also add time as a variable to the analysis of a change on the program’s progress. The use­ful­ness and accuracy of results should also be assessed to avoid acting on negative vari­ances or trends that aren’t grounded in a cost-benefit cal­cu­la­tion. Certain vari­ances can be con­sid­ered ac­cept­able without mit­i­ga­tion as long as the backup jus­ti­fi­ca­tion supports this option in a manner ac­cept­able to the client.

6. Use per­for­mance re­port­ing to further goals for align­ment, stream­lin­ing and rein­ven­tion. Lessons learned will help improve future program per­for­mance and mon­i­tor­ing. As metrics are mon­i­tored and analyzed through the im­ple­men­ta­tion of the program, the needs for all involved stake­hold­ers will be better defined and man­age­able. It’s also im­por­tant to remember doc­u­ment­ing in detail each action taken with each change to the original plan. Re­port­ing these actions only helps to show the pub­li­ciz­ing of good man­age­ment per­for­mance as well as accurate progress.