Memorandum
Date: December 5, 2013
To: Boston Region Metropolitan Planning Organization (MPO)
From: Anne McGahan, MPO Staff
Re: Next Steps in the Development of Performance Measures
This is a follow-up to the memorandum dated March 7, 2013, to the Boston Region MPO from Anne McGahan, titled Development of Performance Measures and Performance-Based Planning. That memorandum provided background and a detailed explanation of performance-based planning; presented an overview of the Boston Region MPO’s work to date in implementing this approach; and proposed next steps for developing and applying performance measures for the region.
This memorandum looks at the next steps in the development of MPO performance measures as they relate to the Boston Region MPO’s own visions and policies. It also provides detail about specific national requirements in developing performance measures, and updates on MassDOT’s work in developing its own performance measures. As a reference, a glossary of performance-based planning terms is included in Attachment A.
Moving Ahead for Progress in the 21st Century (MAP-21) took effect on October 1, 2012, and contains requirements for performance measures when making performance-based investment decisions that support national goals. Performance-based planning and programming will increase the accountability and transparency of the federal-aid highway and transit programs, and improve project decision making.
MAP-21 also requires state departments of transportation (DOTs) and MPOs to develop performance targets for the established performance measures that spur progress towards attainment of critical outcomes for the region. In order to ensure consistency, the performance targets should be coordinated with those of relevant state agencies and public transportation providers. The MPO is required to establish its performance targets no later than 180 days after the date on which the state agencies and public transportation providers establish their targets.
The congressional conference report that accompanied MAP-21 stated that the nation’s surface transportation programs had not provided sufficient accountability for how tax dollars were being spent on transportation projects, and would benefit from an increased focus on key national priorities and goals. The areas of key national goals for performance management are listed below. Also listed is the current MPO vision(s) that correspond with the national goal.
MAP-21 identifies specific performance measures that should be established by state transportation departments and MPOs to make significant progress toward achieving the national goals. The measures should be established under the following programs:
The Federal Highway Administration (FHWA) has not yet issued specific guidance on developing performance measures. The Federal Transit Administration (FTA) has amended Chapter 53 of Title 49, United States Code to include MAP-21 requirements. The specific sections that relate to performance measure development include Section 5326 – Transit Asset Management (requiring the development of an asset management plan and performance measures for a “state of good repair”), and Section 5329 – Public Transportation Safety Program (requiring the development of a public transportation agency safety plan).
The Boston Region MPO is coordinating its work on performance measures with the state agencies, as required by MAP-21. As part of this process, MPO staff met with staff from various divisions in MassDOT to discuss their progress on developing state performance measures to date. The following is a summary of that work.
The Massachusetts Department of Transportation (MassDOT) is in the process of finalizing its statewide strategic multi-modal plan: We Move Massachusetts (WMM). This initiative is a product of the transportation reform legislation and the You Move Massachusetts civic-engagement process. Using a scenario-based planning approach developed for the WMM process, MassDOT will prioritize transportation investments based on performance standards.
The WMM process considers performance targets for asset program components of the transportation system: bridges, pavement, roadways, buses and trains, railroad tracks and signals, sidewalks and bike paths. It will establish ways to measure performance of these critical transportation investments, now and in the future. This process will allow MassDOT and its customers to envision a variety of ways to dedicate current and new revenues to transportation.
The WMM statewide multi-modal plan will help establish and monitor performance-tracking activities; and the various divisions and departments at MassDOT will conduct these activities. The performance measures will be used to develop MassDOT’s five-year Capital Investment Plan for highways and be used as a guide for the MBTA’s long-range capital planning for transit. In addition, the State Smart Transportation Initiative (described below) will help to inform WMM in the development of performance measures that track greenhouse gases. The status of MassDOT’s current activities is described below.
The Office of Performance Management and Innovation (OPM&I) is chartered to expand performance management and innovative strategies across all MassDOT divisions and departments. Monthly quantitative and qualitative data on critical performance indicators allows leadership to evaluate the organization’s progress against specific goals and targets. Specific goals that affect MPO activities were included in the March 2013 memorandum cited on page one. Select information from OPM&I’s September 2013 report is summarized in the table below.
Table 1
MassDOT Goals, Targets, and Current Conditions
| Goals and Targets |
Current |
|---|---|
Prevent the number of structurally deficient bridges from exceeding 463 |
462 |
Maintain at least an 81.98 system-wide bridge health index |
82.36 |
Ensure that at least 65% of total pavement is in good or excellent condition |
64.1% |
Ensure that at least 80% of pavement on the Interstate system within MassDOT’s jurisdiction is in good or excellent condition |
81.8% |
MassDOT’s GreenDOT Implementation Plan was published in December 2012, and included 16 broad sustainability goals supported by tasks to be implemented through 2020. Specific tasks that affect MPO activities were included in the March 2013 memorandum. Currently, OTP is working with the State Smart Transportation Initiative1 to develop specific performance measures for tracking greenhouse gas emissions.
The Massachusetts Bay Transportation Authority (MBTA) is currently developing a Transit Asset Management Plan (AMP) to comply with MAP-21 requirements. The Plan is made up of three components: 1) the state-of-good-repair database; 2) the decision-support tool; and 3) maintenance-management systems for each of the MBTA’s departments.
- State-of-Good-Repair Database
The MBTA is enhancing its existing State-of-Good-Repair Database to help meet the new asset-management requirements under MAP-21. A capital asset inventory will include the entire range of transit assets from vehicles and their major subsystems to stations, track, bridges and structures, power, communications and signals, train control, and fare collection. The MBTA will assess the assets and derive a condition rating for each asset and the system as a whole.
- Decision-Support Tool
The MBTA will develop a decision-support tool to optimize allocating resources and prioritizing proposed capital investment projects in order to achieve the MBTA’s objectives and customer expectations in the best way possible. It will use information from the state-of-good-repair database to rate and prioritize projects for implementation.
- Maintenance-Management Systems
The MBTA is upgrading its maintenance-management systems. This initiative will coordinate information and processes from maintenance and operating departments.
The MBTA expects to complete the AMP, along with established annual performance targets and a monitoring system, in the current year. All of the targets associated with this plan will be state-of-good-repair measures; the plan will not include targets for how well the system operates. However, some information from these measures can be used in evaluating operations of the various components of the system (e.g., an older fleet of buses may result in more breakdowns, therefore affecting on-time performance).
As part of performance-based planning, the MPO’s planning process should be based on a vision for the future of the region as described in its long-range transportation plan (LRTP). As outlined in Section 4.2 below, staff is proposing to revisit the MPO’s current Visions and Policies to reshape and rename them as Goals and Objectives to be more consistent with MAP-21. Goals should address key desired outcomes stemming from the MPO’s vision. Supporting objectives (specific, measurable statements that support achievement of goals) should play a central role in shaping the MPO’s planning priorities.
It is from these goals and objectives that individual performance measures will be developed and used to compare alternative improvement strategies for the region. Targets for each performance measure will also be established to track results of investments over time. Once the MPO has established a set of goals and objectives, it can revisit and revise its central vision statement for the region. The current central vision statement appears in Attachment B.
The Boston Region MPO adopted its current Visions and Policies in 2010 as part of its LRTP, Paths to a Sustainable Region. Since then, the new federal re-authorization legislation, MAP-21, clearly directs MPOs to measure and report the performance of their transportation network based on MPO programming. The MPO’s planning framework should change to comply with these new requirements.
As the Boston Region MPO takes steps to produce the next generation of its LRTP consistent with the new federal regulations and guidance, it also should refine its visions and policies to create a foundation consistent with the new MPO responsibilities in performance-based planning. Because the essence of the federal legislation is performance, the staff suggests that the MPO’s visions and policies be distilled and finely focused on performance; and further, that they be referred to as Goals and Objectives.
For example, the current MPO vision for Safety and Security has four sub-statements, with eight associated policies for achieving this vision. Instead, staff is proposing that these sub-statements be simplified into one statement for each goal, which then could be correlated with quantifiable results—the statements then would become “goals,” with associated “objectives.” Not only would this simpler approach achieve more clearly articulated statements, it also would be better aligned with the new federal guidance.
Staff has identified several themes from Paths to a Sustainable Region’s Visions and Policies to pull into its forthcoming LRTP. These themes reflect the MPO’s current basic visions and form the logical underpinnings of both the proposed goals and objectives and the initial round of recommended performance measures. The proposed MPO themes are shown in Table 2, below; the column on the right indicates how they would align with the key national goal areas.
Table 2
Proposed MPO Themes and Related National Goals
Proposed Theme |
National Goal |
|---|---|
System Preservation |
Infrastructure, System Reliability |
Congestion Reduction |
Congestion Reduction |
Safety and Security |
Safety |
Transportation Options/ Mode Share |
Environmental Sustainability |
Greenhouse Gas Reduction |
Environmental Sustainability |
Economic Impact |
Freight Movement/ Economic Vitality |
The staff recommends that the MPO narrow its focus on these major goals as it establishes its performance-based planning practice. In the future, when the MPO has demonstrated success meeting its performance targets in these areas, the MPO may wish to widen its goals and related objectives to incorporate other topics. Once the MPO has established a set of goals and objectives, it can revisit and revise its central vision statement for the region. The current and proposed approaches are shown in Figure 1, below.
FIGURE 1
Adapting to Performance-Based Planning—Converting Visions and Policies to Goals and Objectives
Below is an example of how the current MPO vision and policies for Safety and Security might be modified for the next LRTP update; and Figure 2 illustrates this example.
FIGURE 2
Performance Measurement Overview:
Safety and Security Example
“The transportation network will provide safe transportation for all users and on all modes and can be depended upon to serve the region in the event of a natural hazard or human security crisis.”
The goal is intended to express the MPO’s values regarding this topic. This goal has extracted the basic and key elements for this topic from the MPO’s current vision statements. Other ideas in the current list of policies are clear outcomes of the updated policy and so, are covered with this simplified statement. Also, the MPO should be able to accommodate safety and security issues relevant to transportation planning in 2014 within this goal statement.
“Give priority to projects that have a major safety or security element and can be expected to produce a positive safety- or security-related outcome.”
This objective would result in a simpler, clearer direction for MPO planning and programming than the current more specific list of safety policies. Specifically, it would guide:
A performance measure is a metric used to assess the progress toward meeting a goal. Under the updated Safety and Security topic, the staff recommends focusing on the MPO’s goal that “the transportation system provides safe transportation for all users and on all modes;” and the objective to “give priority to projects that have a major safety or security element and can be expected to produce a positive safety- or security-related outcome.”
One way to measure improvement in safety in the transportation network is to understand the severity of crashes by measuring the Equivalent Property Damage Only (EPDO) index of crashes in the region. The EPDO index measures the amount of fatalities, injuries, and property damage that occur on roadways. A recommended performance measure related to Safety could then be:
“The Equivalent Property Damage Only (EPDO) index for auto, truck, bicyclist, and pedestrian crash clusters in the Region (LRTP) and the Project Area (TIP).”
A performance target is a specific level of performance that is desired to be achieved within a certain timeframe. To address the safety performance measure cited above, the recommended target for planning and programming funding for investment in the region might be:
“Decrease in the average EPDO index at the top crash locations for autos, trucks, pedestrians, and bicyclists; the locations included in the top crash locations will be prioritized for improvements.”
The MPO’s Needs Assessment identifies the top crash locations in the region based on crash severity using the weighted EPDO index. The EPDO index helps identify locations with the greatest safety needs in order to help prioritize investment decisions. Staff will continue to collect EPDO indices that include all modes (auto, truck, bicycle, and pedestrian) to identify the overall top crash locations. Staff also will collect EPDO indices by mode to identify the top crash locations for trucks, bicycles, and pedestrians.
For example, projects that address safety at a top crash location will be given higher priority in the selection process than projects that do not address safety at a top crash location. By prioritizing projects with the greatest safety needs, the MPO will be able to better achieve its goal of reducing crash severity in the region. Through the Needs Assessment, the MPO will be able to track the average EPDO index of the top crash locations to ensure that crashes are decreasing.
As an example, the EPDO indices reported in the most current Needs Assessment range from a value of 795 at the top crash location to 283 at the 25th crash location, with an EPDO average of 428. The MPO's safety target would then be an EPDO average below 428. The MPO will be successful in meeting the performance measure target if the average EPDO index at the top crash locations decreases from 428. (The MPO can agree to a specific level of reduction, if it chooses.)
The EPDO information in the Needs Assessment will be updated as part of the new Web-based Needs Assessment. In addition, the selection criteria will be updated to reflect the new performance measures as they are developed.
During the development of the TIP, staff also will use the EPDO index to identify projects with the greatest safety needs and inform the prioritization of investment decisions. Currently, projects are assessed for safety needs based on the EPDO index of the project area and whether design improvements would address the safety deficiencies for all modes in the project area. Staff will expand its evaluation to collect EPDO indices by mode to identify the top crash locations for trucks, bicycles, and pedestrians. This approach will be consistent with the LRTP.
Staff proposes two methods of tracking progress toward the MPO’s safety goal and performance-measure target.
Method One
The first is to track the average EPDO index for projects that are evaluated in the TIP process for the discretionary program against all projects that are funded. Using information from project selection between FFY 2012 through 2017 of the TIP, Figure 3 reports the average EPDO from the evaluated projects in the discretionary program versus EPDO from those funded in the TIP. This information shows that the MPO-funded projects had a 37% higher EPDO index than evaluated projects. These results indicate that the MPO selection criteria prioritize projects with greater safety needs, which can move the program toward the proposed performance target.
Applying this method in the development of the FFYs 2015-18 TIP will continue to give the MPO information about whether its investment decisions progress toward the proposed goal and target.
FIGURE 3
Comparison of Average EPDO Indices of Evaluated Projects versus Funded Projects between FFYs 2012 and 2017
Method Two
The second method for analyzing and reporting progress involves additional levels of ongoing tracking and assessment, and will be undertaken in the future as the MPO’s performance-based planning practice grows. Some of this tracking and assessment would be done through data currently used in TIP evaluations, such as the EPDO index. It also could include other not-yet-identified data that could be brought into the TIP evaluations.
For the FFYs 2015-18 TIP, a next step for performance measurement could be to calculate and report the average EPDO indices for the top crash locations, compared to the previous year, to see if the MPO had made progress. The data could be analyzed further by relating the locations of TIP projects in the whole program to the top crash locations. This information could be provided in the draft and final TIP documents.
In developing the UPWP, the MPO would consider funding planning studies and work that address safety and security concerns.
Staff would update the qualitative criteria used to support selection of UPWP studies to conform to the MPO’s safety and security goal and objectives. For a given federal fiscal year, a UPWP may include planning studies that generate recommendations for specific capital projects that would improve safety. Examples in the adopted FFY 2014 UPWP include:
For studies like these, MPO staff will continue to consider EPDO indices or identified top crash locations as part of evaluating locations in the region as candidates for detailed analysis.
The MPO would also continue to advance safety elements, where appropriate, in other studies and activities that are funded in the UPWP. These include the MPO’s ongoing certification requirements, technical assistance programs, and studies that address safety and security issues in a broader context.
The UPWP will support the MPO’s activities to track, assess, and report on the MPO’s progress with respect to its goals, objectives, and measures. This could include before-and-after studies to determine whether MPO-funded capital projects or other actions are helping the MPO meet its performance targets, such as the EPDO index target. If the MPO is not meeting the EPDO or other targets, funding could be programmed to identify and propose further actions that could be taken to meet these targets.
The safety performance measure recommended above would correspond to the national guidance for safety “to achieve reduction in fatalities and serious injuries on all public roads” and the MAP-21 safety performance measures under the Highway Safety Improvement Program to develop measures to assess:
The recommended measure also corresponds to the current MassDOT Performance Measure and Target to “reduce the total number of crashes and traffic fatalities from the previous year.”
The MPO currently funds projects and programs under a variety of investment programs in its LRTP and TIP. These include:
When considering future allocations of the MPO’s target funding, the MPO can prioritize spending in the investment programs that best meet the MPO’s needs for improvement in achieving its goals and objectives. Priorities can be tested as part of the scenario planning that is included in the work program for the development of the next LRTP. This can help to determine if the different investments are moving the region towards its goals. Some performance measures can be used to track the results.
Some performance measures can be measured with the travel demand model through the LRTP scenario-planning work. Some, like the recommended safety performance measure (EPDO) will be measured through other analytical means. All progress will be reported and tracked in the TIP and LRTP updates.
This memorandum laid out the process for developing performance measures and provided an example using the “Safety and Security” theme. Staff will incorporate MPO comments and guidance on this process in an updated memorandum. Once the MPO has agreed to the process, staff will continue to develop the goals, objectives, performance measures, and targets for the remaining themes:
Staff will plan to present the recommendations for goals, objectives, performance measures, and targets for the remaining themes to the MPO in February 2014.
Goal: A broad statement that describes a desired end state.
Objective: A specific, measurable statement that supports achievement of a goal.
Performance management: A strategic approach that uses transportation system information to make investment and policy decisions to achieve performance goals.
Performance-based planning and programming: Practices that apply performance management principles to transportation system policy and investment decisions, including long-range ones; a system-level, data-driven process for identifying strategies and investments.
Performance measure:A metric used to assess the progress toward meeting a goal. Examples include:
Performance target: A specific level of performance that is desired to be achieved within a certain timeframe. Examples of targets for the example performance measures cited above are:
“The Boston region will continue to be a major economic, educational, and cultural hub of New England. It will maintain its high quality of life due to its lively commercial and business enterprises, the strength of its institutions, and its healthy and pleasant environment, all supported by its well-maintained transportation system. Notably, there will be an ongoing transformation taking place in the region’s communities. They will, more and more, be places in which people can have access to safe, healthy, efficient, and varied transportation options and find jobs and services within easy reach of affordable housing. The transportation options will include the transit, bicycle, and pedestrian modes, among others, and will reduce environmental impacts, improving air and environmental quality. The role of the region’s transportation system in making the envisioned future possible will be a result of attentive maintenance, cost-effective management, and strategic investments in the system by the Boston Region MPO.”
1 The State Smart Transportation Initiative (SSTI) is part of the Center on Wisconsin Strategy at the University of Wisconsin. MassDOT’s GreenDOT performance measure work with SSTI is being funded through a grant from the Barr Foundation.