Memorandum

Date       February 21, 2013

To           Boston Region MPO

From      Alicia Wilson, MPO Staff

Re            Job Access and Reverse Commute and New Freedom Federal Grant Program Projects in the Boston Region Metropolitan Planning Organization Area: An Evaluation

Introduction

Information in this memo refers to programs initiated under the previous federal surface transportation act. A new act, MAP-21, with several program changes went into effect on October 1, 2012.

Job Access and Reverse Commute (JARC; 49 USC Section 5316) and New Freedom (49 USC Section 5317) are federal formula grant programs whose eligible recipients are states and public bodies. The Massachusetts Department of Transportation (MassDOT) is the eligible recipient for the Boston Urbanized Area (UZA), which contains the Boston Region MPO and four other MPOs in Massachusetts.

JARC provides grants to support the development and maintenance of projects designed to transport welfare recipients and eligible low-income individuals to and from jobs and activities related to employment. New Freedom provides grants for new public transportation services and alternative forms of public transportation that assist individuals with disabilities and exceed the requirements of the Americans with Disabilities Act of 1990.

Title 49 U.S.C. Section 5316 and Section 5317, as amended by SAFETEA-LU, require fund recipients to certify that projects selected are derived from a locally developed, coordinated public transit–human services transportation plan. The MPO’s plan was developed in 2008 and updated in 2010. The plan provides guidance for improving transportation services in the Boston region for people with disabilities, elderly individuals, people with low incomes, and reverse commuters. The MPO initiated development of the plan by identifying existing resources, transit markets to target for service improvements, and gaps in transportation services.

Proposals submitted in the Boston UZA for funding under the JARC and New Freedom programs are evaluated by the five MPOs, which decide which applications to advance and prioritize them. The selected applications and the priorities are forwarded to MassDOT, which then uses a competitive selection process to determine which proposals will be funded.

There have been five proposal solicitations in the Boston UZA since 2008. This memorandum evaluates the performance of the projects that were funded in the first four solicitations. The first was held in 2008, the second in 2009, and two in 2010. This memorandum does not examine projects from the latest solicitation, which began on February 29, 2012, or use any data related to that solicitation. In 2008–10, nine agencies in the Boston Region MPO area received 16 JARC grants, and twelve received 22 New Freedom grants. Several of these agencies received both JARC and New Freedom grants. Some of the grants funded additional years of previously funded projects.

The funded projects and the organizations implementing them are diverse in nature. Funded agencies include regional transit authorities, state agencies, private non-profits, and private for-profits. Types of funded projects include studies on facilitating coordination of existing transportation resources; identifying resource gaps and developing strategies for closing them; enhancing consumers’ abilities to access and use transportation options; and planning for and operating paratransit services.

The primary purpose of this memorandum’s evaluation of the Boston Region MPO–area projects is to assess the proponents’ implementation of the projects and how successful the projects have been in reaching their stated goals. The evaluation also examines the question of whether some types of projects or proponents have been relatively more successful, because in that case the MPO might consider factoring in that information when selecting applications to advance.

Evaluation Methodology

Data Collection

A printed packet including a letter requesting information, project profile forms, and reporting matrices was sent by U.S. mail to all 16 distinct JARC and New Freedom grant recipients. (A copy of the initial packet is included in Appendix 1.) The same packet was also e-mailed to all recipients. A second request was mailed and e-mailed to each entity that did not respond by the date indicated in the original request. After both rounds of requests and follow-up phone calls, 13 entities had responded (81% response rate).

The project profile form requested information on the primary service/project goal, a service description, major accomplishments, how the service is evaluated by the entity, and lessons learned.

The Federal Transit Administration has sponsored several JARC and New Freedom program evaluations. In keeping with these evaluations, the reporting matrices sent to entities included in this analysis cover three basic project types:

• Trip-based services, which provide transportation directly to individuals
• Information-based services, which provide information about transportation services to individuals but do not provide transportation services directly
• Capital investment projects, which provide vehicles, facilities, and other infrastructure to support transportation services

Federal evaluation-reporting requirements cover five primary program goals. In addition to these, this evaluation includes a sixth goal applicable to planning studies, because of the number of such studies funded in the MPO area. The goals are:

• Expanded geographic coverage of service
• Extended hours or days of service
• Improved system capacity (additional quantities of service)
• Improved access to service or improved connections between services
• Improved customer knowledge (additional resources providing information and/or training)
• Provision of information on specific populations and on how to implement projects (These are goals of planning studies.)

Measures of Effectiveness

In order to measure the success of each project, this evaluation compared the stated and achieved goals of the project, rather than using set criteria. There are two primary reasons for this. First, the JARC and New Freedom programs do not have specifically defined and quantifiable goals such as “create a service that transfers 100 new riders to work sites.” Second, some projects that do aim at results that could be quantified, such as serving trips or making customer contacts, are designed to fill transportation gaps for special populations and therefore cannot be expected to produce results comparable to those of other projects.

In the following analysis, the performance indicator used for trip-based projects is the actual or estimated number of rides (as measured by one-way trips) provided as a result of the project. The performance indicators used for information-based and capital investment projects are, respectively, information gathered and technology and vehicles acquired that impact availability of transportation services. In all cases, the indicator is examined in terms of a project’s realized goals as compared with its projected goals.

In addition to measuring the success of individual projects, this evaluation examined and drew general, qualitative conclusions from the information provided by project proponents on “lessons learned.”

Recipient and Project Overview

As previously mentioned, 16 distinct entities were awarded JARC and New Freedom grants in solicitations occurring between 2008 and 2010. Diverse agencies have received funding. As Table 1 shows, private non-profits are the most common type of agency funded, followed by regional transit authorities (RTAs) and state agencies.

Table 2 lists the entities along with the proportion of total funds awarded to each one. Note that 48% of the funds awarded is JARC funds; 52% is New Freedom. RTAs received nearly three-quarters (71%) of the JARC funds. RTAs also received almost one-third (31%) of the New Freedom funds. In total, RTAs received half (51%) of all funds awarded in the Boston Region MPO. A total of 37 grants were awarded in the solicitations held between 2008 and 2010. Several grants funded additional years of previously funded projects. Some agencies were awarded both JARC and New Freedom grants for the same projects (as a result, the total number of projects is less than the number of grants).

TABLE 2
Proportion of JARC and New Freedom Funds Awarded, by Agency

Table 3 lists the projects awarded, by funding program and by project type. To avoid double counting, two programs that were funded with both funding sources are listed under only one: the first Massachusetts Human Services Transportation Office project is listed here only under the New Freedom program; the Salem/North Shore Career Center project is listed only under JARC. The GATRA project in Pembroke is listed under both programs because it was set up with distinct components to be funded by each program.

Table 4 presents the numbers of projects by funding program and indicates which of the service goals each project aims to achieve. Although several projects have multiple program goals, they are listed under the primary goal only. As one would expect, a majority of the JARC projects are trip-based (57%). The rest are either information-based (29%) or capital investments (14%). The most frequently occurring program goals are to expand geographic coverage (36%) and to improve access and/or connections (29%).

Almost half (47%) of the New Freedom projects are information-based; 40% are trip-based; 13% are capital investments. The most frequently occurring program goals are to expand geographic coverage (33%), to improve customer knowledge (27%), and to improve access and/or connections (20%).

Evaluations of Projects’ Effectiveness

To determine how effective the trip-based projects have been, stated goals in terms of monthly trips served were compared with the numbers of monthly trips actually achieved. Table 5 presents this information. Note that goals are fairly modest; these grant programs were intended to fill transportation gaps and to serve specific populations that are often difficult to serve.

For the information-based and capital investment projects and planning studies, effectiveness was evaluated by comparing stated goals with realized goals, although the goals were generally not quantifiable.

The completed project profile forms, which are included in Appendix 2, provide additional information that cannot be quantified but provides valuable insights into project accomplishments. For example, Greater Lynn Senior Services’ (GLSS’s) program model has gained national recognition, and GLSS representatives have participated in a national webinar.

It should be noted that, because many New Freedom contracts resulting from the July 2010 solicitation were not signed until the fall of 2011, most of those projects were still in their initial stages when evaluation data were requested.

Trip-Based Projects

Four trip-based projects were found to be effective. Of these four projects, two, the Route 1 Shuttle and the East Marlborough Shuttle (both JARC), have met or exceeded projections. Two others, the South Side Shuttle (JARC) and the Logan Sunrise Shuttle North Extension (both also JARC) have almost met their goals. Four projects, all of which are funded by the New Freedom grant program (one is funded by both programs) were not effective, having missed goals by large margins.

In terms of reaching goals, the most effective projects are operated by a regional transit authority (note that two RTAs did not report) and Transportation Management Associations. These entities are experienced transportation providers who generally know their markets. Most of the least effective projects are operated by private non-profits that are new to the transportation field.

Three projects could not be evaluated as they are still in initial stages or were not initiated. There was no response from the Greater Attleboro-Taunton Regional Transit Authority, which operates four projects.

Table 5
Projected and Realized Goals for Trip-Based Projects*

Table 6
Projected and Realized Goals of Information-Based and Capital Investment Projects and Planning Projects*

Information-Based and Capital Investment Projects
and Planning Studies

Table 6 presents projected and realized goals for planning studies and mobility management projects that either did not have quantifiable goals or are equipment

related. All of the responding agencies except for SCM Community Transportation met their projected goals: they bought vehicles or other equipment, conducted planning studies, provided travel counseling, or coordinated with other agencies. SCM Community Transportation has been a transportation provider for years but is new to conducting planning studies. The agency did not meet any of its projected goals and decided it was underfunded. Two agencies did not respond, and two projects are in the initial stages.

General Findings

Successful Projects

All measures of effectiveness that were considered indicate that the most successful projects are operated by entities that both know their markets and have a history of serving them in some capacity. They have either conducted preliminary planning studies before applying for funding or as the first phase of a project, or already routinely plan for and provide transportation services.

For example, as both federal and state designated Area Agencies on Aging, Mystic Valley Elder Services (MVES) and GLSS are mandated to conduct regular needs assessments to identify barriers their consumers face and are therefore familiar with the requirements of their markets. The first phase of GLSS’s Reaching Beyond Borders project was designed to develop strategies for addressing transportation barriers and gaps; to develop transportation options for traveling within the service area; and to develop a coordinated plan for managing mobility among service areas. MVES used the first phase of its project to determine which volunteer driver program best suited its purposes. The Route 1 shuttle was originally funded as a Congestion Management and Air Quality Program project. The East Marlborough project evolved from a transit study.

Less Successful Projects

The Cape Ann Transportation Operating Company (CATOC) and SCM Community Transportation have operated transportation services for years, but they generally offer services contracted by others. They do not have planning experience. New England Paralyzed Veterans of America (NEPVA) has not planned for or offered transportation services; however, it falls into the category of agencies that fill important transportation gaps.

Agencies such as these, without experience in planning and/or operating transportation services, should not be categorically ruled out from receiving funding. Perhaps they can be helped to develop projects that have meaningful results. For example, the MPO might consider conducting pre-proposal workshops to provide technical assistance to entities with good ideas but with little or no experience planning or operating services. Information from “lessons learned” can also be shared at these workshops.

Recipient-Identified Lessons Learned

The project profile sheet sent to all program recipients requested a statement of lessons learned from conducting their projects. The following is a summary of responses.

• Routes that are too long temporally and geographically decrease potential ridership.
• Transportation needs expressed by potential riders do not always reflect trips they will actually make.
• Agencies need to accommodate the business community to ensure that services meet the needs of commuters.
• Facilitating the transfer from the use of paratransit to the use of fixed-route transit for those who are able to use the latter reduces agency costs and allows individuals to control their own travel.
• Partnering with other organizations can help build awareness of the services.
• Know your market.
• Perform in-depth studies and surveys prior to launching future routes.
• Marketing is important. Marketing associated with the introduction of the new route has helped increase ridership on the existing route.
• The most important lesson learned through both travel counseling and community education sessions is that mobility management is about more than just a ride. (A host of issues have to be addressed.)
• Physicians are eager to have travel counseling on-site so they can easily refer caregivers and consumers.
• Coordinated scheduling is more challenging than anticipated because service providers still think that pooling services and/or sharing customers will result in a net loss for them.
• Brokerage transportation systems have the capability of achieving more enhanced coordination and cost efficiencies than stand-alone systems. However, these systems require a committed level of oversight to ensure service quality is not compromised in the pursuit of savings.

From the above, it appears that the dominant theme is that an agency has to be aware of the needs and desires of its potential market. Studies and surveys prior to initiating service yield a more successful service.

Completing This Evaluation Process and
Conducting Future Solicitations

Since a number of projects are still in their initial stages, it would be advisable to evaluate them one to two years from now.

In future solicitations, major considerations in determining which entities will be funded should include whether the entity has conducted a preliminary planning study for its proposal (or will conduct one as the first step of its proposed project), whether it has transportation experience or experience offering services of another kind, and whether it has knowledge of the targeted market. If a proponent has neither planning nor operating experience but is an organization with unique needs and a viable proposal, the MPO should provide it with technical support in preparing its application.

Proponents for projects other than marketing studies should be able to identify in their applications the unmet needs of their target populations and how their projects will address these needs.

Proposed projects that serve social service needs often cannot have strictly quantifiable goals and standards, and the MPO’s solicitations and evaluations should continue to assess such projects using nonquantified measures.

APPENDIX 1

Reporting Packet

(available in accessible format upon request)

APPENDIX 2

Project Profiles

(available in accessible format upon request)