Memorandum
Date June 4, 2015
TO Boston Region Metropolitan Planning Organization
FROM Karl H. Quackenbush
CTPS Executive Director
RE Work Program for: MBTA 2015–16 Systemwide Passenger Survey
Review and approval
That the Boston Region Metropolitan Planning Organization, upon the recommendation of the Massachusetts Bay Transportation Authority, vote to approve the work program for the MBTA 2015–16 Systemwide Survey presented in this memorandum
Technical Support/Operations Analysis Projects
14346
Massachusetts Bay Transportation Authority
Project Supervisor: Laurel Paget-Seekins
Principal: Annette Demchur
Manager: Thomas J. Humphrey
Future MBTA contract
The MPO staff has sufficient resources to complete this work in a capable and timely manner. By undertaking this work, the MPO staff will neither delay the completion of nor reduce the quality of any work in the UPWP.
The Federal Transit Administration (FTA) Title VI Circular (C 4702.1B) requires large transit providers to collect demographic, travel, and fare payment data about their riders using passenger surveys at least every five years. In addition, the results of past MBTA passenger surveys have provided essential data to many different users, including the MBTA, CTPS, consultants, other transportation agencies, academic researchers, and members of the public. Surveys with content comparable to those planned in this work program were last conducted on the MBTA’s rail rapid transit, light rail, bus, commuter rail, and commuter ferry systems in 2008 and 2009. The Silver Line Washington Street bus rapid transit line was last surveyed in 2005, and the Silver Line Waterfront bus rapid transit lines were last surveyed in 2006. Because of the substantial commitment of resources needed to conduct and process a systemwide survey, the results of any such survey will be relied on for several years.
The objective of this work program is to obtain responses from sufficiently large samples of riders on each MBTA service to provide statistically valid results at the level of aggregation at which the findings will be used. Contingent on obtaining statistically sufficient numbers of responses, results will be presented at both mode-level and route-level for buses, trackless trolleys, and boats, while line- and stop-level results will be presented for the other modes. Overall, the survey effort will be divided into the following eight categories:
The surveys for each mode will be designed to be filled out by passengers either online or on paper copies. The information collected will be as comprehensive as practical. In particular, the following types of information about passengers and their trips will be collected:
The work described in the tasks below will be performed for each of the eight categories of service listed in the Objective section, above. The MBTA may designate varying priorities for the surveys of the eight service categories, necessitating the completion of some survey tasks in phases rather than all at once for the entire system. For example, if the highest priority is to complete surveys for the Red and Green lines, the processing of survey data for these lines may occur in advance of the distribution of paper surveys on other transit lines.
The 2008–09 survey was conducted primarily via paper survey forms that were distributed to passengers on all of the MBTA modes, with an option to fill out the survey online. The state of the art in web-based survey platforms has advanced significantly since 2008–09. Software packages available under license from companies such as Qualtrics LLC can now be used to create very detailed web-based survey forms and to efficiently process and manage the data provided by the survey responses. The use of such survey packages can result in substantial cost and time savings compared with past paper survey distribution.
For the new survey, the initial effort will be web-based. However, since past experience suggests that it may be difficult to obtain statistically valid sample sizes for some stations or bus routes using the web-based survey, the initial surveys for those routes will be supplemented with paper surveys.
Both the paper and web-based survey forms for each transit mode will contain a standard set of questions similar to those on past MBTA surveys. The questions will be revised as needed to reflect changes that have occurred since previous surveys and to be consistent with the MBTA’s survey policy. Additionally, questions will be added to address the current federal Title VI reporting requirements.
Each paper survey form will elicit information about the trip during which the respondent received the form. The online form will elicit similar information about the most recent MBTA trip taken by the respondent. For these trips, the questions will include the respondent’s initial origin and final destination locations, the respondent’s means of access to and egress from the transit trip, the fare type and fare payment method, and the respondent’s frequency of use of the MBTA modes included in the trip. More general questions, some of which are required for Title VI purposes, will provide demographic information, including the respondent’s age, gender, race, ethnic identity, English proficiency and language spoken at home, household income, and household size: whether the respondent is a licensed driver and if the respondent had a private vehicle available as an alternative to making the trip using MBTA service. There may also be a section rating various service-quality attributes on a numerical scale, and sections for written comments.
CTPS will solicit input about survey content from a broad range of MBTA departments to ensure that questions are asked in a way that elicits the most useful possible data for future decision-making. Federal reporting requirements, such as those relating to Title VI compliance, the National Transit Database, and before-and-after studies for new capital investments, will also be considered in the development of the surveys. Furthermore, efforts will be made to ensure consistency with the questions in the recently developed bank of standardized MBTA survey questions and, to the extent possible, questions that were asked in past passenger surveys, including those performed by CTPS and the MBTA’s commuter rail contractors, in order to track temporal changes in passenger and trip-making characteristics.
For each transit mode, a separate survey will be developed in both electronic form, for passengers to access online, and in printed form, for distribution on board trains and buses and in stations. The paper surveys will be designed to be filled out by passengers while they are riding in buses or trains. Because of this, the surveys will be printed on card stock in booklet form, and will be short enough to be completed in 5 to 10 minutes. These forms will be preprinted with unique numerical identifiers so that the location of each survey’s distribution can be tracked.
Both the paper and online surveys will be made available in several languages to ensure that the surveys are accessible to passengers with limited English proficiency. The selection of languages will be based on the MBTA’s Title VI Program four-factor analysis. Furthermore, surveys will be available in formats that are accessible for persons with disabilities.
CTPS will work with the MBTA to create a database and a data-entry interface for the surveys using the Qualtrics software package that is licensed to the MBTA. The interface will route online respondents to subsets of questions that are applicable to their trips. The same interface will be used by CTPS for transcribing the results of the paper-copy responses into the database. The responses from each individual returned survey form are referred to collectively as a “survey record.” The Qualtrics software will be used to produce a variety of summary tables from the records in the database.
CTPS will work with the MBTA departments that are most likely to use the results of the survey in order to determine the preferred formats of the summary tables. Some tables will summarize the responses to individual questions; others will provide cross-tabulations depicting correlations between responses to two or more survey questions. To the extent allowed by the Qualtrics software, the summary table formats will emulate those from previous comprehensive passenger surveys, including the 2008–09 systemwide passenger surveys, and surveys conducted by CTPS on MBTA heavy rail and light rail rapid transit, bus, and commuter rail lines in the 1990s.
CTPS will work with the MBTA’s Marketing and Communications Department and Operations Department to develop and implement a comprehensive two-stage strategy for marketing both the online survey and the paper survey. This strategy will include methods for obtaining representative samples of MBTA riders; outreach in languages other than English; and the provision of surveys in alternate formats in accordance with the MBTA’s Title VI program.
The first stage of the marketing program will be focused exclusively on directing riders to the online survey. The web-based survey will be launched during this first stage of the marketing campaign and prior to distribution of paper surveys. The online survey will be available on the MBTA’s website for the duration of the project, and the instructions for the paper survey forms will include the web address for those who prefer to fill them out online instead of on paper. The second stage of the marketing program will begin when online responses start to taper off, and it will target stations and routes for which online response rates were low in order to prepare for the distribution of paper surveys.
Products of Task 3
The strategy for distributing paper surveys will be designed to reach most of the passengers who use commuter rail, rapid transit, bus, and ferry services during the survey hours at the stations or on the routes for which there were inadequate response rates from the online survey. For these services the goal will be to obtain sufficiently large combined samples, from the online and paper surveys, to allow the results to be tabulated separately by boarding stops or, in the case of light rail, by groups of surface boarding stops. For local and express buses and trackless trolleys, the goal will be to obtain sufficiently large combined samples to allow the results to be tabulated separately by route. The statistical errors of the survey results will depend largely on the response rates of passengers, so it is not practical to specify allowable levels of error in advance.
In most cases, the number of survey responses from the online and paper-copy surveys combined will be much smaller than the total number of riders using a particular station or route during the survey distribution time span. This survey response rate tends to vary by mode, length of trip, demographic characteristics of passengers, and other factors. Therefore, as was done for past MBTA passenger surveys, weight factors will be applied to survey returns so that the results can be expressed in terms of the absolute numbers of riders in addition to percentages. This will allow reasonable comparisons to be made between services with varying ridership levels and response rates.
For each route or station to be included in the paper survey distribution program, the distribution strategy will be to try to offer survey forms to all passengers entering that station or boarding vehicles on that route between 6:00 AM and 3:30 PM on one weekday. This has been the standard practice for MBTA surveys since the mid-1990s. It can be expected that most passengers who have already completed surveys online will not take paper forms. Previous surveys that had longer distribution time spans experienced response rates after 3:30 PM that were much lower than those before 3:30 PM. This was believed to reflect the high proportion of trips after 3:30 made by riders who had already taken trips on the same service earlier in the day and had already been offered surveys. However, survey distribution hours may be varied on services for which information from the MBTA’s automated fare collection system shows high percentages of passengers riding only after 3:30 PM.
Because of time and budget constraints, surveys will not be conducted on Saturdays, Sundays, or holidays. The responses to past weekday surveys, as well as informal observations, imply that nonrepetitive discretionary trips account for a high proportion of weekend and holiday ridership. Therefore, responses from surveys conducted on any given Saturday, Sunday, or holiday would not necessarily be representative of weekend and holiday ridership over a longer time span. In addition, with only one Saturday and one Sunday per week, the overall length of a weekend survey project would be much longer than that of a weekday project at similar staffing levels.
CTPS field data collection staff will be assigned to perform each of the survey distribution assignments generated in Task 4. The specific survey distribution method will necessarily differ somewhat between services because of differences in the control of the flow of passengers onto vehicles, average trip durations, and vehicle loads. At stations that have off-vehicle fare collection, CTPS survey distributors will hand forms to passengers as they are channeled through the faregates. Signs will be posted near the outside entry points to alert passengers to expect to receive the forms inside the station.
On services that have onboard fare collection and many stops, surveys will be handed to passengers by CTPS distributors riding the vehicles. (On such services, MBTA vehicle operators cannot be expected to distribute surveys in addition to their regular responsibilities.) This distribution method will apply to local buses and trackless trolley lines, and commuter rail lines. The number of distributors assigned to each trip will be based on several considerations, including vehicle configuration, typical loads, and distance between stops. For express bus routes, survey distributors will be stationed at bus stops rather than riding the buses, and will distribute surveys to passengers as they board or alight from buses.
For water transportation services, the initial survey distribution will be done on board by boat crew members, followed up, as needed, by distribution by CTPS. (The onboard staffing levels required by Coast Guard regulations, combined with relatively long times between stops, usually make it feasible for at least one member of each crew to distribute surveys without interfering with regular responsibilities.)
The passengers who receive paper surveys will be able to return them: (1) to the survey distributors, (2) at designated collection points on vehicles or in stations, (3) by mailing them back postage-free, or (4) by using the online option.
Staff will use the database’s user interface, described in Task 4, to enter survey data from paper survey returns. As described in Task 3, many surveys will have been completed by passengers using the interface online. In those cases, no additional data entry will be required.
It is anticipated that most of the surveys that are filled out and returned by passengers will be complete and comprehensible. In all such survey efforts, however, a significant number of returns include erroneous responses, are incomplete, or both. Some examples of erroneous responses include common misunderstandings of one or more survey questions, reversal of directionality for the trip being described, and misspellings of words in open-ended questions. In addition, some errors occur during data entry, such as responses being misread or entered in the wrong fields. After Task 6 has been completed, records for trips on each mode will be scanned for questionable entries by sorting the records by values in each data field and flagging those with entries outside of the expected ranges. Such records will then be subjected to further scrutiny, including examination of the original survey cards in cases of records not entered directly by respondents.
For each record that has questionable responses, if the intent of the respondent can be determined to a high degree of certainty based on the responses to other questions, the record will be edited accordingly. Alternatively, if responses are nonsensical and the intent of the respondent cannot be determined, a decision may be made to void all or part of the record in question.
In the case of incomplete surveys, a determination must be made as to whether some or all of the data provided would be useful for inclusion in the project database. For example, a survey for which a respondent chose not to provide information about household income, but did respond fully to most of the other questions, would not be voided. In general a higher level of expertise is necessary for verifying or editing surveys than for doing data entry, so most data-entry staff will be instructed to enter responses just as they appear on the survey forms. However, in cases in which a staff member has expertise in both data entry and data editing, some editing may be done during the initial data entry process.
Database containing screened and edited records
The raw data from the passenger survey will be of limited value for drawing conclusions about passenger attitudes and usage patterns without first weighting individual records by the ratio of the total ridership on the corresponding segment of the system to the total number of records from that segment. This process is referred to as “expansion” of survey records. A passenger making a trip using more than one MBTA service will usually represent a different percentage of the ridership on each service, so the database will need to assign different weights to the same records for the summary tables for each service.
For trips on buses, trackless trolleys, and boats, data will be expanded at the route level, while for trips on other modes stop-level expansion will be performed. The survey results generated in this project are intended to reflect ridership on a “typical” day rather than on a specific day, so it is not necessary that the ridership totals used in the expansion process be collected on the same days that surveys are distributed.
The ridership total to which each group of survey records is to be expanded is referred to in this context as a “control total.” The control totals for pre-payment stations will be based on data that will be obtained from the MBTA’s automated fare collection (AFC) system. Control totals for services with onboard fare payment will be based on the most recent ridership numbers available from various data collection methods used by the MBTA. For the commuter rail system, survey distributors will also record passenger boardings by station, or ridership volumes by train, at selected locations. These will be used as cross-checks of reports generated by the contract operator of the commuter rail system. If necessary, supplemental counts will be taken at selected stations at later dates. Control totals for water transportation services will be based on reports for the survey day or another typical day that will be provided by the boat operators.
After weights have been applied to each record, electronic summary tables will be generated in the formats developed in Task 2.
After the data have been processed and standard summary tables generated for each of the eight service categories, staff will compare the results for these categories and subsets thereof, identify findings of interest, and determine general profiles of the ridership for each category. In some cases, certain findings may suggest that additional cross-tabulation of the results would be useful to illustrate the relationship between two specific variables within a category. For example, it may be of interest to highlight the relationship between a demographic indicator and the type of fare paid.
For each subset of the questions posed on the survey forms (for example, demographic indicators or access and egress modes), summary text will be prepared to highlight the most noteworthy findings from the examination of each summary table.
Summary of interpretation of survey results, by service category and question subset
The products of Tasks 8 and 9 will be compiled into one report for each of the eight service categories listed in the Objective section. Each of the standard database summary tables will be included, along with any specialized tables, including cross-tabulations. The analysis and interpretations completed in Task 9 will also be included in the final reports.
Because of the large number of database tables that will be generated as products of this project, general distribution of the eight final reports will be only in electronic formats. Interested parties will be able to download the reports from the Internet or read them directly from a compact disc. The reports will be saved in Adobe Portable Document Format (PDF) so that readers may easily print portions of the documents of interest to them.
Eight final reports, one for each service category
It is estimated that this project will be completed 20 months after work commences. The proposed schedule, by task, is shown in Exhibit 1.
The total cost of this project is estimated to be $1,180,000. This includes the cost of 730.3 person-weeks of staff time, overhead at the rate of 91.82 percent, travel, and other direct costs. A detailed breakdown of estimated costs is presented in Exhibit 2.
KQ/TJH/tjh
Task |
Month | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | |
|
1.
Design the Survey |
From month 1 to 3.5.
Deliverable A, Online and paper surveys delivered by Month 3.5
| |||||||||||||||||||
|
2.
Create a Database |
From month 1.9 to 3.5.
| |||||||||||||||||||
|
3.
Develop and Implement Survey Marketing Plan and Launch Online Survey |
From month 1.2 to 17.4.
| |||||||||||||||||||
|
4.
Develop Sampling Plan and Distribution Schedule |
From month 1.2 to 4.5.
| |||||||||||||||||||
|
5.
Distribute Paper Surveys |
From month 4.2 to 6.3.
From month 7 to 13.
From month 15.1 to 16.5.
Deliverable B, Completion of survey distribution delivered by Month 16.5
| |||||||||||||||||||
|
6.
Enter Data |
From month 7.5 to 17.6.
| |||||||||||||||||||
|
7.
Verify Data |
From month 7.7 to 18.3.
| |||||||||||||||||||
|
8.
Expand and Process Data |
From month 12.8 to 18.8.
| |||||||||||||||||||
|
9.
Interpret and Analyze Data |
From month 13.5 to 19.7.
| |||||||||||||||||||
|
10.
Prepare Final Reports |
From month 14.2 to 20.6.
Deliverable C, Final reports delivered by Month 20.6
| |||||||||||||||||||
Task |
Person-Weeks | Direct Salary |
Overhead (91.82%) |
Total Cost |
||||||
|---|---|---|---|---|---|---|---|---|---|---|
| M-1 | P-5 | P-4 | P-3 | P-1 | Temp | Total | ||||
|
1.
Design the Survey
|
1.5 | 2.2 | 2.2 | 2.0 | 0.0 | 0.0 | 7.9 | $11,607 | $10,657 | $22,264 |
|
2.
Create a Database
|
0.2 | 1.8 | 0.0 | 2.0 | 0.0 | 0.0 | 4.0 | $5,715 | $5,248 | $10,963 |
|
3.
Develop and Implement Survey Marketing Plan and Launch Online Survey
|
1.0 | 0.0 | 0.0 | 1.5 | 0.0 | 0.0 | 2.5 | $3,409 | $3,130 | $6,538 |
|
4.
Develop Sampling Plan and Distribution Schedule
|
0.5 | 15.0 | 0.0 | 5.5 | 7.0 | 0.0 | 28.0 | $38,660 | $35,497 | $74,157 |
|
5.
Distribute Paper Surveys
|
3.5 | 25.0 | 0.0 | 10.0 | 41.0 | 324.9 | 404.4 | $256,428 | $235,452 | $491,880 |
|
6.
Enter Data
|
1.0 | 3.0 | 0.0 | 3.0 | 8.0 | 150.0 | 165.0 | $92,117 | $84,582 | $176,699 |
|
7.
Verify Data
|
1.0 | 5.0 | 3.0 | 2.0 | 2.0 | 0.0 | 13.0 | $18,240 | $16,748 | $34,989 |
|
8.
Expand and Process Data
|
2.0 | 9.0 | 0.0 | 8.0 | 0.0 | 0.0 | 19.0 | $28,129 | $25,828 | $53,957 |
|
9.
Interpret and Analyze Data
|
3.0 | 9.5 | 15.0 | 10.0 | 0.0 | 0.0 | 37.5 | $52,784 | $48,466 | $101,250 |
|
10.
Prepare Final Reports
|
5.0 | 9.0 | 23.0 | 12.0 | 0.0 | 0.0 | 49.0 | $68,190 | $62,612 | $130,803 |
|
Total
|
18.7 | 79.5 | 43.2 | 56.0 | 58.0 | 474.9 | 730.3 | $575,279 | $528,221 | $1,103,500 |