How to measure the MBTA’s operational efficiency
Pioneer has long been interested in the efficiency of the MBTA relative to other systems around the country. As noted in a blog response to an organization critical of our work, we are not interested in ideological playtime with numbers.
The organization, called the Frontier Group, which is allied with advocates of continued MBTA expansion, notes that Pioneer should focus its operational and efficiency analyses on costs per “unlinked passenger trips.” When we began our recent series on the MBTA we had a number of in-house discussions regarding what would and would not provide useful information on operational efficiency. In transit speak and then in basic laymen’s terms, here is why “unlinked passenger trips” don’t tell us much about efficiency, and why measuring dollars spent per vehicle revenue hours and total passenger miles traveled are far better metrics.
Transit Speak
The Federal Transit Authority collects information from all American transit authorities and maintains a National Transit Database. As part of that program, they provide suggestions on ways to measure the cost effectiveness and efficiency of the systems. The linked document provides a useful distinction between cost-effectiveness and efficiency. On page 14, they define cost-effectiveness as “the relationship between service inputs and service consumption.” That is, it’s money used by the system divided by “is the amount of service used by the public.” Unlinked passenger trips, that is, people jumping on and off-board, is one way to measure that consumption.
Costs per unlinked passenger trips is an endogenous measure (money spent) taken against a largely exogenous measure (people who decide to get on and off the system). You could imagine, for example, that the establishment of a SXSW-style series of summer events or the 2024 Olympics would markedly increase unlinked passenger trips. But does it say a lot about operational efficiency? Not that much. By mixing internal and external factors, it does not say much about cost of operation – and certainly not as much as cost efficiency measures described in the NTD document.
That document (p. 16) defines cost efficiency measures as “the relationship between service inputs and service outputs. Service output is the quantity of service produced by a transit operator…” NTD suggests vehicle hours and miles as good denominators for efficiency. Vehicle revenue miles are the miles a transit vehicle travels while available for the public to ride, therefore excluding out-of-service travel to and from garages, yards, and to change routes. Vehicle hours and miles are endogenous measures and therefore operations-focused. They’re good internal measures, and corollaries for solid ways to measure reliability such as “miles between system failures.” Endogenous measures up against endogenous measures.
Putting this in Laymen’s Terms
To make this discussion easier to understand, let’s take it out of the transit world. Imagine you own a giant apple orchard and run a complex apple picking operation. You have your Kubota and a trailer with the requisite hay bales for folks to sit on. You let passengers get on and off at a dozen stops so they can arrive at the apple varieties they most want. Some pieces of the overall ride are shorter, some longer. How do you measure cost effectiveness and efficiency?
Cost effectiveness in this case is simple, because there is one ticket passengers will buy for the day. Measuring the tickets sold will tell you two things: Whether you will make or lose money, and perhaps the relative need to improve advertising, boarding and “offloading” and the comfort of the hay bales. It says very little that’s useful to understanding the cost structure of the service.
To understand your operational efficiency you need to look at very different measures. You’d look at the hours you run the Kubota engine and/or the total miles the Kubota traveled. You would need to measure fuel costs and maintenance costs. And you would have to measure the hours and pay of the person driving the Kubota.
That is in fact why the Kubota has a vehicle hours meter: It is the common-sense way we measure wear and tear on a tractor. Same for a boat. The corollary in a car, truck or bus is your odometer, which measures total miles traveled.
What this debate comes down to is in some ways not a very technical discussion at all. It’s a matter of how people who operate things measure efficiency – in this case hour meters and odometers – versus folks who prefer to kick up a lot of dust on some simple questions. They do that because as we noted in our last run-around with them, they are “attempting to make an ideological point with no relation to the operations of the system.”
Follow me on twitter at @jimstergios, visit Pioneer’s website, or check out our education posts at the Rock The Schoolhouse blog.
