We build bike lanes to make us safer and more comfortable while riding our bikes. Cities all over the world are building painted bike lanes and separated bike lanes. Knowing whether bike lanes are actually safer is important, to say the least. The science of the safety of bike lanes, however, is a bit behind.
The science doesn't have to be perfect in order for us to take some action, but it needs to be helpful. We need to understand how research was done and how the researcher came to their conclusions. When it comes to a Danish study by Danish researcher S. U. Jensen titled "Bicycle Tracks and Lanes: a Before-After Study" we probably should not trust its conclusions that bike lanes and separated bike lanes are unsafe. Or so argues Dr. Kay Teschke, with whom I corresponded by email last winter. (Photo of woman and girl cycling on Copenhagen street by Ian)
I'm looking at this study in particular because there are competing claims to what it actually proves. There are those who have argued that Jensen's study is proof that separated bike lanes (more commonly known as cycle tracks in Europe) are dangerous. But there are also those who have argued that when read properly the study actually shows that separated bike lanes are safe (see postscript below).
I decided to get to the bottom of this and contacted Dr. Kay Teschke of the University of British Columbia to find what an expert in epidemiological research--who also conducted a large research study on cycling safety--has to say about this study. I'm reaching out to Jensen as well and will post his response if I get one.
Jensen, in his Copenhagen study, came to the controversial conclusion that cycling on cycle tracks* is less safe than cycling on streets without any cycling infrastructure. Jensen concluded:
The safety effects of bicycle tracks in urban areas are an increase of about 10 percent in both crashes and injuries. The safety effects of bicycle lanes in urban areas are an increase of 5 percent in crashes and 15 percent in injuries. Bicyclists’ safety has worsened on roads, where bicycle facilities have been implemented.
Safety is worse with bike lanes? Jensen's conclusions are counter-intuitive and don't fit well with the results of a number of other studies, as was shown in a recent meta study of scientific studies of cycling and injuries. (Doing meta-analysis is common in epidemiology, where researchers compare different studies and look for patterns.) Given that Jensen's study comes to this irregular conclusion it would be easier to trust if we had a clear idea of how he arrived there. That, however, is one of the main problems with Jensen's paper: a lack of transparency.
Black box
The main issue with Jensen's study is that it's a black box; an algorithm that he never reveals. Jensen shows us the initial numbers that he measured, then puts them into his black box and out the other end comes the inverse.
Says Dr. Teschke:
Jensen did a very elaborate analysis with lots of adjustments. It is good to take into account factors that might bias unadjusted results, but usually the first analysis is the simple unadjusted one. Jensen does not present the unadjusted results, so I calculated them based on the data presented in his paper. On page 4, he indicates that the before and after periods studied were equally long. In Table 3 on page 9, he presents the observed before and after data on crashes and injuries. He also presents expected after data, based on all the adjustments. But let’s first look at the observed, before and after. In every row, except two (Crashes Property damage only and Intersections All crashes) the observed injuries or crashes after are lower than the observed before. On page 12, he indicates that there was a 20% increase in bicycle and moped traffic in the after period. So a calculation of crude relative risks (RR) for after vs. before:
- Bicyclists and moped riders, all injuries total:
RR = (injuries after/traffic after)/(injuries before/traffic before) = (406/1.2)/(574/1) = 0.59
- Bicyclists and moped riders, intersection injuries:
RR = (injuries after/traffic after)/(injuries before/traffic before) = (285/1.2)/(353/1) = 0.67
- Bicyclists and moped riders, on links injuries:
RR = (injuries after/traffic after)/(injuries before/traffic before) = (121/1.2)/(221/1) = 0.46
All three of the unadjusted results in the after vs. before comparison for bicyclists and moped riders show a reduction in risk (RR=1 means the same risk after and before, RR < 1 means lower risk after, RR > 1 means higher risk after). It is very strange not to report these results in the paper. They mean that over the period studied, the risk for cyclists and moped riders went down in the period after installation of the cycle tracks.
The question that should be answered with the adjustments is whether this reduction in risk is because of the cycle tracks or whether it is just a time trend - perhaps risk also went down on routes without new cycle tracks. If the comparison streets used for the adjustments were really comparable and if all the adjustment assumptions are unflawed, then the answer to that question would be “the reduction in risk is not from the cycle tracks”. But to take the conclusion further than that and say, after all these adjustments, cycle tracks are less safe (i.e., completely reversing the crude results)? This requires a level of trust in the adjustments that is very hard to justify in my view - especially given the difficult-to-follow description of the methodology and the many assumptions involved.
So can we trust Jensen's numbers? I don't think we can. Dr. Teschke's preliminary calculation of risk based on Jensen's numbers came to a Relative Risk of 0.59 compared to a higher risk of 1 for a street without bicycle facilities. In other words, Jensen's raw numbers support the conclusion that bicycle facilities reduce risk of injury. But it would be quite odd, Dr. Teschke explains, that the final, adjusted result would show the opposite of this. Yet when the final result comes out of Jensen's black box they are just that.
Science needs to be transparent and reproducible and this study falls short of that standard.
No one study can be the final word one way or the other. In the much more studied world of health and medicine, epidemiologists are looking for consensus among studies before coming to conclusions. The Copenhagen study has too many problems to serve as the final word on bike lane safety for policy makers.
Postscript
I wasn't the first blogger to question this study, I had also asked Dr. Teschke other questions about this study and how it related to what another statistician, Dr. Lon Roberts, had said about the study.
Another blogger from Texas, Jason Roberts, was also interested in understanding this controversial study better and had asked Dr. Lon Roberts for his opinion on the study. Jason linked to a simplified version of Jensen's study (called “Road safety and perceived risk of cycle facilities in Copenhagen”) that had even less information about Jensen's methodology and thus may have mislead Dr. Roberts. At least, that's my theory.
Dr. Roberts told Jason that "the Copenhagen study shows that the "likelihood an individual bicyclist will experience an accident goes down as the number of bicycle riders go up".
Furthermore Roberts says:
Using Soren’s percentages, here’s an example starting with the assumption that 10 bicyclists out of 10,000 will experience an accident over a certain period of time if there are no bike tracks:
On an individual basis, there’s a 10 out of 10,000 (or 0.1%) chance that an individual biker will experience an accident if there are no bike tracks
When the bike tracks were added, the accident rate increased by 9%. In other words, if there are 10 accidents without the tracks, the number of accidents increases 10.9 (or approximately 11). On the other hand, the number of bike riders increased by 18%, from 10,000 to 11,800. Therefore, on an individual basis the likelihood of an accident with the tracks added is now 11 out of 11,800, or 0.09%, as opposed to 0.1% without the lanes/tracks.
So Dr. Roberts is basically saying that Jensen had accounted for bicycle traffic volume when reporting the numbers (greater numbers of cyclists will always have some effect on increasing crashes/accidents). But Dr. Teschke seemed to be saying that Jensen had accounted for bike traffic. So I asked for more explanation.
I asked:
In your discussion paper you listed RR = 1.10 versus Jensen's estimates of expected injury rates. Does this mean that Jensen had taken into account bicycle traffic volume when he provides an estimate of injuries +10% (Table 3)? I'm assuming you added .10 to 1?
And if Jensen has already accounted for bicycle traffic in his estimate I'm confused about Roberts' calculation. Is Roberts' accounting for the denominator a second time?
Dr. Teschke responded:
Jensen’s formulas indicate that he did take bicycle and moped traffic into account - in more than one direction. He also took motor vehicle traffic into account. The reasoning for the latter is not clear to me. Perhaps he is saying that if MV traffic volumes went down after cycle tracks were installed, you would expect fewer crashes. But if lower MV traffic is one of the pathways to lower bike and moped crashes on cycle track routes, that is a good thing, not something to be adjusted out of the analysis.
You are right, I added .10 to 1.
Robert’s calculation is not very clear to me. It does seem to assume that Jensen did not take bicycle and moped traffic into account. The formulas in Jensen’s paper suggest he did.
But it is easy to be confused about what Jensen did. It is not clear what alpha and beta are, or how he chose the values for those parameters. He mentions “Danish crash prediction models” but does not provide a citation. Although he laid out the formulas for the traffic adjustments, he did not do the same for either the trend or regression-to-the-mean adjustments. When an adjustment method reverses the unadjusted result, it is important that the method be clear and highly defensible.
I also asked Dr. Teschke why Jensen's study wasn't included in their literature review.
Herb:
- Why was this one missed from the lit review? Did it not meet the criteria?
Dr. Teschke:
You are right, we didn’t include the Jensen study in our literature review, because it did not meet our criterion of being published in the peer-reviewed literature. It was published in conference proceedings not a peer-reviewed journal. After the review was published, many people noted that we did not include this study and, when we gave the reason, they argued that the reviews of the Transportation Research Board are more rigorous than those of many conferences, so it should be considered truly peer-reviewed.
We have referenced the Jensen study in subsequent publications, for example in the Discussion of our injury study.
I hope this is useful for some of you interested in connecting the dots between bike lanes, protected bike lanes and safety. It's not easy to dive into the data, but luckily scientists are taking subject matter more seriously.
