Why you shouldn't rely on people with a political stake in the outcome to provide your crowd estimates.
Using a fixed camera mounted in the floor of the plane, the crew made images of the rally from 2,000 feet. The photographs -- taken directly above Market Street and Civic Center Plaza and enlarged -- provide a perspective that allows a discrete count of individuals and a view of the spaces between them, a view that is impossible from ground-level.Both Air Flight Service and The Chronicle examined the photo survey and independently arrived at the estimate of 65,000 marchers at the time the photographs were taken, a figure supported by public transportation statistics.
The flight service says its count is accurate within a range of plus or minus 10 percent.
This verifiable technique, experts say, could replace the current politically sensitive count totals, whereby organizers tend to provide a high number of participants, and police generally provide a lower number.
'MYTHICAL NUMBER'
When told of The Chronicle's survey, Alex S. Jones, the director of Harvard University's Joan Shorenstein Center on the Press, Politics and Public Policy, said, "The number of people (in a crowd) is a mythical number, and now you're going to turn it into a fact, and that won't be welcomed."Jones, a Pulitzer Prize-winning former reporter for the New York Times, added, "There's an old saying in journalism: People only see what they believe. This is an emotional issue, not a factual issue as far as most people are concerned."
Police and event organizers, when told of The Chronicle's numbers, stood by their estimates of 200,000 marchers, though both groups based their figures almost entirely on observational methods and not on a verifiable methodology. (Organizers originally estimated the crowd at 200,000 to 250,000 but now are going with the 200,000 figure.)
They said the aerial survey results were flat wrong.
"Oh my word. Come on, that's ridiculous," said Bill Hackwell, spokesman for International ANSWER, one of the groups that organized Sunday's march and rally. The organizers have another planned for March 15 in San Francisco, coinciding with rallies in Washington, D.C., and elsewhere.
That's ridiculous! What kind of a shoddy methodology is that, trying to get the number of protesters by counting them?
Now, there were certainly people who weren't there to be photographed. But the photograph was taken at the peak of the rally, and as an experienced marcher, I can verify that it's just not possible that three times as many people as were present at the peak marching time between 1 and 2 either showed up very early or very late. Especially since the article reports only 43,000 extra riders on the public transportation system. It's wildly unlikely that the number even approached 100K, much less doube that.
Do you think ANSWER would let me do their books? I could do a lot for a client that doesn't believe in counting. . .
(Via Mean Mr. Mustard)
Posted by Jane Galt at February 24, 2003 11:58 AM | TrackBack | Technorati inbound linksJane,
I think you are overstating your case. First of all, how do you estimate a crowd size? Is it the instantaneous # when you took the picture? The time weighted average number of people? The sum of all the people who came? I would submit that this decision could certainly have a two or three fold impact on your number (say, from 65k who were there at 10:05 am on Sat morning to 200k who came overall throughout the day).
The more important question, though, is does it matter? To you, probably not-you've already decided what the right thing to do is. To me, probably not, because I haven't decided, but my opinion is not important, and is also not particularly effected by what others think. to GWB, certainly not, for reasons that should be obvious by now. But Tony Blair, Jacques Chirac, Gerhard Schroeder... well, now, this just might be important to them. It is not the numbers, though, but the enthusiasm that is the issue for them. They get the numbers from their polling (hint, it doesn't look good for war with Iraq), so what they care about is, how vehement is the opposition.
Anyway, the original point is, this is not an example of innumeracy (i.e. not knowing how to count or estimate) but rather an example of "lies, damned lies and statistics"
The San Francisco Bay Area is a patch work of several public transit systems. The article notes an extra 43,000 riders via BART, and an extra 5,500 riders via Golden Gate Transit. BART is a subway system that primarily services the "East Bay" (Oakland, Berkeley, Contra Costa Country, etc.) coming into downtown and a few of the residential neighborhoods in San Francisco. Golden Gate Transit takes riders from Marin County to the city. The article doesn't give figures for MUNI which is the main bus system that services within the city of San Francisco itself. It also doesn't give figures for Caltrain, which services the area to the south of San Francisco (Silicon Valley) nor does it take into account the people who came in from out of town via private buses or people like myself who walked to the rally. It would seem to me, and anyone else with a knowledge of the Bay Area transit systems and political landscape, that 75% of the marchers coming from BART and Golden Gate Transit would be fairly high. The majority would probably use MUNI.
Photographing people from the air is an estimate the same as counting them on the ground. From what I understand, they counted people within a certain space and then extrapolated that large sections of the crowd had that same density. They didn't count each individual head.
Posted by: Manish on February 24, 2003 12:51 PMWhen I was going to Rutgers, protests were always held at noon in front of the College Ave. cafeteria. That way it always seemed as if the protests were well attended.
Posted by: scooterboy on February 24, 2003 12:52 PMI think Jane's point was not that the bird's-eye-view method of estimating crowd size was definitive, simply that it was much more reliable than relying on the organizers of the march or police to provide an estimate. Both the organizers and the police have an incentive to "guess high" when providing an estimate of crowd size. Naturally organizers want everyone to perceive the rally as well attended. (This is true no matter who is sponsoring the rally. Ever hear of anyone advertising their rally as being focused on a very selective, exclusive group?) Police would prefer that those who set their budget think there is a big need for staff and support to handle large crowds. This does not mean that the police and the organizers intentionally fudge their estimates. They may honestly believe their numbers and still be way off.
When the bird's-eye-view method produces a much smaller estimate of the crowd than that provided by the organizers and by police, there is likely to be some grumbling. But claiming that 2/3rds of the people at the protest were not their at its peak doesn't wash. Sure, some came and left, but 2/3rds? What would that say about the depth of passion of the protestors? Neither is is reasonable to believe that most of the people there just didn't show up in the pictures. Was there some sort of mass potty break or are we to believe most attending the protest were hiding under camouflage netting? No, the most likely answer is that the actual number attending the protest was far smaller than we were lead to believe.
Posted by: David Walser on February 24, 2003 01:34 PMA lot of the uproar is just due to inflation. 65,000 is a lot of people to get in one place, but it's not respected as such. For whatever reason, numbers like 100,000 or 200,000 are the current psychological benchmark for large rallies. (Never mind the ludicrous Million Man March.) But no one really knows what that amount of people looks like. If you're in the midst of a large rally, you can't directly see more than a couple thousand people anyway; the rest is just extrapolation.
Aerial photography really does answer the question better than anything else. The traditional method of counting people in a given area, and using that density for the whole rally, is fundamentally flawed. The density calculation will be done at the center of the action, where people are *much* more packed together than elsewhere.
Posted by: spearmint on February 24, 2003 02:27 PMOne has to favor the most objective counting method possible. The article mentions that this is the first time the aerial photo method has been used to count crowds -- and while its accuracy may not be perfect and will surely improve over time, the use of a checkable quantitative counting method has to be favored by everyone with an interest in the truth.
As an aside, in comparing with previous marches, it might make the most sense to use the outdated counting methods of the past, just to compare apples to apples.
Whatever way that you count, there can be very little doubt that what happened on Feb 15th was unprecedented: coordinated marches all over the world on a single issue on a single day.
The numbers, even if inflated, compare well to the inflated numbers of the past, suggesting that February's march saw historically high turnout.
Posted by: Jim on February 24, 2003 02:53 PMWhatever the numbers actually were, you can't beat Ken Layne's take on the issue:
"Or, to be a bit cruel, the protests attracted about as many people this weekend as the movie 'Kangaroo Jack.'" Ouch.
http://www.kenlayne.com/blogarchives/week_2003_02_16.html#003068
Posted by: Sean E on February 24, 2003 04:16 PMI doubt that this is the first time that aerial photography has been used to estimate crowds. This a mature technology. I did a quick Google search on "aerial photography" +counting and found references to counting marine mammals, prairie dogs, sea gulls, labor day beach crowds, olive trees, parade crowds...the list could go on and on. My point is that the real reason that there is any controversy at all is that some people are blaming the messenger because they don't like the message. I can relate to that. The blasted scale in my bathroom is no good. 20 pounds no good in fact.
Posted by: Ernie G on February 24, 2003 04:53 PMI'll bet more people have seen Kangaroo Jack than the combined number of people who read this (or Ken Layne's) blog, so what's your point? Kangaroo Jack was playing at 2,500 theaters, so that's an average of 26 people per showing.
Factor in the weather variable, and I'm not too impressed with the comparison.
Posted by: Thumper on February 24, 2003 05:17 PMIt's a stupid analogy. Kangaroo Jack is (ostensibly) entertainment. It's indoors (where it's warm), it's escapist, and it's (again, ostensibly) fun.
Protesting and marching outdoor for a few hours is cold, tiring, and risks police ire. While it can be enjoyable, the activity is not about fun, it's about political expression.
You'll always get more takers for entertainment than political expression, and drawing any conclusions about the protests from that fact is a bit facile.
Posted by: Jim on February 24, 2003 05:47 PMWhat's that old saying?
It ain't quantity it's quality.
Numbers are relevant in some respects,
not so in others.
My problem with the events of Feb. 15
is that it was basically "A Happening"
There was no core, just a collection
of agendas. In other words, from an
activists standpoint, FAILURE!
However, politicly a raging success.
We all know in politics numbers count.
The accuracy of the numbers only count IF
they are accurate.
Acuracy of the method aside, it probably is true that crowd counts at protests and the like are exaggerated, or at least wildly inaccurate in some direction or another. There's no good way to count from the ground.
Which was your reaction?
(a) "Hmm, a new more accurate way of counting people. What else do we have aerial counts for? Do ground estimates always overestimate or do they vary both ways?"
(b) "See? Anti-war people are lying scum-of-the-earth after all! Nyah nyah toldya so [etc]!"
I was thinking (a) myself. Lots of questions remain unanswered, but it seems that the lessons to be learned have very little to do with anti-war protestors per se and more about crowd estimates and the optimism of organizers, in general.
If *I* took every piece of critical humor seriously enough to get aggravated about it, at what age could I be expected to suffer a coronary?
Posted by: anony-mouse on February 24, 2003 07:16 PMFor ANSWER's numbers to be accurate, the average time spent at the rally would have had to be around two hours, which doesn't speak that well for the committment of the protesters.
I actually thought it was first of all, an interesting comment on how far off the historical estimates of all rallies had been; and second of all, a funny comment. I wasn't trying to say it meant that all anti-war protesters were evil, nor do I think I said so in the post. It was just funny that the ANSWER spokesman was trying to argue that his methodology was somehow preferable to counting the damn protesters. That's all.
Jim makes an interesting point about comparing apples to apples, and I think that's valid. It brings to mind something I was thinking as I read the article -- that almost certainly what we're witnessing is an inflationary trend in protest numbers. Both organizers and the police have incentives to bias their counts upward; the organizers for obvious reasons, and the police because they not only want to inflate the need from their services, but also fear the inevitable wrangling with pressure groups that comes from placing their estimates too low. The article demonstrates that the previous demonstration becomes the baseline for the next one, so any previous inflation will be built into all future estimates. Over time, the inflationary pressure may have resulted in wildly overestimating the number of people; for example, an actual count of the people in the plaza appears to have yielded a number slightly over half of what the police estimated the plaza held. Moreover, the police weren't even bound by their own estimates; early in the article they state that the plaza can't hold more than 43,000, and later state that an earlier protest had packed 50,000 into that space.
Posted by: Jane Galt on February 24, 2003 07:36 PMThis is just a meme that bothers me. I've seen it reported in several places that crowd estimates are inflated. Big deal.
The editorial that accompanies the article says it best:
"The high-resolution view from the aerial shots does not erase the bottom-line message of Sunday's protest: An impressively large and diverse group of Bay Area residents disagrees with the Bush Administration's rush to war."
Can we put this one to bed already? Yes, organizers inflate numbers and use shoddy methodology. Yes, Kangaroo Jack is a more compelling draw than ANSWER's rally.
It's just a cheap shot at the legitimacy of the anti-war movement from a crowd that has no right to criticize biased embellishment (see "our entire rationale for invading Iraq.")
Posted by: Thumper on February 24, 2003 08:12 PMYes, the Chronicle has long been noted for its pro-war bias and penchant for taking cheap shots at the left in pursuit of its conservative agenda.
Posted by: Jane Galt on February 24, 2003 08:17 PMGee, independent verification that reactionary stalinists lie?
Why should we be suprised? the Soviet Union choked to death on a surfeit of lies.
MonkeyPants
Imperial Falconer
I don't know if aerial photography is the best method or not, but I would trust its estimate a hell of a lot more than the estimate of the people organizing the event, who have a vested interest in inflating their numbers.
Protest "counts" have gotten so ridiculous, that whatever number I hear reported I automatically slash that number by at least 50%. By being less than truthful with their counts, protestors have now left themselves in the situation where they have to inflate their estimates, lest they tell the truth and have people mentally halve those figures.
Some can say it doesn't really matter, but protests are a battle in a PR war, and the number of supporters can influence that battle. So I think getting accurate counts are a relevant factor.
Posted by: Paul on February 24, 2003 08:57 PMThe reason it is a big deal is because inflating numbers has become the standard operating procedure. From protest marches to battle casualties to the size of the fish I caught last weekend. If the numbers aren't really important as many people keep trying to say ("Can we put this one to bed already? Yes, organizers inflate numbers and use shoddy methodology.") then why continue to inflate the numbers? Any method of counting protest marchers is going to have inaccuracies, but the numbers floated around after this rally were way outside of the normal range of +/- you would expect. And this large over-estimate is being given as the "real" number by the sponsers (and the police). If the numbers are being infated on purpose, then they are basically lying to us. If it is just because they don't have a better means of counting, then this article shows us a better more accurate means of doing the count. Shooting the messenger only works if you shoot him before the message gets out. It's too late for that.
That is why this story is important.
Posted by: fred on February 24, 2003 09:01 PMThe reason it is a big deal is because inflating numbers has become the standard operating procedure. From protest marches to battle casualties to the size of the fish I caught last weekend. If the numbers aren't really important as many people keep trying to say ("Can we put this one to bed already? Yes, organizers inflate numbers and use shoddy methodology.") then why continue to inflate the numbers? Any method of counting protest marchers is going to have inaccuracies, but the numbers floated around after this rally were way outside of the normal range of +/- you would expect. And this large over-estimate is being given as the "real" number by the sponsers (and the police). If the numbers are being infated on purpose, then they are basically lying to us. If it is just because they don't have a better means of counting, then this article shows us a better more accurate means of doing the count. Shooting the messenger only works if you shoot him before the message gets out. It's too late for that.
That is why this story is important.
Posted by: fred on February 24, 2003 09:01 PMOoops!! Either I double posted or I am seeing double. Maybe I should be a protest march attendance counter. (cheap shot, I'm sorry)
Sorry for double posting (and wasting bandwidth).
Fred
Posted by: fred on February 24, 2003 09:04 PM...It was just funny that the ANSWER spokesman was trying to argue that his methodology was somehow preferable to counting the damn protesters. That's all...
Absolutimundo Jane
"see what they believe"
I for one know better than to argue
with belief.
I mean really... Evidence over Faith
I don freekin thin so
For the record, I fully admit that Kangaroo Jack vs the rallies isn't a fair comparison. I just thought it was a funny line. Plus, the anti-war protesters seem to take themselves pretty seriously, which makes them extra-fun to tease.
I doubt Mr. Layne meant it as a serious comparison either, since he strikes me as a pretty intelligent sorta guy.
Posted by: Sean E on February 25, 2003 09:33 AMA more interesting numbers to debate:
(a) How many of those protesters were Iraqi?
(b) How many people next year; after we see the celebrations in Baghdad, and the mass graves, torture chambers and child prisons of the ba'ath party; will admit to having been at the protests?
Back in the early 1970s, I volunteered to be a proctor at an anti-Vietnam war march in San Francisco. After the march, I was assigned to watch the press tent in Golden Gate Park, a no-brain job that consisted mainly of just checking for credentials. At one point, a reporter for the San Francisco Chronicle came out of the tent and asked me how many people I thought had attended the march. I essentially knew nothing about organizing events or crowd estimates but the number 150,000 had been making the rounds, so that's the number I gave him. Damned if that wasn't what showed up in the paper the next day...
Posted by: Carl Maniscalco on February 26, 2003 01:16 PM"The blasted scale in my bathroom is no good. 20 pounds no good in fact." Ernie, it's a nationwide conspiracy! All the scales have been exaggerating my weight since I turned 31. (In 1984. Hummm...) And my trousers and belts are in cahoots, too.
Posted by: markm on February 26, 2003 09:58 PMComments are Closed.