Asymmetrical Information

From the desk of Mindles H. Dreck:

"Emergent Stupidity" or How a Bunch of Smarties Can Be Dumb as a Bag of Rocks

Rand Simberg's mischevious mind wandered in similar directions to my own when he read Malcolm Gladwell's latest article, The Talent Myth. Rand contrasts the intelligence of a committee or group as compared to its component individuals, demonstrating how groups of simple things can perform intelligently (ants, for instance), but groups of highly intelligent beings can underperform even the simplest individuals. Using the analogy of calculating cumulative electric resistance, he determines the net cumulative I.Q. of a government agency....

Assuming for simplicity that everyone in a government bureaucracy has the same I.Q....the net I.Q. will be that I.Q. divided by the number of agency employees.

If you add the number of lobbyists and interest groups to the mix, you can drive it down another order of magnitude in value, to the point that it has the intelligence of a lobotomized fern (only slightly smarter than Joe Biden).

I once saw a sign posted on a laboratory door that said "DANGER: 10,000,000 OHMS!". Silly me, I thought it was a just joke until I read Rand's article.

Dreck is no stranger to committees, unfortunately. If you are similarly afflicted, or simply a masochist, you may wish to read one or both of the two sections below. In the first, I look at the competing values that give rise to group-think, and in the second, I examine how these competing values operate in that bane of corporate life - the big-budget technology project.

Competition of Values
A committee's marginal tendency to achieve its mission is a function of the relative weights of four competing considerations:

consensus values;

group values;

individual values; and,

project or mission values.

Nominally, one forms a committee to perform a task. "Project" or "Mission" values should therefore supercede most other considerations. Committees are also formed in order to obtain buy in as much or more than to perform a task. Thus obtaining consensus may be a mission value. However, even if consensus is not a mission value, it often becomes the prime directive. Without resolution, usually achieved by consensus, the committee cannot complete its work. Reaching a solution on which all agree will be more important than the quality of that solution.

In addition to the function of bringing different expertise to the committee, committee members may function as representatives of their various groups (i.e. departments in a corporation, disciplines or "schools" in a scientific or academic committee, agencies in a government commitee). The more animosity or dysfunction between the groups, the more "group values" will intrude. Surely you have all been on a committee where the optimal solution circumvented, overburdened or otherwise offended a particular group. At that point the offended group becomes the sphincter in the committee G.I. track, shaping the conclusion through uncomfortable resistance.

Individual values are, by definition, unique to individuals. However, certain common human foibles emerge in a committee setting. For instance, there is always one member for whom the appearance of committee success is an important career objective. By definition, there will be another on the committee who wishes to see that person fail, for career competitive reasons. Some values are individual but common: Individuals typically like to see their imprint on a project and they generally prefer to play solitaire or even go to the dentist rather than sit in committee. Committee members often have an axe to grind. Whether to make themselves look good, make their dissatisfaction known, or just show the other committee members who's who, committee members have agendas. Individual values usually interfere with mission values. Unfortunately, the more intelligent the individuals, the more individual value baggage they tend to bring. Add a high-pressure environment to a lot of brainiacs and "One Flew Over The Cuckoo's Nest looks like a Quaker meeting.

Human tendencies lean towards avoiding not only work, but also conflict. The degree to which people avoid conflict is directly proportional to their physical proximity. Think about group behavior in an elevator, where people hardly interact as a way of avoiding conflict. Conference rooms can be only slightly less confining. This is true in cyberspace as well. Compare, for instance, how you speak directly to your blog detractors in email relative to your tone when you "fisk" someone with whom you have no connection. For a..um...professional example, look at Maureen Dowd's reaction to the editor of a magazine she panned (as recounted by Blow and cited in a book review by Russ Smith of The New York Sun):

The single most fascinating nugget in "American Son" is Mr. Blow's recollection of New York Times columnist Maureen Dowd panning the first issue of George (Blow was Senior Editor of George). Incensed, Mr. Blow wrote her a note and received the following response: "Don't be mad at me, I'm paid to be a baby curmudgeon, and it's no fun. I'd go back to reporting in a minute. I've subscribed and I promise only plugs from now on."

It's a gas to be a "baby curmudgeon" on the editorial page, but not so much fun when your victims materialize in person, and they turn out to be funny, affable and sensitive, or at least human. But if MoDo hated the magazine so much, why promise "only plugs from now on"? It sounds like her editorial mission just got subsumed by a need to make nice. Either that or she runs a men's hair replacement business on the side.

The point is that consensus values increase in importance when people are forced to interact directly. When you put people together in a committee, the premium on getting along skyrockets - except for the people who don't care about getting along, who gain tremendous power, but only temporarily. Sooner or later, those people are simply removed from power by an ex-committee action of the other committee members, or the committee begins to function behind their back. Being the committee asshole is what we call a high-beta strategy.

While this is all abstract, I suspect you recognize most of the behaviors above. One of the most measurable examples of "Emergent Stupidity" through competing committee values is:

The Big-Budget Technology Project

(DISCLAIMER: Nothing written below describes any project I've been involved with recently. It's all long ago and far away, and the individuals I'm thinking of no longer exist in this universe or any accessible parallel universe)

A large technology projects creates all kinds of hideous pressures. Substantial amounts of money are typically at stake, and management will penalize budget and time overflows. The project in question may involve mission-critical processing (such as re-routing factor workflow or electronic transactions) or it may effect how the entity is viewed by the outside world (a new website, for instance). Furthermore, several constituencies are involved.

Important technology project-specific values include the following: line executives and marketers want a say in how the system looks and feels and its key features, management needs to weigh costs vs. benefits and what additional risks the system may incur, and technology needs to be able to maintain and support it.

Group values include the following: Line executives and marketers want to minimize effort - they don't have a lot of time to spend on a system as they are measured by their production performance. Managers need to minimize surprises to senior management, and have little incentive to allow any new risks from a technology project. Technology professionals want to prioritize workflow to keep their in-house resources working and leverage the technologies with which they are already familiar. These group priorities are not at all likely to be consistent with mission values.

I'm extremely sympathetic to technology professionals. Day in and day out, they are asked to estimate the inestimable, build systems that do nothing in particular, and resolve the contrary phobias of the technologically-challenged. As a people-intensive function, technology professionals are hit hard in a lay off, and the pace of work is unpredictable, resulting in all-nighters, sudden calls during free-time, and just generally a lot of screaming and hurrying. There are few thanks when things go right, and when things go wrong, the line people have righteous hissy fits.

That being said, the still nascent science of "systems project management" has provided several coping mechanisms which allow the technology people to ride herd over committees. The first, of course, is the strict codification of the methodology itself:

1. A whole new set of people ("business analysts") who must be included on the committee, thereby at least doubling the number of technology professionals, increasing the sheer bulk of the IT department and lending tremendous new power to that group. Business analysts don't code, and they aren't users - they are the methodology police.
2. Together with the methodology police, users must complete and sign off on a "Requirements Document" (or "RD"). In effect, the RD is an attempt to create a contract between users and technology describing in the most minute detail the functioning of the system prior to commencing any development. Since the line professionals have neither any idea what can and can't be done with computers, nor any clue as to what it might cost, nor the time or training to produce a 200 page systems specification, this usually stops a large project dead in its tracks for months. Finally, after the project has been gathering dust in the RD phase for a while, the CEO will say "do it at all costs" and all are released from the responsibility of delivering it within budget limits. Whatever goes wrong after this is, by definition, nobody's fault, or everybody's, or just the CEO's. This is committe nirvana- every individual on the committee is served at the expense of the mission.
3. If the project somehow gets beyond the requirements phase without senior management absolution, the project tracking process is invoked. This involves producing an inscrutable project timeline ("GANTT", "PERT" or some variant) that shows the simultaneous chronologies of various initiatives. These can include such crystal clear sub-projects as "eval. of prelim. UAT - test debug", with a pretty colored bar covering several weeks. The civilians on the committee might as well be reading heiroglyphics. Once they get used to these strange colored markings they marvel at how the individual parts of a project are all on time, but the overall project has been extended six months! Truly, the project is more than the sum of its parts, even as the committee is not.
4. Exceptions and enhancements come next. If they are very lucky, the non-technology people involved in the project will now somehow get a sense of how the final deliverable might look. They will suddenly realize it isn't quite right, or that their needs have evolved during the two years the project has been germinating. Worse still, some interoperative system has been replaced, and the system must be redesigned to work with it. Technology must now estimate the costs of each of these exceptions. We discover that the "help" buttons the marketers want cost $10 million and an extra seven months to implement, while the change to a completely different underlying hardware platform (that the new senior project manager happens to be familiar with) is mandatory but inexpensive, incurring only contingent delays to another project.
5. The line people finally fight back. They hire their own commitee padders - the technology liaisons (if they have already been through this process, they are already involved). Now the line has personnel on their own team fluent in RD negotiation and at ease with project methodology. They are able to assert their knowledge of "how things are done" right back at the technology group. Group and individual values become too numerous to count at this point, as the liaisons and analysts compete to demonstrate expertise, have unproductive arguments about their pet technologies and how the "used to do it at their former employer" and generally behave like rats in a garbage can.
6. This is typically when the consultants are called in...

Once all the competing agendas have been beaten to death, the committee starts to suffer fatigue. The project has gone on indefinitely, resources are expended and line attention has moved on out of sheer frustration. Or, perhaps, the line has adapted around the project requirements, typically through an informal system design by an individual working as a stop-gap without a committee. If the project remains critical, the aforementioned "senior management absolution" may be forthcoming, allowing the project management to cease and the project itself to begin, with most of the work concentrated in a small group of developers and interested parties unbeholden to GANTT charts.

Although not a huge technology project, builfing a corporate website for a multi-line company is a good example of a committee assignment from hell. Virtually every group in the company needs a say in the website. God forbid users on the net should see one particular service before the others. God forbid the legal and compliance police should not vet every single word on the website and make sure it poses no legal problem in every single jurisdiction around the world. As for appearance, everyone has an opinion. Furthermore, from day to day it has been very difficult to conceive of all the things the company might want to do on the internet, let alone just how they might be done. The committee is not going to start with a website that works for just one constituency and then move on to the next because then they would have to choose one group over the others. That would be a violation of group and consensus values.

Of course, that might be the best way to go. I have experience programming and writing for marketing. I know that most innovations come with trial and error. If you want to design a good program, you create a prototype of some sort and evaluate it to produce successive more effective versions. When you write for an audience, you get your thoughts down on paper, seek feedback and revise extensively. It's very hard (impossible?) to get it right on the first try. The first version will be far from optimal. Yet the right outcome is extremely difficult to imagine without producing an outcome. Success comes from successive drafts, seeking input from others and individual perspiration. The dominant committee values of a large technology project are entirely antithetical to this process.

No wonder, then, that these technology projects tend to blow up - producing either terrible outcomes or no outcome at all. And no wonder so many corporate websites look like shit and produce no traffic, thereby immeasurably enhancing the reputation of technology in the minds of technophobes in positions of influence.

UPDATE: A colleague writes, rather pessimistically:

All software projects I have seen fall into two camps;

• Run by geeks - All the time spent on coding w/o any direction. A lot of nice codes, but no purpose. The project never ends or delivers


• Run by suits - All the time spent on requirements w/o any coding. A lot of documentation, but not product. The project never ends or delivers

It is a lose-lose situation. We would always chose a model that didn't work.

Posted by Mindles H. Dreck at July 28, 2002 04:06 PM | TrackBack | Technorati inbound links