Story location: http://archive.pressthink.org/2007/10/09/what_i_learned.html
Jeff Jarvis has pulled together almost 200 people able to make progress in pro am journalism. David Cohn interviewed some of them (here’s the one with me.) They in turn asked us to pull together our own thoughts in advance of the summit at CUNY’s J-school, which is named for a related idea— networked journalism. (“It’s about action and next steps, not talk.”)
These are my thoughts prior to the conference, and after the conclusion of Assignment Zero, a collaboration with Wired.com. This post is also background for my session at the summit, thirty minutes on “NewAssignment.Net’s story.” (Conference schedule. )
In March 2007, when we launched Assignment Zero, I said it was trend reporting gone pro-am. That meant Wired and New Assignment asked for help from anyone within their editorial reach who was inclined to contribute pieces of a larger narrative. Open platform means just that: the door is open. Anyone can play who’s a qualified user of the site.
Open platforms don’t work like closed systems in news production, and we shouldn’t expect the two to be similar.
The larger narrative we picked to investigate was a trend story (and a Wired story): we wanted to track the spread of peer production and wisdom-of-the-crowd efforts across the social landscape, including the practice of crowdsourcing, which Wired had on its radar since a June 2006 essay by writer Jeff Howe.
To some critics, that was a geeky and overly meta story to start with. “Great project - crappy assignment,” wrote blogger and ex-reporter Tom Watson at Buzzmachine.
Where’s the news value? Where’s the impact on people’s daily lives? How does this fit into the current discussion of issues? Why so esoteric? Why a subject that will only attract cyber-geeks?
Others, like sumiteer Jeff Jarvis, thought that asking people to help report a big trend story is asking for way too much at the start. “I think they actually bit off a big bite for their first story, their assignment zero, because it’s more qualitative than quantitative, more about interviews and views than numbers and facts,” he wrote.
The numbers and facts approach says: ask each participant for a simple and clearly-stated “bit” of information. Then combine lots of bits—hundreds at first, thousands eventually—into a report that reveals something new. For how it looks in practice, see the Brian Lehrer Show’s Crowdsourcing Project: Are You Being Gouged? It maps grocery prices using commodity items (a six pack of Bud) to see where neighborhood differences may show up. Lehrer uses his midday radio program on WNYC to recruit partcipants and air the results.
I think both criticisms—geeky and self-referential story! simplify it for people if you want to succeed!—have a lot going for them. I accept their counsel.
“Start with clear, simple tasks,” says Assignment Zero contributor Derek Powazek in his own review. (See Did Assignment Zero work?) “This isn’t because the crowd can’t handle complicated ones - they can - it’s because they haven’t decided if it’s worth doing them for you yet.” And: “People won’t do what you say because you just told them to. You have to inspire them to want to participate.”
Both points are dead on.
I remember what Evan Hansen and I were thinking. (He’s the editor-in-chief of Wired.com.) We wanted to be in a situation where readers knew more than we did about the story we were investigating.
Because there were many participants in the open sources spreads out from software story who were within plausible reach of Wired, PressThink, and NewAssignment.Net, we thought there were many potential contributors to Assignment Zero who could furnish first hand accounts or serve as good sources. And because a good number of Wired.com readers already knew something about the open source idea and its roots in software, we figured they would be qualified to assist in a more detailed, updated and fully-rounded coverage of its spread to domains beyond tech.
In announcing its participation in Assignment Zero, Wired.com said it was testing an idea: “That a team of professionals, working with scores of citizen journalists, is capable of completing an investigative project of far greater scope than a team of two or three professionals ever could.” One way to find out is to attempt it. Assignment Zero was essentially that. (Some people got the point. Zero won honorable mention in the 2007 Knight-Batten awards for innovation in journalism.)
Here is what I wrote for Wired.com on July 9th of this year, after the project concluded:
I wouldn’t say it’s easy for widely scattered people working together voluntarily on the net to report on a big story unfolding in many places at once. But we know a lot more about it now than we did when we started, and one of the goals of Assignment Zero was to test whether pro-am methods had potential. I think they do, but we haven’t really unlocked it yet. We are, however, getting closer.
Which brings mean to these points, attempting to get closer still.
Here are the coordinates for the territory we need to be searching if we’re going to figure out how to do “pro-am journalism in the open style made possible by the web,” as my Wired essay put it. I found them by working with Evan Hansen, Amanda Michel, David Cohn, Tish Grier, Jeff Howe and hundreds of other contributors.
Get the division of labor right.
In order to succeed, networked reporting projects have to get the division of labor part right, and that means right-sizing the work in a way that works for the project, as well as the different types of participants who walk through the (open) door. Dividing up the work into tasks people can and will do is among the trickiest decisions the project will have. Expectations have to be extremely clear or a crowd will generate a limitless number of honest misunderstandings.
If pro-am reporting has an art it’s in breaking up the work into tasks doable by the contributors you actually have, in the time they have available. Since the division of labor in “open” reporting systems resembles not at all the system in a professional newsroom, denizens of the newsroom may not understand the challenge at first.
For some participants it’s, “I have two hours a week; tell me what to do.” Fitting into your scheme is fine for them. Others want an assignment where there is more creativity and control. Meeting different kinds of participants where they are permits the project to benefit from different kinds of contributions.
Grok their motivations and they may contribute.
When the gift economy is involved in editorial production the “right” division of labor starts with the motivations people have for freely participating and volunteering what they know. A well managed project correctly estimates what motivates people to join in, what the various rewards are for participants, and where the practical limits of their involvement lie.
Derek Powazek is right: “People won’t do what you say because you just told them to.” It sounds simple, but the difference between Powazek’s Law and a command-and-control system (like a newsroom) is profound and decisive. This is one reason amateur production will never replace the system of paid correspondents. It only springs to life when people are motivated enough to self-assign and follow through. Experience suggests that will happen spontaneously for a very limited range of stories.
Plan for sudden coordination costs.
If they are to succeed, networked reporting projects conducted on an open platform have to be ready with answers to the “sudden” coordination costs generated by the project’s success in drawing participants, who will inevitably have questions, problems, suggestions and demands. Assignment Zero went through this when more than 1,000 people signed up to help. These steering costs go up as the invitation to join in succeeds. Unless they can be absorbed at the edges, shared among contributors themselves, or eliminated by sheer elegance in project design, these costs will collect at the center.
That’s fine as long as the center is ready and budgeted for that. Frequently it isn’t ready, and hasn’t budgeted. A solution common in open source settings—super contributors share the costs of coordinating other contributors, so the project scales—is promising but more needs to be known about how it works in reporting projects. Assignment Zero did not crack that case.
At the other end are projects engineered to have extremely low coordination costs; participants get instructions online and they can send in their content or upload their data without much assistance. An example would be the contest method with cash prizes, which even the New York Times uses.
Where the solo pieces meet the larger puzzle confidence gets created.
At the moment of contribution, it has to be abundantly clear how the individual contributor’s piece of the puzzle is received, and how it fit into the larger narrative. If it is clear, the project has a chance of working. If it is not clear, people won’t find participation satisfying and they are unlikely to return. The strength of the connection between the “little” tasks and the big story is thus critical to success. The science of reception matters. What happens when participants upload their portion tells them whether they have wasted their time, or contributed to a public good. Designers: zero in here.
Shared background makes for an information foreground.
An invisible factor in the success of a networked journalism project is whether a broad group of participants coming into the project shares critical background knowledge that lets the story’s editorial signal stand out from the noise. A good example is TMP Muckraker’s plea to readers as the US Attorneys scandal reached its peak of intensity: TPM Needs YOU to Comb Through Thousands of Pages. (March 20, 2007.) This works only because TPM readers have followed closely along in the story by reading Josh Marshall and his fleet of contributors on the mysterious firing of United States attorneys.
The build-up of background knowledge among widely-scattered users makes them a far more potent force when called upon to help. Here the danger is in under-estimating what readers are capable of.
Communities that already coordinate can distribute reporting.
Networked journalism projects conducted by already existing online or offline communities—people brought together by shared beliefs or a common world of interests—are different in kind from “open call” projects that recruit participants from the online public or from a crowd of unorganized users.
Existing communities, people in already-formed networks, are strong on coordination and mobilization. This is one way distributed reporting might scale quickly: start with a community already able to divide up work, and run the reporting project over existing tracks, as it were. I haven’t investigated that possibility yet, but someone could.
Summary: hitting the coordinates
That is my attempt to map the perimeter: solutions lie within. Division of labor is the key creative decision in acts of distributed reporting. Grok the motivations or it can’t be done. Watch for ballooning coordination costs as ramp up succeeds. Where the small pieces meet the larger narrative the alchemy of the project lives. Shared background knowledge raises group capacity. Extant communities already coordinate well.
When the editors of Assignment Zero hit upon the idea of asking contributors to pick from a list of key sources and do a single interview, returning the results as a cleaned-up but otherwise raw Q and A, the division of labor clicked with participants and the “ask” worked.
People were motivated because it was interesting, challenging and fun to interview a mover and shaker in this story. They could get their minds around that task, and “see” it from start to finish. There were some coordination costs, but like the essay contest posted instructions worked well enough. Contributors understood that their individual interview was part of a larger coordinated act, interview week at Assignment Zero. The connection made sense to them. And by participating in Assignment Zero they had gained enough background to steer the interviews toward common themes without being told what to do.
The result: within days we had our target list assigned to contributors who went to work and got the interviews done. We were within the coordinates I just identified, and the distributed model came to life.
Finally, Assignment Zero’s published results and some earlier evaluations.
After Matter: Notes, reactions & links…
(May 2, 2008). Academics Jane Singer and Louise Thomas present their overview and summary of what’s known about crowdsourcing and what’s been done so far.
“Long Tail” author Chris Anderson says news providers should harness the collective power of their “engaged, smart, informed, opinionated readers” in order to succeed in the digital age.But how exactly can this be done? Is anyone already doing it? And what exactly are crowdsourcing techniques?
A very handy resource, with links.
Why do things like Assignment Zero? (one)
On Oct. 11th I was on the Brian Lehrer show (WNYC) discussing their crowdsourcing (or “group journalism”) experiment on grocery prices, Are you Being Gouged? You can listen here. Brian kindly said New Assignment.Net “inspired a lot of the thinking that went into this project.” He said they were going to keep at it: “The next thing we want to do is a group investigative journalism project… together with many of you.” He mentioned health care and health insurance as possible areas, and also asked for ideas from listeners.
Another guest, New York Times economics writer David Leonhardt, said, “I think this is exactly the kind of thing we should be using the Internet to do: to make information gathering in journalism a more collaborative and interactive process.”
Why do things like Assignment Zero? (two)
From Oliver Luft at Journalism.co.uk.
Trinity Mirror Regionals is to launch a crowd-sourcing pilot project at one of its Liverpool newspaper titles.Speaking at the Journalism Leaders Forum, at the University of Central Lancashire yesterday, Trinity Regionals editorial director, Neil Benson, told attendees the group was taking its first step into crowd sourcing with its ‘Making the News’ project.
Speaking over a telephone link Benson said the project had been inspired by US crowd sourcing site Assignment Zero.
Why do things like Assignment Zero? (three)
Dan Barkin of the News & Observer, in reflections on the summit for his readers, said that “beware sudden coordination costs” made sense to him:
When we asked for help with the speeding project, all of a sudden I had more than a hundred e-mail messages and phone calls from people who wanted to be involved. Contacting them all, editing their contributions — thanking them — took weeks. Their involvement was invaluable, but this was just a baby step in citizen journalism. Fortunately, Rosen’s project and his lessons learned give people such as me a great road map on how to do more ambitious projects with you.
Here’s video of me presenting the heart of this post—the coordinates—to the CUNY Networked Journalism conference. Courtesy of cybersoc, Robin Hamman’s blog.
Here’s a good list of highlights from the conference.
John Abell, a pro-volunteer in Assignment Zero’s pro-am mix, in the comments: “Newsrooms developed as monasteries not because reporters were anointed (though many believe they were and are) but because there wasn’t really any frictionless way to include the public in the reporting enterprise.” Now there is. His calls Assignment Zero an “attempt to use common communication tools to create ad hoc communities of reporting teams, with a twist: There was to be a strict time limit and a finite, pre-selected choice of sagas to pursue.”
Some notes from the morning of the Jarvis summit:
Jarah Euston, founder of Fresno Famous, on what happened when McClatchy bought her site: “The community felt like it had been sold.” She said the same thing has happened at Newsvine with its sale to MSNBC.
Clear from the morning panels that printed newspaper pages is still where the revenue is, even for “user generated content.” Dan Barkin of the News & Observer says that any newspaper that is not doing reverse publishing—content produced for the web, re-packaged for print—is missing a trick.
Robin Hamman of the BBC says that a lot of media companies, including his own, are very excited about social media; to learn about it, however, they have to find people who are not journalists to teach their own troops how social media works.
“We can wave our big email address and say send us stuff.” Hamman says the BBC has done that for big international stories and gets a flood of material, with significant costs for sorting and replying. He thinks a better model is for people to post their stuff on a sharing site like YouTube, tag it properly, and let the BBC know so it can point to the material.