By MICHAEL CIEPLYJULY 6, 2014
The documentary film "The Square," about the Egyptian uprising in Tahrir Square, Cairo. On an impact scale of 100, the film scored an average of 92. Credit Noujaim Films
LOS ANGELES — You watched the wrenching documentary. You posted your outrage on Twitter. But are you good for more than a few easy keystrokes of hashtag activism?
Participant Media and some powerful partners need to know.
For the last year Participant, an activist entertainment company that delivers movies with a message, has been quietly working with the Bill & Melinda Gates Foundation, the Knight Foundation and the University of Southern California’s Annenberg School for Communication and Journalism to answer a question vexing those who would use media to change the world.
That is, what actually gets people moving? Do grant-supported media projects incite change, or are they simply an expensive way of preaching to the choir?
Ultimately, the answers may help determine which projects get financed, which formats are favored and how stories are structured. That could be true for so-called double bottom line companies like Participant, which seek to profit (or at least break even) while creating social change, and also for nonprofits like the Gates Foundation, which increasingly rely on entertainment-style media (like the education documentary “Waiting for ‘Superman’ ”) to drive an agenda.
"The Square" scored extremely high for emotional involvement at 97 out of 100, but dropped to 87 in terms of provoking action. Credit Netflix/Noujaim Films
More immediate, those behind the effort say, new measures of social impact will enable sharper focus and rapid course corrections in what have often been guesswork campaigns to convert films into effective motivational weaponry. That approach would apply to a hit like the movie “Lincoln,” which counseled civic engagement, or to a box-office miss like the antifracking film “Promised Land.” Both were Participant-backed films.
To get the answers it wants, Participant is developing a measuring tool that it calls the Participant Index, assisted in the effort by the Annenberg school’s Media Impact Project. In rough parallel to the Nielsen television ratings, the still-evolving index compiles raw audience numbers for issue-driven narrative films, documentaries, television programs and online short videos, along with measures of conventional and social media activity, including Twitter and Facebook presence.
The two measures are then matched with the results of an online survey, about 25 minutes long, that asks as many as 350 viewers of each project an escalating set of questions about their emotional response and level of engagement.
Did it affect you emotionally? Did you share information about it? Did you boycott a product or company? Did it change your life?
“If this existed, we would not be doing it,” said James G. Berk, chief executive of Participant. “We desperately need more and more information, to figure out if what we were doing is actually working.”
The answers result in a score that combines separate emotional and behavioral measures. On a scale of 100, for instance, “The Square,” a documentary about Egyptian political upheaval that was included in Participant’s first echelon of 35 indexed titles this year, scored extremely high for emotional involvement, with a 97, but lower in terms of provoking action, with an 87, for a combined average of 92.
By contrast, “Farmed and Dangerous,” a comic web series about industrial agriculture, hit 99 on the action scale, as respondents said, for instance, that they had bought or shunned a product, and 94 for emotion, for an average of 97. That marked it as having potentially higher impact than “The Square” among those who saw it.
The documentary "The Cove," which looks closely at dolphin killing in Japan, had worldwide ticket sales of just $1.2 million after its release in 2009. Yet it has repeatedly led to campaigns to protect the Japanese dolphins. Credit Oceanic Preservation Society/Roadside Attractions
Daniel Green, the deputy director for strategic media partnerships at the Gates Foundation, traces the new drive for impact measurement to a Seattle meeting in December 2011 among about two dozen representatives of nonprofits with an interest in social change.
“Grantors didn’t have a lot of sophistication around their analytics,” said Michael Maness of the Knight Journalism and Media Innovation program, a group that attended. He joined Mr. Green last month in describing frustration among nonprofits at their inability to gauge how much change their projects are prompting.
The Seattle gathering led to an association with the Annenberg school’s Norman Lear Center, which early last year established its Media Impact Project. That project then served as a consultant to Participant in creating its index, which received $4.2 million in combined financing from the Knight and Gates foundations and from George Soros’s Open Society Foundations.
The methodologies being used for the index will be provided on an open-source basis to those who are interested — whether on the left or right or in the center of the ideological spectrum.
“We’re developing a set of tools and measures that will be available for any researcher, no matter what their viewpoint,” said Martin Kaplan, director of the Lear Center.
Participant, created in 2004 by the eBay co-founder Jeffrey S. Skoll, is using that methodology to build a proprietary database. It will feature three echelons with 35 projects each, or about 100 distinct bits of media, annually.
The company will lean heavily toward films and television shows of its own, especially those carried on its activism-driven online and pay-television network, Pivot. But it will also index properties for partners, like the Gates and Kaiser Family foundations, and for companies or others who will pay a fee.
Photo
Participant was created in 2004 by the eBay co-founder Jeffrey S. Skoll, left, pictured here with James G. Berk, chief executive. Credit Emily Berl for The New York Times
(Prices have not been set, Mr. Berk said, but he expects to serve nonprofits at cost. He declined to say how much Participant has invested in the index.)
In an inaugural general survey, which polled 1,055 of its viewers in March and April of this year, Chad Boettcher, Participant’s executive vice president for social action, and Caty Borum Chattoo, a researcher and communications professor at American University, found some perhaps surprising results.
Even among the presumably progressive Participant audience, crime ranked near the top of the list of 40 primary concerns. It was cited by 73 percent of respondents as an important social issue, placing it just behind human rights, health care and education.
Gay rights, female empowerment and prison sentencing reform, by contrast, ranked near the bottom of the list, while climate change was stuck in the middle, a concern among 59 percent of respondents. Digital intellectual property issues, at 38 percent, brought up the rear.
Stories about animal rights and food production, it turned out, were the most likely to provoke individual action. But tales about economic inequality — not so much.
Over all, said Marc Karzen, a social media entrepreneur whose company, RelishMix, advises film and television marketers, Participant will most likely affirm what is becoming clear to conventional film studios: Impact can be less about persuasion than nudging an audience to go where it is already pointed.
“You have to embrace your fans, not shout at them,” Mr. Karzen said. “They need to be inspired to spread the word.”
One of the weirdest problems in measuring social impact, and one still unresolved, Mr. Boettcher said, is the paradox of “The Cove.”
That documentary, which looks closely at dolphin killing in Japan, had worldwide ticket sales of just $1.2 million after its release in 2009. Yet it has repeatedly led to campaigns to protect the Japanese dolphins, Mr. Boettcher notes, particularly among activists who are aware of the film but will not watch (and hence, would not be counted under the current methodology of the index) because of its gory content.
“They don’t want to see it,” Mr. Boettcher said, “but they will sign up.”
A version of this article appears in print on July 7, 2014, on page B1 of the New York edition with the headline: Divining Why One Film Spurs Activism, While Others Falter. Order ReprintsToday's PaperSubscribe
