In the age of transparency, reporters will shift from a puzzle-solving to a mystery-solving model of journalism.

Two of my favourite authors – Clay Shirky and Steven Johnson – recently published brilliant pieces on the future of the news industry. The below paragraphs, pulled from their respective pieces, hint at a shared theme both authors stop shy of making explicit: the impending death of journalism.

… in the long run, we’re going to look back at many facets of old media and realize that we were living in a desert disguised as a rain forest. Local news may be the best example of this. When people talk about the civic damage that a community suffers by losing its newspaper, one of the key things that people point to is the loss of local news coverage.... I adore the City section of the New York Times, but every Sunday when I pick it up, there are only three or four stories in the whole section that I find interesting or relevant to my life – out of probably twenty stories total. And yet every week in my neighborhood there are easily twenty stories that I would be interested in reading [about] ...

But of course, that’s what the web can do ... there are close to a thousand bloggers writing about Brooklyn: there are multiple blogs devoted to the Atlantic Yards real estate development; dozens following the Brooklyn foodie scene; music blogs, politics blogs, parenting blogs. [A veritable rain forest of information where there was once a desert.] --Steven B Johnson

When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to. --Clay Shirky

Both Shirky and Johnson’s pieces acknowledge that the trends hitting the news industry are hitting every part of society, but – because they have written articles and not books – they deal with the changes to the news industry ecosystem in isolation. As a result, their analyses account for the death of the newspaper in its current form. However, both shy away from explicitly looking over that bigger cliff – are we seeing the death of Journalism? I, for one, hope so, as it will mean a more profound change may be upon us.

Step back and look at the relationship between news gathering institutions and the organizations they report on. A large piece of “news” has been about one set of opaque institutions – the news organization – covertly gathering information on another set of opaque institutions – government, corporations or non-profits – so as to shine a light on some malfeasance.

To date much of the discussion about the death of newspapers and news media has focused on their business models, arguing that they are faulty. But what if it isn’t just the business model that is the problem? What if it is the underlying structure and values that are eroding, not only among news institutions, but also among the entities they normally cover? What if belief in objectivity and trust in opaque structures are dying? This would mean that the public’s confidence in products, ideas, services, policies and news created behind a curtain – within any opaque institutions – is slowly crumbling. In his Bertha Bassam lecture, this is precisely what David Weinberger brilliantly argues is already taking place:

“Wikipedia is far more credible because it shows us how the sausage is made. Yet this is exactly the stuff that the Britannica won't show us because they think it would make them look amateurish and take away from their credibility. But in fact transparency – which is what this is – is the new objectivity. We are not going to trust objectivity ... unless we can see the discussion that led to it.”

Such a transformation, a reshaping of credibility from objectivity to transparency, would have profound implications for every organization – corporate, non-profit and governmental – in our society.

The trends Shirky and Johnson describe as killing newspapers – the fact that there are more eyes, able to create more information, that is able to flow faster, and freer than ever before – may be making openness and transparency a strategically salient choice for an increasing number of organizations. Firstly, it is simply becoming harder and harder to keep secrets. More and more organizations may decide that, rather than devote energy to hiding secrets that will inevitably see the light of day, why not devote energy to solving the underlying problems that are creating them? More importantly, being transparent allows these organizations to access the long tail of analyses, an additional powerful incentive to being open. Those who share information and invite criticism and analysis may be better positioned to survive crises and challenges than those who don’t. As a result we may see an organizational ecosystem emerging that strongly favours transparency.

Such a shift would forever change journalism in several ways. The first is that opaque news entities – those that don’t make explicit editorial decisions or core assumptions and that fail to allow readers to hold them to account – will themselves be at risk. I suspect this will be true even if some magical financial solution (like the terrible idea of subsidizing news with an internet access tax) emerges. The problem would simply shift from being a financial crisis to a credibility crisis. If journalism prides itself on objectivity, then it had better find ways to be transparent. This means news sites had better engage with legitimate critics; and this means doing more than having columnists who ignore commenters that poke large holes in their arguments or electing to publish retractions on the bottom corner of page 8.

More profound, however, may be that journalism in a transparent ecosystem could look very different than it does today. If journalism has been about uncovering the dirty secrets within opaque institutions, what does it do if an increasing number of institutions are not opaque? What if there are fewer and fewer secrets?

I suspect the ideal of good journalism will shift from being, in Malcolm Gladwell's terms, puzzle solving to mystery solving. In the former you must find a critical piece of the puzzle – one that is hidden to you – in order to explain an event. This is the Woodward and Bernstein model of journalism – the current ideal. But in a transparent landscape where huge amounts of information about most organizations is being generated and shared, the critical role of the journalist will be that of mystery solving – figuring out how to analyze, synthesize and discover the mystery within the vast quantity of information. As Gladwell recounts, this was ironically the very type of journalism that brought down Enron (an organization that was open, albeit deeply flawed). All of the pieces that led to the story that "exposed" Enron were freely, voluntarily and happily provided in reports by the company itself. It's just a pity it didn't happen much, much sooner.

I for one would celebrate the rise of this mystery focused style of journalism. It has been sorely needed over the past few years. Indeed, the housing crisis that led to the current financial crisis is a perfect example of a case where we needed mystery solving, not puzzle solving, journalism. The fact that sub-prime mortgages were being sold and re-packaged was not a secret, what was lacking was enough people willing to analyze and write about this complex mystery and its dangerous implications.

Interestingly this is precisely what many blogs – alone or as part of an emergent network – already do. They take large complex stories, break them down and, by linking back and forth to one another, create a collective analysis that slowly allows the mystery to be decoded. I hope this post is part of such a mystery solving exercise – I’m trying to build off of, and extend, the brilliant analyses of Johnson and Shirky.

Does this mean the death of journalism? Well, in a world where everybody can be a journalist, is anyone a journalist? I don’t know. I’m sure there will always be some professional journalists, but in a world where people distrust opaque institutions, I'm not certain they will reside in organizations that look even remotely like the news institutions of today. Most importantly, in a world of mysteries, perhaps citizen journalists and bloggers, and their role in the news ecosystem, will be less frightening than the one most present-day pundits (especially newspaper columnists) would have us believe.