Why Cameron's open data Big Society will drive hacks to finally face up to technology

('Today We're Social' cartoon by Oliver Wibber - Geek and Poke)

On Friday the iPad finally arrived in the UK. If the frenzy of excitement among journalists on Twitter on the day of Apple's product announcement, and the number of hacks suffering from iPad-delivery anxiety last Thursday are anything to go by, it is possible the entire UK media community now owns Apple's new wonder gadget.

As if to justify their own purchases, a series of articles on "Why every journalist must have an iPad" is bound to follow, just as there have been articles written on why every journalist should own this phone, that camera, and the other recording machine. But must they really?

I am not one of those rushing to get my paws on what Apple-sceptic Charlie Booker called "the world's most expensive rectangle" in a Guardian column, but I have recently started acquiring a collection of other electronic appliances myself with the pretext that "I need them to develop my journalism work".

I do not randomly collect gadgets. I buy them as and if the need arises. The need is a very 21st century one, with tools helping journalists work more efficiently, be more mobile and flexible. But owning a piece of hardware, no matter how cutting edge, is only a first step towards making a tool effectively work for you. An electronic device alone will not write a story, let alone research it or fact check it on your behalf.

Journalism vs. technology
The first News:Rewired conference organised by Journalism.co.uk in January 2010, was an excellent eye-opener to what lies ahead for journalists in the digital age, as more and more tools and skills become necessary to keep up with the speed and complexity of information available.

Open University lecturer Tony Hirst (@pyschemedia), gave a fascinating presentation on data mashing (a summary is available on the News:Rewired site), which caused a collective brain meltdown among the audience of journalists, as he started demonstrating how to manipulate and analyse data using tools such as YahooPipes, Google Fusion Tables, ManyEyesWikifield, and several more. You name it, I hadn't heard of it.

The new  coalition government has since pledged to introduce radical changes in the public's 'right to data', making large volumes of government data accessible as part of their Big Society proposals.

From gatekeeper to masher
In the Media Guardian last Monday, Chicago-based web developer Adrian Holovaty was quoted saying that even though journalists may lose privileged status as gatekeepers of information because of the upcoming torrent of data availability, the more of it there is, "the more essential it is for somebody to make sense of it."

In other words, one of the key functions of journalists is about to shift from being fact reporters to that of also acting as data mashers and analysers. This could entail a hack being able to do more than simple sums and subtractions in Excel, being familiar with CAR (not an automobile but "computer assisted reporting"), knowing the difference between bag pipes and Yahoo Pipes, plus a smattering of coding knowledge thrown in for good measure.

Taking into account that government data will start being released in matter of weeks, not years, I wondered aloud on Twitter how many hacks actually have such skills today. The good-humoured tweet I received in response from James Ball of the Bureau of Investigative Journalism is not that far from the truth:

If geekery is not your cuppa
With so many  journalists coming from a humanities background and not exactly with a mathematician's nor a data analyst's head on their shoulders, this knowledge gap could prove to be a bit of a challenge in times to come.

The new generation of young journalists, on the other hand, may revel in this type of environment. Students of Paul Bradshaw of the Birmingham City University or of Murray Dick (former BBC trainer of online research at the BBC) at Brunel University, for instance, will have been trained in database journalism and are unlikely to shy away from experimenting with data scraping.

We could say all journalism is investigative. Yet if success in 21st century journalism is to be determined by how able you are to fish selectively from a sea of raw data, those who have been specifically trained in investigative technology, rather than techniques alone, are likely to fare better in the long run.

If the continuing recession further increases competition in the job market, will media organisations fill their rare vacancy with an old school hack with top-speed shorthand and traditional reporting skills or a freshly graduated young multimedia journalist, for whom social media tools, data mashup and most online technology may be second nature? Whereas one will have the experience and wisdom youth cannot buy, the other may have the resilience that brains hardened by experience may find awkward to handle.

Age matters
I was delighted to come across a blog by Andrew Brightwell, one of Professor Bradshaw's MA students in Online Journalism, describing how Tony Hirst's teachings had inspired him to use GoogleDocs to analyse swimming pool opening times in Birmingham. The same Tony Hirst, who had me dumbstruck with the complexities of the data mashing world had not intimidated this young journalist in the least but filled him with genuine "joy", as he himself admits in his post.

Those of us who are well past our school days, or not particularly technically-minded, however, may need a helping hand in sorting through and breaking down such great volumes of information. How else are we supposed to logically explain data to our readers, if we cannot ourselves make sense of it, analyse it and present it in small digestible pieces, maybe even with the aid of visuals?

(Photo: Infographics by The Guardian by tripu)

Back to school
Course providers may see a golden business opportunity there to cash in on crash courses for techno-phobe journalists, or those who may hate Excel. Conferences such as News:Rewired, geared towards digital journalists, are likely to become even more important as a place for acquiring technical know-how while networking with those who may have skills and knowledge you need but do not yet have.

While we wait for more specialised training opportunities to become available, some hacks have taken the initiative to spontaneously pursue coding lessons in a friendly, relaxed environment.

Two (or more) heads are better than one 
A group of journalists led by Joanna Geary, web development editor of The Times, and James Ball, started meeting up monthly in a London pub to learn a programming language called Ruby – over a few beers of course. You can follow their conversations under the hashtag #rubyinthepub or #ritp on Twitter.

Judith Townend, of Journalism.co.uk, went along to one of the Ruby evenings and wrote a report of her impressions here. Her general view was that whether the journalists actually managed to learn and digest any of the coding taught by the programmers, it was most significant that the Ruby In The Pub meetups opened up dialogue between the two parties.

The lesson might be that, if you are not nerdy enough to be a geek, you might as well have a geek as one of your friends.

Community
In the US technologists already collaborate with journalists within digital communities, such as Hacks/Hackers, aimed at helping "apply technology lessons to journalism".

The community even has a useful Question and Answer forum (image below) for media technology issues, which I believe might soon turn into a type of wikipedia for journalists struggling to keep up with the technology.

OWNI has published an interview with Hacks/Hacker's co-founder Burt Herman, which illustrates the spirit of collaboration among the founders.

In a dog-eat-dog world that journalism can sometimes be, partnership and teamwork risk being overlooked in favour of solo performances and a single place in the sun. But when outcomes are best achieved through conquering technological challenges not everyone may be comfortable with, forming specialist communities to respond to those emerging needs, to exchange and share knowledge, seems like a no-brainer. The age of journalists living in ivory towers is long gone.

United we stand
Friendship is not a bad place to start for geek wannabies either. Over a conversation about multimedia skills necessary for the modern journalist, Sarah Booker, of the Worthing Herald, and I discovered a common desire to learn how to podcast and are planning to get together in a few weeks' time to trial out our first joint podcasting session.

Once we become confident, it is likely we will each go off and do our own, but it is very comforting to be able to work with another journalist on such collaborative terms. It reminds me of how much fun I used to have as a child going over to a friend's house to do homework together. It may not be Ruby, but it still feels like a gem.

As the data dams open up and government figures come flooding in, it remains to be seen whether journalists, who have so long fought for their release, will die of ecstasy or simply stand frozen in sheer bewilderment. Whatever happens, if the data release will be as large in scale as the government is promising it to be, investigative journalism will  never be the same again.

Will hacks be able to hack it?

It might not be too soon to start joining hands, brains and gadgets, and order some more beer at the bar.