Teaching journalists to code is never easy and impossible in one day, but I wanted the participants to get a flavour of what is possible and where the journalistic pitfalls are. Being a new sphere for innovation in media, data journalism comes with its challenges. Luckily I only had one person run away after the “coding bit”. Others took up the baton and started scraping the very next day. One such person is Arno Kersten who works for the Dutch-Flemish Association of Investigative Journalists and publishes the ddj website Medialab.

He interviewed me after the workshop, adding me to a prestigious list of data journalism folk such as Paul Bradshaw, Geoff McGhee and Andrew Lehren. Find me and many more here or watch the video:

I would also like to thank the Data Science Meetup group in Utrecht for the bottle of wine and Goof van Winkel for organising the workshop.

“You have no excuse not to be making things now. You don’t need funding. You don’t need to be put in the newsroom. Start with the basics, expand it, and make something fun as a prototype.”

I was always told by developers that you will only learn through making. Which is why I believe so many developers are not formerly trained in computer science. My fellows are not. I would liken it to the gap between ‘studying’ journalism in college and what you learn by being part of a newsroom. The former is where you get the basics, the latter where you build your craft.

I have applied to the Knight Foundation Prototype Fund to build a platform and will go to Kickstarter if that fails. And if that fails I will just build in my spare time. Entrepreneurism and start-up life is about, as my fellow fellow Stijn described his journey, a series of failures. I am not afraid to fail because I will just learn and build it myself. Even if I succeed with Knight I plan on having a Kickstarter project this year. I have also applied for funding for a big data driven investigative story. I am not looking to build a body of work but a body of knowledge. You gain knowledge from trying and failing.

When I started an internship at CNN, all those many years ago, the first thing Richard Quest asked all of us fresh-faced interns to do was pitch a story. Here I learnt there was an art to pitching and a successful pitcher became a successful journalist. The beauty of digital journalism is that you don’t need to be in a bustling newsroom to pitch. You can pitch to your audience. You can pitch for funding. You can build your newsroom online and gauge the success.

In that sense, you have no excuse not to be pitching now. Not to be having ideas. You don’t need to be in a newsroom. Build a platform. I have recently bootstrapped my WordPress theme. I installed this free theme and modified the code base. Tinker with products, tinker with code and tinker with the stories you can produce.

I’ll let you know how my funding and project go. In the meantime I will be on a panel at the Polis Journalism Conference speaking about trust in data journalism on 5th April. Sign up here, it’s free!

Don’t use it for what it’s made for, understand its functionality and use that

I said this over a year ago when I was first using social media tools to gather, filter and verify news. I’m sure it is one of the reasons The Guardian chose me to be its 2012 Knight-Mozilla OpenNews Fellow. And I stand by those words just as strongly today.

Gone are the days when a hugely popular blog and thousands of Twitter followers would pave your way into the newsroom. They now allow you to be part of the brand, at the peripheral. You could blog as part of a news organisation, working freelance and communicating every now and again with the editor of that section. Your battle to shape the news, pitch stories and drive the agenda is made more difficult by the “new media” silo you find yourself in.

Blogging is a new breed of the medium. It is the same species. It functions pretty much the same. The key to extending the news sphere is cross breeding. Find a species bred for an entirely different purpose, understand its functionality and use that. For instance, I came across this graphic by The New York Times’ graphics editor and resident statistician, Amanda Cox:

As you can tell from the topic, this was published in 2008, long before data journalism became such a trend. I love this because something like this cannot be fathomed by a journalist. Amanda Cox sees data but she also sees a tool for analysis and can shed light on what factors affected whether Obama or Clinton won a state. I like this as well because I have leant to do it! (from a free online data analysis course run by Johns Hopkins University). Using R:

setwd("directory_your_data_is_in") # you need to set your working directory to where your data file is located

my_data <- read.table("name_of_data_file", header = , sep =, quote = , dec = ,  ...) # for instance if you have a csv file with the first row being the column names, entries surrounded by " if they contain "," and "." as a decimal point then header=TRUE, sep=",", quote="\"", dec=".". Or you can just use read.csv()

install.packages("tree") # installing the tree package so you can access its functions
library(tree) # loading the functions into R

# To predict, say which Candidate (column name) wins from all the other columns in the dataset
predict_tree <- tree(Candidate ~ ., my_data)

# To print out the prediction tree
plot(predict_tree)
text(predict_tree)

# I think the data in the NYT example needed a lot of manipulation, turning numerical 1's and 0's into factors for example. Also finding the percentage splits on things like education involves looking at that subset as its own branch

Rather than being creative with the medium why don’t we start thinking outside of the box in terms of the message. Analytical journalism is something Twitter, Facebook and Google can’t compete with. They can be faster, they can be more diverse, they can be more averting. But your friends and followers are not going to take the time and effort to analyse raw facts. That is where the value of new journalism lies.

Linear regression could be used, for example, to see which more strongly predicts levels of school truancy, social or educational factors. But thinking outside the box needn’t be so demanding as a university course in data analysis. Instead of a static graphic or even an exploratory interactive, why not make an animation. With HTML5 video, 4G networks and Popcorn.js to tie video to web elements upon us already, web is fast becoming a playing field for broadcast journalism. Here is a brilliant example of building a data narrative through animation:

For those who play with code, why not build robots to help you report? You can build a web scraper to monitor a data source and tweet/email you if certain conditions are met. The possibilities are endless.

If you are interested in news and journalism look outside the reporting sphere, gain analytical skills and ask yourself “how does it function?” and use that.

Data journalism has sprawled into an all-encompassing term meaning new digital journalism excluding social media/community. In other words, news generated from a newsroom which doesn’t solely consist of text print. However, there is a wide variety of fields interwoven and interrelated into the term data journalism.

So when asked about getting into data journalism, one has to clarify which branch.  The first split from the trunk has to be the broader areas of investigative data journalsim and data visualisation. As newsrooms moved from print to digital, few of them had the chance or expertise to experiment and so the whole tree fell under the data journalism canopy. Thus they were made of fullstack developers (most were designers and developers with an editorial leader, such as the NewYork Times team run by Aron Pilhofer).

Now data journalism is reaching its tipping point. A data journalist can no longer dapple with Excel, Google Fusion tables, HTML and JavaScript and expect to be in the game. I was not lucky enough to be able to attend this year’s National Institute for Computer Assisted Reporting (NICAR) conference, #NICAR13, but following the twitter feed has revealed this shift.

Seeing resumes of people who can do a little of everything, but not having specialty or small set of specialities worries @pilhofer #NICAR13

— Greg Linch (@greglinch) March 3, 2013

I specialise in the backend of data journalism: investigations. I work to be the primary source of a story, having found it in data. As such my skills lean less towards design and JavaScript and more towards scraping, databases and statistics.

What I Need To Do It

There is no clear way to get into data journalism. Firstly you must decide what area of this wide field you are interested in. You must find where your passion and curiosity lies. You can never get to a level of training where you feel you can satisfactorily do your job. That’s the difference between journalism and data journalism. The web evolves in 3 month cycles and so you are always playing catch up, whether that’s retrieving data or making the latest unique interactive.

The most important thing you need is drive. Because it is hard. I have learnt an object oriented programming language primarily for scraping, Python, a database language MySQL (ElasticSearch if the data is very large) and an analysis software language, R. I want to get at a story, I want to find an interesting pattern, a tantalising trail.  For that, I need to be able to get, clean, sort and analyse data in order to understand it.

The problem of “what skills do you need to know?” depends on what you want to do, says @markhorvit. Understanding data is big part. #NICAR13

— Greg Linch (@greglinch) March 3, 2013

But it’s not just the the skills that are an important part of the journey. That’s just what you need to get ready. It’s what you learn along the way that is your greatest asset.

How I Do It

I work in a virtual world. Literally. The only software I have installed on my machine are VirtualBox and Vagrant. I create a virtual machine inside my machine. I have blueprints for many virtual machines. Each machine has a different function i.e. a different piece of software installed. So to perform a function such as fetching the data or cleaning it or analysing it, I have a brand new environment which can be recreated on any computer.

I call these environments “Infinite Interns“. In order to help journalists see the possibilities of what I do, I tell then to think about what they could accomplish if they had an infinite amount of interns. Because that’s what code is.  Here are a couple of slides about my Infinite Interns system:

To use this system I have another repository called Skel. This is the skeleton layout for all my data driven projects. Every time I start a new project I download Skel. This also has folders for the various components of my investigation, the source data, the transformed data, the analysis, etc. With each intern I bring to life I transform the data and put the results on my own machine. After that I kill it. A benefit of having virtual interns is that no labour laws apply!

All my processes are coded, as are my environments, so I can write a script that will automatically run the investigation from start to finish in one command. In that way it is completely transparent and reproducible.

Why I Do It

It has taken me a long time to develop this process. It’s built on what I’ve come to learn as an OpenNews Fellow at The Guardian. I use code to do journalism, to find and report stories. Thus my process of development is not built around agile or responsiveness or all the other ways developer teams are run. My process is built around transparency and reproducibility. It is meant to stand up in court.

I practice this on every scale. For instance, I have a three step process when I scrape. For a large enough dataset, there will be a search function on a webpage which will pull out sections of the database. I write a scraper to pull out every section, paginate through the entries and store the URLs for each of the individual entries. Then I write another scraper to go to all the URLs and store the HTML for the page. All of it. Only after I have done that do I write a third and final scraper to collect all the necessary details and store it in a structured database.

Having recently taught this process to a group of developers at Al Jazeera English, one of them asked me why I have this second interim step. Surely it wastes time and computing space. You need it just as a journalist needs their notes. When you scrape a website you are fetching information from a site that has fetched it from a server. The entity you are investigating has access to that server and can change that information. It will change on the site and what proof do you have that you did not fabricate the data or retrieve it incorrectly. If you have the HTML you have proof of what the website retrieved at that point in time.

I am a journalist by training and am learning developer skills. But I am creating my own processes based on what I need as a journalist not as a developer. It is an ever evolving process but that insight is the result of two years delving into the world of data journalism. As long as you have an insight of your own, a unique way of getting done what you need to get done, then you are a data journalist.

Late last year there was a call out by Alex Howard from O’Reilly Radar. In it he asks:

…how many data journalists are working today? How many will be needed? What are the primary tools they rely upon now? What will they need in 2013? Who are the leaders or primary drivers in the area? What are the most notable projects? What organizations are embracing data journalism, and why?

I spent last year as a Knight-Mozilla OpenNews Fellow at The Guardian, working within the Interactive Team, in the hope of getting where a journalist in the age of big data needs to go. It was not just about learning to code, which I started a year pervious at ScraperWiki, but understanding the digital story cycle, the editorial decisions, and the internal politics. Most importantly, my journey was and still is the road less travelled by. I had to completely retrain myself. With no syllabus, no certificate and no notion of what would be needed to don the robes of the data journalist. So here is my two cents.

How many data journalists are working today?

That depends on your definition of a data journalist. After the dawning of the era of the citizen journalist we no longer have a clear definition of journalist! This is a moot point but I would say it is more of a legal point. Rather than egotistically heralding the bastions of truthfulness, impartiality and objectivity; I would say a journalist is someone who can argue a case for public interest defence in court. A data journalist is therefore anyone who can argue a case for public interest if he/she is brought to court upon producing content for the web.

Typically a journalist could argue a case for public interest upon publication and these were traditionally of the form of print, radio and TV. And traditionally , the process toward publication began with collecting documents and documenting the process which transformed these documents into its resultant mediated format. Now, documents are data. We live in a digitized world. Therefore all journalists are becoming data journalists. It’s just journalism adapting to new media going out and new media coming in.

To answer the question (whilst not really answering), the number of data journalists today are the number of journalists working in an organization which will survive the digital transition and find a stable business model, and who will not be fired before these things happen. The number of data driven investigative journalists will be one or two per survivor successful enough to afford specialist desks. I make this distinction because all documents are data and all journalism is built atop documents, however journalism in its simplest form involves mediating already processed data whereas data specialists will be processing and analyzing raw data in order to get to the stories within.

How many will be needed?

That depends on who survives the digital cull. Journalists will need produce for the web first. Once news organizations settle on a content management system I believe there would be very little computer skills needed. Training to use the CMS would be more than adequate. I believe that the big stories, the special features and editorials are going to need that extra pizzaz. As such, small investigations coupled with video, graphic and/or interactive features will become desirable, especially amongst those editors who want to boast at conferences.

To create a long form digital piece you of course need a multi-skilled team. For these teams to work efficiently requires each member to be familiar with all the processes and to have a vast range of skills. So a most specialized data journalist with coding skills will be desirable but only at a select few organizations. But I believe supply will be very low as few journalists are interested in this route and even few institutions teach computer-assisted-reporting to an adequate standard for the job requirements.

What are the primary tools they rely upon now?

Currently there is no standard and tools vary according to institutions. The very basics are spreadsheets and fusions tables. Web standards are HTML and CSS. The more data intensive (and ambitious) interactive desks work with R. This is a legacy of training as statisticians and data scientists coming out of University should be R trained. Also R is free. Assume everything they use will need to be free. One object oriented programming language, Ruby or Python, is used. Some say either is fine but others such as ProPublica state they are Ruby houses. If a team dictates which one will be used as standard, this is most probably a good sign. It means they are sharing code and plan on reusing it.

The trend I think you will see this year and for years to come is an evolution towards JavaScript and HTML5 houses. These will be the hard skills looked for. Simply, everything will more towards being browser native so that there can be quality control across all devices. Responsive design rather than singular apps. Third party tools will be phased out very quickly the higher up you climb the digital ladder. The web changes too quickly to rely on them. The tools that will be relied upon most will be people-based: creativity, problem-solving, ingenuity, design architecture, accuracy, etc. All the things for which we are yet to make an algorithm.

Who are the leaders or primary drivers in the area? What are the most notable projects? What organizations are embracing data journalism, and why?

These are all sort of the same question. The primary drivers are the tje organizations embracing data journalism and they are creating the most notable projects. The Guardian, The New York Times, The Boston Globe, Texas Tribune, BBC, ProPublica, etc. The majority of the big players in the US are in the game and playing mostly to swap and/or steal people between them rather then fostering their own personnel or attempting a different approach. The UK has a couple of major players and some crouching at the starting line. The US has a much longer history of CAR and a more go-get-’em attitude that has allowed Aron Pilhofer and Scott Klein to adopt their own team management and structure.

Even though “data journalism” is on the rise (as a term anyway) getting to where Aron and Scott are is extremely difficult, being successful at it even more so. Aron and Scott were given the opportunity to form a team when news organizations were going through a frantic burst of evolution, straining to cling on to life. ProPublica is one fo the more elegant species to arise from this turbulent time. Now, those who feel they survived the crunch are battening down the hatches and hoping to stave off the winter by being conservative. Because managers and directors have now heard the terms “big data”, “interactives”, “data journalism” they feel someone has figured it out and all they need to do is steal them or if that fails find a cheaper copy.

Sadly, most news organizations are embracing data journalism because The New York Times has. Managers feel they should be sprouting buzz words at meetings and having coffee with Google reps. They need to be seen to be embracing the future. Even if they have no idea what that entails. Many send editors to conferences but few send journalists to training courses. Most want to find out what free tools they can use, who they can partner with and who is the next Twitter. At the core of digital journalism development is incentive. It still doesn’t make money but it can make careers. It has to stop being about the newsroom ‘ooooh’ factor and more about getting the right story.

What will they need in 2013?

Experience. They will need people attempting these projects on a small scale, therefore they need non-developers to have some skills and a thirst to learn new ones. They need more people and more models in order to experiment as “data journalism” is not a solid thing as yet. They need the internal structure (and a stable one) in order to allow the team the time it takes to mature. They need the to money to pay developers, good ones. They need the internal structure of their institution to undertand what it is they are doing and to adopt the same fluidity.

They need to keep experimenting.

I optimistically decided to run two learning labs. Both teaching non-coders to hack and hackers some media literacy hacktivities and interactive video with Popcorn.js. So there is something for everyone! These will be put into a hacktivity kit but until I find a home for making my workshops into online lessons I’ll be putting all the links right here.

First a word of note, if you’re attending my workshops or learning labs bring a laptop and not a tablet. I’m very hands-on!

HTML for Journalists

Using Mozilla’s new webmaker tool, Thimble, you can code and see what the browser sees. In this hacktivity you markup and style a news article, learning the bascis of HTML and CSS. For those of you already web savvy there is a media literacy game suggested. Try it out!

Location-based Storytelling Using Popcorn Maker and Popcorn.js

This was really fun to make and teach. Popcorn is the project that is going to get me to further hone my JavaScript skills. It is the most applicable webmaker project to journalism. Here non-coders use Popcorn Maker to replicated a BBC Interactive and those who code for the web can recreate it easily using Popcorn.js. Choose a track and try it for yourself. For non-coders I suggest you do the HTML course first, go through the web fundamentals and JavaScript fundamentals track on CodeCademy and then have a crack at the Popcorn.js assignment!

Besides running the above workshops, which was great fun and so many journalists attended, all the OpenNews 2012 fellows also gave a one minute presentation on something they got up to this year. Watch it here (25 minutes in). The 2013 fellows were also announced. It was great to meet almost all of them and I look forward to following their amazing journey next year. It’s mind-blowing to think that such talented and diverse people are offering their skills to newsrooms around the globe. Counter to what most people would think, I think now is an incredible time to be in journalism. People are telling their own stories, instantly and to the world. The fabric with which media people now work with is truly intricate and interwoven through the public sphere. Now we should be experimenting with new types of stories that can be told.

Shaking things up, moving things around and exploring new territory is how we have survived as a species for so long and this crisis will stand to prove that those who adapt will survive. Nowhere is this more evident than the tech industry. Nowhere is this easier to do than the web. The web is a new ecosystem to explore. It is the new medium not just for monetary transactions but for social internactions. So why is the move from print to web proving so difficult for the news industry?

I have been fortunate in the last two years to get a glimpse at the greener side of the road. As part of the OpenNews programme developers are put into news organizations. I, on the other hand, was in news. I trained in news and worked at Channel4, BBC and CNN International. For me, the exciting new prospect was not just to work with developers to hone my data skills but to be part of Mozilla. I didn’t realize how much a part I would be until last month where I found myself at the All-Hands (AGM) with 50 other Mozillians.

It was a real eye-opener and a real privilege to be there. It was my chance to learn more about Mozilla and the projects they are undertaking. Before I tell you about those I want to list my observations on the way Mozilla is run which I think reflects the web-based tech industry as a whole. Mozilla are:

• Inclusive of every employee: everyone had a say, everyone could get involved and contribute to decision-making
• Promoters of creativity: success on the web is built on ideas and ideas can come from anywhere and anyone
• Hugely varied in their employees backgrounds and skills: from designers to teachers to musicians, there was a wide range of personalities
• Risk-takers: they see themselves not just as making fun things for the web but as taking on the likes of Facebook and Youtube
• Concerned about sustainability: even though the foundation has funding they are still looking at how they can support their work in perpetuity
• Mission-driven: every project must have a purpose and this must fit into the Mozilla mission statement of an open and inclusive web
• Moving to coherency: for their message to be loud and clear they are creating threads between all their projects

The modern newsroom is still faction-based between digital and editorial. There is still too much of a top-down approach to management. Your product is still your content, no matter what medium it is on. If that content is ripped from wire copy it is information when contained in a touch and take newspaper but has very little value on a web of social and global links. People need to touch a paper and people need to be touched by the web. There needs to be creativity (and humanity) in the creation and presentation of content. The medium is the message.

For the audience to feel a connect to the news product there needs to be a real connection in its creation. News cannot be produced in a factory and it cannot be produce in code (the aggregators will lose their audience to a truly digital news platform). There needs to be a passion for journalism, real journalism in the newsroom for a news organization to survive. In that sense, hack days should not just be for developers, they should be for journalists also; to hunt down the stories that have passed under their nose in the process of producing copy. Developers and journalists should have access to as wide a range of skills as possible to make these ideas reality.

Managers should take risks at news organizations. Now is the time to do things differently as your product has to do its job differently. Be agile and be lean, just make sure you have learning metrics so you know the results of the changes you make. Turn to your developers, turn to your journalists and turn to your supporters. You no longer have readers or an audience or even users. You have supporters. They can be supporting you financially and in other ways. Your product is not a chair, it serves no definitive function. It is ideological. Your supporters choose to support you because they want you to continue working to your ethos and principles. In that sense, you do not have a brand. You have a mission. Managers in newsrooms speak of brand far too much, and building products around said brand. No. You have a mission and you find ways for your supporters to connect with and further their support in your mission.

On the surface, I want news organizations to be more like tech organizations because I want 20% time, I want the high spec computers and great internet connection, I want the cereal bar and soft drink filled snack bar, I want the fun events and travel. But I think it goes deeper than this. I think they should adopt the ideology.

The Mozilla Festival 2012 will be in London 9-11 November. I will be running two workshops in the journalism track, Location-based Storytelling using Popcorn.js and Popcorn Maker (yet to be released but watch this video) and HTML for Journalists using Thimble. Do come and join me.

To look at this angle I scraped the Dfid contracts from contracts finder and looked at which contracts were won by UK companies. The article had the data but ScraperWiki has the code for any of you interested in digging up contracts.

Firstly, you should scrape all the links to the individual contracts from the search result page. Here is the one for Dfid. Click on “Copy” to get your own and change the “search_page” variable to the URL of your search. To make sure you get all the URLs change the page size in the URL to make sure they are all on one page.

Next, go into each URL by attaching the data from your search results scraper into a new scraper which extracts the HTML and pulls out the necessary information. Here is the one for Dfid. I have used the HTML scraping library BeautifulSoup. You can find the documentation here.

So take a look at the source, take a look at the code and take a look at the documentation. Open data and open up the news.

Being immersed in a passion, an upcoming field, a new area of exploration is of course exhilarating. But living in a bubble of code, data and journalism can lead you to adopt certain assumptions and to fall into naive paradigms. This is often geo-located. I, like most in the West, tend to look to the West for innovation and progress. Even though I am not completely Western it is easy to get enveloped in your immediate online community and not those occupying the next node on the web.

This became apparent to me on my recent trip to Argentina for HacksHackers Buenos Aires. Not only is the journalism-coding community thriving in this bustling South American city but the hub of activity is something every newsroom developer and webmaker journalist in the West should be envious of. They organized a star-studded lineup of new media composers for talks and workshops. A three day media party with over 700 attendees. If that wasn’t impressive enough, they had live translators so that both Spanish and English speakers could attend.

I was included in the keynote and workshop run my the members of the Interactive Team. What I said was as follows:

Journalists used to work with information in print. Now we work with data in the digital age. Now “we” means not just the journalist with pen, camera and microphone but anyone with a phone and access to the internet. Now we all aggregate, curate and cultivate in the age of big data in the hope that not one person can dictate.

Now philanthropical bodies like Knight and campaigning foundations like Mozilla are enabling news organisations to openly embrace data driven investigative journalism by funding projects, training and education centres. Even giving headstrong idealists the opportunity to work with the new multi-skilled teams being fostered in newsrooms around the globe.

Because what newsrooms typically have no longer works. It does not work for collaboration, for visualisation, or for big data. To be digital, news organisations have to now move at the speed of web. And with the advantages of legacy come the disadvantages of rigidity.

So how do we engage with data? How do we move forward in our understanding of the typical news story? How do we pitch a story without a headline until the majority of the work has been done? How do we decide how to tell the story before we have it? How come it is already happening all over the world simultaneously?

How do we strive for data integrity when the structure we know is the sentence and the paragraph? How can we ensure accuracy when we only have one source, the data? How can we interrogate data on a scale that cannot be consumed by a human being? And how can we find stories in data whilst upholding the cornerstones of impartiality, accuracy and fairness?

I don’t know the answers but I do know that to do all of this we need tools. We all need to collectively and openly share ideas, data and code. We are no longer news-makers on the web, we are news-makers of the web. And I have had the great fortune of seeing The Guardian team and indeed the news industry tackle these challenges.

This went down well with the crowd but what I failed to communicate was how impressed I was with them. Before the media party kicked off I had the great fortune of meeting up with the data team at La Nacion. Not only were they present and active participants but they brought along an entire university class whose students are taught on campus and in the newsroom. What a brilliant idea!

Another inspiring concept incubated by the data team is that of the digital journalist notebook. The government of Argentina publish reports of spending, expenses, contract awards, etc in paper bulletins which are available online as PDFs. Each region has it’s own take on the general layout. The team scrape all these PDFs and have a search for the contents of the documents.

In this way they have made all government reports into a digital library where, using their journalistic hat, they can connect who is who and who gets what. Hard-hitting investigative stories have already emerged. With data “more open” in the West, newsrooms there shold be taking a leaf from their digital notebook.

So I’ve made a GitHub repository called skel (for skeleton) for all data driven journalism projects. Whenever I have a new project, I’ll make a zipped copy of the skel folder without git (see pdf instructions below), giving it the project name and build up my scripts and data in the various folders.

skel contains the vagrant I talked about in my last post so it has iPython, BeautifulSoup, MySQL and Elasticsearch. With every project iteration I will be adding to the vargrant (and possibly repackaging it) and refactoring my skel repository. So if you a journalist interested in data projects, get on GitHub, clone the repository and watch it for any updates. Below is my very simple guide to using GitHub for non-developers:

How to Use GitHub