WebPA: the road to sustainability

by Elena Blanco, Ross Gardler, and Nicola Wilkinson on 30 November 2007 , last updated 8 November 2013

Introduction

OSS Watch is funded by JISC to provide support and guidance on the use and development of free and open source software and licences. One of the ways that this can be achieved is through detailed one-on-one consultations with projects. OSS Watch has now conducted a significant number of consultations and is extremely pleased to report on some genuine results from that activity.

A significant consultation in early 2007 took place with the WebPA project. This JISC-funded project will make an open source version of an existing online peer moderated marking system and is especially interested in building a community support structure. Following a consultation with OSS Watch, the WebPA project, largely driven by Nic Wilkinson, took active steps to build a community support structure and it now seems that all that hard work is paying off. As a result WebPA makes an interesting case study as a model for all projects wishing to go down an open development route.

The WebPA blog

WebPA set up two project blogs as a mechanism to keep the community updated on what was happening. One blog concentrated on the technical development of the project (the technical blog) and the other on pedagogic research and the development of framework for the project (the research blog). The technical blog is a treasure trove for anyone interested in open development as it provides real life examples of the issues faced by the project and the practical steps taken to address them. More importantly for OSS Watch it provides a concrete example of how a JISC funded project can successfully take our advice and tailor it to their specific case. To illustrate this, consider these blog excerpts, listed here alongside OSS Watch’s specific advice to which they relate.

I never imagined that choosing the potential licence that we would use could be so difficult. This is all leading up to the writing of the consortium agreement and the future open source release of WebPA. _2 January 2007_Advice: Sort out your licence and IPR position early. In an ideal world, the licence will be decided before the project starts or at least as soon as software dependencies have been identified.
We have also managed to get some very basic presentations together as an introduction to the project…. Our other major mile stone is getting a JISCmail list. 17 January 2007
The WebPA Project has started to disseminate further afield in ‘DLib’ magazine. _8 February 2007_Advice: Make sure your potential community is aware of you and can communicate with you from day one; sure they won’t come flocking to your door yet, but creating a brand is the first step. Simply ensuring that all disseminated materials are identified as coming from your project goes a long way towards cementing your brand.
Yesterday we had a very informative meeting with Ross and Rowan from OSS-Watch. 7 March 2007
All this to do and two open source books to read! _23 March 2007_Advice: Do your research, building a community led project is not hard, but it is quite different to any other kind of project structure and it won’t happen overnight for you. Understanding best practice and figuring out how to apply it to your work is the key to success.
As with everything related to agreements and IPR, there is a distinct lack of understanding as to the amount of work that needs to be completed. There is also a distinct lack of understanding as to why this can be so important to the project and its potential success. _24 April 2007_Advice: Recognise that you don’t know everything and get advice early. Once you have got that advice make sure that it is communicated effectively to both the project team and the community.
Just to make sure that we are getting our moneys worth out of OSS-Watch (not that we pay) I asked some colleagues here at Loughborough if they would like to join the WebPA team for a couple of hours and sit in. Everyone who attended to find out about Communities has given positive feedback. People have said that they wished they had been able to give more time to attending the afternoon session, to find out more. _10 May 2007_Advice: Always help those who help you and give credit where it is due.
It has also highlighted to the WebPA team some of the routes that the project can take as exit strategies or (a better term) project sustainability from when the funding runs out. _10 May 2007_Advice: Plan for sustainability as early as is practical. OSS Watch encourages you to consider it during the bid phase and can help with this. The sooner you start planning for it the more likely it is to happen.

Sustainability is concerned with the ongoing survival of the project, or the project’s outputs, after the current round of funding. There are many sustainability routes available to projects including donating all outputs to a suitable parent project, exploiting the outputs commercially or depositing in appropriate repositories. It is vital to note that in the case of software it is not sufficient to simply make the source code available in a “forge” such as SourceForge.

However, this is now giving me a headache! I have made it overly complicated. _15 May 2007_Advice: Keep it simple. Make it do what users want it to do and nothing more. Keeping it simple not only helps you, but it also helps anyone coming to the code as a potential contributor.
After a fairly swift poll of the project partner and the other potential pilot the opportunity to use LDAP arose…. The first problem I encountered was… _25 May 2007_Advice: Be open about issues you are facing and how you intend to address them. By now you have a small audience, someone out there may have the answer, or may stop you making the wrong decision.

Best practice suggests that issues should be discussed on a mailing list rather than a blog since the intention should be to encourage contributions of all kinds. People can only comment on blogs, they can’t create new posts so their contributions are limited to the topics you choose. However, some community members may be more likley to engage with a blog post than a mailing list and vice versa so an interesting compromise could be to deal with a particular issue on the mailing list and then to re-iterate the point with a follow up blog post. This will maximize exposure within your target audience.

Who is our community and what exactly will the wiki be used for? _7 June 2007_Advice: Don’t use technology simply because it is “cool”.
A wiki is not the correct tool for discussion to be carried out on. We have a mailing list, which anyone can join and the achieves are openly available. Hence the mailing list is the correct tool and a wiki is an incorrect tool. _7 June 2007_Advice: Use the right tool for the right job.
In this process we hope to get more institutions to host WebPA for their academics. From this we hope to build a community of users. _15 June 2007_Advice: Look after your users first; without your users there is no project. From your users your contributors emerge. Remember that contributions take many forms, it’s not just about code contributions, and contributors may have different roles. Feature requests, bug reports, bug fixes, and user feedback are all contributions, as is evangelism and generating attention in the wider community.
Yesterday I attended a workshop run by OSS-Watch on building communities. For us as a young open source project it was really useful to get the opinions of the community that we work within but never really think of as a community! _21 June 2007_Advice: Ask OSS Watch for a consultation!
This leaves one route to go down, which is to change attitudes. Whether this is the best course of action only time will tell _21 June 2007_Advice: If current practices are failing and you think you may understand why speak loud and clear. Don’t be arrogant, don’t tell “them” they are wrong, just lead the way and those who agree will come and help. Those who don’t agree will challenge you and will help you ensure that you have considered all options, thus you are more likely to choose the right path.
My next delve [into SourceForge] was to set up a list for feature requests and set up the few other lists for people to use when the project moved forward again. _6 August 2007_Advice: You absolutely must have an infrastructure that allows your community to engage with you. It doesn’t take long to do and once done it actually makes project management much easier, even if you don’t have a community yet.
There are people out there trying to change both the system and the cultures, mainly thanks to the work of Randy Metcalfe, and now Ross Gardler, and the lads [and ladies] at OSS-Watch. Due to them I no longer feel isolated as they have set up a community which is thriving. _14 September 2007_Advice: Where possible engage with existing communities.
Have we a break through, have I finally stopped talking to my self on the JISCmail list? _21 September 2007_Advice: Building a community is slow, it can be frustrating, it can be lonely, but it is worth the effort. Be patient.
I am in the process at the moment of supporting a number of institutions with their own installations of WebPA. 8 October 2007
I released a version of WebPA as a download on Friday 5 October 8 October 2007
I have in the past twenty four hours been asked about the projects plans to develop integration modules for a particular VLE. 16 October 2007
The road map document lets the projects community members see where the project is intending to go for the next phase of work… In order to ensure that the community needs are reflected… _22 October 2007_Advice: Understand your users, engage with your users, satisfy your users and make it easy for users by providing early downloads. Without your users your project will not be sustainable; the more users you have the more likely you are to find a sustainability route.
This leaves me to add the information to the trackers myself. At the moment this is okay, as there are not too many requests. I am encouraging the users to use the SourceForge system, so every time I email back with the solution I endeavor to include tracking information, as well in the hope that at some point they will use the system. _8 October 2007_Advice: As the number of users grows, so will the demands on your support activities. You must encourage your users to use the proper channels so that the support load can be spread across the community. This may require a change in behaviour for your users so aim to support them in adopting processes and procedures that may be new to them.

Remember users are usually the best people to support other users. The aim is to create a culture in which users who got free support from you are willing to give free support to others. Failure to do this means that you will eventually become a victim of your own success and will never have time to do development work.

It [drive-by contributions] is a way of building a project where people can contribute the small element they need to and then leave the project. Unknowingly I facilitated this type of action _26 October 2007_Advice: Consider third party contributions as just reward for all your hard work on community development.

To fully understand why these small contributions are important consider this WebPA case. It was a small bug that would only occur in a specific configuration in the authentication system so it was unlikely WebPA would find it.

If this bug went unfixed, how many potential users would try to use the software but give up because they couldn’t log in to their initial installation? Since every user is a potential contributor, each lost user is potentially a lost contributor. Furthermore, each lost user could be a lost paying customer for Loughborough or indeed any other entity that might want to offer a paid product or service based on WebPA. In other words each lost user results in a decrease in the chances of reaching sustainability.

These small fixes are the lifeblood of an open source project not only because they ensure a higher number of satisfied users, but also because having had one patch accepted the contributor is more likely to submit another, then another and another. Eventually you have a new developer to vote into your project and you are on your way to sustainability.

One piece of advice we were given was to make a demonstrator and make it available to potential users to see what the software is about. Well this was realised at the end of October. Within this first week we have had a phenomenal response. _1 November 2007_Advice: Make it as easy as possible for users to evaluate your software.

If someone tries an online demo and likes what they see they are far more likely to spend the time downloading and installing your software. If your software cannot be evaluated online then create a series of screenshots and/or screencasts (actually they are useful for software that can be evaulated online too).

Does this hard work guarantee sustainability?

WebPA has done an incredible job of building a community support structure around their code, they are even starting to see genuine community activity.

Will all this effort make the project sustainable? It is far too early to tell. Building sustainable communities takes a long time. For example, this graph of activity on The Apache Software Foundation’s mailing lists shows that it took around four years before the Apache community took off.

Although it is too early to say whether WebPA will reach sustainability it is possible to say this: if the users continue to appreciate the value of this software and the WebPA team continue to proactively support them in this way the chances of reaching sustainability are very high since all options are now available.

WebPA is certainly giving itself the best chance possible.

Current status?

As part of the work to integrate WebPA more closely with VLEs, the system was been used as a case study in the JISC-funded ceLTIc project, which looks at implementations of the IMS Learning Tools Interoperability (LTI) specification. Version 2 of WebPA has recently been developed to improve LTI integration, and saw its first release in April 2013, with a second release following in August of the same year.

Acknowledgements

Our thanks go to Nic Wilkinson of the WebPA project for posting the blog entries around which this article revolves.

This document contains a link to a graph generated by MarkMail, a free service for searching public mailing list archives.

OSS Watch provides unbiased advice and guidance on the use, development, and licensing of free software, open source software, and open source hardware.