Wookie: a case study in sustainability

by Sander van der Waal on 15 March 2011 , last updated 9 May 2013

Introduction

Apache Wookie is a Java web application that allows you to upload and deploy widgets for your applications, based on several W3C widget specifications. Several other APIs are also supported by Wookie, such as OpenSocial. The project started off as part of an EU-funded project run at the University of Bolton, but was proposed and accepted as an incubator project by the Apache Software Foundation (ASF) in July 2009.

Since this case study was first published, Wookie graduated as an Apache top-level project in November 2012.

Project history

The University of Bolton started the Wookie project as part of the TENCompetence project, funded by the European Union via its Framework 6 programme. The TENCompetence project ran from 2005 until 2009, and focussed on the support of life-long development of abilities of individuals, groups and organisations. This was to be achieved by developing and promoting the most suitable technical and organisational infrastructure. A central element of the project was the Personal Competence Manager, a portal web application that allowed people to discover competencies they would like to develop, and subsequently register for the courses that are required in order to develop this competency. Wookie formed an element of that portal. It allowed the deployment of widgets, providing functionality within the portal and making it possible to integrate this functionality into other applications, such as Moodle or Wordpress.

The project team realised that the use of an upcoming, open standard could encourage uptake of the software and could make it easier to integrate with external systems. Also, some members of the project team at the University of Bolton were already engaged with the W3C through their involvement with CETIS (Centre for Educational Technology and Interoperability Standards). So they decided to implement the W3C Widgets standard for Configuration and Packaging and the W3C Widgets standard on The Widget Interface. W3C defines widgets as:

Small client-side Web applications for displaying and updating remote data, that are packaged in a way to allow a single download and installation on a client machine, mobile phone, or mobile Internet device.

The project team collaborated early on with other projects that implemented the standard, such as the IST PALETTE project¹. The Wookie project operated as one of the candidate reference implementations of the W3C widget standards. Over time, it achieved one of the highest success rates among the implementations in passing the conformance test suite of the specification².

Growth and development

The Wookie project team recognised at an early stage that the potential applicability of the widget server was much wider than just the web portal for which they were developing the application. As a result, they managed to keep the widget server a discrete, generic application that could easily be re-used outside of the TENCompetence context. They considered the sustainability and continuing development of the project and contacted OSS Watch for advice on this. Ross Gardler from OSS Watch acknowledged that the project would be a good fit for the Apache Software Foundation, who have a strong history of hosting projects that act as reference implementations for open standards. At this stage, there was already some interest in the project from outside the core team, which made it even more important to move the project to a sustainable home.

In collaboration with OSS Watch, the project team made a plan for continuing the project after the EU funding ran out at the end of 2009. They sought a core budget in order to ensure that some of the time of the initial team of committers, consisting of three people, would be guaranteed. They estimated that about 0.2 FTE over the period December 2009 to September 2010 would facilitate a smooth transition to the Apache Software Foundation.

The structure of the ASF is such that new projects cannot directly become a fully acknowledged, or ‘Top Level’ project, right from the start. Instead, a project has to be proposed and accepted into the foundation’s Incubator first. Projects can only be proposed if they are championed by an existing member of the Foundation. A champion is basically the leading mentor on the project. Having an ASF member as a champion ensures that there is an interest in the project amongst the existing ASF community. In order to find a champion, one usually can simply go to the mailing list of the Incubator to informally propose your project and ask for people who are interested in the project. During its time in the incubator, the project needs to take care of its due diligence. That is, the project needs to make sure that its source code is properly relicensed (if necessary) to the Apache License V2.0 and that all included external libraries that the project depends on are compatibly licensed. Additionally, all people that have contributed code are required to sign a Contributor Licence Agreement (CLA) to grant a copyright licence to the ASF. If they do not wish to sign this document their code will have to be removed from the codebase.

The Wookie project was proposed to the Apache Incubator community in July 2009, with Ross Gardler as the ASF champion. It got an overwhelmingly positive response and was accepted on 14 July 2009. The three main project team members were also the original committers of Apache Wookie (Incubating), who took care of the IP clearance. They each signed an Individual Contributor License Agreement (ICLA) and sought CLAs and from a few external contributors that had contributed code to the code base. One Bolton employee who worked on the project did not want to sign a CLA; their code was therefore removed from the repository, with not too much effort.

Moving the project to the Apache Incubator also meant moving over the mailing lists and source repository to Apache infrastructure. But it was definitely worth the trouble, because being a part of the ASF helped the project in attracting more attention than they otherwise would have been able to. As a result, new external contributors came along, which resulted in new committers becoming involved in the project.

For example, a developer from Estonia added connectors in several programming languages, making it possible to integrate Wookie widgets in applications written in other languages like C-sharp, Ruby or Python. Another very important contribution was made by Randy Watler. This developer had not been active on the Wookie developer mailing list before. But he stepped up when the issue was raised that Hibernate, one of the underlying components Wookie depended upon, had a licence that was incompatible with the Apache License. This dependency had to be removed, which was a major task that had a steep learning curve for the existing team. Randy was able to do it much faster than the team would have been able to and replaced the dependency on Hibernate with other code that was compatible with the Apache License. This was an important step for Wookie, because it was necessary to ensure that all the code is compatible with the Apache License before a release could be made. Later, Hoang Minh Tien joined as a committer after comtributing an OAuth implementation.

At this point, Wookie had reached a point where the original developers from the University of Bolton were no longer the majority of committers, and several releases had been made, and so the project decided it was time to graduate from the incubator. A vote was held and in November 2012, Wookie became a top-level Apache project with more responsibility for its own governance.

Project structure: sustainability

The Wookie project’s acceptance into the Apache Incubator was a significant step in the sustainability of the project. The large TENCompetence project received a total of €8,796,000 in public funding. Because software development formed a significant part of that budget, and all the code was to be released as open source, one would expect that the sustainability of that software and the continuing development of the tools after the end of the initial funding would have been addressed. However, enquiries at the time of writing has revealed that the software is not being developed further. Instead, all the software components, that have been developed as part of this project, have been made available on SourceForge for download only. No development activities have taken place there since September 2010 and the number of downloads has been decreasing steadily. Had Wookie remained part of that software stack, the chances of the Bolton team attracting attention to their work would have been very slim.

Instead, by recognising the usefulness of the software and choosing the right standards to implement, the project team created an excellent opportunity to preserve the software. They also made sure that the investment of public money is not wasted, but will instead help to further develop the software and sustain the community around it.

Also, by focussing on open standards and open development, it has been far easier for other projects to incorporate Wookie into their software stack; for example, the Apache Rave social portal project includes Apache Wookie as one of its components.

Community building

Scott Wilson from Bolton University and CETIS (and now Service Manager at OSS Watch!) has been a key figure throughout the project. He liaises with the W3C working group for the Widget standards, he is one of the most active developers on the Wookie project and he also works closely with the OpenSocial foundation, trying to harmonise the W3C widget standards with the OpenSocial standards.

CETIS have also organised a number of workshops on widgets for the UK academic community, such as the 2011 WidgetBash. The ASF itself has also provided key community building support, for example, there have been Wookie related meetups at the both the 2009 and 2010 ApacheCon Conferences in the US. This also led directly to a related project joining Apache via the incubator, Apache Rave which provides a portal engine using widgets and gadgets.

Ross Gardler, in his role at the time as service manager of OSS Watch, also played a key role in the sustainability of Wookie. He championed the project and provided advice and guidance during the process of moving the project to the Apache Software Foundation. But he also helped out with the more technical side of the project and provided useful code contributions. As a result, he was also voted in as a committer on the project. Although Ross is an Apache member and Vice President of Community Development in the Foundation, Ross strongly denies that being an Apache member has anything to do with the getting a project into the Apache Incubator. And he is right: anyone can bring a project to the ASF, as long as the software is of genuine value. Anyone can go to the Apache Incubator mailing list and make people aware of an interesting project, and as long as interest can be generated among existing Apache members, a project can become part of the Foundation.

Finally, OSS Watch has helped to generate interest in the project in several ways. For example, it organised a training day aimed at developers who were interested in developing widgets. OSS Watch also helped generate interest in Wookie during the Dev8D 2010, a developers’ conference organised by JISC’s DevCSI project. Two Wookie widgets were developed during the conference that won prizes in developers’ challenges.

Reflections and future

The Wookie project turned out to be a huge success for the project team at Bolton University. As a result of taking the project to the ASF, the people at Bolton became much better known to the wider community and their expertise in building widgets and working on standards became a valuable asset for new collaborations. They also managed to secure funding for new projects in the Framework 7 Programme of the EU for a total amount of about £700k over a period of three years. The only extra investment the department needed to make was the 0.2 FTE over the course of 10 months.

The team members themselves gained a great deal of experience through this process, and brought this to bear in subsequent projects; for example, in the OMELETTE EU-funded project they were instrumental in the consortium adopting open development and working with existing open source projects.

In terms of the future of Wookie, the groundwork has now been done for creating a viable community. New contributions are continually added and all dependencies to external libraries that had licences incompatible with the Apache License have been removed. At the time of writing, the project is making regular releases and continues to add more committers. Furthermore, the user base is growing but not all users are part of the project community. The mentors of the project will need to address this. It looks like there is a bright future for both the Wookie project and the team that brought it to the ASF.

When OSS Watch team member Sander van der Waal presented about the Wookie project at the Apache conference in 2010, a renewed interest in projects around web standards like the one used in Wookie was becoming apparent. Thanks largely to the efforts of Ross Gardler of OpenDirective and Ate Douma of Hippo, this eventually resulted in the start of a new Apache project, Rave, in March 2011. That project incorporates Apache Wookie as one its core components, and itself graduated from the incubator in March 2012.

This is clearly an example of how an academic project can become a success story in terms of sustainability beyond its initial funding, without large institutional or commercial investments.

OSS Watch provides unbiased advice and guidance on the use, development, and licensing of free software, open source software, and open source hardware.