Crossing the Chasm: open source software comes of age

by Paul Anderson, Intelligent Content on 8 June 2006 , last updated

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

A report from the OSS Watch conference, Open Source and Sustainability, held at the Said Business School, Oxford, 10-12th April 2006, by Paul Anderson, Intelligent Content.

As any teenage starlet will tell you, growing up in public can be hard to do. 15 years after Linus Torvalds sent his famous first e-mail (to the Minix operating system newsgroup) announcing his work on Linux, open source software is going through its adolescent phase. Open Source Software (OSS) or FLOSS (Free/Libre Open Source Software) as it is also known, is coming of age and taking its place in the landscape of technology. There remain sceptics, of course, but anyone who doubts that it is having an impact, particularly in Higher Education, may have second thoughts on hearing the news that the Open University (OU) is launching a Moodle-based open source e-learning course management system in the summer of 2006.

So it is clear OSS is having an impact, but how is it dealing with the growing pains: what about its long-term sustainability? Is it here to stay? Are there emerging technical and commercial models for long term viability? In essence: how does OSS cross the technology chasm that exists between early adopters and technical hackers and its more widespread take-up in mainstream organisations? These are the growing-up questions now exercising the open source community and they formed the subject of the recent OSS Watch conference in Oxford entitled Open Source and Sustainability. A range of speakers from Higher and Further Education, businesses, software developer communities and open source organisations spent three days debating three major aspects of the sustainability agenda: what do we actually mean by sustainability, what does it consist of, and how do we go about developing it?

What do we mean by sustainability?

The first question exercising the speakers was what we actually mean when we talk about the sustainability of open source projects and the associated software code. Julie Walker (MIT’s DSpace project), made the point that projects need to be clear about what it is they’re trying to sustain, for how long and for whom (i.e. which group of users). Pointing out that sustainability is fluid throughout a project, she said: you don’t reach a point in the project when you can say, OK, we are sustainable now. It is really an ever-changing, ever-adapting plan.

Jim Farmer (Georgetown University) and Paul David (Oxford Internet Institute) both brought an economic perspective to the discussion. Paul said that economists are intrigued by the sustainability of OSS communities, particularly the mobilization and efficient co-ordination of disparate developers working on long-term projects. Jim Farmer provided a different viewpoint, stating that economic sustainability was about focusing on constancy and permanence and providing medium- to long-term resource planning capability. He said that a lot of the discussion around what sustainability means can be centred on the user and their needs, as well as demands such as reliable software, long-term product support, good quality documentation and training, integration with other systems and reasonable costs. Open source software that can deliver this is much more likely to be economically sustainable.

Mark Taylor (Open Source Consortium) thought that sustainability should be considered as inter-linked on several levels: technical (level 0), commercial (level 1) through to issues of government and educational policy (level 3) and finally, widespread public acceptance at the highest level. In his view, only when the latter stage is reached can FLOSS be truly considered sustainable.

What does sustainability consist of?

Many issues surrounding the concept of sustainability were raised at the conference. These included explorations of models for achieving sustainability, legal issues, community development, meeting user needs, purchasing OSS, funding OSS development, software usability and the role of innovation within software development.

Communities

Many speakers thought that it’s the open source communities and their future development that are crucial to the long-term viability of OSS. Andrew Savory (Luminas) strongly emphasised the role of community: Does it have a community behind it? If there is no community, then it is not Open Source. How the community functions was seen as important: there is both a need for diversity in the participants (different kinds of users, vendors, technical staff, documentation experts etc.) and for efficiency in the manner in which it is run. Niall Sclater (Open University) added that it is also important to look at the levels of activity within the community and the number of people involved. Kit Blake (Infrae) argued that networking within the community is super important to an OSS business and you need to get clients who use the same software system to talk to each other and work towards collaboration, although he acknowledged that this isn’t always easy. Jono Bacon (OpenAdvantage), backed the need for diversity and balance, using a political analogy: you need people on the far left who are only into the ethics perspective as much as you need people on the far right who are into it from the technical [perspective].

But size is not all. John Norman (CARET, Cambridge University) felt that there is a wide range of community types that operate in different environments and require differing levels of organisational complexity. He contrasted a single developer marathon like Exim (a mail message transform system) with the team relay of the large international DSpace project. Jim Farmer agreed, particularly in the context of Higher Education, where if you have a sharply focused community where the participants have identical needs, and I interpret that to be the major research-led universities, then you can have a very effective OSS development produced by a smaller community.

Skills

Jim Farmer felt that within Higher Education, the availability of suitably skilled staff is a potential source of concern. He contrasted the large research-led institutions that can afford to employ staff with in-depth knowledge of OSS and communities with the smaller colleges, particularly in America. Jono Bacon felt this was also an issue for the business sector, where too many software developers are not engaged in OSS development in their paid employment but are doing it in their spare time.

Leadership skills within communities was recognised as being vital and it was noted that it can be difficult to find talented people with these kinds of skills. Kit Blake noted that: successful OSS projects all have a clear leader, an individual, and this person responds to every query, to every newbie question. Every possible opportunity [to communicate] is taken. Few people are prepared for the level of personal commitment needed.

A number of speakers emphasised how OSS communities nurture skills and develop talent. Paul David (Oxford Internet Institute) looked at the process of skills improvement, turning software package triers into skilled code committers and noted the importance of these training processes in the recruitment and retention of staff in the larger projects where both a sense of community and the opportunity to acquire software skills through social interaction are important. Professor David agreed with other speakers when expressing his concern over the supply of both skilled OSS developers and community leaders. He was particularly concerned by the statistics on women programmers working in these projects: 3% to 8% at most, compared to around 20% of women programmers in the ICT industry as a whole.

As far as sourcing people with the right skills is concerned, Stuart Yeates (OSSWatch) felt there are four sources: raised from the user community, raised from the academic community, conversion of closed source developers or poaching/sharing with other OSS projects. It was mentioned that OpenAdvantage (an RDA-funded business support agency) worked by developing an eco-system of skills within the existing ICT business community.

Governance

Throughout these discussions it was clear that the governance of a community and responding to and meeting users’ requirements and needs are of paramount importance. Remaining open to new ideas, new input and new people (Stuart Yeates) is important whilst also maintaining good resource and expectation management. Some leading examples of large OSS projects had spent considerable time and effort on sorting out organisational and governance issues. A clear and well-established process for the contribution of code to an open project was important. Stuart Yeates said: one of the signs [of a healthy community] is that there is a means by which positive contributions find their way into the code base. In addition, Bill Olivier (JISC) noted that there needs to be agreed interfaces for communication between different parts of the code and Paul David pointed out the requirement for modular, tree-like architectural structures for new code additions.

Innovation

A key issue for sustainability was that of the innovation adoption curve. A number of speakers referred to Geoffrey Moore whose 1999 work Crossing the Chasm has had a major influence on high-technology businesses. The book outlines a model for how technology is adopted throughout a population and presents a set of different categories of adopter which it maps on to a technology’s diffusion through target markets. The diffusion pattern forms a bell curve, and for software, the left hand side of the curve represents innovators and early adopters who swiftly make use of new software and are prepared to weather any bugs, lack of documentation and other features symptomatic of a lack of productisation. The remaining, far larger, segment of the population is separated from this group by the chasm, which has to be crossed by companies who want to sell new technology to the mass market. This chasm is as much an important factor in the sustainability of OSS technology development as it is in other software development.

For Bill Olivier, the chasm is a very important issue for educational software in general. However, it is proving to be particularly important for projects that JISC has funded through their R&D phases and now need to be made into recognisable products for diffusion to a wider audience. Other speakers, including Kit Blake and Paul Everitt (Zope Europe) felt that their businesses had actually crossed the chasm and were developing systems that had some mainstream uptake. Paul Everitt highlighted the uptake of OSS in local government (49%) based on Richab Ghosh’s FLOSS study 1. He then extrapolated from that, saying if you [can get] 50% [penetration] in a fairly conservative market than you’ve crossed the chasm.

James Dalziel (LAMS) highlighted the fact that some OSS projects are innovative in that they have developed something entirely new, rather than a clone of an existing, proprietary idea or package and he cited the Learning Activity Management System (LAMS) as one example of this kind of innovation. Paul David argued that SourceForge, the project home of many OSS projects, was a key component of the innovation process and could be viewed in economic terms as a virtual incubation centre.

There was some concern that Europe was being left behind both in the productisation of OSS packages after initial R&D development and in the commercial development of OSS businesses. Paul Everitt noted that there is a discrepancy between the percentage of OSS developers that come from Europe (70%) compared to the number of businesses or legal enterprises created from OSS projects, where only two out of eleven major initiatives are European, and he asked why is it that Open Source seems to start in Europe and finish in the US?

Models

James Dalziel: There is no one model of sustainability. We are seeing a wide range of models for the sustainability of Open Source.

A large part of the discussion at Oxford centred on institutional and business models that could be created or applied in order to support the development, growth and adoption of OSS. The speakers outlined a wide range of models with a variety of nomenclatures. The main models are summarised in the following:

Community Development

A community of user-developers works together to provide a solution – managed by a variety of governance procedures – which is often founded by a grass roots group of volunteers and uses a wide range of self-selecting participants. This is the classic Open Source development model, characterised by Eric Raymond as the Bazaar 2.

Examples include Apache, Linux

Inter-institutional Collaboration

A group of institutions agree to collaborate on and invest in a focused, but open, development project. This builds on the ideas established in the community development model but uses staff and resources from institutions more than individual volunteers. While the code may be open to review by anyone, only members of the ‘club’ make decision and contribute directly. There is often a host institution or two that takes the lead, with others pooling support and resources. It is also commonly referred to as the community source model 3 or focused open development (Jim Farmer). Funding is often secured from charities or educational foundations. Can be considered as a blend of cathedral and bazaar models as described by Eric Raymond 4.

Examples include Sakai, APLAWS

Foundations

The further development of an existing Open project (often an inter-institutional collaboration) is supported by the launch of an umbrella legal entity – a foundation. This provides a governance structure and legal space for collaboration.

Examples include DSpace, LAMS, OSAF

National adoption

The government or a national agency mandates a certain OSS solution. This forms a government-sanctioned monopoly solution which is therefore inherently sustainable.

Examples include Shibboleth

Commercial Development

An OSS project converts to a commercial development.

Examples include Luminis Content Management Suite (and uPortal)

Service Businesses or Open Source Support Providers (OSSPs)

A consultancy that provides support, advice, training and customization for an OSS product or suite, which has been unbundled from the code licence and its developers.

_Examples include LibLime on Koha, Infraea, Luminas _

Sponsorship

A large company sponsors an OSS project for brand development or other reason.

Examples include the Google sponsorship of Firefox

Spin-out

A large company or public agency develops a system and then spins out their proprietary code as an OSS project for wider community 5 development.

Examples include DSpace, Eclipse (IBM)

Dual licence

Software is open under GPL or other licence but also has a commercial, revenue-generating licence which helps fund development.

Examples include mySQL

Suite Packagers

A company packages together a suite of OSS products and provides certification and configuration.

Examples include Spike Source

Shrink wrap

Neatly packaged OSS along with the provision of manuals, training and support.

Examples include Redhat Linux, SuSE

Top sliced

Core development is funded by top-slicing a percentage of revenue from a network of OSSP partners who support the project.

N. B. It is important to note that there are many examples of OSS projects that blend more than one of these models into hybrids and there are also examples of companies who mix open and closed source software (e.g. Novell’s mixed source environment).

Considerable attention was devoted to two of these models in particular: inter-institutional collaboration and foundations. This was partly because these models represent cutting-edge work within the international education community on ways of supporting OSS projects. In general, OSS is seen as a potential way forward for developing software applications within education because of the reduced total cost of ownership and the ability to provide more innovative and flexible specialist solutions than proprietary systems.

The Inter-institutional Collaboration model (a.k.a. Community Source model) is closely related to the more general Community Development model in which a community (often of volunteers) shares the open development of code. The community model is widespread throughout the OSS world and, according to Rebecca Griffiths (Ithaka) is viewed by many as the preferred option.

An example of the Inter-institutional Collaboration model is Sakai6, and John Norman outlined the project’s development. Sakai has been leading a collaboration, since 2003, between a small number of research-led universities that includes MIT, Stanford and Cambridge to develop a Collaboration and Learning Environment (CLE). A total of ten Higher Education institutions and two commercial partners are involved, each contributing various levels of developer resource and other commitments. Some additional funding is provided by the Mellon Foundation 7. The code base is open to all, but partners who pay and sign-up to the Sakai Partners Program (SPP) get additional documentation and support, a say in the roadmap of development, and access to early releases of code. An example of the inter-institutional model developed from elsewhere within the public sector was provided by Aingaran Pillai (Camden Council). The APLAWS+ Content Management System is used to develop information portals for local government.

Foundations were also discussed by a number of speakers. Foundations are non-profit umbrella organisations or legal vehicles for overseeing and, in particular, governing OSS projects. They are regarded by some as a key tool in the development of sustainable, long-term OSS projects.

Within the small (but highly specialised) market that Higher and Further Education represents, it is difficult for OSS communities to sustain themselves because resources are scarce, so more attention is needed with learning and research-related OSS projects than perhaps is the case with conventional, purely technical OSS projects. The development of new models and, in particular, blended models and Foundations to support sustainability and provide a safe harbour for code systems, was discussed by a number of speakers and delegates. There was considerable interest in international developments in America (through the evolving DSpace and Sakai foundations, and the activities of the Andrew W. Mellon Foundation), Australia (the LAMS foundation) and Canada (Simon Frasier University’s blended OSS models). It was suggested that such developments could lead to a wider Foundation, possibly on an international scale, that could house and govern a series of major OSS initiatives within HE.

Julie Walker outlined in detail the processes by which DSpace, a library system, developed from a closed, in-house system being developed by MIT and HP Bristol, through the opening up of the code, into the launch of a separate foundation. Although it had always been part of the plan to open up the code (the spin-out model), the development of a foundation represents a recent change (March 2006) in response to governance issues and the needs of the user community. Other examples of the foundation model include the Open Source Applications Foundation (OSAF) which is developing the Chandler Personal Information Management (PIM) system, and the Learning Activity Management System (LAMS) 8, which was outlined by James Dalziel. The latter is a blended model in that a non-profit LAMS Foundation handles code R&D, and this operates in parallel with a limited liability OSSP company (LAMS International) based around services and support for LAMS (and which also helps fund new code development).

Funding

The funding of OSS development and support for user communities was tied to that of business models and to issues surrounding the emergence of new innovations. Many projects in education started life as initiatives funded by time-limited research grants and therefore need to develop new funding models and income streams. Several speakers touched on the potential role of Venture Capital (VC) investment for OSS development but it was generally felt that there are significant difficulties with this model. Sarah Porter and Bill Olivier, both from JISC, raised the issue of JISC project funding for OSS developments and the increasing need to demonstrate that projects have sustainability beyond the initial development and R&D funding. Colin Smith (VistA) outlined the ways in which the issue of obtaining predictable funding is having implications for the sustainability of OSS development in the healthcare domain.

Software procurement

Related to funding, and an issue raised by several speakers and delegates, was that of the process of software procurement (especially in education) and how that affects OSS businesses and foundations. Open source businesses attempting to develop business models that offer sustainability are often pitching against closed source proprietary products. G.W. Brian Owen (Simon Fraser University) outlined the difficulties entailed in responding to RFPs (requests for proposals) with OSS solutions. Procuring institutions, used to the large sales teams available from closed source corporates, do not understand why OSS organisations are not able to send in teams of sales staff. Similarly, trying to explain to a potential purchaser that there is no paid-for software licence, only an on-going support contract, can be complicated. As Jim Farmer said: Procurement processes have to change. When you choose a procurement process you have [in effect] automatically chosen whether you are going to do Open Source or whether you are going to do commercial. This was seen as an issue that requires policy development within the education and public sectors.

Developing sustainability

Having outlined a wide range of issues surrounding sustainability there was no shortage of debate and discussion over possible ways forward and suggestions for the open source community. The following summarises some of the key suggestions.

Strategy
  • Develop a culture of software sustainability throughout the HE community
  • Increase levels of senior management buy-in to OSS projects in HE right from the beginning
  • Plan and implement a strategy for sustainability from the start of projects e.g. issue roadmaps
  • Manage benefits and risks of a solution and its model throughout the life of a project
  • Plan to cross the innovation chasm – most HE and public agencies don’t have the staff to cope with installing buggy code and poor documentation
Funding
  • Much proprietary software is funded by VCs, and methods are needed to allow VC-type funding to resource OS
  • Grant funding should be tied to sustainability
User Community
  • Increase the focus on meeting end-user needs
  • Require community building activities as part of new-start development project plans
  • Get customers of OSS networking together
  • Run sprints (bootcamps)9 at customer sites
  • Be aware that it is important to convert interested others to becoming users and developers
New institutions and organisations
  • Explore the idea of a wider OSS foundation for UK and/or international F&HE
  • Investigate the potential for the incubation of HE/FE source code projects through JISC
  • Explore the need for an open source advocacy unit in American HE along the lines of OSS Watch
Policy
  • Change procurement policy in institutions and public agencies
  • Develop Quality Assurance metrics for OSS
  • Look at funding policy for development programmes (e.g. JISC)
  • Develop an information gathering system for finding existing good quality code and solutions (hidden gems) within HE/FE with a view to opening them up
Communication
  • Improve open source marketing skills
  • Improve OSS documentation and be more aware of its value in helping to cross the chasm
  • Improve usability (particularly installation of software)
  • Engage with commercial entities
Legal
  • Explore ways of providing OSS projects with better legal and business advice e.g. Software Freedom Conversancy (USA) 10
  • Explore ways of dealing with the use of 3rd party code and the sheer number of licences in larger projects
  • Explore the Dual Licence as a legal model
Models
  • Develop new, blended models of OSS sustainability based around foundations, business services, and community with the key theme of collaboration.
  • Look at ways existing projects can be incorporated into wider co-operating projects and suites
Skills
  • Develop community leadership skills
  • Develop OSS programming skills (particularly help women programmers to engage with OSS)
  • Develop an eco-system of skills (coders, documenters, communicators etc)

Where next?

Although there were many suggestions there is perhaps one overriding theme: the need for open source projects within education to be aware of sustainability and its issues right from day one and to plan a strategy to handle the process. Should sustainability, and a clear roadmap for delivering it, be part of the requirements that JISC and other research grant fund-holders request before agreeing to contribute to a project?

This was tied up with another question: should JISC or another public agency take on the role of leading the actual incubation of innovative OSS education projects? Perhaps this should include looking after the code and helping (providing tools for) the development of the associated community. Is there a role for a formal code repository and community tool, a Source Forge+ for HE? How far could a publicly funded JISC go in helping individual projects across the innovation chasm?

Conclusion

The conference tapped into a rich vein of issues associated with the sustainability of open source. The fact that it was even taking place shows that OSS is growing up; trying to find its way across the innovation chasm into the mainstream, and to seek some models of permanency. There may be many reasons why these issues are rising up the agenda, but Jim Farmer and Paul David highlighted one: money. If Moodle charged each client in the same manner as its proprietary rival Blackboard then its market value would be around £649 million (942 million euros). And it can be estimated that replacement costs of the Debian Linux system could be as high as £7.5 billion (11 billion euros). In software production, as much as in tinsel town, money talks. And even for a teenage starlet this is big money.

Further reading

Links:

Related information from OSS Watch:


  1. http://www.flossproject.org

  2. Raymond, E. 1999. The Cathedral and the Bazaar, O’Reilly.

  3. Wheeler, B. 2004. Open Source 2007. EduCause Review. July/August 2004.

  4. Brooks, L. Values of Community Source Development. At: http://campustechnology.com/Articles/2004/09/Values-of-Community-Source-Development.aspx?p=1.

  5. For a discussion on this model see: West, J., Mahony, S. 2005. Contrasting Community Building in Sponsored and Community-founded Open Source projects. Proceedings of 38th Annual Hawaii International Conference on System Sciences. Waikoloa, Hawaii, Jan 3–6, 2005.

  6. http://sakaiproject.org/portal

  7. http://www.mellon.org/ Note that this is not the same type of foundation as the Open Source foundations under discussion elsewhere in this article.

  8. http://www.lamsfoundation.org/

  9. Intensive sessions at which code developers work side-by-side with end users to develop and refine a solution

  10. The Software Freedom Conservancy http://conservancy.softwarefreedom.org/ provides an umbrella legal and corporate entity which OSS projects can join in order to benefit from group benefits such as tax exemption, legal entity status, single tax filing, copyright consolidation and accounting services for project funds.