Sakai: a case study in sustainability

by Dr. Bradley C. Wheeler, Indiana University on 5 June 2007 , last updated

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

In the latter half of 2006, the Joint Information Systems Committee (JISC) commissioned a study via its Teaching and Learning committee to examine the issues surrounding sustainability of open source software. The resulting report drew together seven case studies of successful but very different open source projects and examined each project’s sustainability model. Each of these case studies has been told from the point of view of the lead developer or one of the key personnel and gives a fascinating insight into the factors that have determined the success of each project. These case studies are now presented by OSS Watch as stand alone documents in a series.

This case study, examining the Sakai project, has been written by Dr. Bradley C. Wheeler, Indiana University.

Brief description

The Sakai Project produces Collaboration and Learning Environment (CLE) software that is distributed under the Educational Community License, an open source licence. The enterprise-scale software is mostly used by colleges and universities to support online instruction for faculty and students, research projects, and other forms of group collaboration. Ownership of the Sakai software rests with the independent Sakai Foundation that was formed as a not-for-profit entity in the United States to help co-ordinate and sustain the Sakai software and community. The software is available for use, modification, redistribution, and even sale without fee. The Sakai Foundation has over 100 education and commercial members who contribute US$10,000 annually to support the work of the foundation. July 2006 marked Sakai’s fifth major release in just two and a half years, and the Sakai CLE continues to benefit from many new tools that are being contributed by the community.

The Sakai developer community maintains a set of evolving technical documentation for the Sakai software. Most of the user documentation is in the form of online, context sensitive help and user training/tutorial materials. Indiana University has freely shared all of its Sakai training materials and animated tutorials for Sakai software, and other partners are developing, refining, and sharing their materials too.

Introduction

The Sakai Project represents an interesting case study in directed, collaborative development. Sakai uses the term Community Source to describe its model of directed resources that were tendered to a formal partnership to develop software based on open source practices. Economically, sharing development and support costs among institutions has obvious appeal, but higher education has a rather mixed history of success with voluntary collaborations. Sakai’s first two and a half years offer some lessons for project governance, community development, transitions from grant-funded project to community-owned foundation, and the possibilities of sustaining global communities for open source application software.

Project history

The Sakai Project was formulated at an impromptu dinner in Ann Arbor, Michigan with Joseph Hardin (U. of Michigan), Amitava Mitra (MIT), Jeff Merriman (MIT), Charles Severance (U. of Michigan), Lance Speelmon (Indiana U.), and Brad Wheeler (Indiana U.) on 22nd September 2003. The project built upon prior code-sharing collaborations between Indiana, Michigan, and Stanford (with Lois Brooks) universities. Michigan, MIT, and Stanford had also previously collaborated as part of the Open Knowledge Initiative (OKI) Project. All four institutions had decisively chosen not to use a commercial VLE and were independently pursuing development of a next generation set of software tools to support education and research. Thus, Sakai was born through a merging of efforts for home-grown systems with a vision to scale the collaboration to a vibrant open source community. The project agreed to combine the ‘best-of’ software tools and intellectual property from its founders to create the Sakai software.

The Sakai Project was announced at EDUCAUSE in November 2003 (Figure 1). The University of Michigan, Indiana University, MIT, and Stanford University agreed to collectively contribute US$4.4m in staff to the Sakai Project. The OKI Project and uPortal, via JA-SIG (with Carl Jacobson), also joined as Sakai co-founders. In December, the Andrew W. Mellon Foundation added US$2.4m to the Sakai Project based on the strength of the partners and their commitment to implement Sakai. In February 2004, the William and Flora Hewlett Foundation seeded the Sakai Educational Partners Program (SEPP) with US$300,000 as half of its first year start up budget. SEPP was announced in March with 19 founding educational partners.

Figure 1 Sakai Plan Presented at EDUCAUSE November 2003 (original slide)

The Sakai Project began in earnest in January 2004 with an aggressive date-driven delivery approach. The CHEF software from Michigan was the seed design and technical framework. Staff members from the four institutions had to quickly learn how to work together, make decisions, and develop quality software. Sakai 1.0 was released in June 2004 at the first Sakai Conference, which had 165 attendees. Mara Hancock (U. of California, Berkeley) and Vivian Sinou (Foothill Community College) were appointed to the Sakai Project Board and added their development resources. Four commercial firms became Sakai Commercial Affiliates to provide for-fee services around the open Sakai software. Sakai 1.0 was rolled out for production at the U. of Michigan in August 2004, and Indiana U. was the first to launch a large pilot with Sakai 1.5 in January 2005. Ian Dolphin (U. of Hull) was appointed as the ninth member of the board.

Sakai 2.0, released in July 2005 after 450 people attended the third Sakai Conference, represented a considerable 18-month milestone for the grant, software, and growing community (68 members). Indiana University rolled out Sakai 2.01 to its entire user community of 100,000 members as its default learning environment in August 2005. Many other institutions began formal pilot projects, and Michigan retired its legacy system and ran completely on Sakai. The Sakai Project accelerated its work to shift from a grant-funded project to a self-sustaining community through an extensive governance discussion at its conference. Jutta Treviranus (U. of Toronto) was appointed as the tenth member of the board.

In October 2005, the Sakai Foundation was formally incorporated, and the Sakai intellectual property was granted via a copyright licence agreement from the universities to the Sakai Foundation. In December 2005 the grant chapter was brought to a close with 565 attendees at the Austin, Texas, Sakai Conference and the release of Sakai 2.1. The Sakai community had elected three new members to the Sakai Foundation board, including one from a commercial firm, as three founders completed their term. The Sakai Foundation topped 100 members with 12 Sakai Commercial Affiliates including Apple, IBM, Harvest Road, rSmart, Sun, Unicon, and others. The University of South Africa went live with Sakai for 70,000 students in January 2006 and many other institutions began pilots.

Structure and governance: the grant

The Sakai Project began with a board of six—one board member from each investing institution, OKI, and uPortal. The board members each faced the responsibility for implementing learning software at their respective institutions and had tendered their development staff to the direction of the Sakai Project Board. There were no formal agreements between the institutions other than some subcontracting (of a portion of the US$2.4m grant to each partner from Michigan) for hiring additional development staff and travel. The glue for the collaboration was that each institution wrote a ‘Memorandum of Intent’ from the university president/provost to the Mellon Foundation. The memorandum asserted four things:

  • the institution would tender five development staff members to the control of the Sakai Project Board
  • granted the project use of specific software or intellectual property, e.g., the CHEF software from Michigan or CourseWork from Stanford
  • the institution agreed that project work products would be distributed under a BSD-style, open licence with no restriction on commercialization
  • the institution would implement the Sakai software when completed.

The tendered staff positions were all 100% assignable by Sakai, and they did not include fractional contributions of time or board member effort. The grant proposal document itself was the unifying instrument that held the partnership together. The approach of tendering staff to line management control of a multi-university project board represented a new approach for enterprise application software development in universities. Sakai became a virtual organisation with distributed resources that mostly got their pay cheque from someone else.

Structure and governance: the process

Sakai appointed leaders for the Tools Team to focus on gaps and functionality, a Technical Team to make architecture choices, and a Chief Architect to work with both. The teams met periodically face-to-face and worked extensively through e-mail lists and a Confluence wiki. The work of the project teams and their communication was not publicly accessible in order to limit the number of participants while trying to hit aggressive release dates.

Collaboration is not easy, and it was especially difficult as these staff members learned how to work together and shift from local institution decision priorities to collective priorities. There were the inevitable challenges between the design/functional team and the technical architecture team. There were extensive debates within the architecture team regarding various technical choices, and many of these were highly inter-dependent. The board rarely exercised its formal power regarding staff assignment, and most disagreements were ultimately resolved through debate. Over time, a meritocracy developed — like in any open source project — that accrued referent power and the ability to effectively influence.

Early in the second year of the grant, Sakai decided to co-ordinate the Tools Team, Tech Team and Board meetings to have an All Hands Meeting. This proved as beneficial as it was challenging. Tough decisions were being made, and many of these had real implications for home institutions and staff. The approach of date-driven development brought considerable pressure to keep things moving, even if imperfectly at times. So the board and team chairs put considerable energy into social development via dinners and other means for staff to build relationships with each other beyond just professional duties.

There was only one occasion in the project’s history which necessitated that a difficult decision be escalated to the Board’s chairman. There was irreconcilable and conflicting advice from within the project leadership regarding the timing of a particular feature and its readiness for the 2.0 release. Once the decision was made, however, the teams unified around the decision and quickly moved on.

Structure and governance: the foundation

The ability to sustain the software was part of an extended conversation with the Mellon and Hewlett Foundations before funding, and the Sakai Board engaged earnestly in efforts for post-grant sustainability within the first two months of the project. This meant that the Sakai Project began thinking about a sustainability plan even before the grant and partnership were finalized and with hindsight, most view this early start on the partners’ program as one of Sakai’s best decisions.

The Sakai Educational Partner’s Program was designed as an annual membership costing US$10,000, with a three-year commitment. Members received no special rights to the software over and above those of the general community via the open licence. They did, however, receive access to some SEPP workgroups and discussion lists, and a waiver of conference registration fees to send as many attendees as desired to Sakai Conferences. SEPP funds were used for community development, documentation writing, conferences, and member support for general Sakai information. SEPP membership did not initially give access to many of the workgroups that the investors used to co-ordinate the project.

In July 2004, the Sakai-Developer list, which up until then had been closed to all but the Sakai Project members, was opened as a public list. The board had wrestled with decisions regarding how much to open up, to whom, and how fast. There was some desire to protect the value of SEPP by providing some ‘gated’ resources that would offer more than would normally be available to the public. There were also perceived risks in engaging too many voices in the formative days of the project while striving to hit aggressive release dates.

However, openness prevailed and many good things followed from the early opening of the Sakai-Developer list. By July 2005, all Sakai lists had been made open to anyone. SEPP membership continued to grow, and this early opening up proved instrumental to a smooth transition at the end of the grant period. Members of the community were able to take on roles for bug fixing, requirements refinements and governance decisions, and as the project shifted from a start-up to a community, openness to track growing investments by others was essential. By opening up and expanding community engagement before the end of the grant, the number of direct participants in the Sakai Community who had developed specific skills and relationships for sustaining the community was readily in place.

In March 2005, the project began a discussion among the SEPP membership regarding the structure of post-project governance. This extended debate pulled in many views. Some favoured focusing Sakai on larger institutions that might pay more, while others preferred low fees and a diverse membership. Some argued to include commercial firms as full members of the Sakai Foundation while others thought Sakai should be ‘of, by, and for higher education’, with that criterion restricting membership to colleges and universities. There were discussions regarding ownership, control, and licensing of the Sakai intellectual property over time. A full day of the June 2005 Sakai Conference was devoted to governance discussion with additional follow up by e-mail list. Brian Behlendorf, co-founder of the Apache Foundation, was a keynote speaker and contributor to the discussion. The results of these discussions shaped the Articles of Incorporation and By-Laws for the Sakai Foundation.

Sakai ultimately chose that all members of the Sakai Foundation are equal members without distinction to educational and commercial members. All can vote in Sakai elections, and anyone can be nominated by a member for the Sakai board. The first election, in November 2005, yielded 16 candidates for three seats. It is noteworthy that, worldwide, educational institutions held 88 votes and commercial firms held 12 and yet a member of a commercial firm was elected to the board of directors. The member and his company had been active on the discussion lists and had contributed in many substantial ways to the work of the project. The membership acknowledged that in the election without distinction.

Finally, the Sakai Project had to devise a way to transfer the Sakai software with its joint copyright, held by the four founding universities, to the newly incorporated Sakai Foundation. This work required several months of negotiation and resulted in the Sakai Copyright License Agreement (CLA) for contributing code. All contributions to the Sakai Foundation software must have a CLA on file.

Since the end of the grant, the Sakai community has operated much more like a traditional open source community except that Sakai’s overall leadership and decision rights are very distributed rather than resting with one, key individual. There are many developers who have earned committer rights, and many institutions have now added tools to Sakai. The community maintains three levels for new tools: Contrib is a place to put new tools so others can download and see them. These tools have not yet been reviewed for quality and meeting Sakai standards. Provisional is the place for tools that have been in production at a home institution and that make use of Sakai standards. Provisional tools are usually high quality and are ultimately promoted to the Core release of Sakai with full Quality Assurance and Integration work.

Sakai 2.2 was released in July 2006 as the first post-grant, community-owned release. It went through 15 rapid rounds of quality assurance work by over 80 members of the community, and by all accounts was the smoothest and best release yet. Additional sites are rolling out Sakai 2.2 for full production use in the latter half of 2006.

Reflections and future

Without question, the achievements of Sakai in its first two and a half years are imperfect. It has succeeded and grown quickly due to the tenacity of its contributors and the faith of its community. For Indiana University, one of the early signs of Sakai being the right strategy was rolling out the Wiki Tool developed at the University of Cambridge. For years, IU’s effective work to develop its own educational software was of no benefit to others, and IU received no benefit from the work of others. The Wiki Tool was the first addition to IU’s Sakai implementation that met a local need, was developed entirely by another institution, and could be readily snapped into Sakai. Many more tools are now in development across the Sakai community, and the common Sakai architecture makes it easy to add tools developed by others. This is essentially the path to leveraging the innovation of others.

The early Sakai Conferences were largely developer centric. They have now made a full transition to a ‘big tent’ with sessions for lots of different roles beyond just software development, including information sharing for training materials, user support, server configurations, help desk, and many of the other needs for implementation. As the community has grown, the Sakai Founders have, by design, played a smaller role at each gathering of the community. Thus, more than anything else, the Sakai Project and now the Sakai Foundation are a lightweight co-ordination mechanism to ease voluntary collaboration among an ecosystem of institutions and corporations.

This path has proved to be highly efficacious, but founders must be willing to relinquish control to the community to realize this potential. Sakai’s choice of an open BSD-style licence, free access to the software, community discussions for anyone and equal rights for any member of the foundation—large or small, educational or commercial—have proven a magnet for rapidly developing both enterprise-scale software and a global community.

Members of the Sakai Foundation have also demonstrated their ability to prioritise functionality requests and requirements. A wiki and a voting tool, developed by a member of the community, were used in February 2006 to document and prioritise functionality needs through a sophisticated voting algorithm. Non-members of the foundation could also vote, but the system did display separate counts for members. Everything is open to anyone to see.

There is a requirements list in existence, but it operates more as a co-ordination and communication mechanism across the community rather than something that directly drives the development of new features. This is largely due to the foundation’s choice to maintain a very small staff and to second personnel from member institutions as its preferred path for getting paid work done, with the foundation playing a co-ordinating role. This means that most software development, except for some of the core architecture, purposefully remains at member institutions and is therefore close to the faculty and student users of the system.

The Sakai Foundation has also demonstrated its ability to serve as a home for other projects too. The Open Source Portfolio (OSP) Initiative has transferred its software to the care of the Sakai Foundation, and the OSP community runs its discussions and Quality Assurance work on Sakai infrastructure. There are considerable efficiencies in these types of shared services for related projects while they also continue to maintain their own identity and community.

There will always be healthy reflections and questions around how Sakai might have done better or worse if it had used a more traditional open source model in the beginning. For those who lived through the tornado of the start-up period and the challenges presented by having only six partners, it is difficult to imagine that more voices with differing levels of investment would have made those times easier. It is possible, however, that the opportunities and challenges would have simply been different—not better or worse. Sakai’s purposeful, but gradual, opening up throughout its two years of grant funding enabled a very smooth transition to a broader community with the Sakai Foundation. A different development model might have yielded a greater community of contributors and adopters earlier on, but it is not clear what could have been accomplished in just two years to yield enterprise-class software. Some ask, when is the Community Source model as used by Sakai applicable for developing educational applications? Was it the scale of the project that made it a good candidate or was it also growing dissatisfaction with commercial alternatives that drove some of the rapid growth of Sakai?

One insight is that Sakai’s approach to hundreds of decisions in shaping the Community Source model for a project and community has already proven replicable for administrative systems. The Kuali Financial System (KFS) project has mirrored Sakai in much of its governance and project execution and has made some improvements based on lessons learned with respect to project governance. Kuali has already created the Kuali Foundation and began developing its membership base even before the first software release, during the grant period. A second large project, Kuali Research Administration (KRA) is also being launched using the same Community Source approach as Sakai.

The next steps for the Sakai Foundation will be to continue to improve the user’s experience of the software, add additional tools for desired functionality, continue to expand the size and strength of the community, and work hard to interoperate with other projects and data standards. With these objectives in mind the Sakai Foundation appointed Dr. Charles Severance as its Executive Director in June 2006 to develop community leadership for these areas.

Current status

Since 2006, when this document was written, Sakai has reached version 2.7 with the next major release, version 3.0, now an official product with a June 2011 release date planned. Sakai is in use at more than 350 educational institutions, in production settings ranging from 200 to 200,000 users. The Sakai foundation still has around 100 members in its Sakai Partners Program, predominantly academic institutions but also some commercial organisations. A new Executive Director, Ian Dolphin, has been in place since June 2010.

Further reading

Links:

Related information from OSS Watch:

Acknowledgements

The sustainability study from which this case study is taken was commissioned by the JISC Learning and Teaching committee and funded from HEFCE’s IT Infrastructure funds. The Learning and Teaching committe is responsible for supporting the learning and teaching community by helping institutions to promote innovation in the use of ICT to benefit learning and teaching, research and the management of institutions.

The sustainability study was edited by Gaynor Backhouse of IntelligentContent and her editorial guidance has contributed in large part to the excellent result.