Documentation issues in open source

by Stuart Yeates on 1 November 2005 , last updated

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

Many open source projects face significant challenges generating and maintaining high quality, end-user documentation. Some of these challenges are well known in software engineering and common across the computer industry in both open source and proprietary realms. Others are peculiar to open source projects.

Generic challenges to software documentation

The causes of poor or lacking software documentation have been widely studied within academic and industrial software engineering. The lack of progress suggests that there is no easy solution even when the causes can be pinpointed, and workarounds are hard to come by. They include:

  • Direct benefit: the value of documentation is inversely proportional to an individual’s ability to write that documentation. As a result developers are often reticent to fully document their work.
  • Skills: writing documentation requires a specific set of skills that are not commonly found within open source development communities.
  • Time: writing documentation requires time, which is often lacking, particularly in the race to complete software by a deadline. Software developed using the open source model may suffer less than proprietary software in this regard, because it has less need to focus on deadlines. However, many open source projects, especially end-user focused projects, commit themselves to a strict release schedule.
  • Change: during development, software changes frequently, requiring matching changes in the documentation. This makes it tempting to leave the documentation writing until the software is finished and no longer changing. Of course by that time many details will be forgotten and the time left before release will almost certainly be very short.
  • Libraries: much software is written on a component or library basis, whereas it is experienced by the user as an entire system. Even very good documentation of a component or library can be irrelevant or unhelpful in the context of an entire system.
  • Level: many programmers have been deeply immersed in the software industry for years. They have a deep knowledge not only of their own software, but computers in general. This can make it challenging for programmers to write effectively for end-users because their world-views differ on so many important points. Imagine a world-class mathematician attempting to teach arithmetic in school. Of course there are special individuals who can succeed at both levels, but it is rare indeed.

In traditional software engineering contexts, the normal response to these issues is to employ technical writers, who have dedicated time, the appropriate skills, and a more generalist world-view than software developers. Technical writers are brought in towards the end of the software development life cycle, when the structure of the software is clear but before the software is released to beta testers, to mitigate against software change issues.

Challenges to documentation in open source projects

The issues that exacerbate the problem in open source projects include:

  • Perceived focus on developers: open source projects sometimes appear to revolve around software developers to the exclusion of other contributors. This is unfortunate as the best people to write user documentation are the users themselves.
  • Excitement: writing documentation is not perceived as an exciting or sexy activity. In open source projects where contributors have significantly broader freedom over what to contribute, very few of them contribute to anything that doesn’t excite them.
  • Diffuse information: the bulk of information generated by open source projects is often in mailing lists, forums, and chat logs. Few projects have mechanisms for integrating the useful information into formal documentation.

Meeting the documentation challenge

These issues can be potentially offset within an open source project by consciously and explicitly valuing documentation. Some of the ways different open source projects do this include:

  • Requiring structured documentation along with every contribution of source code. This is used by many communities, such as Perl and Plone, to maintain the quality of their user-contributed libraries. Unfortunately, while it works well for library level documentation, it only works for new contributions, and so cannot be used at the system-level.
  • Making mailing lists, chat logs, bug reports and as much other project information as possible accessible to search engines and bookmarking systems. Mailing lists and chat logs contain a wealth of information about the project, and by exposing these to search engines and other tools, users can help themselves to already-answered questions.
  • Encouraging new, but articulate users, to contribute documentation as their first contribution to the project. New users are ideal for writing system-level documentation aimed at new users. They have the point of view and experience with the current software at the system level rather than at the library level. The most common evidence of this is FAQ lists, in which issues repeatedly raised on mailing lists and chat sessions are answered. Wikis, bug tracking systems, and FAQs are great first steps.
  • Allocating explicit resources to documentation writing. Some projects with access to external funding (Eclipse, OpenOffice, SuSE Linux, etc.) use some of those funds to employ specialist technical writers and these projects tend to have polished end-user documentation.

Perhaps the most important aspect of documentation in software engineering is listening to end-users’ questions and problems. End-users are the best (and in many cases only) form of feedback that many projects get. Documentation can be improved by first answering end-users’ immediate questions and then stepping back to examine and address the underlying causes of the problems.

Further reading

Examples of open source software that provides good documentation include:

Related information from OSS Watch: