0% Complete

About the Openness Rating

Is your software open or fauxpen?

This is the question that OSS Watch, in partnership with Pia Waugh, developed this ‘openness rating’ tool to help you find out.

Unlike earlier models designed to evaluate open source projects, this model can also be applied to both open and closed source software products.

Use the questions under each of the tabs to get a quick assessment of the "Openness Rating" of a project. These questions are based on the paper Foundations of Openness by Pia Waugh and Randy Metcalfe

OSS Watch use this as a diagnostic tool in our consultancy work. For more information, and to engage our help, see our services page.

We'd love your feedback too - just send an email to info@oss-watch.ac.uk.

Data Formats and Standards

Full public disclosure of formats is the only practical way someone can implement the standard properly in another system. It is also the only way to future-proof data and systems and ensure there is always a mechanism to replicate and access the data or systems.

Proprietary standards can limit a project's potential and interoperability. There are of course some projects that use proprietary standards for interoperability, however this question is about whether a project could not work without the proprietary standard depended upon.

Costs associated with either acquiring the standard documentation or in implementing the standard are a barrier to entry that limits the potential of the standard.

Industry, de facto and published standards such as the Microsoft doc format can be popular, however arguably less open than a standard which has gone through a process of peer review, support and publication by a trusted and international standards organisation.

Managed processes make it much easier for third parties to see where they can engage with the project.

Unicode support means better opportunity for multiple language support.

Knowledge

Multiple documentation and communication components are indicative of at least the opportunity for project knowledge to exist. There are certainly cases where too many avenues of knowledge can hurt a project.

If major project communications are encouraged to be done through the main channels, then the chance to lose major decision making processes and information dissemination in private conversations is limited.

Making a decision in an environment that is not accessible to all interested parties limits awareness of those decisions and prevents participation in the decision making process.

The intent to keep knowledge private is not great for knowledge openness, however there may be specific reasons to keep the knowledge private, such as is the case for legal or privacy concerns.

Apart from any knowledge defined as private, the ability for anyone to acquire project knowledge is important to their ability to participate as well as for the sustainability of the project.

If there is any legal or financial barrier to accessing knowledge, then that provides a barrier to entry and participation, and thus makes the project less open. Note that requiring the purchase of proprietary software in order to access project knowledge is an example of a financial barrier, as is the need to pay for access.

If there is any technological barrier to accessing knowledge, then that provides a barrier to entry and participation, and thus makes the project less open. Examples of such barriers include DRM or deliberate limitation to a specific operating system.

If the knowledge is stored in proprietary formats then it is more likely the data won't be accessible in the long term which is a risk to the long term openness of the project.

The previous questions are determining whether there are any artificial limitations to accessing project knowledge.

Understanding who can contribute to the knowledge is a good way of understanding the potential for participation in the knowledge.

If there is a single point of loss or failure in the knowledge base, then projects put themselves at risk of massive and possible permanent interruption.

Without clearly documented processes for data recovery it is more likely that data archiving and recovery will have full coverage. In addition, reinstating data in the event of failure will be slower and potentially flawed without a documented process to guide recovery.

The usability of knowledge is a difficult thing to measure. This could probably be captured through extensive subjective questions, however asking people to rate the quality is a good start to understanding and differentiating between base standards of good documentation and projects that really excel in this area.

The usability of knowledge is a difficult thing to measure. This could probably be captured through extensive subjective questions, however asking people to rate the quality is a good start to understanding and differentiating between base standards of good documentation and projects that really excel in this area.

The more external sources of documentation, the more knowledge there is available that is likely done professionally or to cater for extra use cases. This could be external community documentation or professional publications.

Governance

Leadership may be an individual or group such as a board. Clear leadership means a project has a better chance of clear direction and purpose, and is more likely to avoid leadership contesting and the sort of committee based decision making which can slow a project down to a grinding halt.

Public documentation of a project structure and policies increase transparency and trust in a project. This should include all of leadership structure, decision making proceses, the process for becoming a contributor and maintainer and the licence of the software.

Provides a public reference by which project members are held accountable. Encourages a good working environment that is productive and welcoming to newcomers.

The availability of such documentation encourages use and contributions to a project, so it is important to facilitate new interest in a public fashion. If a project doesn't make this available it simply makes it more difficult to get involved in any capacity.

Elected leadership indicates a more openly participatory project. It is true that many projects have only one maintainer, and thus they are handicapped by this question, however it is also true that smaller project do not require as open a governance as larger projects.

Contributing to the software is defined as any activity which adds to the project. Includes code, patches, bug reports, documentation, and translation. A project may choose to limit who can contribute to a project for various reasons of control, however this limits the potential of the project. Note that we do not mean "who can gain write access to the resources", contributions can be in the form of ideas, documentation, patches, bug reports, feature requests, testing, major software additions etc.

It is critical for a project to ensure that all code can be released legally under the chosen licence. Contributor licence agreements enable the project to show they have taken all reasonable steps to ensure this is the case.

This is a person who commits code/changes to the primary project source. Understanding who is able to become a committer is indicative to the openness of a project to share responsibility at the code level. By "commit rights" we mean those who have write access to project resources.

Single point of failure/control refers to both the case of a single individual and the case where all committers work for the one company. A single point of failure/control for committing changes to the codebase provides the opportunity for massive project disruption, whether it be through an individual not having time or in all committers working for the one company and introducing the risk of hostile takeover. If there is a single point of failure, either individual or company-wise, a documented succession plan can make the difference between seamless progress of the project or a major disruption.

The terms of use in the licence of a project does not mean that in actual practice the software is publicly and openly available for public use. the more barriers to entry for use of the software, the less open.

Enabling users to quickly download, install and test a software product ensures that the project generates additional interest in the project and thus increase the chances of new contributors being attracted to the project.

A project that is predictable and consistent is more likely to encourage regular community participation and interest than a project that is inconsistent and unpredictable.

If the codebase is unable to be openly forked, then the project can be held ransom to bad leadership or hostile takeover.

If there is no avenue for recourse, the project relies on the good will of the maintainer, and thus opens up the likelihood of forking under bad leadership.

Market

The higher the costs of setup, the higher the barrier to entry for creating an open market around the project. Also a set cost is far less an overhead than ongoing costs such as royalties or patents.

Technical barriers to entry, such as DRM or proprietary hardware, reduce the ease of building an open market around the project.

If so this gives one company a potential market advantage over competitors.

The more contributors that are able to work on a project with business support, generally the more market ready it is.

If the software is only applicable to one industry, the market opportunities are limited.

The more revenue models that are available the better the opportunity for building a market. The more businesses that are already involved, the more the project is already succeeding in the broader market space.

The more business are already offering services around the project, the more open it most likely is for a new project to come along and build a business.

Your results

Standards

Knowledge

Governance

Market

Share your results