Open source: facing a skills shortage

by Ross Gardler on 7 July 2009 , last updated 11 February 2013

Introduction

Open source software has emerged as one of the most important IT movements in recent times. According to a study carried out in 2006 by analysts IDC, ‘Open source is the most significant all-encompassing and long-term trend that the software industry has seen since the early 1980s’, and it is gaining momentum. Evidence of this can be seen in all sectors in the UK, but a severe shortage of skills in this area could limit its future. This document looks at the growth of open source in the UK, and considers the possible effects of this skills shortage on the long-term success of open source, and how the problem might be addressed.

Open source in the public sector

In the European Union, the use of open source has, for many years, been seen as an appropriate means of providing maximum economic value. A survey conducted by Maastricht University in 2005 concluded that nearly 80% of European local governments were using it, consciously or unconsciously. Today, governments across Europe are still actively encouraging, or even requiring, future IT projects to consider open source as an option. The European Interoperability Framework v2.0 is aiming to level the playing field for open source software by specifying requirements for the specifications that allow for proprietary and open source implementations alike. Some argue however that these conditions are not favourable for free software licenses.

The UK government, reflecting the trend towards open source software, defined a policy in 2004 that aimed to deliver value for money by ensuring that procurement in the public sector considers open source alongside closed source. It also stated that software resulting from publicly funded research should be sustained through commercialisation and/or open source licensing. This policy was re-iterated in 2009 and 2010. In 2011, the government released a toolkit to help dispel myths about open source and help levelling the playing field further.

Open source in the commercial sector

The for-profit sector is adopting open source even more readily than the public sector. For example, a survey of nearly 1000 IT staff in the UK, Germany, France and North America, commissioned in 2008 by Actuate, revealed that 54 per cent of UK businesses responding to the survey felt that the benefits of open source outweighed any negative aspects. This is a significant increase over 45 per cent in 2007. This upward trend appears to be based on actual experience, rather than on expectations, with 43 per cent of responding businesses in the UK already using open source. A similar trend was identified by the aforementioned IDC study. This study found that of the 5000 survey respondents, 71 per cent of developers are using open source software, and it is in production at 54 per cent of their organisations.

This growth seems set to continue. IT research firm Gartner predicted in 2008 that ‘By 2012, 80 per cent of all commercial software will include elements of open-source technology’. The firm quickly revised this figure to 90 per cent, stating that open source technologies ‘provide significant opportunities for vendors and users to lower their total cost of ownership and increase returns on investment. … Embedded open source strategies will become the minimal level of investment that most large software vendors will find necessary to maintain competitive advantage during the next five years.’ Gartner have subsequently revised this prediction saying “By 2016, at least 95% of IT organisations will use non-trivial elements of open source software in mission-critical IT portfolios.” IDC, apparently bearing this out, believes that open source will eventually play a role in the life-cycle of every major software category. Open source will be an important part of the development of software and, as a result, will have a direct influence on the cost of software development. This will result in a reduced total cost of ownership, for the customer, of all software, regardless of the licence model used. Even Microsoft, historically one of the most vocal opponents of the open source movement, has been using open source components since Windows 2000. Today, open-source-related announcements come from Microsoft on a regular basis, and the company has recently been engaging directly with a variety of open source development projects, such as Apache POI and Hadoop. In addition, it has had two open source licences approved by the Open Source Initiative and hosts and releases software under those licences. Also, Oracle recently announced a partnership with Cloudera to base their Big Data Appliance on Apache Hadoop products.

Open source in education

The adoption of open source software in the educational sector has been similar to that in the public sector, and is also on the increase. The most recent OSS Watch National Software Survey in 2010 concluded, that about 60% of Further Education institutions and 75% of Higher Education institutions explicitly mention open source in their procurement policies.

However, a look at the current landscape of software actually installed on servers and desktops in FE and HE reveals that most of it is still closed source. This has largely to do with the inability of knowing how to evaluate open source and specifically knowing how to calculate the Total Cost of Ownership of open source solutions. OSS Watch is working on tools to help assess open source projects, for example with the Software Sustainability Maturity Model.

One particularly successful, and encouraging, exception to this pattern of limited uptake in education can be found in the adoption of virtual learning environments. Here, the open source application Moodle is in use in the majority of institutions.

To help inform procurement decisions in educational establishments, OSS Watch maintains a list of Open Source Options for Education, containing open source software which provides solutions specific to education for consideration alongside proprietary alternatives.

Skills shortage

We have seen that the use of open source in the UK is growing in every sector. But all is not well in this increasingly open future. In research conducted for Actuate, six in every ten respondents mentioned ‘serious problems finding the right IT skills to implement and manage open source solutions’. This research is borne out by the OSS Watch survey mentioned earlier, which found that a lack of staff expertise was the main reason for not adopting open source on the server, and the secondary reason for not adopting open source on the desktop. As more open source is adopted across all sectors, this skills shortage will become more acute, since there are currently very few educational or training organisations offering courses that focus on open source, open standards and open development.

This lack of skills does not reflect a lack of user skills, since users need to learn to use any new system, regardless of the software’s licensing and development model. Instead, it reflects a lack of skills on the part of IT staff, IT managers and procurement officers. This is a basic lack of understanding about how open source is developed, marketed, sold, licensed and supported, and of the benefits of engaging with open source projects. The result is that open source solutions are often not considered during procurement exercises, and when they are, their evaluation is flawed. Open source and closed source solutions cannot be effectively compared using existing techniques and skills. While the software products themselves can be compared on a feature-by-feature basis, the ‘softer’ aspects, such as quality of support, security, flexibility and sustainability of the solution, cannot easily be compared like for like.

The reason for this lack of skills is simple. Very few courses in the use of open source are available, and the people who do understand open source licensing, development and deployment are able to command higher than average salaries in the for-profit sector. They are therefore not drawn to the education sector. Consequently, there is a shortage of teachers and trainers to equip new graduates and existing staff with the skills required by employers and vital for the ongoing development of open source.

How do we tackle the problem? We need to increase the level of available skills through education and training and we need to change the software procurement process to ensure open source options are correctly evaluated. Simon Phipps, who headed up Sun Microsystems’ open source programme at the time of writing, suggests one potential model for a level procurement playing field through adoption-led approaches, and warns about how the existing process can be abused.

OSS Watch has been addressing these needs in the education sector for some time and, while we have had considerable success in some areas, the statistics show that we are only just beginning to see the fruits of our labours.

Conclusion

The use of open source is undeniably on the increase in all sectors: it is seen by the public and education sectors as a way of providing better value for money in public spending, by the for-profit sector as a way of cutting costs in product development, and by the software development sector as a way of better serving its customers. However, this growth cannot be sustained without a fundamental change in the procurement process and an increase in the skills necessary for implementing, managing and engaging with open source projects. Without these skills, most policies and action plans will fall on fallow ground. Or, worse still, open source will be rolled out in a cultural environment that is not conducive to the success of the project as a whole. To get the most from any engagement with open source, it is essential to understand the whole model from beginning to end. This will ensure that a complete and thorough evaluation of the options presented by open source can be made. Open source is much more than a licensing model, and failure to understand this will result, at best, in a sub-optimal open source experience.

OSS Watch provides unbiased advice and guidance on the use, development, and licensing of free software, open source software, and open source hardware.