Governments achieve lowest operational costs and most flexibility when they embrace a small number of popular and widely supported open standard formats. Governments should not expect files in any single standard to be readable or editable by every potential recipient. Remaining flexible and choosing applications which best support their range of open standard formats offers the prospect of engaging with citizens by allowing for a degree of latitude in how that collaboration takes place. By focusing on "how to get the job done", rather than "how the job is done", governments are able to improve the quality and value of the services they provide for citizens.
In our original response to the Open Standards consultation, we articulated why we don't believe that choosing and mandating single standards offers the most flexibility, the opportunity to secure the lowest operating costs for governments (or, indeed, for any organisation) or provide the best experience for users.
Indeed there is plenty of evidence from around the world to show that where single standards are chosen and mandated, operational costs increase and the decision or the mandate is ignored as users take a pragmatic approach in order to "get the job done".
With document formats, there are some important principles to consider when deciding on the use of one format or another. Many organisations have already faced and dealt with this challenge. They found that aligning the organization on one "suite" of products from a single vendor lead to economies of scale in procurement, guaranteed interoperability within the company and workforce efficiencies through familiarity.
But companies still needed to communicate with their customers, their partners, the government and, perhaps, the public. They also had to decide what to do - at the point of change - with all the "legacy" documents that were held in one format or another spread and stored throughout their corporate ICT infrastructure. Many companies developed and then shared a set of best practice principles. It is, perhaps, almost 20 years since such first appeared, but they may be as relevant today, to the government, when considering document format standards.
Here's a summary of the important principles. Considering creators or recipients INSIDE an organisation.
- You should know what the range of applications are in use and what range of standards they support. Further, an organisation should be able to choose the applications it needs in order to most effectively support the range of its business process operations at the lowest total cost of ownership.
This means there could be a range of applications in use to create, edit and share "content" and they may support a range of standards. Indeed, pragmatic organisations will default to a small range of standards that have maximum utility across the organisation, but avoid mandating any in order to retain maximum flexibility. Today, popular productivity applications support the full range of standards, providing the needed flexibility. Allowing the users some choice permits (indeed, encourages) innovation and flexibility in order that they can "get the job done".
Considering creators or recipients OUTSIDE an organisation.
- You can make NO assumption about the ownership, the vendor, the version or even the installation of any given productivity tool by ANY recipient outside of the organisation.
This means that in today's rich and varied information technology environment there will always be someone, somewhere with whom you need to communicate who may not have the ability to receive, open, read and/or edit, save and send back a document, regardless of content type, in any format you may choose.
This is true for a government department communicating with any member of the public or a company communicating with a supplier, partner or customer. The larger the recipient base, the more likely it is you will find someone who does not have any application that could process a document you send them. So the often quoted example of a small business having to spend money on software it didn't otherwise need in order to open and deal with a document sent it by a government department will not be prevented by selecting a single standard with which departments must communicate with the world outside. Indeed, one can argue that selecting a single standard will only make such stories more likely.
The best way to reduce such stories in the future is to embrace a range of open standards acceptable to the majority of recipients and to provide users with flexible tools that allow them to easily respond to any request which might say "I can't open the document you sent me, can you send it in another format?"
Sharing or collaborating on government documents
Organisations generally use the same application and the same format for "viewing" and "sharing/collaborating" and trust that recipients intended only to "view" (i.e. read and understand) the content would not edit it and claim it communicated some alternative meaning.
Organisations that require the content and layout to be reproduced faithfully, including for printing, but would prefer to eliminate the risk of un-intended changes will typically share that content in open standard formats like ISO/IEC 32000 or ECMA-388.
Having considered "viewing" documents in editable format, we can turn to sharing and collaborating proper, by which we assume that the original creator of the content may choose and expect others to be able to change both the content and the layout (and other metadata) associated with the document.
What is the user need?
Clearly, like documents intended only for viewing, there are perhaps conflicting requirements for the different types of users of a document intended for "sharing and collaborating".
- Does the creator expect all content and layout to be editable?
- Does the creator expect only the content to be editable (form filling)?
- Does the creator expect only the layout to be editable (like a spreadsheet where the data is fixed, but the user can "pivot" the data or render it into a number of different graphs)?
The corollary to each of the above also applies, i.e. does the recipient expect to be able to change both content and layout, which need not match the expectation of the creator.
Collaboration also takes a number of forms. You may have "full" collaboration, where multiple editors can create their own content and edit content (and layout) created by others and you may have "contribution" collaboration, where each editor can create and edit their own content, but not edit the content created by others. And in yet other cases you may need collaborators to be able to comment on content created by others, but not to edit it directly (i.e. they can see it, but not edit it nor add their own content).
For example, a government department could use an open standard format to create a form to be completed by citizens (form recipients). There are two collaborative processes here. The creation of the form and the completion of the form with content by the form recipient.
Can any open standard format guarantee support for both phases of collaboration around the one document (a form)? Or will users have to create the form using one process in one open standard format and then, when it is complete, publish it to another open standard format to enable it to be understood and completed by recipients?
Experience indicates it’s best to allow those users at each stage to choose a format which best supports that part of the process on which they collectively and collaboratively work and then allow the possibility of retaining the format or switching to another format for the next stage of the collaborative process.
Those who have tried to guess or to force a single format have most often failed (and reverted to a more pragmatic tolerance of multiple open standard formats). Those who are still trying to promote or force a single format are either ignoring the pragmatism of their users or tolerating (but denying) higher operating costs. We know of no governments who have tried to standardise in this way who have not reverted to a pragmatic tolerance (sometimes explicitly declared) of multiple open standard document formats after a few years.
Governments achieve lowest operational costs and most flexibility when they embrace a small number of popular and widely supported open standard formats.
Governments should not expect files in any one standard (or another) to be readable or editable by all recipients. Remaining flexible and choosing applications which best support their range of open standard formats offers the prospect of engaging with citizens by allowing for a degree of latitude in how that collaboration takes place.
By focusing on "how to get the job done", rather than "how the job is done", governments are able to improve the quality and value of the services they provide for citizens.
An approach of supporting a range of open standard formats will enable users to share and work on editable government documents without requiring the vast majority of users to buy or install new software. Such an approach also allows citizens to submit documents to government in a manner that enables government users to re-use data and text when appropriate.
An approach of supporting a range of open standard formats is the best way to ensure that the functional needs set out in the challenge can be achieved as detailed in our response to the user need section.
Governments may consider setting up an interoperability testing programme, to demonstrate the success or short-comings of its chosen standards against the intended purpose. While this is entirely possible, it cannot be recommended because the costs of such a programme could be considerable and the outcome uncertain. Even testing just one format against a small number of applications that can deal with that format can quickly escalate into many tests, not only due to the combination of applications (one to another and back again), but also because of the many features supported by any format, which may – or may not – be supported by design by all applications. In general, we would recommend leaving interoperability testing to application vendors closely engaged in the relevant standard setting processes. Vendors work closely with the relevant standard setting bodies (indeed many vendors volunteer staff to work in the technical committees of those same standard setting bodies) to ensure the standards do what they claim and that their applications follow the standard faithfully. Governments (and others) are welcome to participate in many of these bodies and the processes they follow, so it would seem redundant for a government to need to set up its own interoperability testing programme.
Indeed, in a world where we now routinely use many different devices to view and edit content (even if we are performing only one role in the creation and collaboration process), document fidelity across multiple devices must now be considered.
This is partly a consequence of people in the modern world embracing multiple devices using them in different situations and contexts and partly that each different device will carry a portfolio of applications that deal with different formats in different ways. Because of different device designs (and intended uses) there is no guarantee that any one device and the applications (or “apps”) installed on it will deal with any document format in the same way as a different device even in the same device class or even with versions of the same application ported or re-written for that different device, let alone different applications.
Many countries and their governments have already faced the same or similar questions to those posed here. It may be of interest to consider the choices made by near neighbours to the UK.
Several (but not all) European countries have set a list of standards to be used for document formats (and other topics). While we have little visibility of which formats are routinely used in practice, it is perhaps informative to see that most choose to embrace multiple standards.
We offer that there is anecdotal evidence to suggest that even those who choose a single format are pragmatic enough to use multiple different formats at different stages of the creation and collaboration process, resorting perhaps to one of their chosen formats only as a “last step”.
Please find below a list summarising the choices made by a representative selection of relevant EU Member States.
Germany has chosen OOXML + ODF (SAGA, page 36, section 7.7) which came into force on 3 Nov 2011
France has chosen OOXML + ODF (RGI page 61, section 3.2.1) which came into force on 12 May 2009
Spain has chosen OOXML + ODF (Catálogo de estándares, page 7 and 8) which came into force on 31 oct 2012
Netherlands has chosen ODF (https://lijsten.forumstandaardisatie.nl/lijsten/open-standaarden?lijst=P...) which came into force in 2008
Poland has chosen OOXML + ODF (page 11 and 12)
European Commission has chosen OOXML + ODF, with a stated preference for OOXML for inter institution exchange (see “conclusions on document exchange formats”, page 3) which came into force on 25 Jun 2011
In conclusion, it can be seen that many European countries have decided to support several document formats. Often other simple standard formats are also supported, such ASCII text (TXT) or Comma Separated Values (CSV), which may well contain ASCII text.