Update: The time period for commenting on this proposal has been extended to 17:00 GMT on Friday 28 February.
Citizens, businesses and delivery partners, such as charities and voluntary groups, need to be able to interact with government officials, sharing and editing documents. Officials within government departments also need to work efficiently, sharing and collaborating with documents. Users must not have costs imposed upon them due to the format in which editable government information is shared or requested.
Users in the context of this proposal include citizens, businesses and delivery partners who need to share information with government in editable formats. Users are also officials within government departments who need to share and work on information together.
As technology progresses, government’s production of editable information in formats traditionally associated with documents will become less important for users.
Government services are being redesigned to make them more straightforward and easier to use by making them digital by default. This will diminish the use of traditional government document formatting even further as information is published or collected directly on the web.
This proposal recognises that changes in technology and service delivery will therefore mean that document formats become less important as collaborative editing and transactions increasingly become an online experience. However, documents formatted in office software are still prevalent amongst users of government information and the formats used by government should meet user needs.
Users need to:
- Open, edit and save information online and offline
- Submit information in response to a request, to perform a transaction or to access a service
- Share information with specific people
- Publish information online so that a wide audience can access and work with it
- Edit information and be confident that it remains usable and editable when saved and shared with other users
- Create a new document with the same style as documents previously created
- Export the documents created in a non editable format so that they can share a document as they intend it to be presented
- Export the documents they create in a format compatible with other software so that other people can use the information
- Share information so that they can gather feedback
- Share information so that they can respond to a request for information
- View/edit the information shared with them so that they can read/act upon the content
- Provide input on information created by someone else
- Copy and paste content from one source to another so that they can quickly collate pieces of information in one place
- Edit information created by an integrated system they work with so that they can add additional information
- Gather feedback on information they have drafted so that they can apply other people’s recommendations to the content
- See version updates so that they can be sure they’re working on the latest version of a document
- Access information from any appropriate place so that they can get on with their work
- Their devices not to be clogged up with downloads
- Ensure integrity of specific documents, e.g. audit trail for editing, versioning
- Use the information on the device and platform of my choice, for example laptop, tablet or smartphone
- Be able to use accessibility tools with information in online and offline formats
- Users are able to efficiently share and work on editable government information
- Users are not required to buy new software to submit or work with government information
- Users are able to re-use data and text, where licences permit
The format should support:
- Characters associated with Unicode 6.2 for text based file formats (in accordance with the standards profile for cross-platform character encoding)
- Digital continuity - having implementations that enable support for import of older formats
- Use of metadata
- Imports and exports to/from other applications
- Fonts and graphics that are reusable in other formats
- Creation of templates
Citizens, businesses and delivery partners must be able to interact with government officials and services, or those working on behalf of government, sharing appropriately formatted, editable information.
Users should be able to work on their device of choice and must not have costs imposed upon them due to the document format in which government information is provided or requested.
Documents should be editable on different devices without loss of integrity - the information should not become spoiled. Documents in this context include:
- Word processed text
When dealing with citizens, information should be digital by default and therefore should be published online. Browser-based editing is the preferred option for collaborating on published government information. HTML (4.01 or higher e.g. HTML5) is therefore the default format for browser-based editable text. Other document formats specified in this proposal - ODF 1.1 (or higher e.g. ODF 1.2), plain text (TXT) or comma separated values (CSV) - should be provided in addition. ODF includes filename extensions such as .odt for text, .ods for spreadsheets and .odp for presentations.
For statistical or numerical information, CSV is the required format, preferably with a preview provided in HTML (4.01 or higher e.g. HTML5).
Forms and information exchanges should be digital by default where this is enabled, therefore use of office formats should not be encouraged for the completion of forms.
For information being collaborated on between departments, browser-based editing is preferable but often not currently available. Therefore, information should be shared in ODF (version 1.1 or higher e.g. ODF 1.2). The default format for saving government documents must be one of the formats described in this proposal.
To avoid lock-in to a particular provider, it must be possible for documents being created or worked on in a cloud environment to be exported in at least one of the editable document formats proposed.
Information that is newly created or edited should be saved in one of the formats described in this proposal. There is no requirement to transfer existing information, unless it is newly requested by a user and shared.
- A government body must not refuse to accept or supply a document in at least one of the open formats described in this proposal
- Documents may be shared in other formats but only in response to a specific request from a user
- Existing documents should be migrated to the formats specified in this proposal if they are re-opened for editing or are requested by a user
- Government bodies should avoid bespoke implementations which may limit their ability to migrate information or to share it with other users
- Macros should be avoided wherever possible, particularly when sharing documents.
- Government officials should engage with interoperability testing initiatives for document formats
- Government officials should engage with standards bodies associated with the maintenance of standards that are agreed for document formats for use in government
This proposal, if agreed, would apply to information produced by or on behalf of central government departments, their agencies, non-departmental public bodies (NDPBs) and any other bodies for which they are responsible. These government bodies would need implementation advice to give clarity about when to use particular formats, the user needs they meet and the interoperability that can be expected.
A document metadata profile is outside the scope of this proposal, although this may be the subject of other challenges taken through the Standards Hub process.
Assessment of tools that can be used for providing multiple formats from a single standardised format are also outside the scope of this standards challenge.
- HTML 4.01 - ISO/IEC 15445:2000 Information technology - Document description and processing languages - HyperText Markup Language (HTML)
- HTML 4.01 Specification
- ODF 1.1 - ISO/IEC 26300: 2006/Amd 1: 2012 Open Document Format for Office Applications (OpenDocument) v1.1
- ODF 1.2 - Open Document Format for Office Applications (OpenDocument) Version 1.2
Enables citizens, businesses, charities, voluntary groups and government officials to share and efficiently work with government documents. Users are not required to pay for specific software.
This would apply to information produced by or on behalf of central government departments, their agencies, non-departmental public bodies (NDPBs) and any other bodies for which they are responsible.
The Technical Standards Panel’s recommendation is to:
Adopt a single format for offline editable documents.
Adopt ODF as the standard for sharing and collaborating for UK government documents.
The Panel did not reach a conclusion on the use of HTML as a standard to meet this profile.
Note from the Panel Chair on use of HTML and CSV:
The Panel discussed the difference between the use of HTML to present an interface for sharing and collaborating and as a definitive interchange file format.
The Panel was keen not to rule out online collaboration and the use of digital tools (which were strongly supported) but there was no agreement amongst panel members as to whether they felt able to support the recommendation as is.
While not formally an output of the panel, my sense from the panel’s conversation was that members would be content with using HTML in the context of a digital collaboration tool, however significantly more discussion would be needed to agree HTML for use beyond this, taking clearly into account the difference between an interchange file format and the presentation layer.
I would therefore appreciate a steer from the Board on whether it would be content to adopt the standards as they are currently described or whether they would require changes and/or further assessment by the Panel. My suggestion would be to remove HTML 4.01 and HTML5 from this standards profile, whilst retaining the preference for online editing.
On the use of CSV, there was a strong view from a minority of the panel that CSV should be strictly “comma separated values”. The panel agreed that it would be best dealt with via guidance (potentially expressing a preference for comma separation but ultimately not mandating it). I agreed that I would take this issue to the Board given the strength of feeling on the matter. I should, however, stress that the majority of the panel felt that character separation was more appropriate.
The panel considers that UK government requires one document standard in order to properly meet the needs outlined in this standards challenge. For reasons of interoperability and integrity of documents, adoption of multiple standards would be sub-optimal.
The panel considered the standard proposed and also provided scrutiny on the choice of ODF over OOXML. Following a number of challenges to the representative of the challenge owner and review of the analysis of each choice against the open standards policy, the panel agreed that ODF was the most appropriate standard. There was concern from a minority within the panel that transitional OOXML should have been proposed as a standard; this was assessed by the panel as having larger market take-up than strict OOXML, however failed to meet other criteria.
The scope of this standards profile is new documents produced by UK Government. The Panel discussed different scenarios, such as interoperability with other institutions, which might require different standards. The Panel agreed that these edge-cases were best met through the “comply or explain” process.
The Panel agreed with the draft standards profile that highlighted that the proposed standard met the requirements listed under these topic headings.
The Panel felt that the answers supported adoption of the standard, with the exception of positive impacts on the environment or on privacy, and around transfer of maintenance if the maintenance organisation ceased to function. In these cases, this was due to an absence of positive evidence rather than any evidence that there would be a negative impact in these regards. Given this, the panel was content to recommend adoption despite this lack of evidence.
There was a strong debate both through the Standards Hub and at the Technical Standards Panel regarding a multiple format approach for offline editable documents. In particular with reference to the use of OOXML along with ODF.
There was a vast majority of public support for a single format approach. The Panel was cautious not to be swayed by popular appeal and considered this issue carefully, particularly with regard to impact on users and on costs to government.
The Panel agreed with the proposal and reached a consensus that having two standards within government for offline editing of documents would not be appropriate for the reasons of document integrity or for exchange.
The vast majority of support in the public comment period was for ODF. There was a small minority suggesting use of OOXML too. The Panel considered the analysis of ODF and OOXML in view of the UK Government definition of open standards. The Panel reached a consensus that agreed that the challenge owner was correct in proposing ODF over OOXML.
Other standards were also suggested including:
PDF (for sharing and collaborating)
File compression standards
Standards for complex data
PDF - An interesting scenario of dynamic document generation using content derived from databases to create PDFs was described. The Panel agreed that this does not fit with the current use cases in the creation of government documents.
SDF - The Panel agreed that this format does not have sufficient uptake for us to adopt at this stage.
XML - The Panel agreed that XML with document specific semantics for element and attribute names, may satisfy other user needs and could be the topic of another suggested challenge.
The Panel agreed that standards for file compression and complex data were outside of the scope of this challenge.
The Panel agreed that there would be a long term benefit to service delivery on implementation of this standards profile. In the short-term there might be additional work and cost to implement the standard.
The Panel requested that the Government Digital Service provide appropriate guidance and / or design patterns for departments so cross-government cost is minimised and that implementations are properly planned to avoid disruption to service delivery.
The Panel discussed the evolution of the standard from 1.1 to 1.2 and felt that the backwards compatibility of the ODF standard was important to ensuring that users’ needs were met. The Panel acknowledged that backwards compatibility with existing product sets was important and was heartened by the data that showed that existing products could use ODF.
The Panel noted that there would not be immediate conversion of older formats to ODF and the ability to maintain access to legacy document formats would be an important part of implementation.
Changes to the emerging HTML5 specification may create uncertainty for implementers.
There may be a threat from competing standards that could limit adoption of ODF, however it has so far proved resilient and adoption levels are encouraging.
ODF 1.2 will shortly be submitted to ISO. Work has already begun on ODF 1.3 in OASIS.
There will be increased interoperability for users in government
Users will not have to pay for specific software to view government information
Users will not be digitally excluded due to the technology choices made by government, improving engagement, transparency and accountability of government
Government officials will be able to work more efficiently, by collaborating on interoperable documents
There will be clarity for suppliers and transparency in public procurement regarding which standards are in use in government, encouraging a level playing field
Less converters and bespoke integration are likely to be needed over time
The selection of an open standard will stimulate competition for government contracts and help to decrease costs for government of office productivity tools. Total cost of ownership (including exit costs) will continue to be considered by government organisations on a case by case basis
The cost of lock-in will be reduced over time
For new procurements, implementation should begin as soon as the Open Standards Board has reached agreement.
Government bodies will be required to work with GDS to publish implementation plans to signify when implementation will be completed in existing systems, including with relation to integrated systems.
GDS should coordinate guidance to help departments and users to transition.
Government officials should consider engaging with standards bodies associated with the maintenance of standards that are agreed for document formats to ensure our users needs are considered.
Government officials should engage with interoperability testing initiatives for document formats to ensure that our users needs are considered.
- Additional challenges, for example on fonts, graphics and images should be considered through the Standards Hub process.
There are tools available to test certain conformance criteria of an ODF document. Validators are provided through the ODF ecosystem.
ODF uses the ISO Relax-NG standard as a formal description of an ODF document. http://relaxng.org/
Plugfests have been a mechanism for conformance testing for ODF between application vendors in the past.
The OASIS ODF Interoperability and conformance testing TC (closed as of 2013) was responsible for helping implementors create applications that conform to ODF. Examples of validators are provided: https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=oic
An Interoperability demo during OpenOffice.org conference in 2010 highlighted what was working well with ODF 1.2: https://www.oasis-open.org/news/pr/oasis-interoperability-demo-features-eight-open-source-and-commercial-implementations-of-ope
Existing fora, consortia and standards body specifications are being referred to, notification was not therefore required. However, for standards from OASIS and W3C, these will be referred to the Multi Stakeholder Platform.