This site is currently in beta and more functionality and content will be added over the coming months. We welcome your comments. Please click here to provide feedback.

Challenge: Describing and sharing our information

Short description: 

If the Government is to make maximum use of the information we hold we must be able to find it, share it responsibly, trust it, know what we publish and allow users to locate and consume that information. We require rules to make sure we describe our information in a standardised way to support sharing and re-use, and to enable us to understand the information created by others.

User need: 

Citizen - finding and using government information using semantic web search will be enhanced.

Government business - managing, storing and sharing government information will be made easier and more effective with a shared standard including machinery of government changes and systems migration.

Government partners and suppliers - a clear information/data interoperability standard used by government will help systems integrators, suppliers, service providers and partners to create usable systems and reduce costs of re-engineering.

Consumers need to be clear about the ownership and licensing conditions associated with government information and data, particularly when it is published.

Expected benefits: 

Secure and cost-effective management of the full range of information we create and hold (documents, datasets, images etc.)

Effective management of our information for the full life-cycle, from creation, sharing and re-use to timely disposal or archiving.

Interoperability, enabling migration between different environments. For example, sharing or publishing, movement between internal systems, transfer between departments for business use or, ultimately, for permanent preservation as the historical record of government.

Discovery of internal information through effective search and categorisation (collaborative working environments, content management, document management, intranets, etc.)

Government is publishing increasing amounts of information in different forms, from official publications, to open data. Descriptions of information is increasingly used by search engines like Google (with initiatives like schema.org) and improves discovery.

Clear indications of the licence and any third party rights associated with our information.

A single government standard for describing our information will streamline procurement activities, enabling departments to tailor or re-use a common approach to data management, data description and data sharing when defining their requirements.

Functional needs: 

We require rules to make sure metadata is created in a standardised way. Metadata is simply a structured description (of a document or collection of information). Without metadata, discovery and reuse of information is much harder. The purpose of metadata is to help users to locate data and validate information. This supports sharing and re-use, and enables us to understand the information created by others.

Any mandatory elements should be:
1) Low barrier
2) Included by default in most business systems (including cloud based, social media, Software as a Service, etc)
3) Support reuse and transparency across government, including lower costs for data migration activities at system end of life.

The standard should be open, flexible, future-proof, fit for purpose and require no or minimal maintenance by the Government’s standards authority.  It should be tailored to UK Government business requirements. For example, a set of minimum elements for description of digital objects such as images, datasets, geospatial data, sound files, documents. It should define a set of fields that can be used and the format of those fields so that government bodies can ensure a standard presentation that can be used across systems when moving, sharing or searching for data.

The Government’s current metadata standard (e-GMS v3.1, last updated in 2006) is outdated and needs to be revised to support transparency and reuse of data across government and public sector bodies.

The e-GMS v3.1 includes guidance on encoding schemes. In 2012 the Public Sector Information Domain - Metadata Standards Working Group agreed to remove the requirement for the use of IPSV and recommended SKOS. The full announcement can be accessed on the Wikipedia e-GMS page. Other small, incremental changes to the current version of the standard would be possible; however, a more significant revision is required to deliver the benefits described below.

Status: 

Archived

Archive Reason: 

The Open Standards Board recommended in September 2013 that this challenge and its associated proposals and draft standards profile should be archived, and a new challenge be developed which more clearly defines the use case and user needs.

Related meeting minutes: 

Most value from government data will be enabled by timely, accurate and complete data made available in the most expedient and lowest cost format, preferably as an inherent output of existing business processes, with minimal intervention and sufficient metadata to allow consumers to understand its meaning. Multiple standards to publish data permits agility, flexibility and choice.  Single standards constrain flexibility, compatibility, innovation and limit solution choice. Governments and citizens use data and documents in different ways, most governments embrace a policy of neutrality using multiple standards, allowing data producers and consumers to choose the formats that best meet their needs.

 

[Incorporated in a proposal]

Consistent use of standards can be applied to both ‘unstructured’ information i.e. documents, and even moreso, to structured data.  By applying the right metadata, and semantics, we can make joins to discover, describe, and share both documents and  data.

[Incorporated in a proposal]

Standards for metadata should be embedded in standards for data publication since it is the presence of the metadata that creates information from the data by allowing it to be interpreted. A good standard for data publication is one that appears to be emerging as a de facto standard, specifically linked data.

Apply metadata using Dublin Core Terms (http://dublincore.org/documents/dcmi-terms/) and mandate some terms for some circumstances by defining appl;ication profiles.

Populate the subject property with concepts from Simple Knowledge Organization System (SKOS) compliant vocabularies with each conept having a URI that de-references to its precise meaning. 

Express metatadata in a variety of formats, including as triples in 5-star Linked Data  Publicise schemas for data structures that are richer than simple metadata.

[Incorporated in a proposal]

There is a hierarchy of standards for systems interoperability.

Top Level: Modelling principles and paradigms, a common language to use.

Second Level:  Logical Integration Model - Identifying semantics about things and types.

Third level:  Mapping data fields between systems for standard reference data eg metadata for URIs 

Fourth Level - Mapping transactional exchange data - templates for operational interoperating systems 

Fifth Level:  Implementation following the standards above - OUT OF SCOPE.

 

Data from multiple sources and aimed at multiple platforms has been the bug-bear of government since the early 1980s. It stems from the misconception that 'competition is healthy' when viewed by the ordinary layman. This is demonstrated by the NHS Primary Care system that has over six principal service providers - each with a mutually incompatible data system for managing patients. Data needs to be identified and subsequently standardised in order to bring order to chaos. To do this, you need to identify who 'owns' the data and then make that entity responsible for ensuring that their data meets the expected standard. 

This proposal includes a set of minimum mandatory metadata elements for use with documents, information assets and datasets and outlines work to produce guidance for recommended and optional elements as well as how to express the metadata for different types of information assets (HTML syntax for example). The implementation guidance would include approaches to building and linking federated controlled vocabularies that together with metadata support information discovery, management and processing. The approach also ensures that these information assets are published in re-usable form as part of the web of Linked Data.

[Incorporated in a standards profile]

This profile is for describing minimum mandatory metadata elements for use with information containers; it applies to containers of information rather than to individual data.

It outlines work to produce guidance for adding further recommended and optional elements and how to express the metadata for different types of information. The implementation guidance will include approaches to building and linking federated controlled vocabularies that together with metadata support information discovery, management and processing.