Beyond INSPIRE: Using the Scottish Spatial Data Infrustructure (SSDI) as a metadata store.

On September 12th 2017 at the One Scotland Mapping Agreement (OSMA) Annual Conference, the Spatial Hub presented a case study on the semi-integration of the Scottish Spatial Data Infrastructure (SSDI) in the Spatial Hub data publication workflow. This blog post includes a summary of this presentation, the slides shown to the attendees, parts of the code used to deliver the SSDI semi-integration, and what we think needs to happen to allow the SSDI to fully integrate with our organisation.

Presentation summary: In a technical context the Spatial Hub includes software & tools which deliver the following objectives:

(a) Communication: Two way communications with Scottish local authorities to support community building
(b) Data collection: collect the local authorities spatial data,
(c) Data processing: amalgamate all individual Scottish local authorities spatial data
(d) Data publication: publish the amalgamated national datasets using Web services & popular GIS formats.

The Spatial Hub delivers the above objectives using a technology stack which is founded on our technical values; using open source software and re-using software. To support these technical values the Spatial Hub administers & maintains CKAN, PostgreSQL, Python, WordPress, and Geoserver to collect, process, and publish data, and re-uses the Knowledge Hub (KHUB) and SSDI to supplement data publication & communication with the local authorities’ community and to make data discoverable.

During the Spatial Hub data publication process one of the team members manually creates a new metadata record or updates an existing metadata record in the SSDI. The metadata held in the SSDI make the Spatial Hub data INSPIRE compliant (i.e. ticking the box of making the data discoverable) and can also be re-used to support other applications such as the Spatial Hub “Get Data” web page. The purpose of the “Get Data” page which is part of the Spatial Hub WordPress instance is to inform the Spatial Hub data consumers on the published data and their associated metadata in a user friendly way. The information which are needed to populate the “Get Data” are stored within the SSDI and the Spatial Hub Geoserver instances. Once sourced from SSDI & Geoserver these data can be re-used instead of duplicating that information within WordPress. The OGC compliant Catalogue Service for the Web (CSW) which is supported by GeoNetwork (i.e. SSDI) is used to source the held metadata and the Geoserver REST API is used to source the published web service data. For example in the Spatial Hub Get Data page the record “Air Quality Management” includes a “Description” paragraph and a “Metadata” link which are both sourced from the SSDI using the CSW before being populated. The CSW request to source all Spatial Hub published data is HERE. Clicking on the CSW request the browser will return all the records that the Spatial Hub has published in the SSDI in an XML format.

Slides:  The slides which were shown to the attendees at the OSMA event can be found at OSMA Event – September 2017 – Spatial Hub & SSDI

Code: The python programming language is used to source the SSDI information before being re-used. This code has been published under GNU General Public Licence in an open Improvement Service GitHub repository.

The way forward: There is an opportunity for organisations to use the SSDI as their key metadata store instead of using it only as a one off tool to comply with parts of INSPIRE. This will remove the need from organisations to install, maintain, and administer their own instance of GeoNetwork or other metadata stores to support their metadata needs. To achieve this goal the SSDI needs to invest on a sustainable API which will be built on top of the GeoNetwork functionality and also produce extensive documentation to help organisations use it. The SSDI API must allow HTTP operations including CREATING, UPDATING, DELETING, and GETTING information to assist with the automation of metadata record management. For example to fully integrate SSDI as the metadata store to support the Spatial Hub & any other organisations’ operations HTTP POST, HTTP PUT, and HTTP DELETE requests must be available to allow automatic metadata record creation, update, or deletion whenever there are changes in the data linked to the metadata record. Currently we have found that this is only possible with manual effort.