Technical Development

A major part of the Mapping Museums project was the development of a knowledge base (the Database), a web application that enables users to engage with this knowledge base, and the project website. This development work was collaborative and involved all members of the research team. It was also iterative, comprising successive cycles wherein we repeatedly identified the humanities researchers’ requirements, and then researched, designed, implemented and tested the evolving system.

The database and web application were developed alongside the archival research and data collection. This meant that they had to be flexible enough to encompass new or changed data, and to continue to do so as the sector changes or as new information was established. We therefore opted to use semantic technologies to describe and store our data. Semantic databases are also known as ‘Triple Stores’ and they store pieces of information in triplets of the form Subject-Predicate-Object. For example, the fact that the Science Museum is located in London would be stored as the triplet Science Museum-hasLocation-London. This kind of data storage can easily be extended with new triplets as new data and knowledge accrue, and it allows us to describe in fine detail the different relationships between entities.

The database is designed so that users can browse in a structured way through categories such as accreditation, governance, location, size, subject classification, year of opening and year of closing, and see the results on a map or in a list view. Alternatively, users can submit a detailed search that allows them to filter results by combinations of categories, or they can generate visualisations of how the different types of museums have emerged over time or create tables showing how the various categories inter-relate. At any point, it is possible to scrutinise the details of individual venues.

Technical Implementation

The figure below shows the architecture of our system. It is a three-tier architecture comprising a Web Browser-based client served by a Web Server connecting to a Database Server.

The web server, Apache/WSGI, uses the Python Flask framework with Jinja template views to deliver HTML/Javascript pages to the client. The pages are styled with Bootstrap and CSS. Communication with the database server is through the Python SPARQLWrapper library.

The Map view supported in Browse and Search has been developed using the Leaflet Javascript library together with the MarkerCluster plug-in to create the clusters. The maps themselves are dynamically loaded using the Openstreetmap service.

For the Visualise facility, the Bokeh library is used to produce the line graphs, bar charts and heatmaps for delivery from the server to the client.

Overall, the web application has been kept simple with a stateless approach free from sessions and cookies. It comprises some 27,000 lines of Python code, as well as additional scripts consisting of some 26,000 lines of javascript, numerous HTML pages and additional source files.

The database is implemented as a Triple Store and it supports a SPARQL end point for communicating with the Web Server. The database is built using the Virtuoso database engine and managed using the Virtuoso Conductor HTML based Administration Console version 07.20.3217.