Almost there with data collecting!

Many thanks to the various people who have provided data for the Englaid project over the last month. It really feels as if we are coming to the end of the data gathering stage of the project now and can begin to work with all this information, which is quite exciting.

With a few exceptions (mainly due to changes of personnel and HER closures) we now have the HER data for the vast majority of England (see below). We are particularly grateful to Julia Wise (Buckinghamshire), Ian Scrivener-Lindley (Chichester), Gill Stroud and Nichola Manning (Derbyshire), Claire Pinder (Dorset), Pete Boland and Jennifer Mincher (Dudley), Alison Bennett (Essex), Andrew Armstrong (Gloucester City), Mark Bennet (Lincolnshire), Susan Lisk, (Oxfordshire), Alison Yardy and Richard Hoggett (Norfolk), Chris Addison (Northamptonshire), Ben Wallace and Caroline Rann (Solihull), Ken Crowe (Southend on Sea), Suzy Blake (Staffordshire), and Ben Wallace and Caroline Rann (Warwickshire) all of whom have sent HER data through to us over the last few weeks. Thanks, once again, to Keith Westcott at Exegesis for his continued efforts to assist HER professionals in processing the Exegesis data query.

Further thanks go to Simon Crutchley and Poppy Starkie at English Heritage who have provided additional tranches of NMP data (we now have NMP data for all but the north west and north east of England), and to Caroline Keay and Sara Larman at the National Soil Resources Institute (NSRI), Cranfield University, who have kindly provided digitised maps of the 1983 soil survey of England, which (as you’ve probably gathered from Chris Green’s recent postings) have provided plenty of food for thought!

Processing raster NMP tiles (part 2)

In my previous post on vectorising raster NMP tiles, I concluded that for certain purposes it would be more analytically useful to create a version that consisted of lines rather than polygons (in order to attempt to look at the topology of field systems in particular).  I said that I would look into using the Thin algorithm in GRASS GIS, but I then noticed that ArcGIS also has a Thin tool, so I decided to play with that instead!

After some experimentation, I came up with a process for performing this conversion, implemented as three iterative models in ArcGIS.  The first stage reclassifies the raster tiles so that white areas have a value of ‘NoData’ and then runs the Thin tool on each tile (to, as you might guess, thin all of the lines):

Model stage 1

The second stage then iterates through the results of the first, performs the same reclassification (as the Thin tool seems to output a binary result with values of 0 and 1), then converts each raster tile into line shapefiles:

Model stage 2

For the final stage, we then make sure the projection is defined correctly, trim any small lines that are likely to be artifacts of the process rather than real features (less than 3m in length), and then run the Smooth Line tool to try to improve the aesthetic quality of the result:

Model stage 3

The results of this process look quite good on first appearance, and removing grid marks / lines worked even better than with the polygon results (using the same methodology as previously described):

brickwork fields
Line version of vectorised raster NMP (brickwork fields in Nottinghamshire).

However, the results are significantly more problematic than when converting to polygons.  As an example, areas of rig and furrow are represented in the raster NMP drawings as dotted outlines with arrows showing the direction of the furrows.  These show up fine in the polygon version, but the dotted outlines disappear in the line version.*  Yet the arrows remain.  Therefore, if studying the line version in isolation, there is no way to tell that the lines representing the arrows were once arrows and are not drawings of archaeological features.  Something similar happens with text on the maps.

Comparison of line (red) and polygon (grey / black) vectorised versions of raster NMP: note the disappearance in the line version of the dotted outlines of rig and furrow areas.

As such, on balance, this line version is less useful than the polygon version.  However, for areas of field systems it will be simpler to use this version for the study of topological relationships.  In order to do so, however, it is necessary to extract the field systems from other features and fill small gaps in the drawings of what are clearly continuous features in reality (which presumably exist due to gaps in the crop marks / earthworks).  The former could only be done manually, but I did make an attempt to extend the lines automatically to fill gaps.  The tool took 15 hours to run for a small section of Nottinghamshire and, although successful in some cases, on the whole produced a very messy (and useless!) result:

The rather messy result of trying to automate filling gaps in lines (originals in black, extensions in red).

As such, if I am to use this data to study the topology of field systems, I will have to make all these edits manually: to eliminate small gaps in continuous features and to remove features that are not themselves part of the field systems.  I will experiment with this and report again at a later date.

Chris Green

*  Incidentally, although this result is problematic for most purposes, it may provide a route into converting these areas defined by dotted outlines into polygons covering the extent of the enclosed area, by creating buffers around the line result and erasing the buffered areas from the polygon result (and then processing this somehow into filled polygons: I’m not sure how!), but that is something that will require a lot more thought / experimentation.