Before we all sign off for Christmas, let us take this opportunity to thank the following HERs for kindly providing us with data over the last few months: Leicester City, Canterbury, Yorkshire Dales, Lake District, Plymouth, Nottinghamshire and Portsmouth. Special thanks must also go to Keith Westcott of ExeGesIS for helping with running the query in a few instances. This makes our HER data for the entire country almost complete!
Thanks as well to Helen Saunders from Essex HER for providing us with NMP data, and to Simon Crutchley, Lindsay Jones and Poppy Starkie for completing our NMP / NRHE data supply, and to Simon Crutchley and Mandy Roberts for taking the effort to visit us in Oxford. Finally, many thanks to the organisers of the HLC conference in London earlier this month for inviting us.
Last but not least, the EngLaId team wishes everyone a VERY HAPPY CHRISTMAS, and a wonderful start of 2013!
We are now in receipt of all the NMP data (and associated NRHE data) currently possessed by English Heritage, alongside a couple of regions which were kindly supplied directly by the local HERs (Norfolk and Essex), and we would like to extend our thanks to Simon Crutchley, Lindsay Jones and Poppy Starkie for their work in pulling together these datasets for us.
I have previously discussed methodologies for processing the scanned (raster) maps which represent the results of the earlier NMP surverys (1) (2). I am reasonably satisfied with the polygon result, but one issue that I have discussed with Simon Crutchley is whether it is possible to convert areas of rig and furrow (drawn with a dotted outline) into polygons representing their extent (rather than individual polygons for each dot). Here is an example of the raster NMP:
Raster NMP example
And here is the same data converted into polygons (with grid marks removed):
Vectorised polygons generated
The first stage in converting the dotted outlines into filled polygons is to generate the line version of the same raster input data:
Vectorised lines (red) generated, overlaid on polygons (click to enlarge)
We then create a 5m buffer around these lines (i.e. total width 10m):
5 metre buffers around lines (red)
And then use this buffer layer to delete most linear features from the polygon version of the data (using the Erase tool in ArcGIS):
Buffered areas erased from polygons
Most of the remaining objects are associated with areas of rig and furrow. However, we can further improve the result. First, we recalculate the areas of each polygon and filter the layer down so that we are only dealing with polygons of between 3 and 30 square metres in extent:
Erased result filtered down to polygons of 3 to 30 sq. metres
This removes a few remaining linear features that were not previously erased. Next, we generate centroids for each polygon (using the Feature to Point tool), run the Near tool on the result to get the distance to each resulting point’s nearest neighbour and filter (red dots) out those that are above a certain distance from their nearest neighbour (blue dots). In this instance, I chose 40 metres, but I think a smaller value would have been better (probably 30m):
Centroids generated for each polygon, filtered down to those within 40 metres of another point (red); blue points are those eliminated
We now have a point layer which for the most part represents the vertices for creating our rig and furrow polygons. We can run the Near tool again, this time asking it to give the spatial location of the nearest neighbouring point for each point, and use the Calculate Geometry tool to insert two fields into the layer giving the location of each origin point. The XY to Line tool can then create lines between each point and its nearest neighbour:
Lines generated from points to their nearest neighbour
This result is getting fairly close to what we desire, but has some considerable problems. First, none of the lines perfectly enclose the polygons, making it currently impossible to immediately process this result into a polygon layer. It might be possible to fill some of these gaps using the Extend Line tool, but that is a very computer intensive task and liable to still produce an imperfect result (I left it running for 24 hours on this relatively small dataset before I gave up and cancelled it). Second, in some instances, two lines of parallel dots can be closer to each other than the dots are within each line. In this instance, the generated lines are drawn as links between the two parallel lines rather than along each individual line.
As a result, currently, if we were to try to convert this result into polygons, we would first need to fill in all of the gaps manually using editing tools and possible also delete all of the small sections of line that have no association with rig and furrow (albeit we could ignore most of these as they will not produce closed polygons in the result or, if they did, they are likely to be small in area and thus possible to easily filter out). I do not think it is possible with out-of-the-box tools in ArcGIS to improve upon the line generation result, although it might be possible to develop a new tool to do so (perhaps with a directional bias towards its nearest neighbour to encourage linearity?).
I shall keep thinking about how to improve this process.
English Landscape and Identities 